Category Archives: PHP

A Guide to Base Changing For Short URLs

Some time ago, I developed a VERY simple way to fake a bit.ly-style short URL. On any server that uses any form of an integer to identify an article (either in the database or the URL), on an Apache server that supports mod_rewrite, you edit your .htaccess file like so:

RewriteEngine Off
<ifmodule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !^index.php$
RewriteRule . index.php [NC,L]
</ifmodule>

This essentially tells your server to redirect anything that isn’t a file or directory to index.php.

Then index.php looks like this:

$url = str_replace("/","",$_SERVER['REQUEST_URI']);
if(isset($url) && trim($url)!='') {
$id = base_convert($url,36,10);
//this is where you either query your database for a slug or build the URL
$uri = 'http://your-site-goes-here.com/path/to/article';
}
 
if($uri) {
header('HTTP/1.1 301 Moved Permanently');
header('Link: < '.$uri.">; rel=shortlink");
header('Location: '.$uri);
exit;
} else {
header("Location: http://your-site-goes-here.com/"); exit;
}

How do you get your short links? That’s easy. Just run this function:

$shorturl = base_convert($id,10,36);

However, this isn’t the most compact way to condense. Obviously, this is base36, the highest PHP can go. But what about uppercase letters? And other characters?

So I set out, for some reason, to build a better condenser.

This is the result of several hours of work, mostly wasted, on some intellectual pursuit that was more a case of simply not letting it defeat me. A few notes: I’m quite confident that given enough time, and if I cared, I could make the code cleaner and more efficient in some places. I’m also aware that on 32 bit machines, it maxes out at the integer limit. I does support signed integers though, from min to max.

$ft_str = '0123456789abcdefghijklmnopqrstuvqwxyzABCDEFGHIJKLMNOPQRSTUVQWXYZ';
# uncomment the next line if you prefer to use potentially non-URL-safe base96
# $ft_str .= '0123456789abcdefghijklmnopqrstuvqwxyzABCDEFGHIJKLMNOPQRSTUVQWXYZ_@$!#%^&*()=+\|}{][,;:~'; }

$powers = array();
for($p=0;$p&lt;=10;$p++) {
$powers[$p]=pow(strlen($ft_str),$p);
}
 
function ft_unconvert($str) {
if(substr($str,0,1)=='-') { $pfx='-'; } else { $pfx=''; }
global $ft_str,$powers;
$base = strlen($ft_str); $q=0;
$s = str_split(strrev($str)); $len = sizeof($s);
foreach($s as $k=>$v) {
$sp = strpos($ft_str,$v);
$decimal += pow($base,$k)*$sp;
}
return $pfx.$decimal;
}
 
function ft_converter($int) {
if($int > 0) { $pfx='-'; $int=abs($int); } else { $pfx=''; }
global $ft_str,$powers;
$base = strlen($ft_str); $q=0;
$p = str_split($str);
krsort($powers);
foreach($powers as $k=>$v) {
if($int>=$v) {
$timesinto = floor($int/$v);
$digit .= $ft_str{$timesinto};
$int = $int % $v;
} elseif($int>0 && $k==0) {
$digit .= $ft_str{$int};
$int = $int % $v;
}
}
return $pfx.$digit;
}
 
function ft_convert_demo($num) {
global $ft_str;
$ftc = ft_converter($num);
return "Converting ".$num." into base ".strlen($ft_str).": ".$ftc."<br />Unconverting ".$ftc." to base 10: ".ft_unconvert($ftc);
}
 
echo ft_convert_demo('50687');

I’ve Been Hacked! WP-Lytebox Sucks

Hacking WordPressThis is the third time I’ve had to spent serious time “fixing” WordPress.  Say what you will about my old “Small Axe” or “Flip” solutions, but I never had a problem.  With WordPress, I have found regular issues.

This time, I traced the problem back to wp-lytebox, and I’m ashamed to say I’ve had to fix the same problem before.  It all started when I couldn’t load the post page in the backend of firsttube.com.  Digging in, I eventually found a file called sys.php in the root of the site, and it listed the contents of my site and had a form that allowed someone to add a page, chmod a page, or delete a page.  Killer!

I found that it was defaulting to /path/to/WP/wp-includes/plugins/wp-lytebox, and sure enough, digging into that directory revealed several other fun scripts, all of which gave someone the ability to access all the files on my site.  Fun!

I found that I already had replaced this plugin before, so I decided to get rid of it altogether, this now proving it wasn’t a misconfiguration, but rather, a problem with the wp-lytebox itself.

In this process, however, I was unable to fix my issue.  Visiting /wp-admin/post-new.php still rendered only a page footer, and nothing more.

So I starting fooling around in my directories looking for files that had been modified more recently than when I did my 2.9.2 upgrade.  One of the files? My .htaccess file.

This be odd,” I thought to myself, “I’ve changed this not, methinks.

Sure enough, there was a rogue line within: RewriteCond ^/default/$ /wp-admin/includes Huh?

I dug into that folder, and the .htaccess file there was recent too? It’s contents? DefaultIndex users.php

Of course, I immediately opened users.php and found, as you might have guessed, a bunch of Russian crap. Savvy WP hackers will know, it’s not a real file, there is no users.php in the real wp-admin/includes directory.

I also found a folder that had two large files, both named core.XXXX where XXXX was a 4 digit number, and a massive 40 MB error_log.  Yikes.

I thought I had everything cleaned out, and I truly believed that the way in was wp-lytebox.  Then I found this.  And sure enough, all of the listed files were compromised.  So I nuked all the files, and replaced them all.  D’oh!

So, if you’re arriving via Google or Bing or Yahoo!, do NOT use wp-lytebox.

Behind the Scenes at OSNews

I just started putting together a series of articles I will be publishing on OSNews.  I’ve only roughly sketched it out, but in short, it’s going to discuss how OSNews works, how the PHP is structured, why we made certain architectural decisions, why we don’t use tried-and-true CMSes like WordPress, Slash, or Joomla!, and how, during peak traffic times, we have survived 30,000 unique visitors per hour on a single server.  OSNews didn’t happen by mistake: over a series of months, arguably years, we took a constant read of performance, hits, server load, and usability with the mission to continually improve load time, performance, and UX.  We’ve just recently begun testing some new data presentation methods that I intend to include in my little exposé.

If you’re interested in some revealing behind-the-scenes info, feel free to ask questions now.

Flip 3.0.1

I never expected to release another version of my old weblog project Flip, but while searching my own name in a new search engine, I came upon several vulnerability reports for Flip 3.0. I’ve known about them for awhile now, but having dropped Flip in favor of another project (which I’ve since abandoned, for the most part, in favor of WordPress), it seemed pointless to bother. However, since there is an active exploit, I thought I’d release an update and a patch.

I don’t believe anyone out there is still using Flip, but if there is, this is how to defeat the script: simple add this line at line 102 of account.php:

 if(strstr($_POST['em'],"][")) { die('Fail'); }

and this at line 162:

 if(strstr($_POST['nem'],"][")) { die('Fail'); }

Alternatively, you can download the modified file here or download Flip 3.0.1 here.

It may sound odd, but I would highly recommend that you do *not* use this code. It’s now 7 years old and the web is a much different place. The code here is really not suited for running a website today. That said, it was odd to unzip and install it and see that it actually works. The rendering of most of the “themes” is weird (Fudge works great), but otherwise, everything worked.

If you are still a Flip user, I recommend you update your account.php page immediately, and if you have the time and inclination, upgrade to 3.0.1. The following files have some minor changes:

  • account.php
  • index.php
  • inc/config.php
  • README.html

Once again, this code is aged not particularly well suited for today’s web.  If you want a simple weblog, I recommend WordPress.

Posting Your Latest Tweet in WordPress

Although I posted yesterday how to add your latest tweet to WordPress without a plugin, I made several changes to the script before I posted it to make it more “generic” and re-usable. Since I’ve changed it quite a bit, I decided to repost it. This new script also autolinks @usernames and #hash tags.

Directions are this easy: set the path of $tw_File with a static, writable file.  Set $tw_userid to your Twitter user id.  Done. 

Download firsttube.com “get latest tweet” php snippet.

Sorting a Multi-Dimensional Array with PHP

Every so often I find myself with a multidimensional array that I want to sort by a value in a sub-array. I have an array that might look like this:

//an array of some songs I like
$songs =  array(
		'1' => array('artist'=>'The Smashing Pumpkins', 'songname'=>'Soma'),
		'2' => array('artist'=>'The Decemberists', 'songname'=>'The Island'),
		'3' => array('artist'=>'Fleetwood Mac', 'songname' =>'Second-hand News')
	);

The problem is thus: I’d like to echo out the songs I like in the format “Songname (Artist),” and I’d like to do it alphabetically by artist. PHP provides many functions for sorting arrays, but none will work here. ksort() will allow me to sort by key, but the keys in the $songs array are irrelevant. asort() allows me to sort and preserves keys, but it will sort $songs by the value of each element, which is also useless, since the value of each is “array()”. usort() is another possible candidate and can do multi-dimensional sorting, but it involves building a callback function and is often pretty long-winded. Even the examples in the PHP docs references specific keys.

So I developed a quick function to sort by the value of a key in a sub-array. Please note this version does a case-insensitive sort. See subval_sort() below.

function subval_sort($a,$subkey) {
	foreach($a as $k=>$v) {
		$b[$k] = strtolower($v[$subkey]);
	}
	asort($b);
	foreach($b as $key=>$val) {
		$c[] = $a[$key];
	}
	return $c;
}

To use it on the above, I would simply type:

$songs = subval_sort($songs,'artist'); 
print_r($songs);

This is what you should expect see:

Array
(
    [0] => Array
        (
            [artist] => Fleetwood Mac
            [song] => Second-hand News
        )
 
    [1] => Array
        (
            [artist] => The Decemberists
            [song] => The Island
        )
 
    [2] => Array
        (
            [artist] => The Smashing Pumpkins
            [song] => Cherub Rock
        )
 
)

The songs, sorted by artist.

PHP Weirdness

Beware: this post is definitely not for the feint of heart. It includes a lot of code. You have been warned.

I wrote an application some time ago for my company that looks up the longitude and latitude of an address for use in our geocoding initiative. It relied on yahoo_geo(), a function written by PHP creator Rasmus Lerdorf and the Yahoo Maps API. It was largely dependent on this function:

function yahoo_geo($location) {
	$q = 'http://api.local.yahoo.com/MapsService/V1/geocode?appid=rlerdorf&amp;location='
		.rawurlencode($location);
	$tmp = '/tmp/yws_geo_'.md5($q);
	request_cache($q, $tmp, 43200);
	libxml_use_internal_errors(true);
	$xml = simplexml_load_file($tmp);
	$ret['precision'] = (string)$xml-&gt;Result['precision'];
	foreach($xml-&gt;Result-&gt;children() as $key=&gt;$val) {
		if(strlen($val)) $ret[(string)$key] = (string)$val;
	}
	return $ret;
}

This function worked for over two years for us with no problems at all. Then suddenly, in the last month, it started getting spotty. I fixed things by commenting out the caching parts of the function and forcing each execution to run again. Then I got errors about the libxml_use_internal_errors() function, so I commented that out. But today, the function just flat out failed, every single time returning the same error:

Warning: file_get_contents(http://XXXXXXXXXX/XXX) [function.file-get-contents]: failed to open stream: HTTP request failed! in /home/intranet/html/fetch.php on line X

What the heck? This code is all over the web. I’ve tried a million permutations of this function, including using fopen() and ob_get_contents(), and none have worked. And most frustratingly, I could load the URL successfully in Lynx and eLinks, so the machine could quickly and easily fetch the URL.

So I ventured into a sandbox I’ve never really played before: cURL. cURL is an interesting animal. But the interesting thing is, once I got it working, it worked faster than ever! So, without further ado, here is the new and improved yahoo_geo() function:

function yahoo_geo($location) {
	$q = 'http://api.local.yahoo.com/MapsService/V1/geocode?appid=rlerdorf&#038;location='.urlencode(trim($location));
	$ch = curl_init($q);
	curl_setopt($ch, CURLOPT_HEADER, 0);
	ob_start();
	curl_exec($ch);
	$stream = ob_get_contents();
	ob_end_clean();
	if($stream) {
		$xml = simplexml_load_string($stream);
		$ret['precision'] = (string)$xml->Result['precision'];
		if($xml) {
			foreach($xml->Result->children() as $key=>$val) {
				if(strlen($val)) $ret[(string)$key] =  (string)$val;
			}
		}
		curl_close($ch);
		return $ret;
	} else {
		return FALSE;
	}
}

Note: If you’re reproducing these functions elsewhere, be careful – WordPress may have converted the quotes into smart quotes that will need to be fixed before this script will run properly.

OSNews vs. WordPress

I’ve spent quite a bit of time, over the last 5 or 6 days, diving into WordPress and learning what makes it tick.  Parts of WordPress are really impressive – just flat out cool. The way some of it works is fairly complex and deciphering it sometimes means reading page after page after page to understand an entire routine.  But sometimes, when you finally see, end to end, how something in WordPress works –  I mean really see individual bits of the engine – you have to admit it teaches you a little about PHP.  WordPress, underneath it all, is a pretty big beast and its strength and ubiquitous presence comes largely, I think, from the fact that it can do virtually anything.  The really sweet plugin system, the ways hooks work, “The Loop,” the dynamic options panel – it’s all very educational.  

The interesting thing here is that I’ve browsed the source of Slash, Scoop, phpNuke, and now WordPress, and all of them are definitively more complex and much heavier than the entire OSNews codebase. Now, before you jump all over me – firstly, Slash and Scoop are Perl, and I don’t really read Perl, so I can’t speak as an expert there.  Secondly, WordPress and Nuke both are very portable and dynamic, whereas OSNews has a narrow focus and, location-wise, is very static.  But that aside, OSNews has withstood simultaneous link bombs from Slashdot and Digg.  As amazing as WordPress is, it’s mostly amazing that it functions at all and loads in less than 2 minutes per page with as much going on as I can see behind the scenes.   That’s not a cut on WordPress, by the way.

In fact, if anything , what is really impressed upon me is how smooth and simple OSNews code is, if I may be so bold.  OSNews runs superfast due, in part, to lots of creative caching, some on-demand, some via cron.  But it also does so because of highly efficient queries that are measured for speed on their JOINs, meaning in some cases, it’s faster to do 20 simple queries than one complex one, or build a long and scary chain of “OR x=a OR x=b OR x=c OR x=d…”  Watching WordPress code in action is really fun for me, but watching OSNews work knowing what I now know about how much work PHP can cram into its threads is even more fun.

Integers on the Intertubes

Some time ago, I wrote an application for my company. Like most weblets I’ve written, this used PHP and either MySQL or MSSQL for the backend. This particular application logged all phone calls. As part of the record, it would record the caller’s account number, which is a 5 or 6 digit integer.

So, I got a phone call from the director of our customer contact department this week. He was concerned about the reports. He made a decision last week that when a call came in that was a lead – in other words, a non-customer, that his people would fill the phone number from the caller ID into the account number field. But when he ran his export reports, he found that hisn techs had entered this phone number for ALL of the calls: 429-496-7295. That’s weird, he said. So he called me and asked why that was. I checked all the calls and most were from one woman, so my first instinct was “Check if her browser has autocomplete turned on”. But he swore that he tried it too and gotten the same result.

I checked the database and sure enough, it was right there: 429-496-7295, in all of the fields. So I went back to the code. In short, I took the input from the form, and declared it like this:

$accountnum = (int) $_POST['accountnum'];

Pretty straightforward: explicitly declare the type. So, I started my debugging by attempting to manually enter the data into the database. Sure enough: the account key field showed this: 4294967295.

So, I went back to the PHP and started by dumping out the raw SQL query:

INSERT INTO calls ('','x','x','x','4294967295','x','x');

What? So the database automatically converts it to this weird phone number and PHP does too? Suddenly it occured to me. One of the benefits of 64-bit computing is the ability to address more memory. There are limits to what can be done in 32-bit computing, and one is that integers have a limit! In this case, a database field called “integer” is limited to numbers between -2,147,483,648 and +2,147,483,647. It just so happens that the number is the same length as a US phone number – 10 digits. Changing the db field to “BIGINT” allowed me to manually run the SQL query and it worked. But the app still didn’t.

PHP’s int() and (int) $var syntaxes both conform to the integer limit. So I devised a work around:

$ac = $_POST['accountnum'];
if(!is_numeric($ac) { $ac = (int) $ac; }

It’s not gorgeous, but it will more than suffice for an internal app. We web programmers don’t usually have to deal with big integers, so it’s totally possible that web developers would never have had occasion to handle a situation like this. Here’s looking forward to native 64-bit for our next server, though.