Tag Archives: Code

Creating a JSON API

I was assisting someone recently in building an API for their website and it occurred to me that while the current trend is generally using XML/REST/SOAP for APIs, there is quite a bit of benefit to using plain old Javascript and JSON.  Most users won’t venture into API territory, so if your goal is to make your API accessible – and this goes double if your primary purpose is embedding content in a third party site – it’s hard to argue with Javascript.

The Phish.net API, for example, is a simple HTTP request to an endpoint that returns JSON.  If you provide a callback function name as an argument, and then pre-define that function, it will return the contents wrapped in a function call.  In short, if you define a function called “example()” that accepts JSON an array/object as an argument, then by requesting the API with a callback of “example”, the response will be returned like so:

example(json response);

The benefit, of course, is that it allows a user to embed your code easily. If you host your own callback functions, you can very easily walk a user through a data embed. For example, on Phish.net, we offer the band’s latest setlist.  So we host a callback called pnet3setlist().  Then we offer the API response.  Embedding the setlist is literally as easy as this:

<script src="http://api.phish.net/callbacks/pnet3-setlist.js" type="text/javascript"></script>
<script src="https://api.phish.net/api.json?method=pnet.shows.setlist.recent&amp;amp;callback=pnet3setlist" type="text/javascript"></script>

I’m going to dig more into this later, but the gist of this is that the API can be constructed in a way that allows novice users to interact with it in their WordPress- or Blogger-blogs with little to no modification, it allows semi-skilled users to modify it slightly and tweak it for display purposes, and it allows advanced users to control virtually all aspects. Since virtually every major web language can encode and decode JSON (Javascript, PHP, Ruby, ASP, etc), it’s a near-universal way to exchange data. It doesn’t carry the overhead or complexity of XML, nor does it have the limited scope of something like serialized PHP.

Javascript-based APIs may not be the ultimate solution to a fully interconnected web, but they’re certainly going to be one of the best and simplest methods of data exchange for the foreseeable future.

A Guide to Base Changing For Short URLs

Some time ago, I developed a VERY simple way to fake a bit.ly-style short URL. On any server that uses any form of an integer to identify an article (either in the database or the URL), on an Apache server that supports mod_rewrite, you edit your .htaccess file like so:

RewriteEngine Off
<ifmodule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !^index.php$
RewriteRule . index.php [NC,L]
</ifmodule>

This essentially tells your server to redirect anything that isn’t a file or directory to index.php.

Then index.php looks like this:

$url = str_replace("/","",$_SERVER['REQUEST_URI']);
if(isset($url) && trim($url)!='') {
$id = base_convert($url,36,10);
//this is where you either query your database for a slug or build the URL
$uri = 'http://your-site-goes-here.com/path/to/article';
}
 
if($uri) {
header('HTTP/1.1 301 Moved Permanently');
header('Link: < '.$uri.">; rel=shortlink");
header('Location: '.$uri);
exit;
} else {
header("Location: http://your-site-goes-here.com/"); exit;
}

How do you get your short links? That’s easy. Just run this function:

$shorturl = base_convert($id,10,36);

However, this isn’t the most compact way to condense. Obviously, this is base36, the highest PHP can go. But what about uppercase letters? And other characters?

So I set out, for some reason, to build a better condenser.

This is the result of several hours of work, mostly wasted, on some intellectual pursuit that was more a case of simply not letting it defeat me. A few notes: I’m quite confident that given enough time, and if I cared, I could make the code cleaner and more efficient in some places. I’m also aware that on 32 bit machines, it maxes out at the integer limit. I does support signed integers though, from min to max.

$ft_str = '0123456789abcdefghijklmnopqrstuvqwxyzABCDEFGHIJKLMNOPQRSTUVQWXYZ';
# uncomment the next line if you prefer to use potentially non-URL-safe base96
# $ft_str .= '0123456789abcdefghijklmnopqrstuvqwxyzABCDEFGHIJKLMNOPQRSTUVQWXYZ_@$!#%^&*()=+\|}{][,;:~'; }

$powers = array();
for($p=0;$p&lt;=10;$p++) {
$powers[$p]=pow(strlen($ft_str),$p);
}
 
function ft_unconvert($str) {
if(substr($str,0,1)=='-') { $pfx='-'; } else { $pfx=''; }
global $ft_str,$powers;
$base = strlen($ft_str); $q=0;
$s = str_split(strrev($str)); $len = sizeof($s);
foreach($s as $k=>$v) {
$sp = strpos($ft_str,$v);
$decimal += pow($base,$k)*$sp;
}
return $pfx.$decimal;
}
 
function ft_converter($int) {
if($int > 0) { $pfx='-'; $int=abs($int); } else { $pfx=''; }
global $ft_str,$powers;
$base = strlen($ft_str); $q=0;
$p = str_split($str);
krsort($powers);
foreach($powers as $k=>$v) {
if($int>=$v) {
$timesinto = floor($int/$v);
$digit .= $ft_str{$timesinto};
$int = $int % $v;
} elseif($int>0 && $k==0) {
$digit .= $ft_str{$int};
$int = $int % $v;
}
}
return $pfx.$digit;
}
 
function ft_convert_demo($num) {
global $ft_str;
$ftc = ft_converter($num);
return "Converting ".$num." into base ".strlen($ft_str).": ".$ftc."<br />Unconverting ".$ftc." to base 10: ".ft_unconvert($ftc);
}
 
echo ft_convert_demo('50687');

I’ve Been Hacked! WP-Lytebox Sucks

Hacking WordPressThis is the third time I’ve had to spent serious time “fixing” WordPress.  Say what you will about my old “Small Axe” or “Flip” solutions, but I never had a problem.  With WordPress, I have found regular issues.

This time, I traced the problem back to wp-lytebox, and I’m ashamed to say I’ve had to fix the same problem before.  It all started when I couldn’t load the post page in the backend of firsttube.com.  Digging in, I eventually found a file called sys.php in the root of the site, and it listed the contents of my site and had a form that allowed someone to add a page, chmod a page, or delete a page.  Killer!

I found that it was defaulting to /path/to/WP/wp-includes/plugins/wp-lytebox, and sure enough, digging into that directory revealed several other fun scripts, all of which gave someone the ability to access all the files on my site.  Fun!

I found that I already had replaced this plugin before, so I decided to get rid of it altogether, this now proving it wasn’t a misconfiguration, but rather, a problem with the wp-lytebox itself.

In this process, however, I was unable to fix my issue.  Visiting /wp-admin/post-new.php still rendered only a page footer, and nothing more.

So I starting fooling around in my directories looking for files that had been modified more recently than when I did my 2.9.2 upgrade.  One of the files? My .htaccess file.

This be odd,” I thought to myself, “I’ve changed this not, methinks.

Sure enough, there was a rogue line within: RewriteCond ^/default/$ /wp-admin/includes Huh?

I dug into that folder, and the .htaccess file there was recent too? It’s contents? DefaultIndex users.php

Of course, I immediately opened users.php and found, as you might have guessed, a bunch of Russian crap. Savvy WP hackers will know, it’s not a real file, there is no users.php in the real wp-admin/includes directory.

I also found a folder that had two large files, both named core.XXXX where XXXX was a 4 digit number, and a massive 40 MB error_log.  Yikes.

I thought I had everything cleaned out, and I truly believed that the way in was wp-lytebox.  Then I found this.  And sure enough, all of the listed files were compromised.  So I nuked all the files, and replaced them all.  D’oh!

So, if you’re arriving via Google or Bing or Yahoo!, do NOT use wp-lytebox.

Javascript Ninja!

Thank you, John Resig. Because of you, I’m learning about Javascript the way Andy Gadiel taught me HTML. In the days before server-side scripting, I learned my first bits of HTML largely by viewing the source of Andy Gadiel’s Phish page (which, for some reason, remains largely unchanged since ~1997).  By reading Gadiel’s HTML, I slowly pieced together my own understanding of HTML.  It was Joe Burns’ fantastic Javascript Goodies that first had me dipping my n00b fingers into client side active scripting.  I picked up CSS all over the web.

Resig’s jQuery is so powerful and so easy that even with basic knowledge of CSS and Javascript, anyone can be a virtual scripting master.  It’s so easy, that I’ve slacked on learning about javascript objects, inheritance, closures, anonymous functions, prototypes, and scores of  other Javascript staples that I should’ve long since mastered.   I just discovered John’s new web app, cleverly titled “Learning Advanced Javascript“, and so far, so good!

I wrote this myself and understand why it works, which is much more than I could say yesterday.

var ninja = {
	walk: function(steps,turn) {
		toDo = 'Walking '+steps+' steps forward, then turning '+turn;
		return this;
	  },
	star: function(action,distance) {
		toDo = toDo+' '+action+'ing star '+distance+' feet'; return this;
	  },
	then: function() {
		toDo = toDo+', then '; return this;
	},
	doIt: function() {
		log(toDo);
	}
}
ninja.walk('7','south').then().star('throw','50').doIt();

Output:

> Walking 7 steps forward, then turning south, then throwing star 50 feet

It’s clear to me – and has been for some time – that the future of the web, for better or for worse, rests heavily on the mighty shoulders of client side scripting.   Building on powerful, extensible frameworks like jQuery and MooTools, the next generation of web apps is sure to compete with the desktop.  The ability to understand how to utilize the frameworks when necessary and hack together powerful scriptlets for other purposes seems essential to success in the future web. I know I’ll be investing in “Secrets of a Javascript Ninja” just as a result of this tutorial.

Behind the Scenes at OSNews

I just started putting together a series of articles I will be publishing on OSNews.  I’ve only roughly sketched it out, but in short, it’s going to discuss how OSNews works, how the PHP is structured, why we made certain architectural decisions, why we don’t use tried-and-true CMSes like WordPress, Slash, or Joomla!, and how, during peak traffic times, we have survived 30,000 unique visitors per hour on a single server.  OSNews didn’t happen by mistake: over a series of months, arguably years, we took a constant read of performance, hits, server load, and usability with the mission to continually improve load time, performance, and UX.  We’ve just recently begun testing some new data presentation methods that I intend to include in my little exposé.

If you’re interested in some revealing behind-the-scenes info, feel free to ask questions now.

Flip 3.0.1

I never expected to release another version of my old weblog project Flip, but while searching my own name in a new search engine, I came upon several vulnerability reports for Flip 3.0. I’ve known about them for awhile now, but having dropped Flip in favor of another project (which I’ve since abandoned, for the most part, in favor of WordPress), it seemed pointless to bother. However, since there is an active exploit, I thought I’d release an update and a patch.

I don’t believe anyone out there is still using Flip, but if there is, this is how to defeat the script: simple add this line at line 102 of account.php:

 if(strstr($_POST['em'],"][")) { die('Fail'); }

and this at line 162:

 if(strstr($_POST['nem'],"][")) { die('Fail'); }

Alternatively, you can download the modified file here or download Flip 3.0.1 here.

It may sound odd, but I would highly recommend that you do *not* use this code. It’s now 7 years old and the web is a much different place. The code here is really not suited for running a website today. That said, it was odd to unzip and install it and see that it actually works. The rendering of most of the “themes” is weird (Fudge works great), but otherwise, everything worked.

If you are still a Flip user, I recommend you update your account.php page immediately, and if you have the time and inclination, upgrade to 3.0.1. The following files have some minor changes:

  • account.php
  • index.php
  • inc/config.php
  • README.html

Once again, this code is aged not particularly well suited for today’s web.  If you want a simple weblog, I recommend WordPress.

Posting Your Latest Tweet in WordPress

Although I posted yesterday how to add your latest tweet to WordPress without a plugin, I made several changes to the script before I posted it to make it more “generic” and re-usable. Since I’ve changed it quite a bit, I decided to repost it. This new script also autolinks @usernames and #hash tags.

Directions are this easy: set the path of $tw_File with a static, writable file.  Set $tw_userid to your Twitter user id.  Done. 

Download firsttube.com “get latest tweet” php snippet.

How to Add Latest Tweet to WordPress (Without a Plugin)

I decided to add my latest “tweet” from Twitter to the sidebar of my WordPress blog. Rather than use yet another plugin that adds yet another hook – and there are many that do this with lots of code, I decided to use a homegrown solution, dependant only on PHP4+ and cURL  (most webhosts already have cURL compiled in, if not, you should request it).  Adding the following to any of the files in your WordPress theme will print out your current Twitter status and cache the results so you don’t hammer their system.

First, snag your Twitter user id.  Then, open up your theme file.  I put mine in sidebar.php found in /wp-content/themes/<THEMENAME>/.    Use the below code.  If you want the output wrapped in a list, you would need to put <ul> and <li> tags around this code.

Carefully set your variables.  The cache file should be writable.  Note that you can use a decimal value for $tw_BlankAfter and $tw_Minutes if necessary.   That’s it.

Due to what must be a bug in WordPress, please ignore the closing “</text></created_at>” at the end of this post.  It’s trying be smart and “fix” broken tags, but the code is right.

NOTE (2/20/09): I have updated the below code.  The new version can be found at “Posting Your Latest Tweet in WordPress“.

/* ~~~~ Custom Twitter Bit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */
/* ~~~~ Adam S, firsttube.com, twitter @sethadam1 ~~~~~~~~~~~~~~~~~~~~ */
 
$tw_File = '/path/to/a/static/writable/file/twitter.html';
$tw_Userid='XXXXXXX'; //set to your Twitter user id
$tw_BlankAfter = 30; //blank out status if it's older than this many days
$tw_Minutes = 10; //minutes between reloads
 
$tw_Offset = FALSE; //leave as is
// uncomment below time if you want to allow a manual reset via ?twitter-reset
// if($_SERVER['QUERY_STRING']=='twitter-reset') { $tw_Offset=0; } 
 
/* Do not edit below this line */
if(filemtime($tw_File)&gt;time()-floatval($tw_Offset)) {
	include $tw_File;
} else {
	if(is_writable($tw_File)) { $tw_iswritable=1; }
	$tw_time = (86400*floatval($tw_BlankAfter));
	if($tw_Offset) { $tw_time=$tw_Offset; }
	$tw_hyperlinks = true;
	$tw_c = curl_init();
	curl_setopt($tw_c, CURLOPT_URL,
		"http://twitter.com/statuses/user_timeline/"
		.intval($tw_Userid).".xml");
	curl_setopt($tw_c, CURLOPT_RETURNTRANSFER, 1);
	$tw_src = curl_exec($tw_c);
	curl_close($tw_c);
	preg_match('/(.*)&lt; \/created_at&gt;/', $tw_src, $tw_d);
	if(strtotime($tw_d[1]) &gt; time()-$tw_time) {
		preg_match('/(.*)&lt; \/text&gt;/', $tw_src, $tw_m);
		$tw_status = htmlentities(str_replace("&amp;","&amp;",$tw_m[1]));
		if( $tw_hyperlinks ) {
			$tw_status = ereg_replace(
			"[[:alpha:]]+://[^&lt;&gt;[:space:]]+[[:alnum:]/]",
			"<a href="\">\\0</a>",
			$tw_status);
		}
		$tw_output = $tw_status;
	} else {
		if($tw_iswritable==1) {file_put_contents($tw_File,''); }
	} 
 
	if($tw_iswritable==1) { file_put_contents($tw_File,$tw_output); }
	echo $tw_output;
}
/* ~~~ /Custom Twitter Bit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ */

Please note that portions of this code come from the twtter_status() function that was not written by me, but is available from various sources online.

Update: Removed function and put code inline.

Sorting a Multi-Dimensional Array with PHP

Every so often I find myself with a multidimensional array that I want to sort by a value in a sub-array. I have an array that might look like this:

//an array of some songs I like
$songs =  array(
		'1' => array('artist'=>'The Smashing Pumpkins', 'songname'=>'Soma'),
		'2' => array('artist'=>'The Decemberists', 'songname'=>'The Island'),
		'3' => array('artist'=>'Fleetwood Mac', 'songname' =>'Second-hand News')
	);

The problem is thus: I’d like to echo out the songs I like in the format “Songname (Artist),” and I’d like to do it alphabetically by artist. PHP provides many functions for sorting arrays, but none will work here. ksort() will allow me to sort by key, but the keys in the $songs array are irrelevant. asort() allows me to sort and preserves keys, but it will sort $songs by the value of each element, which is also useless, since the value of each is “array()”. usort() is another possible candidate and can do multi-dimensional sorting, but it involves building a callback function and is often pretty long-winded. Even the examples in the PHP docs references specific keys.

So I developed a quick function to sort by the value of a key in a sub-array. Please note this version does a case-insensitive sort. See subval_sort() below.

function subval_sort($a,$subkey) {
	foreach($a as $k=>$v) {
		$b[$k] = strtolower($v[$subkey]);
	}
	asort($b);
	foreach($b as $key=>$val) {
		$c[] = $a[$key];
	}
	return $c;
}

To use it on the above, I would simply type:

$songs = subval_sort($songs,'artist'); 
print_r($songs);

This is what you should expect see:

Array
(
    [0] => Array
        (
            [artist] => Fleetwood Mac
            [song] => Second-hand News
        )
 
    [1] => Array
        (
            [artist] => The Decemberists
            [song] => The Island
        )
 
    [2] => Array
        (
            [artist] => The Smashing Pumpkins
            [song] => Cherub Rock
        )
 
)

The songs, sorted by artist.