One place I have made some changes is in the RSS and Atom feeds. Although I advertise my feeds at feeburner, our source feeds are at firsttube.com/feed and firsttube.com/feed/atom. They have received some stylesheet love and are much more readable by the human eye.

If you want to make a mirror of your blogsome blog, you can use a single very powerful command to generate a snapshot of it from any Linux machine or Windows with Cygwin installed:

wget -k -m -r http://url

But this will only create a static HTML mirror of your website. It won’t allow you manipulate content or put it into another database. That leaves only one way to do it – request the entire site page by page, and parse each page individually. RSS is not reliable here, as most people have it set to only 15 or so items and parsing an enormous page make make PHP or your server run out of memory or alloted script execution time.

It’s a multi-step process, to be certain. It was actually painful to go through the process. Requesting over and over, debugging the script line by line. It takes several steps to get things right. But, eventually, I did it. I was able to export a Blogsome blog in its entirety – every entry, all the categories, all the comments with emails and websites …everything.

Slashdot was the same. It took some tinkering, but eventually, I was able to backup a very lengthy Slashdot journal. Again, in its entirety. I got every post, the date, etc. It was not simple, but it worked well. And I not only got them backed up, I merged them into a single database serving up… Small Axe 0.6 (which was a whole adventure of its own, taking current firsttube.com code and “neutralizing” it). Suffice it to say, when I saw all entries working and served in Small Axe, I had a huge smile. Turns out that the person I was doing this for decided against Small Axe (only because it doesn’t yet offer all the bells and whistles WordPress does, even if it is a beast). But it was irrelevant – the hard part was getting the data properly, and that’s done. Migrating to *any* blog database is possible if you have the time, inclination, and skill to write a SQL export/import script.

Here’s how it works:

* Cycle through each page of the blogsome blog. On each page, we get the entries, the URL, the postid, and other relevant info. We set a flag on each item to 0.
* As we retreive the items, we correct the path to images, spacing, smilies, etc.
* Then we cycle through each page individually. We have all the URLs already, so we go through each one and parse comments. It’s important to know that comment owner comments are marked up differently. As we get the comments, we upodate the flag and let the script run on its own. Our 900+ items took about an hour by meta-refreshing the fully rendered page every 3 seconds.
* As we go through the comments, we strip tags we don’t want, we fix emoticons, we fix internal links, spacing, etc. We must expose emails temporarily if we want them to transfer over.
* Finally, we import them all into a central database with an agreed schema.

If you have a Blogsome blog and/or a Slashdot journal you need backed up, I can help you do it. It’s not a simple process, but it is very accurate and preserves whatever data is exposed via HTML. So for the right barter, I would be very motivated to help. If I can simplify the process, I may create an open script to do this. But for now, I’ve got the code.

On a related note, I’ll probably release an updated version of Small Axe sometime in the not too distant future, because the amount of changing I’ve done and all the features I’ve just implemented are killer. Small Axe is FAR from WordPress caliber tested, but it’s SUPER simple and can do all the basics of a normal blog, including templating, smart per-domain caching, blocking by ip, username, email, or keyword, gravatar support, tons of configuration options, RSS and Atom support, threaded commenting, post locking, post expiring, browser identification, slug-based permalinking, and much more.

- Small Axe Encryption (which is VERY breakable (being as though the decrypt code is actually available, but a new version, which uses a password to decode, will eventually be out).

- Small Axe PHP XML-RPC API. YIKES. What a mouthful. It’s by far the easiest API out there. I have searched high and low for an API. Version 0.1 is so simple it’s crazy.

- Dango 2.1. A simple discussion board. Version 3 will be a complete rewrite, so this is a nice mini-step.

Also, I’m going to be rewriting the administration panel of Small Axe shortly. It’s not because it’s dirty, because it’s not, it’s actually modular and easy to write new plugins. However, It’s not ideally organized. I’ve updated the icons from a mix of random icons and Tango icons to mostly Echo icons from the forthcoming Fedora Core 7. I really like the new ones.

As you can see, I’ve finally added preference panes(!) for some features that I’ve had for awhile, such as User Agent review. The database cleanup link doesn’t work yet, nor does the Recent Votes link, but both are functions I will be implementing shortly.

Now, your basic operations are read, print, and comment.

So, the URLs work like so:

http://firsttube.com/read/Slug-Transition-Complete
http://firsttube.com/print/Slug-Transition-Complete
http://firsttube.com/comment/Slug-Transition-Complete

This requires no tweaking of Apache (other than permitting .htaccess files). There is no mod_rewrite going on here. It’s all in PHP. Huzzah.

It doesn’t work.

Above all things, it doesn’t work right now. The build system is now set to rebuild the demo site every 30 minutes, and the code is changing very quickly, but there are still major problems. All in all, I think I should have it available within the next month or so. All I’m trying to do is fix the features that already work here on firsttube.com. However, they must all be made completely portable, which is a challenge.

Anyway, it’s exciting, because I’ve been working on a lot of stuff I hope to release by year’s end. It’s good to be coding again. Feels good.

God, I’m a nerd.

Now, in truth, this build will have all of the features I had planned for 0.4, 0.5, 0.6 and probably 0.7, but since I haven’t released anything since 0.3, I think I’ll tag it 0.4. Then I’ll try to get to 0.5, a bug fix build, quickly.

0.4 will be another MySQL only build. I will reinstate the flatfile and SQLite support as we approach 1.0.

So let’s aim for a late September 0.4 final build. There’s A LOT of work that will have to go into making this code portable again, so I’m going to take my time.

I wrote the piece on Israel earlier because it was on my mind, not because I wanted to start a debate or turn my blog into an anti-Israel platform. So when I saw the tide turning that way, I locked the thread administratively.

It wasn’t pretty, it just spit out undescribed errors. So I’ve finally gotten around to spiffing up the UI, and it now displays on the front page, as well as the comment page, that the thread is locked. It won’t accept new comments.

I’m also doing a quick audit for additional features needed.

Anyway, here are some screenshots of the new backend. Click each one to see a larger version:

I have also removed a lot of the password prompts. Every single admin page load, including the AJAX requests, require your cookie for authentication, and even the cookie is just random data. I’m fairly certain we haven’t sacrificed much in the way of security by only forcing the login once. Even Gmail doesn’t ask for a password with every session.

That said, the backend is now mostly complete. I do need to add one more feature, which is image uploads, a portion sorely lacking. This has been needed for some time, as when I need an image, I need to FTP it up, which is lame. So I will incorporate image uploads shortly, and then break for the wedding and honeymoon. Then I’ll come back and release Small Axe 0.4 and get cooking on the next batch of features!

When I get back from the honeymoon, I’ll be packaging a new release of Small Axe, which will be, by far, the best one so far.

Today I added a new AJAX backend for the admin section and you can now edit topics via AJAX. If this is Small Axe 0.4, 0.5 will have to have a backend rewrite. It’s just ancient. There are several things that need to be fixed to prevent future bugs.

I also changed the RSS feed to obey the “related” switch. If you’re set up to use tags instead of topics as your related basis, then the RSS feed will display the first tag instead of the topic as the “category.” So all posts in my RSS feed as now going to be using tags instead of topics. I don’t know what I will do about topics in Small Axe or on firsttube.com right now. On smallaxesolutions.com, the topics mean something. Here, they are completely random. So, for now, not entirely sure. Read on for more.

I’ve thought about some things I’d like to see in future incarnations of this code. I’d like the backend to be faster and more dynamic. I’d like to branch off the function files into admin functions and user functions for optimization. I’d like to restructure the Javascript in the admin backend. One challenge I’d like to see is the ability to host multiple Small Axe blogs in a single database. This is something I’ve had in mind for awhile. In theory, there’s no good reason why you could have 5 users running their blogs all from the same database, and in the meantime, have their own front ends AND a common front end. It would be extremely easy. This is more of a 0.6 or 0.7 goal.

Another thing I’ve thought about is the ability to add an RSS feed DIRECTLY into your blog. This means items would simply show up in your blog (either automatically or manually, that could be an option), but it would require cron.

I’d really like to integrate an weblog endpoint API, and I’d like to eventually learn how to parse an email so you could email an entry in. These are for much further out, probably post-1.0.

Anyway, those are some things I’ve been tossing around.

So, I added a new configuration option. Now you can choose in the backend whether or not you want related posts to be “related” by topic or by tag. By toggling the option, suddenly, related posts were — get this — actually related! This is entertaining the hell out of me, and because of that, I will be forced to tag THIS post “Nerd.” Sigh.

So I wrote it to use a single query, and guess what I found? The implementation sucked. It was slow. It was dirty. It meant editing tags was a precise business. And searching was lame. So, I rewrote tagging (while preserving links, etc) and – boom! – fast. Yes, it added a query for every story on the front page, but dammit, it works, it’s clean, it’s easy to read, and best of all, I’m going to implement some flickr-style tag editing on the backend. So, that’s cool.

I guess we’ll see what happens in the next week or two.