firsttube.com » Spam

Trackback Spam Gateway

Adam S — Thu, 13 Mar 2008 20:33:46 +0000

It’s over. My referrer experiment is over… at least, in its current form. Today, I roll out firsttube.com referrer gateway version 1.0. That makes it sound fancy, but it’s not. Basically, it’s PHP to prevent trackback spam.

Traffic at firsttube.com has grown steadily, for some reason, and the logs reveal it: we get a TON of traffic from search engines, and the most popular terms are surprising – sensitive readers beware – here are the terms that most frequently drive people here:

cumtube, red-tube, uporn, adult youtube, milf, gay tube, tube 8 and many more equally odd terms.

You know why? Because, in a shrewd move that search engines seem to love, I display links back to my referrers, thinking they are trackbacks. But when it’s not from Google, Yahoo, Live.com, or OSNews, it’s most often spam. Why? Because not only are we using the name “tube” in our title, but with each erroneous entry, we tell the search engine it’s a good thing by back-linking to that search. In short, I’m perpetuating the problem. As a result, dozens of spammers have begun issuing basic GET requests in the hundreds placing their sites in my referrer lists.

Some time ago, I began the battle by adding rel=”nofollow” to all outgoing links not added via the admin section. But alas, that wasn’t good enough, the spammer didn’t care, so I implemented a pre-check, whereby referrers are, via regular expressions, matched against a list of known crap. As of today, there are 36 terms that I actively filter. In time, this will be performance intensive, if it isn’t already.

Thus, a gateway. Now, *all* referring traffic goes into a temp table, and each entry must be approved. I wrote a nice tool to batch import, batch delete, or even approve based on certain filters, such as domain or term. As it matures and I get an idea of time, I will “whitelist” certain domains that can immediately post to the referrer table. In the meantime, I need to decide if I want to filter referrers with obscene unrelated terms or just leave them and let the magic run its course; after all, these are not “spam,” they are simply organic mistakes. An argument could be made that it’s interesting, and therefore, mostly the reason to post referrers, to see what terms and sites around the internet drive traffic to a site.

Anyway, spammers, take note: I gotcher number! Stop referrer spamming me! That means you , you stupid lyrics sites!

Trackback Spam, Again

Adam S — Wed, 23 Jan 2008 15:30:34 +0000

Once again, I am dealing with trackback spam, aka referrer spam. Since firsttube.com records the pages that refer hits to us, I’ve had to deal with jerks who issue HTTP requests so that they get a link back. Too bad they don’t realize that every referrer gets a rel=”nofollow” attribute (more here).

So, I had to issue these SQL statements to the database today:

DELETE FROM user_agent_table
WHERE (referrer LIKE 'http://mp3%' OR referrer LIKE '%mp3.com%')
 
DELETE FROM user_agent_table
WHERE referrer LIKE '%musicforum.org%'

Musicforum.org has some asshole posting all sorts of links that pass a GET variable with a firsttube.com URL in it, which appears to do nothing other than ping the page. So, effective immediately, we run a regex validator on referrers and will be doing a more frequent clean up.

Hear that spammers? Take your crap elsewhere.

ReCaptcha is a Smart Ass!

Adam S — Fri, 21 Sep 2007 11:13:31 +0000

ReCaptcha thinks it knows me, huh? It started calling me names today.

Suckered by a Spam Text Scam

Adam S — Thu, 31 May 2007 15:58:33 +0000

I got scammed.

I got a text message yesterday that said “Free Msg from Verizon Wireless: We have new calling plans for UNLIMITED text to anyone in the US. Requires new customer agreement. Call 877-256-XXXX. To opt out reply X.”

This type of message is not completely unseen on my phone network. I’ve gotten a few “notice”-type emails in the few years I’ve been with Verizon, so before I even really knew what I was doing, I quickly replied “X” and hit send. And then I saw it: “Sending message to 9000XXX03671″

I realized what had happened. I had just texted 900-XXX-0036. And unfortunately, 900 numbers can charge, according to Verizon up to $25 or more for this type of thing. I IMMEDIATELY dialed 611 to talk to Verizon. Much the same way that if I complain with American Express, they can withhold payment from a merchant, I wondered, can Verizon refuse to square up on what was obviously a spam text scam?

Yes and no, apparently. The helpful rep I spoke with, “Sarah,” told me she could “flag” my account for a follow up when the billing cycle ends, but the agents cannot see live data and couldn’t see a text message within the last few days, let alone minutes. She promised to credit me whatever the message cost.

So I asked her the same question I’m broadcasting now: “Why do cell phone companies allow incoming text messages from 1-900 numbers?” Even if they were limited to just replies it would be an improvement. What possible reason could I have for a 1-900 number to send me an unsolicited message?

Is this the future of cell phone? SMS spam from 1-900 numbers? This is very dangerous ground, and I see where it’s heading: advertising spam (which has already started), “verified” senders, block lists, and finally an option to accept SMS only from your contacts. A new age is upon us. May the people who perpetrate these scams rot in the blackest pits of hell.

But Verizon is going to take care of me this time, and I won’t get suckered again. The moral of the story is: don’t reply to text messages from people who aren’t your contacts, period.

–

Within seconds of hanging up with Verizon, I got another message from 900XXX003671: “Thank you. If you want to eliminate all future SMS informational and marketing messages from Verizon Wireless, reply Q.”

Pwned!

Adam S — Fri, 16 Feb 2007 09:50:51 +0000

This morning, I checked my Gmail and found that I had received 203 comments on firsttube.com last night. In case you don’t realize how odd this is, I typically get 1-2 comments a day max, unless a particular discussion gets moving, and then it might be 10 at best.

These 203 comments were generally grouped, about 30 to a group, with identical information. The name was some kind of drug spam (you know the typical Viagra, etc) and the IP was blank (which I may soon require to comment). But the odd thing: it was almost like it was custom written for my blog: the thread kept changing, the parent comment changed, the content changed, and it passed the captcha… or so it seemed!

So I changed my captcha. I think I might have found a bug in it whereby if your browser wouldn’t begin a session, it might pass the captcha. So that’s fixed. Plus I added several little checks to the comment page. It will not only reject lots of comment variants (all of which are unlikely to occur with real comments), it will actually ban some users automatically if certain strings are found, etc.

It’s kind of ironic that my last entry was on spam, and then my site gets spammed.

Trackback Spam

Adam S — Tue, 13 Feb 2007 14:51:00 +0000

There is a new trend out there, one that hasn’t received much coverage, but it’s a big deal, and it’s getting bigger. As user generated content becomes more and more prevalent, we have a new type of spam out there: trackback spam. On my blog, beneath all of the entries (above the comments), there is a section that shows you the user agents that loaded that page as well as the refering pages. I recently discovered something: people gaming the system. Read on…

The thing is, I sometimes follow those links to see who is linking to my site. There are several reasons I do this, but part of it is the effect on search engines. Anyone who links a blog entry to my particle immediately is linked back on subsequent page load. It’s good search engine karma. That said, I started seeing some sites that didn’t have a link back, but somehow referred someone to my site. And then I realized they were unrelated – car insurance, casino, etc. Typical spam crap.

All were coming from different IPs. All had different user agents. All had different referer links. This is spam, pure and simple. It’s someone trying to piggyback off of my pagerank.

These are the jerks who spammed me:
theonlineslotsmachine .com
online-casino-special .com
adencitycasino .com
onterminsurance .com
ontermlifeinsurancerate .com
onslotmachinesonline .com
actoncasino .com
onusinter .com
iloanmortgageonline .com
scrail .net
ppplastic .com
mysteryclips .com
e-z-ly-treat-e-d .com
onhomecontentsinsurance .com

And that’s just a small slice of the pie.

I’ve removed all of the spam links I could find, added some tighter controls to try to avoid recording these faked headers, and also added ‘rel=”nofollow”‘ to the links, which means I still reward referrers with a link, but bots won’t follow them, so they get no pagerank bonus until I manually change it.

Trackback spam is going to be a big problem, particularly as people continue to use commenting engines that allow you to link your name to a URL. It makes sense to start posting fake comments just to get that link on a worthwhile website with a high pagerank. So combating this early will be important.

I’ve thought about some ways we might combat this, and was thinking that on OSNews, I might only light up your blog/homepage link if you have a positive “trust” level. Otherwise, it will be just plain text. Or maybe add the “nofollow” to links of untrusted users. Not sure yet.

Either way, trust me, though the subject may catch on with a different name, you haven’t heard the last of trackback spam.

Reinventing E-mail

Adam S — Mon, 03 Mar 2003 19:15:00 +0000

There’s been a lot of discussion in the last few months about the rapid increase in spam e-mailings, defined loosely by one website as “mass-delivered, unrequested advertising delivered via email or discussion groups.” Many have wondered what should be done, but few have proposed anything truly worthwhile.

One of the problems with the internet is that is got big really quickly. So quickly, in fact, that many poorly designed architectures were implemented and adopted. In truth, a better, fairer way to say it is that many of the recognized standards were designed without the knowledge of the degree to which they would used and depended upon. Protocols like telnet, ftp, nntp, and http are insecure plain text transfers and are routinely manipulated, cracked, and exploited. Only recently have extensions or replacements to these, ssh, ftps, nntps, and https, become commonplace. In fact, in some cases, they are really just extensions, not designed from the ground up with intentions of security or administration. Though these were all designed by brilliant and talented engineers, they didn’t have the knowledge we now have to plan properly for the ways people would abuse their project.

This is common all over the internet. I, surely, do not propose to be smarter or more inventive than the designers of the internet. In comparison, my skillset is supremely dwarfed. However, I think everyone can agree, all of what has been adopted on the internet was done so without the knowledge that it would catch on so quickly and so intensely. Even the central protocol that powers the internet, IP, needs to be upgraded to a new version to accommodate the growth rate and security needs unforeseen by those who put the ball in motion 30-odd years ago.

That said, one nasty habit of interneteers has been to “jiggle” the current internet to fix problems as they arise. Of course, it’s impractical to create a new internet that does things properly. In addition, replacements are hard because getting everyone to switch over is an arduous and tedious process. But some problems simply don’t have simple or practical solutions, and the most obvious problem is spam.

Spam affects nearly everyone with an e-mail account. It’s been estimated that around 3% of all internet traffic is SMTP, or Simple Mail Transfer Protocol, the TCP port on which e-mail is sent. Further, this website reports that “Brightmail, an anti-spam software maker, reported that spam will account for 40 percent of e-mail traffic in 2003.” 40 percent! That’s nearly half of all e-mail transmitted! They also estimate that each piece of spam, at the least, takes about 4.5 seconds of our time. For the average person, simply receiving 10 pieces of spam a day takes, at the very least, a minute of their time. Multiply this times the many billions of messages sent out each day. And some of us are fortunate enough to receive up to 100 spam mails a day.

Read about recommended solutions to the spam problem and you’ll get the same few answers:

Blacklists, which block known spammers, but are virtually impossible to keep up.

Whitelists, which allow only approved e-mail, are also nearly impossible to maintain at the ISP level.

Of course, there are client side whitelist tools, but first, they only work with client software (not webmail), and second, most people don’t add after market spam protection.

There’s the “sender pays” solution, but I already pay my ISP for access and don’t want to pay on a per e-mail basis.

This got my mind going. I’m not an internet engineer, so I’m not knowledgeable enough to propose a 1-2-3 solution. Instead, I have an idea and I’d like to generate some discussion and entertain the feasibility of a next-generation replacement to the way things are currently done.

I’m not sure the exact details of SMTP, and truthfully, I’m not even sure that what I propose should be implemented in a protocol, perhaps it’s just intelligent client software. Either way, I think it might mostly solve the spam problem.

I imagine this measure as a “Mail Transfer Handshake Protocol (MTHP).” First off, we take SMTP as is as a starting point. The way e-mail is transmitted is not a problem – it works, though the protocol is unencrypted. The next step is that we add some “authentication” to the mix. Every e-mail transmission is, by default, disallowed. E-mail servers will not accept e-mail from anyone who doesn’t have the right authentication. Each domain is then granted a key of some sort – perhaps it’s similar to a “trust certificate” or, better yet, a PGP (or GnuPG) key. Each mailbox is then granted a unique id on that domain (some sort of random ID number/letter combination). When an e-mail is sent to a mailbox, it queues at a MTHP server, which holds it for a specified “drop dead period.” While in queue, the mailbox owner has the ability to view certain information about it – the display name, the originating e-mail address, IP address, and any other information contained in the packet headers. Before the e-mail is viewed, the mailbox owner can then choose to send the originator their individual key that includes the domain key and the mailbox id. When this transaction is complete, an e-mail may now be sent. Further, from then on, there need be no further key exchange, so long as the key remains valid. From then on, it resembles simple SMTP traffic. Should a person not respond within the wholly configurable queue time, the MTHP server simply dumps the message.

At this point, the ISP could be building a list on their MTHP servers of messages designated as safe and those rejected. Perhaps the rejected ones, once they reach a certain number of blocks, can be sent to a central agency that maintains blacklists that an ISP might use to find their own offenders, or, if they choose, find addresses which they can blacklist from even their MTHP servers, so that spammers couldn’t even get into the queue. Maybe even a central propagating feedback system similar to eBay’s that can be distributed via a method like the root DNS servers.

I’d anticipate that after a few months, you would rarely need to visit your MTHP queue for approval. Most of the time, people you know would have your key. In fact, an MTHP server could be configured to send you a periodic digest of mail you have in your queue waiting to be seen so that you’d never need to think about “checking” something else. The system would deliver a heads up, not for each message, but every so often, to let you know what’s waiting.

There are a number of other details to be considered. First off, it must be possible to create “open mailboxes” that can receive e-mail without a key. Another possibility would be mailboxes that simply have a general public key. Since they would be few and far between, it would hardly be worth hunting for those few to spam them. Another important point would be how to transmit this “handshake.” Clearly, the handshake would have to be encrypted. Should the rest of all e-mail then be encrypted? This will definitely slow things down.

Also, we need to consider the security of the key. What if it falls in the wrong hands? Though it might be easy for me to re-key my own domain” with all three of its POP mailboxes, I sincerely doubt users of Hotmail or Yahoo! mail would be very excited about re-approving all of their safe list each time the key is compromised. What if I’m a spammer and I sign up for a Yahoo! mailbox just to send myself the key and then dissect it? We’re talking about some serious work that must be done to keep each individual key safe, otherwise any work done would be in vain. The best solution, as I see it, is to make each key contain a bit about the sender’s address that was granted express approval. Then, you at john.doe@company.com couldn’t take my certificate and use it, or worse, sell it, to spam@nigerian-scam.org and use my key.

Of course, right now, people are vulnerable in real life with social security numbers, ATM codes, and credit card numbers, so it’s unfair to assume that people would be unable to keep their keys safe.

Now, admittedly, implementing a new system like this is not a “weekend project.” We’re talking about changing the way the whole internet works well into this project. The question I’d ask is, how long will it be until we are FORCED to start over? At some point, we’ll be limited by the constraints of certain current methods and architecture and be forced to use replacements, much the way that we’ve had to increase storage capacity in our datacenters, multiply bandwidth at our places of work and homes, and regulate allowed transport methods of network traffic. At some point, IP will be replaced, much the same way that IPX has been made extinct, with another new shiny suite that better suits the internet as it stands at that time, which might be well in the future. Maybe IPv6 is the savior – maybe it will give developers the tools to overcome much of the abuse that takes place today. However, the primary benefits on IPv6 are less on the workstation and more on the network side of things, implementing multicast and IPsec as mandatory rather than optional, limiting routing table sizes, and network autoconfiguration, amongst many other things. We should be planning a “second internet” using lessons learned now. While the name internet 2 is already in use, the idea is one we should take seriously.

It’s been suggested that simple filters, like “spammer” and “nonspammer” acheive the same goal, but consider this: the Gartner Group guesses that spam click through rate is about 1% (many rate it less). We can probably safely assume that most people won’t grant spammers their key, so even if 1 out of every 100 people were to grant their key, that means that it would take a hundred times as many e-mails to generate the same response as today. We know spammers will only continue to spam as long as their is a response and it is economical. Personal lists still deliver the mail and count on you to file them, most don’t use any filtering system. MTHP generally would prevent it from ever reaching your mailbox.

As any project gets larger, there become participants that disproportionately further its cause and those who disproportionately detract from it. The detractors are the abusers, and in this case, spammers. It’s up to the technology savvy to be innovative and propose and share their ideas, even when they’re incomplete, if only to inspire the ones truly destined to change the world. Here’s my idea. Whatta ya think?

X10 cameras – f them.

Adam S — Wed, 04 Sep 2002 07:12:00 +0000

Do people actually buy the X10 web camera? If so, do they actually buy it from pop-up ads? That’s the question of the day.

Honestly, that friggin ad follows me all over the damned internet. You’d think it was just that necessary, “Hey, buying a CD…why not pick up an X10 too?” “Say, checking your e-mail….need an X10 while you’re at it?”

Seriously, I need to know – who is buying these things?