nameandnature | Entries tagged with spam

I was getting too much spam, so I've turned off anonymous comments. LJ's anti-spam system was correctly flagging a lot of it, but I can't help feel that it should just bin the more obvious stuff (with a rejection message so humans know what happened).

LJ's code base being what it is, this may also disallow logins via OpenID and TwitonMyFaceSpace, I'm not sure.

I'm still planning on moving away from LJ as soon as I get some spare hours to do it in.

How Modern Spam Works | MetaFilter: Researchers try buying stuff advertised in spam, following the money. Link is to the mefi thread since one of the researchers turns up to answer questions in the comments.
(tags: internet economics spam)

Many years ago, I signed up for Bloglines. It's a service which aggregates the feeds from various blogging sites, so you can read them in one place without having to do the rounds of your favourite sites looking for updates. (On LiveJournal, your friends page serves the same function, and you can add the feeds of external sites if you're a paying customer).

I left Bloglines for Google Reader when Bloglines became unreliable. Google Reader is nice: it looks clean, and there's an app for it for my Android phone. I recommend it over LiveJournal, which is dying of spam; and Bloglines, for the reasons I'll now get into.

A while back, Bloglines was taken over by a company called MerchantCircle. They sent me an email to say they were the new owners, which is fair enough. As far as I remember, I hadn't logged into Bloglines since I moved to the superior Google Reader service, so I just ignored it.

Yesterday I got an unsolicited bulk email (spam) from MerchantCircle advertising a service not related to Bloglines. Worse, the link they offered to unsubscribe from their mailing list didn't work, as it required a login and password (first mistake: removal links from mailing lists should authenticate the user sufficiently to get off the list). Worse still, giving the email address to which MerchantCircle sent spam to the "forgot password" box gave an error saying that the address was not known: MerchchantCircle don't even know who they're spamming. Logging back into Bloglines doesn't give an "unsubscribe" option either.

I consider Bloglines/MerchantCircle to have gone rogue. I've removed the "subscribe with Bloglines" buttons from my blog, and advise anyone else who still has those buttons to do the same. Use Google Reader instead: Google don't spam.

Edited to add: MerchantCircle have emailed back to apologise, saying they had a "weird glitch" in their email system which caused some Bloglines users to get MerchantCircle emails. In recognition of this, I'm downgrading them from "rogue" to "incompetent".

LiveJournal coughs to their crimes, sort of

So, LiveJournal finally sort of owned up to getting blacklisted for helping spammers, as mentioned previously. This posting is their response to the situation. They say they're doing the right things, although you do have to wonder what took them so long.

They didn't name Spamhaus or properly explain why they'd been blacklisted, so I explained in the comments.

The spice must flow

Notifications are coming through now because LJ have changed the IP address of their outgoing mail server from 208.93.0.128 (the address of www.livejournal.com) to 208.93.0.49 (which calls itself mail.livejournal.net, but isn't accepting inbound mail). The blacklisting for the old address is still in place. The spammy journals specifically mentioned in the SBL listing seem to have been suspended, though.

It's not clear if this change of IP address is part of some agreement between Spamhaus and LJ or whether LJ think they can avoid the blacklist and continue to ignore complaints. If it's the latter, I'm fetching popcorn. It's the work of a few keystrokes for Spamhaus to block LJ's entire address range, and I vaguely recall they've been happy to do that in the past for people who've taken the piss.

(Disclaimer: I'm not Spamhaus, I just used to hang out on news.admin.net-abuse.email in the 1990s, when it was cool).

Postings in

news have been a bit cagey about what's going on with comment notification emails. They've mentioned that there's a "third party" involved. It turns out that LiveJournal have got themselves blacklisted by the Spamhaus Block List for providing spam support services, in this case, hosting websites for spammers.

This is why comment notifications aren't getting through: the SBL is a widely respected and widely used email blacklist. They're not saying LJ are spammers or indeed sending spam email, they are saying that LJ aren't taking down journals set up by spammers, so they're effectively helping the spammers to spam. Most email spam directs the mark to a website, so providing those websites is a serious matter to Spamhaus.

This is worrying: it means LJ probably aren't responding to complaints about hosting the spammers' sites. I think Spamhaus would have tried sending email to abuse@lj, though possibly not under their own names, as you want to be sure that reports from ordinary users are handled correctly, same way as restaurant reviewers don't book saying "I'm Jones from the Times". The detailed information from Spamhaus lists a huge number of spammy journals, and at least a couple of them were still there when I tried them. This doesn't bode well for LJ's future, to my mind.

livredor brought this to my attention. There's a thread on a

news posting discussing the problem.

azurelunatic (who is head of anti-spam for Dreamwidth) has more here, and I've commented on their posting.

I look up potential interviewees on Facebook (as well as Google, obviously). Unlike the proctors at Oxfrod, I don't care whether you've been photographed covered in flour or shaving cream, as long as you look like someone who's smart, and gets things done.

livredor recently posted an entry in which she talks about online privacy, linking to Charlie Stross's essay on the subject. I think Stross has this article on teenagers and online privacy in mind when he talks of a generation growing up with the idea that you have no privacy online and it doesn't matter anyway.

livredor is coming to the conclusion (which I share, see my replies in the comments) that she "should just make everything open and take care never to post anything that I could be ashamed or embarrassed about".

As the comments on her posting point out, the problem is working out what you could be embarrassed about. The problems mentioned in the Times article are partly the result of a generation gap between people who aren't surprised that some of their peers have put their lives online, warts and all, and the staid elders who are shocked to learn stuff that proctors, employers and parents didn't previously find out about. I suspect that absence of evidence of shaving cream was never really evidence of absence, but it's going to take a while for the elders to work that out. It seems sensible for the younger people to be a little circumspect in the meantime, so it's not surprising that many existing Facebook users are tightening up their privacy options. Relying on privacy settings is another risk, because you're trusting your e-friends and the site you're using, but at least you're keeping your embarrassing university antics out of sight of indexers and archivers, and you're not assuming that the elders cannot join the site you're using.

livredor also mentioned the possible problems which might be caused by people migrating away from email to the messaging systems offered by sites like Facebook. Gervase Markham has some thoughts on the subject. Conventional email is a lot less slick than, say, Facebook's internal messages, and faces a greater spam problem, in part because email is distributed but Facebook has centralised control. These proprietary systems have their downsides too, of course: balkanisation, and a single point of failure when Facebook gets shut down by a law suit.

I think there's some mileage in building an email system which is a bit more like Facebook's walled garden. When I say spam in its current form is a solved problem, what I mean is that you can solve it by only accepting messages from well-behaved parts of the Internet. What I mean by well-behaved is stuff like not being in space given to cable modems and the like (Spamhaus PBL, checks on the presence of reverse DNS and that the hostname does not contain some variant of the IP address), not being a known baddie (Spamhaus SBL and XBL or your own email providers local list of scumbags), and not sending bulk email except by prior arrangement (DCC with whitelisting for mailing lists).

Alas, not all badly-behaved emailers are spammers, some of them are just managed by incompetents. Sometimes these incompetents work for large companies who aren't going to change, so you have to start making holes in your garden wall to keep your users happy. However, an inbound email gateway for a hugely popular site like Facebook could enforce these restrictions by fiat without losing anything, since their users are using the internal system to send each other messages anyway, so anything else is a bonus (you could also make a nice interface for whitelisting legitimate bulk senders by requiring them to produce a Facebook application, say). If Facebook does take over the world, it needn't mean the death of email. It might just bring the incompetents into line, we can but hope.

For a while now, I've been getting comments on my LiveJournal which apparently aren't spam, but rather are questions which are totally out of context. For instance, I got one the other day which said "Hi. I find forum about work and travel. Where can I to see it?"

I recently got some more comment spam advertising something called XRumer, a clever and nasty program for spamming bulletin boards and other forums (like LJ), which is brought to us by some evil Russians ("No Meester Bond, I expect you to die"). One of the things the authors claim it can do is a crude form of astroturfing. They say you can configure it to post a comment asking about something, and response apparently from another user mentioning the site you actually want to advertise. It looks like this feature doesn't quite work, and that the questions I've been seeing are examples of it misfiring. Mystery solved.

The spammers seem to favour certain entries of mine, so I'm screening anonymous comments on those entries (and on this one too, since I imagine it might attract undesirables). I don't want to do that for my entire journal, as I get comments from people who aren't on LJ but who say worthwhile things. In an ideal world, the way round this would be OpenID, but that's not in widespread use yet, possibly because people who have an OpenID often don't know they do. [Attention LJ users: you have an OpenID. Congrats. You've got a Jabber instant messaging account, too. See how good

bradfitz is to you?]

A system which allows easy communication between two people who have no previous connection to each other is susceptible to spam. The trick is to keep this desirable feature while not being buried in junk (you could go the other way and remove this feature, of course, as many some IM users have, or make a virtue of it with social networking sites, but that's not really an option for public blogs). Anything an ordinary user might to do create an identity, a spammer can do too, so cryptographic certificates aren't a magical solution. Legislation doesn't help, because the police don't care and anyhow, spammers are in Wild West states like China or Russia, or at least run front operations there.

Most spam is still sent via email. Email spammers have been subject to an evolutionary arms race. The remaining effective spammers are bright and totally amoral. They'll hijack millions of other people's computers to send their spam or even to host the website they're advertising, making it hard for blacklists to keep up (and they'll use these computers to flood centralised blacklist sites with traffic in an attempt to knock them off the net). They'll vary the text they use, to defeat schemes which detect the same posting lots of times. They'll use images rather than text, or simply links to those images, to defeat textual analysis. You can bet that blog spammers will learn from this (some of them are probably email spammers too).

What's working for email spam, and will similar ideas work for blog spam?

Banning mail sent directly from consumer ISP connections is the single most effective thing I do (you can do this with the Spamhaus PBL and with a few checks for generic rDNS to catch what the PBL misses). You can't do that with blog comments, as spam or not, they almost all come from consumer ISP connections.
Banning mail sent from IPs which are known sources of spam is also effective. You can do that with blog comments, but you either need to be big enough to generate your own list (as LJ might be) or have the resources to run a centralised list like Spamhaus (which will be attacked by spammers). There are currently no IP blacklists devoted to blog spamming, as far as I know, although some spam comments I've seen came from IPs which were in the Spamhaus XBL.
Filtering on ways in which spamming programs differ from legitimate SMTP clients (greylisting, greet pause) is currently effective, but only as long as these methods don't become so widespread that it's worth the spammers' while to look more like a legitimate sender. Still, this doesn't seem that likely. Incompetent admins aren't in short supply, and I don't have to outrun the bear, only outrun them. This sounds promising against blog spammers. Apparently simple minded schemes are pretty effective.

What else can we do with a website that we can't do on email?

CAPTCHAs are popular, but a bit of a bugger if you're blind. The evil Russians claim to have defeated most of the deployed ones which use obscured letters, though that still leaves the "click on the picture of a cat" variant.
Proof-of-work or hashcash schemes are currently very effective, suggesting that blog spammers don't yet have the huge amounts of stolen computing resources available to email spammers, or that they don't have the knowledge to implement the hashcash algorithm in their spamming software. By using proof-of-work, we can at least drive the weak blog spammers to the wall.

You can think of proof-of-work as a variant on the tactic of differentiating spam programs from real humans. Spammers can defeat simple-minded checks on how long a user has been reading a page before commenting without slowing their spamming rate up by much (how to do this is left as an exercise to the prospective spammer), but if a web browser has to do a computation which takes a fixed time and send the result along with the comment, the spammers have to slow down or do the work in parallel on many computers. If you can work out a way of doing the calculation in the background as the user looks at your page and writes their comment, so much the better. If you can dynamically generate the code you send to the browser to make it prove it's done some work, you stop the spammers writing something equivalent in a real programming language and force them to run it in Java or Javascript. That'd really show them who's boss.

This hurts people who've turned off Javascript or Java, but it's time for those dinosaurs to join the web 2.0 world, right?

Background: news.admin.net-abuse.sightings is a newsgroup for posting copies of spam, so that the domains and servers involved become public record. Gradwell, run by the eponymous Peter, currently host noctua.org.uk. Peter Gradwell objected to my posting copies of spam to the newsgroup because his machines appear in the headers of all my email. ( Now read on... )

Current Mood: irate
Current Music: Slide - Dido

There is a confusing multitude of spam filters out there. I once wrote an article listing all the ways of filtering spam I could think of. If you're confused by all this, here's what I do, along with ways of doing the same thing on both Unix and Windows systems.

( Read more... )

Current Music: CLUB 977 The 80s Channel (HIGH BANDWIDTH)

Kevin S. Wilson writes in NANAE:

You just don't get it, do you? WE ARE PISSED, VENGEFUL, AND UNSYMPATHETIC. You helped to create the mess that e-mail has become, invading the privacy of millions of people and generally making an annoyance of yourself on a GLOBAL scale. Ultimately, you may have helped to render e-mail unuseable. You think anyone cares that you can't find hosting for a vanity domain? Instead of looking for sympathy here, you ought to be thanking your lucky stars that someone sick of your spam hasn't hunted you down and broken your arms, or worse.

I'm sure we all feel that way some days. (If ASR is the scary devil monastery, what does that make NANAE, I wonder?)

In other good news, Microsoft, AOL, Earthlink and Yahoo are going after some of the most prolific spammers. A quick look at the example emails in the lawsuit documents shows that many of the obvious suspects are in the frame. They're filed against "John Doe" (the US legal equivalent of "John Smith") as this allows the plaintiffs to get ISPs and other organisations to disclose the identities of the people behind the spam, but the targets here are well chosen, so I think the plaintiffs know who they expect to end up bankrupting. The mills of justice grind slowly, but we may hope they grind exceeding small.

There is a conspiracy theory which says this is just large commercial interests getting the porn'n'pills people out of the way, leaving the field clear for mainsleaze. Even if this is the plan of people like Microsoft, there are strategies in place for dealing with spamming from mainstream companies. Such companies can't afford to use the deceptive and criminal tactics of the worst spammers, so blacklists and bulk email detectors like the DCC should see them off.

When you get your shiny new cable modem, you usually configure your mail program to send email via your ISP's server at smtp.ntlhellworld.com (or whatever). smtp.ntlhellworld.com then sends on your mail to the destination at its leisure (or in NTL's case, doesn't). There was no particular reason why a clever enough computer couldn't just connect to the destination directly, especially if it's a computer which is left on most of the time, so that if the destination is down or busy, it can try again later. This is what my computer did. But now lots of servers are blocking mail sent by my machine. This is because of spam.

Know, O King, that the modern porn'n'pills spammer uses open proxies to send email advertising his website. His website is hosted in China or Brazil (Spammy himself is actually a resident of Florida, and the mail originates from his machines in China, but the trail goes cold at the proxy, so it's hard to prove this). Most of these open proxies are on machines connected to cable modems. Sometimes the proxy has been installed without the owner's knowledge, perhaps by one of these "virus" things you Outlook users are so keen on. Sometimes, the owner installed the proxy themselves to share their cable connection with a local network, but misconfigured it. Misconfiguration is easy when your chosen software is insecure by design. Marc Thompson, author of the AnalogX proxy, must surely be a prime candidate for first trials of makali and jwz's famed audio-cock technology.

But, anyway, the solution adopted by some servers is to block any cable modem (or technically, any machine with a dynamic IP address) from sending them mail directly. That's why my mail bounces: my IP address is on a list of dynamically allocated IPs. I can advocate that the admins use the Spamhaus XBL instead, since that only lists the addresses of insecure machines. But then someone will point out that my address is right next door to someone who is compromised, and, being a dynamic address space, I could get that address tomorrow.

So, I'm going to start using Gradwell's machines to relay my mail (they'll let me do this as they also host my domain and incoming mail). They're a lot more clued up than NTL, so their relay machine will probably be up most of the time and will probably ensure my email reaches its destination. But still, it's a shame. It takes that little bit of control away, as I can only tell when something has left here, not when it's been finally received. And it breaks something that wouldn't need to be broken, were it not for those pesky spammers.

Very much so to you all. I've had a good weekend. Watched Pirates of the Carribean and thought it was a good silly film. Explored Milton Country Park and took some photos. Links to follow.

The dancing is back in full swing. GD on Friday was packed, but packed largely with nice people who it was good to see again. I also tried Clive's lessons last night, which were hard but fun.

Had tea with Safi beforehand. The subject turned to Old Testament prophecy and how the prophets predicted JC, hence the Bible is miraculous in predicting stuff before it occurred and I should reconvert at once (I exaggerate a little). I said I thought that things like Isaiah passages referred to by Matthew show that Matthew thought Jesus was the Messiah, but they don't constitute "prediction". There's another opportunity to ask my Jewish readers here: what do you think Isaiah 8 and Isaiah 53 are about? (I need a Vim macro to link to gospelcom's site for references under the cursor, to go with the Google one).

I would like to big up (and also give mad propz to) the combined Spamhaus SBL+XBL blacklist, which is catching stuff which slips by the Distributed Checksum Clearinghouse (this does mean I miss out on my daily dose of poetry, but it's a small price to pay). Windows people who collect email using POP3 or IMAP can use the list via Spampal. I'm passing stuff through rblfilter as well as dccproc.

Now that, as a paid user, I have the full power of LiveJournal's S2 style system at my disposal, it might be time for a revamp round here. I've not yet decided whether to change the format to dark blue text on a black background (and go on about how goth I am, naturally) or merely to have a sodding huge picture of Sarah Michelle Gellar occupying most of the screen with the text in a small strip down the right hand side (I'd need to increase the font size, too). Vote now.

I seem to be dancing quite a lot lately. I'm enjoying Dancesport B lessons. Sadly had to give up on the Intermediates on a Monday night as I think 4 nights a week is just a little bit too much. The word on the CDC grapevine is that some people think the Dancesport classes are too easy. Not sure where that's coming from. The web page reckons people with more than a few terms of medals should be going to the B classes, and Alf never did arms and whatnot in his Latin teaching. Possibly dancing one-upmanship going on? Who knows.

Off to Rome in a week or so, which should be fun. Anyone know any good restaurants and suchlike?

People on the SpamPal support forums are getting a bit excited over the idea that someone might commercialise the thing. This would probably involve producing a better installer so the average Windows user can use it without having to fiddle with Outlook Express settings and so on (note to Windows users: please stop using OE, I'm bored of getting copies of viruses. Try Thunderbird or something instead). James Farmer, the author, has licensed it under an open source licence, so he presumably doesn't mind this. But some people who wrote plugins, manuals and foreign language support for it are up in arms about the evil megacorps appropriating their stuff. I've said that I don't mind my meagre plugin going in to such an effort. I'm not convinced that it'd be a commercially viable venture anyway, but people have packaged up SpamAssassin in a similar way. If people want to trade money for time spent fiddling with it, I can't really see the harm in it.

The company also want to make a free DCC plugin, which would be nice, as the chances of me getting off my arse and finishing mine seem quite small at the moment.

This posting has been brought to you by the number 42 and the ACRONYM tag.

I confess that I underestimated the enemy rather badly. I underestimated both the enemy's level of sophistication, and also the enemy's level of brute malevolence. I always knew that spammers had no principals and no ethics, but up until recently, I had no idea that they could or would stoop this low, or that they would engage in quite this level of criminality. I guess that, naively, I just never thought hard enough about how much money was actually at stake (in the spamming trade) or what that might mean in terms or the determination of spammers to win at all costs.

Ron F. Guilmette announced that he was giving up the fight against spam in the face of massive Distribued Denial of Service (DDoS) attacks. This, in the wake of the attacks which forced Joe Jared off the net, is rather worrying.

( Read more... )

A quick glance at the folder where my computer puts stuff it thinks is spam shows I'm getting a lot more of it lately (about 5 or 6 spams per day, right now, with more at weekends). The amount coming into the "spamtrap" addresses which report spam to the Distributed Checksum Clearinghouse for the benefit of the world at large seems to be going up too. The DCC works by counting how many times a particular message text has been seen in the wild. Messages with a high count are either spam or legitimate bulk email, so you need to whitelist the mailing lists you're on so as not to filter them by accident. Every other bulk email that you didn't ask for (and whitelist) is spam, by definition.

I'm actually doing some good with the spamtraps, as my computer is getting stuff which doesn't already have a high count and maxing out its count as it sees it.

I can't imagine what it'd be like for someone whose address appears somewhere on the web or Usenet and who doesn't have any filters. It looks like it's going to get worse before it gets better.

Vernon Schyrver is being typically acerbic in news.admin.net-abuse.email about whether this Anti Spam Research Group is ever going to go anywhere. Given Vernon's annoying habit of being clever and right, it seems likely that non-techies are doomed to get spam until their mailbox collapses under the weight. When I work up the courage, I'll ask him whether he thinks the hashcash idea might work if there were some way for people on slow machines to pay their ISP to do the computation for them. This seems more likely to work than the mythical micropayments systems which people always suggest, since you're dealing with an organisation with which you already have some kind of billing arrangement.

Danny O'Brien (of NTK fame) linked to a posting by someone who gets why I write stuff to filter the crap that works on Windows for people who aren't very technical: This time we said it would be different, remember? If I can manage to concentrate this evening, might do some more work on another Spampal plugin.

Seeing as a few friends seem to be getting into LiveJournal, I'll mention that Friendster seems to be the new SixDegrees (anyone remember that). I'm tempted.

So, Thom gave me a magic cookie and I made myself an account. Not entirely sure what I will use it for yet, but I expect I'll think of something.

Things done today:

Work
Successfully got Sainsburys listed in SPEWS, as well as their spamming friends (well, I may have had a bit of help, but I claim that posting to uk.net and news.admin.net-abuse.email with the subject line "ATTN SPEWS" might have tipped it). Major supermarket chains quake at my words, I tell you.
Went to Ed and Jacqui's engagement party in some forgotten sub-basement of Trinity. There was glitter and dips and friends and people I've not seen for a while, so that was good. Danced one cha-cha but was wearing wrong sort of shoes so could not spin.