April 30, 2004: |
I figured something out today that's been a mystery for a long time: Why PocoMail will sometimes bring down messages with headers but no bodies. The damaged messages always contain this line: X-Poco-Decode-Error: Cannot find boundary It only appears to happen when I have POPFile installed and working between my POP server and Poco. Long-time readers may recall my struggling with this issue last fall (see my entries for November 20 & 28, 2003) and guessing that it was Poco's problem, because mail accounts that do not pass through POPFile get an occasional message munged as well. I had a hunch that something in Poco was getting out of sync with something else in Poco, and usually when parts of a program get out of sync, it's because one something is taking a long time and the other something can't wait for it. My intuition was that Poco was taking an increasing amount of time testing for blocked senders in junksenders.txt, and checking message bodies for text strings that I had stored in junkbody.txt. These two files have grown enormously in the past year: 4,174 lines in junksender.txt, and another 2,366 lines in junkbody.txt. So I did something I should have tried ages ago: I blanked out both files. I don't need them now, since in only three days POPFile has reached 98.94% accuracy, and training POPFile is way easier than trying to stay ahead of the spammers, who are constantly buying new domains and retiring their old ones. (I kept copies of the full files, and the last updates are available if you want them from ftp.duntemann.com.) Mirabile dictu! Since I blanked the two filter files, not a single message has come in munged. Not one! I was getting about one in twenty before, which doesn't sound bad, but when it's a real message, well, that's bad. So it looks like I can now use POPFile and Poco together without losing any message bodies. I'm ok with that, heh. I'm going to pass the information on to Slaven and his gang and see if they can find the hotspot in the code, but in truth I don't much care anymore. POPFile does the job. I don't need to create a client-side black hole anymore, though it would be an intersting exercise. Interestingly, the next release of Poco Mail will contain a built-in Bayesian filter, and it will be interesting to see if it can match POPFile for accuracy. This seems to be a trend; mail client The Bat! has a Bayesian filtering plug-in, though it took a great deal of screwing around to figure out how to work it. Mozilla Thunderbird has one too, though in my testing it didn't learn very quickly nor very well. BTW, I will be on the radio again tonight, talking (as usual) about spam and promoting Degunking Windows. Tune in to KLAV-AM in Las Vegas (1230 kilocycles—I love saying that!) or try their Internet radio feed. It's a whole hour, from 6 PM Pacific time until 7. I'll be talking about POPFile, POPTray, and the futility of keyword filtering. See you there! |
|
April 29, 2004: |
I just stumbled across one of the most original tech concepts I've seen in years: Poco Mail's EmailVoyager, which is basically a USB flash drive with a copy of Poco Mail Portable Edition preinstalled, along with some proprietary technology that hooks into the USB driver and launches Poco Mail PE when the device is inserted in any USB port. For years I've done various things to schlep my mailbase around with me when I travel. I can't just leave everything on my server until I get home; I'm constantly pruning and sorting and organizing the mailbase, which (even after a catastrophic crash last fall) still represents about 15,000 messages. Not pruning and sorting and organizing is how my mailbase got so big to begin with, so I'm pretty conscientious about it now. Usually I transplant the whole thing from my desktop to my laptop using Poco's built-in backup/restore features, by writing out a backup copy over the network to my laptop and then restoring the laptop's copy of Poco from the backup. At least while the laptop's networking machinery was functional this worked fine. (After that I used ZIP drives.) When I got back home, I did the same thing in reverse. It's awkward, but it works. With Email Voyager, I could still use my laptop to read mail when I travel, but I wouldn't be tied to the laptop. I could use my nephews' PCs while I'm visiting them in Chicago, or the desktop machine I keep in my mother-in-law's basement. I could use a machine in a library (assuming it was new enough to have an accessible USB port) or at an Internet cafe. Plus, if somebody swiped my laptop, they wouldn't necessarily get my mailbase. The Web site is a little vague on how you sync an EmailVoyager with the standard version of PocoMail, but if they're smart they'll figure it out and make it seamless, and that would indeed be a formidable gadget. I'm probably going to buy one and write it up for the book. (Yes, buy. I have a policy of not seeking out nor even accepting "review" materials for my books. We're not talking king's ransom here, and I like not being beholden to vendors. I pick 'em, I buy 'em, and I call 'em as I see 'em.) In the meantime, does anybody know how to make an app launch on plugging in a USB drive? I'd like to do the same thing with Aardmarks and my bookmarks database, which I have long kept on an old 16 MB Q flash drive. (See my August 27, 2001 entry.) |
|
April 28, 2004: |
Some odd lots before I get back to work on Degunking Email, Spam, and Viruses:
|
|
April 27, 2004: |
My Web hosting contract with Interland runs out next month, and earlier today I established a new account at SectorLink, which costs less money and gives me a lot more power. Interland wouldn't even give me a Perl interpreter, much less MySQL; SectorLink gives me all of that plus PHP, five simultaneous domains, and lots of other things I don't fully understand as yet. I will thus be able to pull a few of my other domains out of the parking lot and actually use them. Kris Sotelo, who used to lay out the late and lamented Visual Developer Magazine, will be redesigning my Web site for me later this year. I'll be posting a site for Copperwood Press (see my entry for April 1, 2002) and with any luck at all will begin selling the first of several books I intend to publish myself, including my novel. Also, my primary email address will change. I'm a little tired of fighting spam the hard way, and some time in the next month or so will probably turn off duntemann.com for a few months to see if spammers really do clean their databases. I will be emailing all my regular correspondents to let them know what my new address will be (it will be under the copperwood.com domain) and will also post a bitmap representation of it here. I'm intrigued by PHP and hope to try it out, though I'm not sure what the project may be. And finally, I'll be able to use a real Web diary program and not this ancient hand-bashed lashup that I've been using since early 2000. Should be an interesting year here. Stay tuned. |
|
April 26, 2004: |
The problem with high bookshelves is that you have to be able to reach them. We have the rolling ladder on our library wall, and it's a fine thing, too. But we also have high shelves in the pantry, as well as a smaller book wall here in my office, with a shelf at just under 8'. So I bought a Kramer Kik-Step stool (left) and we love it. It rolls under the bottom shelf in the pantry with an inch or so to spare...but while it's built like a battleship, it contains so much genuine rubber that it makes the pantry smell like a Discount Tire showroom. (It's currently exiled to the back deck to air out a little.) I lucked out in dealing with the high shelves here in my office. While shopping for a coffee table, we happened upon a weird little item that suggested Shaker origins, though it was actually manufactured in China, like almost everything else these days. It looks like a small wooden chair and can function as one if needed; it's currently the "guest chair" here, unless you count my reading chair as a guest chair. On the other hand, the chair is split on a diagonal beneath the seat and hinged: When you grab the top rail and flip it forward, the chair becomes a stepladder. It's the guldurndest thing. And although it's not well balanced and has to be flipped with care to avoid tossing the whole thing on its face, it looks cool and really does let me get up to the high shelf with ease. The most astonishing thing about it was the price: $29.95 list, on sale for $25.95. I had to assemble it, but in doing so I saw clearly that it's made of a good dense wood and lots of it, with key joints fastened via allen-head screws (hidden beneath wooden plugs) rather than glued. I'm not sure where you'll find them, but it's the coolest piece of low-tech I've acquired in a very long time. |
|
April 25, 2004: |
I ran across a ZDNet article this morning on how Google's SafeSearch content filters indiscriminately block any domain with the letter sequence "sex" in it, even when the site isn't remotely about sex. All the usual dirty words are filtered out, plus a few surprising ones. Examples of clobbered domains include: PartsExpress.com (contains
"sEx") and so on and on, not to mention anything relating to Sussex in England. Although Google claims that SafeSearch "uses advanced proprietary technology that checks keywords and phrases" and filters out only Web pages "containing pornography and explicit sexual content" it sure looks like a completely idiotic text-literal search algorithm that any brain-damaged CS freshman could knock out in an hour, if that. (Sounds like five lines of Delphi to me.) I don't much like server-side spam filtering for this reason: Most server-side spam filters err on the conservative side, and the server people don't care. Why should they? There is no way to check on them; no local logs; no way to tell if you're not getting useful messages. At least with SafeSearch you know when your domain is being nukedand sooner or later, either bad press or lawsuits will force Google to rethink their filtering techniques, and maybe put some genuine "advanced technology" into the effort. Filtering images is hard. I first noticed this problem a while back, when I first discovered the image search features on booth Google and AllTheWeb, both of which have filters that claim to knock out porn from the search results. They're damned useful when you have an odd word in hand and want to know what its namesake looks like. Search on "bugia" or "ostensorium" and you'll get a quicker look at the artifact in question than you would if you had to hunt down a dictionary of obscure Catholic liturgical paraphernalia. The problems came in when I was looking for pictures of some odd transmitting tubes. Search on "8877" or "815" with content filtering off, and much of what you see will be hardcore porn images, along with hordes of other irrelevant images of almost every kind, because of numeric strings embedded in digicam-generated file names. (Watch out for AllTheWeb in particular—not for the squeamish!) On the other hand, when I turned content filtering on, a lot of the legitimate images vanished too. No kids here, and porn bores me, though I get irritated paging past it looking for something pertinent. So I grit my teeth and deal with it. Still, I keep coming back to the same conclusion: Filtering cannot be completely automated. Why do we continue to try? |
|
April 24, 2004: |
Networking hasn't been working well on my laptop for some time now, and I was never able to figure out why. Everything looked fine from the Physical up through the Transport layer, and it would work the Internet as long as it had a connection, but Windows Networking hostnames would not propagate either from it or to itexcept very occasionally, when my other machines could see the laptop but not browse its shares. Most of the time, the laptop was network-invisible and network-blind, and something was clearly clobbered in the Windows network machinery itself. Unlike most PCs, the laptop—an IBM Thinkpad X21didn't come with any kind of Windows OEM or reinstallation CD. Instead, there is a hidden partition on the hard drive containing the OS and hardware-associated software and drivers in compressed form. If your Windows installation dies the death, you press F11 during POST and the recovery program takes over, which then re-creates your Windows instance, just as it was when the machine came out of the box. I got fed up last night and after spending a few minutes copying some data out to a SanDisk USB flashdrive, I turned Product recovery loose. It took almost two hours, just like the manual said, and it encompassed a huge amount of console-mode grimbling and scripting and automated screwing around, but as clumsy as the system looked in operation, it worked as advertised, and I now have a reconfigured laptop that networks perfectly and runs way faster than before, probably because it has a brand-new and uncorrupt Registry. My suspicion is that some spyware that I deleted messed over Windows Networking (the trial version of the Snood game comes with that damfool Bonzai Buddy) and when that happens, reinstalling Windows is pretty much your only recourse. I mention this in detail because in recent weeks, my #2 spam category has become OEM software CDs. (Painkillers are #1, and penis pills have fallen to #3.) As I understand it, a court case somewhere gave people the right to resell OEM CDs if they want to, and as with the painkillers market, black market goods join the gray market for legitimacy's sake, and I suspect some OEMs are quietly selling some disks to the spammers to make a little money. Having your recovery "CD" in a partition on the hard drive that only the recovery utility can access gives the consumer nothing to duplicate or sell. If your hard drive as a whole fails, IBM will sell you a new one, and replacing it can be done without dismantling the whole device. Me, I think I'd rather have the CD, but if I had the CD I'd have to put it somewhere and then remember where that somewhere is. All in all, it's a pretty interesting solution to a problem Microsoft and Adobe moan about a lot, and I wonder why in this age of supertanker hard drives, more PC vendors don't do it. |
|
April 23, 2004: |
A few odd lots on this why-is-it-snowing-like-hell-at-the-end-of-April day:
|
|
April 22, 2004: |
One of the world's most notorious spammers has come up with a diabolical new idea: Pay people to act as relays for its spam, making it harder to block by domain or address. Sendmails Corporation, which may or may not be based in Manchester, NH, has created a system called VirtualMDA (Mail Delivery Agent). VirtualMDA does with money what a lot of spammers do with viruses: Puts a mail proxy on some starving slob's PC. Unlike the virus spammers, it then pays according to machine cycles expended by that PC sending out spam. It's not a lot of money; the rate is currently $1 per CPU hour. That in itself is deceptive, since it may take a full day to rack up a CPU hour. (Networking delays don't count as CPU time, heh.) Sendmails is quite clear on its intent: To scatter a spam campaign across many email addresses and make it harder to block. What they don't say to the idiots who will sign up is that it may get their Internet service cut off, and may, though it's still not clear, put them in legal jeapardy if any CAN SPAM violations occur within the mailing. (Not that we're enforcing CAN SPAM at all.) The press hasn't touched on the worst part of all this: With their spam affiliates programs and now VirtualMDA, spammers are recruiting desperate people to do their heavy lifting, hoping that public outcry will not settle on single mothers who are about to be evicted for nonpayment of rent. An unmarried young woman with two preschool children asked me a few years ago what she could do with her computer that would bring in some money, any money at all. I had no good suggestions for her, but it's even money she's now a spammer. My fervent hope is that one or more of those desperate young women will lose her Internet account over VirtualMTA and sue the bastards. The public would be more than willing to pillory Sendmails, and with all the money to be made in spam, one can imagine that at some point the tort lobby will smell blood in the water and get to work on the rest of them. |
|
April 21, 2004: |
Shucks. Something's going to happen on June 8 that hasn't happened in 122 years, and I'm gonna be on the wrong side of the damned planet. On that day, Venus will cross in front of the disk of the Sun, creating what might (if we were a whole lot closer to Venus) be a sort of Cytherial eclipse. (I don't think I'll use the term "venusian" orworse"venerial". I'm not sure what real astronomers say in such a case.) Transits of Venus happen quite rarely. The last one was in 1882. I recall reading in my Golden Book of Astronomy as a kid in 1959 or so that the next transit of Venus would happen in 2004, and I thought, Wow! That's the Future! with visions of spaceships and aliens dancing in my head. We're 0 for 2 on that score, unless you consider the Space Shuttle a spaceship. (I was thinking of something that lands on its tail, with retractable legs, just like in all the movies.) Anyway. On June 8, the disk of Venus, which subtends an angular diameter of about one arc-minute, will cross the disk of the Sun, which is about 31 arc-minutes. The crossing will take about six hours. There are subtleties associated with diffraction of sunlight through Venus' atmosphere that will have observers salivating. There were reports in earlier transits of a silvery arc on Venus' edge after the planet had begun its ingress to the Sun's disk, and also the "black drop" effect as the planet leaves the Sun's disk, which we now know is an optical illusion and not a true astronomical phenomenon. (They were still arguing about that in 1882.) The real problem with this year's transit is that by sheer dumb luck, the best place to see it is in central Asia. The eastern US will get a tantalizing glimpse at sunrise on June 8, but Colorado is on the other side of the cutoff, which falls on the Nebraska border, sigh. Europe is reasonably well-placed, but one doesn't usually go to England to do solar observations. There is a saving virtue: Transits of Venus, like certain other good and interesting things, always come in pairs, and for transits of Venus, almost precisely 8 years apart. The last pair was in 1874 & 1882. The current pair happens in 2004 and 2012. On June 6 of the month in which I will celebrate my 60th birthday, we'll get another chance, and this time, the US will be a lot more favorably placed. Alas, the Mayans, the ancient Egyptians, and most New Agers think the world will end that year, so we may miss it anyway, dang that pesky Armageddon. Web sites focusing on the transit have long been in place. This one is my favorite, and will provide a lot of additional information and further links. This map shows where the 2012 transit will be visible. We here in North America don't get the whole show, but I plan to be in Hawaii that day, celebrating both Venus' dance across the sun and my dance into early middle age. Book early. We're gonna party like it's 1999! |
|
April 20, 2004: |
I install and torment Linux every couple of years, to see how far it's come along its long path toward Granny's desktop, as everybody likes to say. Two years ago it was Red Hat 8, and I was pleased, though I think Granny would still be pretty bollixed if she tried to use it. Yesterday I installed SUSE 9, and I'm still pleased. It may even be ready for Granny...if somebody else installs it for her. SuSE's Installer YAST (Yet Another Setup Tool) was pretty impressive, though installing the whole thing took almost two hours, much of which was apparently spent probing my hardware to find out what my system had. And it did well in that department, nailing almost everything exactly. I haven't been able to spend as much time as I'd like trying it out, but so far I've found only two operational failings:
|
|
April 19, 2004: |
The house is built, but we're going to be stubbing our toes (or lancing them) on the construction debris scattered around outside for a good long while. A week or so ago I took my big magnet outside the back door, rubbed it around in the dirt, and came up with half a pound of various pointy-ended hardware, mostly staples and bent nails. I recovered a small fortune in deck screws that way (which found their way into my stash) though the rest was garbage. However, by all margin the oddest thing we've turned up in the dirt is shown at left. We have no idea what it is, but Carol immediately called it a crutch for hamsters. "An emphatic crutch, at that!" I added, with emphasis, if you get what I mean. Interestingly, it looks like it's been in the dirt for a very long time, like many of the bent nails. I can only assume that the fill dirt they brought in was dug up from somebody else's construction site, and we've inherited as many rusty nails (and other odd things, including half a roller skate) as our project generated over the past ten months since we broke ground. But in truth, the thing is mysterious. It's almost exactly 4" long, by and inch and change wide at its widest point. The legend "ELL" or "ELLI" is embossed on it. I'm sure it's intended to be driven into something with a really big hammer. Is it a climber's piton? Anybody got any ideas? |
|
April 18, 2004: |
In the several months since I discovered Skype, I've gotten numerous Skype calls from people in other countries, none of whom I knew prior to the call. In each case, they were testing out the system by calling a randomly selected person. This morning, I got a call from a person in South Africa who told me he was explicitly calling people in different countries on different continents, trying to "collect" as many countries and continents as possible. As if amateur radio didn't have enough forces arrayed against it (deed restrictions have basically made ham radio illegal in virtually all new housing built in the last twenty years) now we have people creating a Worked All Continents program on VoIP. I vaguely recall a humorous article in one of the ham radio magazines from the mid-1960s, describing something called the Special Subscriber Service. The (wholly fictitious) SSS was a feature the phone company offered, which, when selected, would connect you to a randomly chosen person somewhere else in the phone system, just for a rag-chew. It was described in bombastically overdrawn language as a serious threat to amateur radio, and all readers were encouraged to write their congressman to protest. It was in the April issue of the magazine in question, I recall. I'm sure it was considered a scream by people at the time. Yet in looking back, it was first the advent of personal computers, and (more significantly) the Internet that corresponded with the aging of amateur radio. The median age in the ham radio community is now somewhere in the high fifties, and even in older housing where hamming is still legal, younger people, perhaps realistically, are looking at today's heavily computerized, pushbutton radios (some of which actually use your Windows display for all control functions) and asking, Why is this different from the Internet? Deed restrictions are certainly an issue, but even where they don't apply (as in mobile radio) interest among the young is at best lukewarm. Perhaps ham radio stalled when it got too easy. Like a lot of pursuits, you can make it as challenging as you choose, but the critical point is the difficulty of the bottom rung. Modern, all-in-one broadband radios require almost nothing of the operator beyond the ability to read some relatively simple instructions. I found that the case with my dazzling Icom 736: You select a frequency, and the built-in tuner matched my length of random wire to the chosen frequency. You then grab the mike, push the button, and call CQ. A lot of the struggles I had overcome when first starting out in 1973 were simply not there, and much of the fun had gone out of the game. About 1995, I more or less lost interest in HF, and continued on where the challenges were a little more engaging: Building my own gear from loose parts. The downside is that the types of radios you can realistically expect to build are either limited to Morse code or use modulation techniques that nobody else uses anymore, like AM. In other words, you can build an AM transmitter, but there won't be anybody on the air to talk to you. The legendary Jim Kyle told me that as soon as he saw the Internet, he set ham radio aside forever, and Jim has been a ham since before I was born. It's depressing, but it may be inevitable. We don't encourage kids to build things of any kind anymore (Heavens! They might hurt themselves!) and absent that part of the hobby, the rest is increasingly commoditized and boring. My longstanding suggestion of setting aside parts of the ham bands and then forbidding the sale of commercial gear for those sub-bands has gone nowhere. Beyond that, I have no suggestions. It's grim. |
|
April 17, 2004: |
Evidence continues to accumulate that fertilizing the southern ocean with certain chemicals needed by phytoplankton can pull huge amounts of carbon dioxide out of the atmosphere and trap it on the ocean floor. A recent test by Moss Landing Marine Laboratories sprayed some soluble iron compound onto the ocean surface near Antarctica, and then measured the phytoplankton bloom that resulted. From their little test alone, 30,000 tons of CO2 was deep-sixed. I first heard about this line of research a few years ago, and I was struck by the leverage obtained between materials and effort expended and the results obtained in terms of sequestered CO2. I think it was Bill Higgins who quoted a scientist (whose name I have lost) as saying that "Give me an oil tanker full of soluble iron, and I'll create an ice age." Wow. What this means, of course, is that for what for the Federal government would be petty cash, we could sequester an arbitrary amount of CO2 on the ocean floor, billions of tons of it if we chose. I think it should be done, but although I've seen no reaction from our ever-so-vigilant environmental lobby, I'd lay odds that they would attempt to block any systematic attempt to fertilize the southern ocean on a large scale. The danger they would point to, obviously, is that we could create an ice age this wayor at very least trigger climatic change that we can't predict. The danger I would point out in response is that we're triggering climatic change ever day that we continue to burn carbon while slandering nuclear power and letting mansion trash like the Kennedys NIMBY wind farms to death. It would, in fact, make a great action-adventure not-quite-SF novel: Gallant pragmatic environmentalists battle homicidal Luddite Greens on the high seas as six rusty cargo ships steam toward the Antarctic, with only one containing the iron slurry. (Yes, I'm only half serious. It's up to you to guess which halfand hey, George, why not write that novel anyway? I'd even publish it!) The larger question is this: Should we try to fix global warming by finding Gaia's pressure points and giving her a noodge? Or must we grit our teeth, stop burning carbon, and accept the societal consequences? It's an important question, since it's far from clear that we're running out of hydrocarbon fuels as quickly as the doomsayers have predicted. Natural gas, in particular, seems to be seeping up from parts unknown and recharging gas fields thought to be depleted years ago. This is a good thing, if we can somehow manage the carbon that comes out of the waste side of the equation. The nice thing about the phytoplankton is that they eat carbon and die, sinking into the ooze on the ocean bottom, to be eventually folded back into the crust from which the carbon originally came. What's not to love about that? |
|
April 16, 2004: |
While looking around the Net for pictures of my favorite bomber, the XB-70, I stumbled across the phenomenon of scan collecting, and from it took an interesting insight: The record companies should be glad that there's no unique identifier for individual songs published on CDs. Scan collecting seems to be centered on a Usenet group called alt.binaries.amp, which has a huge pass-through of image files. About 30-40% of them are soft-core porn, basically Playboy-type images of naked girls leaning on trees. The rest cover a very broad range: Fine art, postage stamps, trains, aircraft, classic cars, and so on. It's not random, either. There's a system to it: Some fanatic somewhere creates an index of a certain number of images, which is large but finite, usually a few hundred. The index is in .csv format, and gives the name of each scan in the collection. Collectors then scrounge around, looking for the files by their index name, until they have them all. Consider the concept of collecting hi-res images of stamps. They're generally very well scanned, and can be blown up to fill your entire screen. Who needs a magnifying glass? For that matter, who needs stamps? It's unclear where most of the images come from, though some collections (like images from Playboy's Cyber Club) are explicitly named. I'm sure they're almost entirely pirated copies of clip-art collections or somesuch. Stamps are an interesting example, because there are several authoritative indexes of stamps, each with unique catalog numbers. The Scott catalog is the most famous, with a unique number for every US stamp ever issued. (There are catalogs of foreign stamps as well, but I remember the Scott catalog from years ago, when I collected US stamps myself.) The insight I had concerns such catalogs: If individual copyrighted files can be given unique file names according to some kind of catalog, collecting them through newsgroups could be completely automated. A usenet robot could scan newsgroup headers, find file names fitting a certain collection template, and download those that are not already in its database. The record companies have not paid much attention to newsgroup file swapping because it's not selection-driven. In other words, it's like listening to the radio: You can't choose what's "on;" you have to be content with what comes by. To find a specific track or image that you're looking for takes a lot of effort entailing large amounts of time and eye-crossing scanning of newsgroup header lists. If there were a unique naming system for music tracks, such robots would appear quickly, and would just lurk on the newsgroups until they found what they were looking for. Newgroup servers that don't keep logs of who does what (and they're not rquired to) are reasonably anonymous, assuming posters forge certain parts of posting headers. A Usenet robot could listen for requests from other Usenet robots, and post those files from its own database that have been requested. I'm a little surprised such tools don't yet exist, and I think the key reason is the lack of a standard catalog for music tracks. If such a catalog ever happens, Usenet music swapping will become automated, and it's considerably tougher to discern who does what on newgroups without server-side logs. The file-swapping wars are not over, and it'll be interesting to see how they evolve in the next five or six years. |
|
April 15, 2004: |
Some odd lots on this beautiful spring day:
|
|
April 14, 2004: |
As I've repoted here previously (see my entry for March 14, 2004) there is an astonishing turnover in spam "payload domains" (domains for which there are hyperlinks within the bodies of spam messages) on the order of 12% per day. In other words, 12% of my spam every day comes either from or carrying domains I've never seen before and haven't yet blocked. I have noticed that there are standard patterns in some of those payload URLs. A typical payload URL from one of my worst spammers looks like this: http://wjusytd.com/py/ I rarely see the same domain twice from this guy. However, the substring ".com/py/" occurs in every one. Poco's message body filtering is sharp enough to spot that substring; however, I worry about false positives. You Python guys out there, tell me if this URL structure is some sort of standard convention for Python work, or if the /py/ is just something the spammer made up. I suspect it's a spammer thing, as I get a fair number (if not quite as many) payload URLs in this form: http://wjusytd.com/v/ This is mostly curiosity on my part. I'm getting tired of spending hours hunting for commonalities inside increasingly bland-ized and randomized spam, and will be moving to something differentprobably including a new primary email addresssometime soon. Keyword filtering has simply failed, and I'm sure I've worked on it harder than just about anybody. |
|
April 13, 2004: |
Some odd lots as the snow melts and (maybe) Spring arrives...
|
|
April 12, 2004: |
While researching email tools for my new book, I stumbled across an interesting software gadget: the POPTray email notifier. It's free, open source, and (most wonderfully) written in Delphi. What it's for may take a little explanation: POPTray sits in the Windows taskbar tray, and every so often (you set the interval; mine is set at one minute) it goes up to your POP server and looks at the messages waiting for your regular email client. You can set rules based on several parameters, primarily the sender address. If POPTray sees a message from a specified sender, it will pop up a balloon message to let you know, or (if you enable it) sound an audio announcement. This is useful if you want to get mail from certain people as soon as it arrives, but don't want to be fooling with your email client every ten minutes. Once POPTray sounds off, you can launch your regular mail client, or even take a peek at the message in POPTray's own window. Basically, you can read your email only twice a day, and still get those priority messages as soon as they arrive. POPTray's other use is to delete messages from known spammers before you waste time downloading them to your mail client's inbox. Eyeballing my spam indicated that a relatively small number of domains generated a great deal of spam. I often get as many as thirty spams a day from OptinRealBig megaspammer moosq.com alone. By setting up only about 25 rules, I eliminate about 40% of my spam before I ever download mail, which makes the downloads go a lot more quickly. POPTray is a great deal like what I had envisioned for my Aardmail email robot. Since I have the source, I intend to look into adding a client-side black hole feature. I have bits and pieces of the code already done and tested; how hard could it be to pull it all together? (Don't answer that...) |
|
April 11, 2004:Easter
on my deck
|
Easter Sunday. (And we have almost a foot of snow on the ground! Do people here dream of a "White Easter"?) Continuing the thread I've been following these last few days of Holy Week: The real challenge to those of us who are convinced that Christianity is a good thing is to "out" the lie of Manichaeism anywhere we see it. Doing that requires that we recognize it for what it is, which in turn requires a little background. The early Christian teacher Mani (216-277) was simply laying out his interpretation of an ancient wisdom school that originated in Persia before the time of Christ: That the True God is completely spiritual, and so removed from this world of matter that He cannot be apprehended by human beings in any way. The physical world was created by a sadistic godling called the Demiurge and his demonic minions the Archons, as a trap for pure spirits (us!) within physical bodies that seem incapable of accomplishing good, or doing anything but suffering. This school of belief goes by the technical term Gnostic dualism, and although it existed long before Mani, he systematized it and popularized it, extending it from its roots in Persia throughout the known world. The Christian Church rightly condemned Mani and his theology. What the Church could not anticipate is the hold that Mani's worldview had upon the poor beset human beings of his time. It was the perfect explanation for the conundrum of theodicy: How can evil and suffering exist in a universe created by a benign and loving God? Easy! God didn't create the universe! It's a trap set by the Bad Guy! The real God didn't have anything to do with this filthy, nasty, flea-infested, plague and famine-beset world of suffering! So although Manichaeism as a religion or even a coherent theology perished over a millennium ago, the Manichaean worldview stuck hard to the bootheels of humanity. That worldview can be simply expressed: Matter Bad. Spirit Good. And oh, the trouble that this has caused. The medieval Church absorbed Manichaean ideas to the extent that it emphasized the freeing of humanity from matter altogether, in that this world was to be stoically endured but not loved until such time as God freed us from matter's torments in death. Echoes of the Demiurge can be found in the Anselmian notion of an angry God the Father who would have roasted all of us in hell for the sin of simply having been born unless Jesus hadn't bought His Father off with His suffering and death. The notion of healing is alien to the Manichaean worldview. There can be no healing because matter is irredeemable and implacably opposed to spiritual goodness. Sex was frowned on and in some places demonized, echoing the Manichaean teaching that sex traps divine spirits in corrupt bodies and thus should be avoided at all costs. Because most pleasures are physical pleasures (and rooted in our interaction with matter) Manichaeism found pleasure of all kinds suspect at best, and often condemned. Matter Bad, Spirit Good begat Pleasure Bad, Suffering Good. As though to throw it back in the face of the implied Demiurge, suffering was elevated to a kind of perverse virtue, and monastics thought they were sanctifying themselves in God's eyes by beating themselves with whips. Almost unbelievably, this practice was a commonplace until the Second Vatican Council in 1960. Garry Wills recounts in his autobiographical work, Why I Am a Catholic, that as a young Jesuit seminarian in the 1950s he was required to beat himself with a small whip called "the discipline" and wear chains around his waist that rubbed his skin bloody. (My suspicion is that a revolt against such Manichaean excesses was what caused the Jesuits to morph into one of the most liberal of Catholic religious orders by 1970.) Monasticism has a natural tendency toward Manichaeism, and Henry VIII's dissolution of the English monasteries, although entirely venal in intent, had the accidental benefit of eliminating one of the major breeding grounds of Manichaean excess from the English Church. It took until 1960 for voices within the Catholic Church to dare claim that the mission of Christ and the Church was one of healing, and not one of lopsided Divine justice that required infinite suffering to atone for finite transgression. We will be centuries getting that message across; I guess it's no failure that we haven't entirely won the war in only 40 years. When today's malcontents rail against Christianity, they're really railing against those aspects of Christianity that still cling to Manichaean cruelty. Many sects within Christianity have given it over almost entirely, but those that don't get a lot of press. Mel Gibson's penitentialist church is tiny, and yet it makes national news because of its benefactor. Those Catholic sects that have disavowed Manichaeism labor mostly in obscurity. I first saw such a sect when I discovered Old Catholicism, and am discovering another in high-church Episcopalianism, which in some places (like St. Raphael's parish where we currently worship) are almost indistinguishable. Ultimately we will save Catholicism from Manichaeism, but it will be a long climb. There's a lot more to say on this topic, but I'm going to give it a rest for awhile. Just writing about it makes me depressed, and Easter Sunday is no time for depression. Easter is a feast of exultation and renewal, healing and hope. If it weren't snowing I'd fly a kite (see my entry for April 15, 2001) but at this point, sitting down in my cozy chair with a good book will more than compensate. The time for exultation is coming; only about a hundred more boxes to go! |
|
April 10, 2004: |
Holy Saturday. Continuing with the thread I began yesterday, sparked by several inquiries about my feelings about Mel Gibson's hugely popular (and terrifyingly gruesome) film, The Passion of the Christ. Gibson's film has resurrected a really, really stupid and loaded question: Who killed Jesus? The implication, of course, is that the Jews killed Jesus, which is what most people are actually asserting when they ask the question. This was in fact the historical Christian position for almost 2000 years, and many Jews were tormented and killed over it. Stuff like that dies hard. Until the dawn of the 20th century, there was a prayer in many Catholic liturgies for Good Friday called "The Reproaches of the Jews," which was basically a scold asking that Divine punishment be visited upon the Jews as "Christ killers." It is true that a literal reading of the accounts of the Passion in the Gospels would give that impression. The objective truth is a lot more complex. Until the Jewish War of 66-70 AD, when the Romans destroyed the Jewish Temple and scattered the Jewish people, Judaism was an officially sanctioned and much-admired religion within the Roman empire. The Romans liked religion of all sorts, because it was a factor in social stability, which they valued highly. After the Spartacus slave revolt in 73 BC, the Romans became (justifiably) paranoid about political agitation. Spartacus had practically shattered the empire; had he been a better leader, Italy might have been split from its northern lands, and the Empire might have collapsed centuries before it eventually did. The Holy Land in the First Century was not the tranquil realm suggested by the Gospel accounts of Jesus' public ministry. The secular histories of the time paint a picture of a land in turmoil, full of minor riots and viciously suppressed uprisings against Roman rule. In 63 BC the Roman general Pompey had (after killing over 12,000 Jews) desecrated the Jewish Holy of Holies by riding his horse into the middle of it, which was a blasphemy unlike anything the Jews had seen for many centuries. The Jews did not suffer Roman rule especially well, and insurrection was always just a hair below the surface of Jewish culture. By 66 AD, the Romans had had enough. Nero gave the order to level Jerusalem, and the order was carried out. After that, it was almost as dangerous to be a Jew as it would later be to be a Christian. The Romans would tolerate any religion...if that religion's followers would grant the Roman Emperor primacy. After all, to the pagan Romans, a god was a god was a god. This is why the Gospels give no mention of Jewish unrest in the 30s AD, and tread so lightly on the Romans: The Gospels were written after the destruction of the Temple, in the 80s, and the Romans were very twitchy about the Jewish Wars. The truth is this: The Romans killed Jesus. There was doubtless some complicity in certain Jewish quarters, particularly among those largely Hellenized Jews who administered Jerusalem and the Temple, and had the most to lose in case of a Jewish uprising and the inevitable armed Roman response. However, in terms of religious feeling, those could barely be called Jewish, and were often held in contempt by observant Jews, some of whom (like the SicariiLatin for "daggermen", assasins) would hide knives under their robes and murder those barely-Jewish Roman collaborators on dark streets when they felt they could get away with it. Why do we know this? First from the secular histories of the time:
There are also some hints in the Gospels themselves:
Basically, Jesus had the observant, religious Jews of Jerusalem behind him all the way, and it's likely that huge numbers of Jews died at Roman hands between the mid-30s and the Jewish War of 66-70 AD. Had the Gospel accounts told the unvarnished and unsoftened truth, they would not have come down to us today, and many historians feel that the Roman persecutions of Christians in later centuries were at least in part a consequence of the long Roman memories of Jewish uprisings. For the first hundred-odd years, Christianity only gradually shed its identity as a breakway Jewish sect, and to Roman eyes, that breakway was much less clear than it was to the Christians themselves. The Gospels shifted the blame from the Romans to the Jews because they had to if Christianity were to survive. It did, and survived to first swallow the Roman Empire and later most of the known world. There is nothing about the Gospel accounts of Jewish agitation against Christ that are central to the Christian message, and it's high time we set that ancient dodge aside. It was the Romans, and the Romans are gone. Let's get over it! |
|
April 9, 2004: |
Good Friday. Because I'm a theist (and a Catholic of some stripe, though the color of the stripe is still in flux) people have been asking me what I think of Mel Gibson's The Passion of the Christ. That's a complicated question. First of all, I haven't and won't see the film itself. I simply do not see films depicting vivid cruelty, religious or otherwise, and apparently Gibson has outdone himself depicting the cruelty of the Crucifixion. No thanks. Just hearing the Passion according to St. John read in church earlier today brought tears to my eyes. I can imagine it well enough. Nobody has to paint me a picture. I've read enough about it, however, to go a little bit higher in perspective, and state my position generally on what I call "penitentialism," which is the subculture within Catholicism (and probably places in Protestantism) that emphasizes the sinfulness of mankind above everything else, and sees all of Catholic theology through that lens. That Gibson belongs to the penitentialist crowd is pretty obvious. I have special feelings about penitentialism for a couple of reasons:
What bothers me about films like The Passion of the Christ is more than just the SFX of hooked whips tearing chunks of flesh from Jesus' back. The core of my discontent is that the Passion is meaningless outside the context of the three Great Miracles of the life of Christ: The Transfiguration, the Resurrection, and the Ascension. Christ's pain was both physical and emotional, and any human being can potentially feel the same degree of agony that He felt. By itself, the Passion is just another human tragedy. Unless set in the matrix of the Incarnation and appreciated as part of that ineffable organic whole, the Passion alone is simply a celebration of human cruelty and a condemnation of human nature. I'll have none of it. If you'll grant me a few more days to speak of matters religious (which, as with politics, I strive to keep to a minimum here) I'll follow this thread a little further. |
|
April 8, 2004: |
Some software products appear to succeed in spite of the fact that they're lousy. Qualcomm's Eudora email client is a perfect example. I used it years ago and thought I'd gone to heaven when I moved to Outlook Express, egad. Well, I'm back in Eudora land, because it's supposedly the third most popular email client (after Outlook and Express) and I have to cover it in Degunking Email, Spam, and Viruses. I bought the full version of Eudora 6.1 for $50 (at least it's deductable) and I've been poking at it for a day or two. The product hasn't changed much since 1995 (it was first released in 1991!) and its clunky UI is at odds with almost every other major application out there, including all the most popular email clients. Only Pegasus Mail is less intuitive. Let me catalog a few of my discontents:
|
|
April 7, 2004: |
Catholic spirituality, prior to the "opening up" of Vatican II, was a very insular culture, and odd in ways that few remember these days, even those of us who were there. Back at Immaculate Conception Elementary School in Chicago, the bell rang every hour on the hour, and no matter what else we students happened to be doing, we stopped cold and in unison said a very brief prayer: "Most sacred heart of Jesus, all for thee: Jesus, Mary, and Joseph!" The prayer was said quickly, rhythmically, and emphatically, and in looking back, what it reminds me of most was a football cheer. Prior to 1960, the technical term for this prayer was ejaculation. Really. And if you look in older dictionaries, that's what you'll see: "A short, emphatic prayer put forth in time of temptation," according to my three-volume 1936 Century Dictionary, which I keep precisely for occasions like this. (Arcane vocabulary, and literacy in general, went into decline after WWII.) As for the temptation, well, if you were a Catholic in the 1950s, every moment was a moment of temptation, since virtually everything entering the mind of a Catholic was a temptation to one sin or another. When I was in third grade (1960) the nuns quietly changed the jargon, and we no longer ejaculated every hour. The holy football cheer was the same, but was now called an aspiration. I note with some amusement that this meaning of "aspiration" is not in my Century Dictionary nor anywhere else, unless one bends the meaning of "lofty intention" to mean "a short, punchy prayer that sounds like a football cheer." All this came to mind after reading a short piece in the Wall Street Journal this morning, citing new medical research indicating quite clearly that frequent ejaculation reduces the risk of prostate cancer in men. Now, this other (medical) meaning of "ejaculation" was a frequent topic of the Passionist priests who ran our parish, in the confessional and on occasional "boys only" pep talks in seventh and eighth grades, the prime purpose of which seemed to be to scare us away from masturbation. Back then, masturbation was a peculiar obsession among Catholic priests (projection, anyone?) and if I hadn't heard them talk about it in school I truthfully wouldn't have known what it was. (I remember riding my bike down to the Carol Roden branch of the Chicago Public Library in 1965 to do some further research.) If the two species of ejaculation were in any way related, and if frequent ejaculation really does prevent prostate cancer, Catholic boys my age should be absolutely immune. The research cited in the Journal claimed that "frequent ejaculation" was twenty times per month; heck, we did it six times a day. And far from being "self-abuse" (as the older priests called it, without ever explaining why it was "abuse") if masturbation does in fact ward off the second-most prevalent carcinomic killer of men, the only moral thing for the Church to do would be to de-criminalize it. (I won't insist that they endorse it, or teach it. That would be cruelto the Church.) Alas, it took the Church 500 years to forgive Galileo for teaching that the Earth moves around the Sun. Masturbation may get out of the penalty box someday, but it's gonna be a while. |
|
April 6, 2004: |
Damn. The resistors won't go back into their bins. Back in 1992, I bought a rotating parts tower from Turnkey to bring my chaotic electronics parts storage system into some kind of order. A parts tower is a rotating stack of 12 multibin cabinets, and although it took me a couple of months to get my small parts out of milk jugs and cardboard boxes (my larger parts like tube sockets and relays are still in milk jugs) it made a tremendous difference. Moving it, however, was a pain. I took each little bin (there are over 200) and dumped its contents into a sandwich-sized Glad Bag. For partially full bins, the bagged parts would go back into the bin. For full bins, parts plus bag would not fit, and so I left the bin empty and put the bagged parts into a separate cardboard box. Once the bins were all bagged, I took the tower apart and stored it all (with abundant newspaper for padding) in two large boxes, where it sat for well over a year. As time allows (it's a low-priority project, at least for now) I've been un-bagging my parts and getting the parts tower back into full working order. The problem lies with those full bins. The bagged parts wouldn't fit back into the bins—so why would I assume that after dumping the parts back into the bins, the bins would fit back in their cabinets? I had originally placed the resistors into the bins one or two at a time, as I sorted them by value. As a chaotic wad, the resistors are about 30% larger than they were when they were all neatly placed, with leads in alignment and scarcely enough space between them for an SMT diode. Rattling and shaking the bins made little difference. I may have to dump them all out on the counter and put them back in as they went in to begin with: one at a time. Uggh. |
|
April 5, 2004: |
Some odd lots on a pretty busy day: (but aren't they all?)
|
|
April 4, 2004: |
Perhaps the solution to outsourcing IT jobs to India is to outsource IT jobs to...Goodland, Kansas. Rick Tompkins sent me a link to a Businesseek Online article indicating that IT people would work for lessmuch lessif only given the chance. It makes you wonder if businesses actually prefer hiring in a foreign country, perhaps hoping to duck our lawyer-crazed employment scene. Hard to tell, but the question that arises from the article is this: Can an IT professional live happily on $40,000 a year? Depends. In Silicon Valley, forget it. But there are small towns all through the Great Plains and the West where you can still buy a decent 3-bedroom house for less than $100,000. As broadband gradually makes its way into sparsely-populated areas, a guy who doesn't need to drive a Porsche could probably do well in a small town. I actually suggested this in one of my editorials in Visual Developer Magazine years ago, and ignited a firestorm from a significant number of people who hate small towns so virulently that I had to shake my head and grin. What do small towns have (or lack) that makes them so hateful to bicoastals? I wish I knew. My suspicions: no raves, no piercing parlors, and no population of clueless young women allowing guys so inclined to have sex with a different one every weekend. Gawrsh! What kind of life is that? The standard complaint that small towns are "boring" comes mostly from young people. Middle agers like me would as soon turn off the PC at the end of the day, cook and enjoy a nice dinner with our spouses, and then read a good book until bedtime. So given that middle-aged ITers have borne the brunt of unemployment in the current IT depression, the idea may have some merit: Start an IT consulting firm that works cheap, but first get your asses out of the big city. As American firms begin to bid up the price of Indian IT guys who know Oracle, small-town American may well begin to compete. |
|
April 3, 2004: |
For the most part, I loathe animated GIFs, but Pete Albrecht sent me one unlike any other I'd ever seen: He created an animation from the many individual photos he took of the shadow transit of several Galilean moons across Jupiter last Saturday. (See my entry for March 28, 2004.) Your Web browser should be animating it automatically, unless you have GIF animation turned off, which some browsers (including IE) support. The instrument was a 12" Meade LX-200, with a Phillips ToUCam webcam, set up out in Pete's backyard in Orange County. Instruments like the Meade LX-200 series have turned amateur astronomy inside-out over the past couple of years. New software is appearing constantly (oddly, much of it comes out of France) and one of Pete's more recent finds sights on a star and measures the periodic guide error stemming from imperfections in the main drive worm wheel, so it can apply a cancelling bias to the scope's drives. Some time this summer I'm going to perform some surgery on Pete's ToUCam and mine to allow them to take long-exposure photos. This won't help me much (both my scopes are junkbox specials and lack sidereal drives) but I'm licking my chops anticipating the deep-sky work Pete will be able to do then. |
|
April 2, 2004: |
The code I allowed myself a few hours to write the other day was something I've wanted to do for a long time: Create a system to resolve IP addresses for the immense collection of spammer domains I've gathered since about December 1, 2003. It wasn't just one program, and the three short utilities I wrote weren't very tidy, but their job was to process the PocoMail junkenders.txt and junkbody.txt files to knock out anything that wasn't a domain name, and then build a master-detail database of domains and IPs. The goal was to see how "concentrated" domain name ownership is among spammers. The total number of spammer domains I had gathered was 5,033. I wanted to know whether those domains represented 5,033 different IP addresses, or if most spammers had several or many IPs. I also wanted to know how many of those domains had been abandoned and were no longer DNSed to an IP address. So here are the results:
I wasn't horribly surprised by this. I was a little surprised at how many spammers have only one domain for an IP, and it brings up the question of how long a given domain "lives" (that is, points to an IP) before the spammer abadons it, registers another domain, and attaches it to the IP address. Domains (especially .biz and .info) are sold very cheaply in bulk, so using a domain only a handful of timeseven once!can pay handsomely, and it makes filtering much more difficult. If anybody actually wants to see the database, I'll try and find some time to build a self-explanatory database browser for it and put it up on my FTP site. I'll be tinkering the database and the utilities as time allows, and ultimately I still hope to create the Aardmail email robot to automatically nuke spam out of my POP mailbox before it gets down to Poco. A client-side IP black hole in Aardmail would be useful, but without knowing the life of an IP (that is, how long do spammers keep a static IP) I can't yet tell how useful. More thought and tinkering are needed. Stay tuned. |
|
April 1, 2004: |
As Michael Covington spoke of recently in his Web diary (March 13, 2004), we all get "hack attacks" periodically. I use "hack" here in the old and honorable sense, that is to say, to joyfully mess around with programming because it feels good to do so. It's only very recently (like, the last couple of days) that I've been able to spare any moments for hacking, though I've felt the itch for many months. So yesterday, I gave myself a fair bit of time (several hours) and just played around. Hoo-boy, was I rusty. I forgot a key fact about Delphi that drove me nuts, until Ben Oram reminded me that exception handling works differently when you're inside the IDE. My face was very red. I knew that...somewhere. But I had forgotten. I thought, at first, that it was about being 51, then thought better of it. I have a different theory, and (as many or even most of you are recreational programmers) I thought I'd run it past you to see what you think: When I first learned programming 25-odd years ago, programming was about languages. A language (for me, primarily Pascal, though I've worked in many others) is a structure made for the expression of logic. Languages convey the philosophies of their creators, and once you read up on those philosophies (or reverse-engineer the philosophies from the languages themselves) the syntatic and semantic details of the language stick better, because they're embedded in a larger context. I learned Pascal quickly and waved its banner passionately throughout all the years that it was a power in American computing. Even today, I've forgotten almost none of my Pascal, and Pascal wasn't what was giving me grief yesterday. It was APIs, and IDE details. Programming today is not to any great degree about languages. (You need to know the language you work in, and you're expected to pick it up almost immediately. I took a full year to get good at Pascal. I wouldn't have that option today.) What programming is really about is stitching things together, using language statements as glue. Although your code can be well-written, well-commented, and even elegant in its design, the code that you write is in fact a small part of the program that you create. You have little control over the other parts, and in most cases you don't have source for them. But worst of all, the jungle of API calls is completely chaotic, and has no overall structure that you can use to help remember the whole thing. And the APIs are legion. Object Pascal is a relatively small part of Delphi programming these days. You need to be able to keep a multitude of chaotic details in the forefront of your brain while programming: Component methods and properties and and RTL features of various kinds, and a miles-deep IDE (anybody recall the elegance of Turbo Pascal v1-3?) with hundreds of non-obvious options and gotchas. This may be necessary today. Programming is not what it was 25 years ago, given the richness of I/O, ubiquitous networking, and everything else. But I was appalled how quickly the chaotic details flowed out the back of my head over the three or four recent months when I didn't produce any new code myself. The only way to keep it all in your head is to do it all the time. Being an "occasional programmer" is much tougher today than it used to be. I intuit that there's a book in there somewhere, and if I can find the book in the issue I will write it. BTW, the hack attack was an interesting one indeed, and I'll describe what I did tomorrow. |