The current state of e-mail is one of the travesties of the Internet in my opinion, the excerpt below pretty much sums this up"
Email: Killer App Or Just A Killer? While many people consider email to be the "killer app" that brought the internet into homes and businesses, now some are saying that email has become annoying and costly. The most obvious issue is with spam, but there are other things as well. Maintaining an email server is a pain, and keeping email free from viruses is an additional cost. For companies that monitor email (and there are more and more), that's yet another expense. Finally, since there are questions about email security, some companies are telling employees not to use email for sensitive material. Thus, for many companies, email is only useful for informal communications, and you can only find those messages once you wade through all the spam and viruses - or so this article would have you believe. It's really not that bad - and there are reasonable technology solutions that should be able to keep most companies afloat with minimal costs. Yes, it's annoying, but the benefits of having email certainly outweigh the annoyances associated with it. [via Techdirt]
One of the reasons for e-mail enabling Virtuoso (circa. 2000) was to set the stage for addressing what I anticipated would ultimately become the Spam Monster. This is how the solution was envisioned.
Build a Driver/Sink that could be attached to the SMTP Agents such as Sendmail, Excim, Exchange etc.. such that the mail received is actually stored in a DBMS Engine (in this case Virtuoso or an ODBC accessible database). Once the mail is in the database it is then possible for Triggers to handle filtering of the Mime headers and mail body (using regular experessions). The end result being that Spam and Virulent mails are already filtered prior to POP or IMAP retrieval.
With the emergence of Bayesian Spam Filters and other Anti Spam solutions there remains a possibility for this pursuing the best of both worlds. Enhance the DB Engine via its extensions API (In the case of Virtuosowhich supports Python, Perl etc..), or enhance the Mail Driver/Sink by extending it in a similar manner (a little more work if extensibility isn't part of the original Mail Sink design). My preference is obviously to handle this at the database level so that the Bayesian spam filter becomes a Trigger on the table into which the mail is stored.
With a database in the mix I pretty much have a rules based engine for e-mail and also a pretty flexible mechanism for dealing with false positives (nothing's perfect!) since they remain in the database too, but not automatically part of the IMAP or POP retrieval process.
At the end of the day e-mail is data and we simply need to look at data