CNS Spam-Filter FAQ

Spam (also known as UCE; "Unsolicited Commercial E-mail) is the name given to unsolicited e-mail. It’s junk mail that wastes your time and system resources. This is email that you did not ask for and don't want. It’s the "Nigerian account" scam, pornography, or “bargains” on prescription drugs, to mention but a few.

Basically, it all boils down to money. Someone hopes to get some money, and you are their target. In some cases, they are selling a real product. In many cases the offer is either fradulent, illegal, or both. In all cases, it is an unwanted, time-wasting nuisance. But it costs the spammer next-to-nothing to send millions and millions of these messages. No postage, no printing, no "business reply mail" envelopes... Even if only one person in ten-thousand responds, the spammer is 'way ahead. So, while it would be cost-prohibitive to try to pitch these advertisements via "snail-mail" or telephone-solicitation, every eager (and perhaps unscrupulous) person with a product (or scheme) can afford to blanket the world with their "offer."

There are as many ways for spammers to acquire addresses as you can imagine. Perhaps from an e-mail list to which you subscribed, or an open forum where you posted a question or a helpful reply. Perhaps a business from which you made a purchase had its customer data hacked/stolen. Perhaps such a business even *sold* its customer data. Perhaps they found an address of one person on an e-mail service, and just started guessing what others might be. They are limited only by their own ingenuity.

Some computer viruses will even scour your computer files looking for e-mail addresses, and send them to random people--some of whom may be spammers. Even if your computer never had one of these viruses, if someone you once sent an e-mail to got one, the virus could grab your address from the other person's computer.

There have recently been some attempts to regulate spam (see "http://www.cauce.org/ for up-to-date information). However, the Internet is international; and spammers can easily operate outside the jurisdiction of nations which attempt to regulate their activities. Therefore, it is unlikely that anything effective can be done from a legislative/law enforcement standpoint in the foreseeable future.

You probably cannot stop spammers from sending unwanted e-mail to your mailbox. But you can take measures to limit the amount you receive, and, more importantly, to limit the amount that you have to *see*. The CNS Spam Scoring service enables users of certain e-mail clients to configure Spam Filters to automatically handle (e.g. move to a separate folder) e-mail which fits certain "spam profiles."

A Spam Filter is a function/ability which some e-mail programs have, allowing them to scan incoming e-mail as it arrives, and take some action (such as moving it to a separate folder) if it meets certain criteria. The user (you) generally has to turn the function "on" via some configuration setting, tell the filter what to look for, and what to do when e-mail meeting the criteria is found.

CNS scans all* e-mail coming through the "@UFL.EDU" mail servers, using a program called SpamAssassin. SpamAssassin uses various rules to assign each message a "score" which represents the probability that the message is "spam." Special headers are added to the message as a result of this process. One header contains the "spam score," and another lists the tests the message met in order to earn this score.

If you have an e-mail program capable of reading and acting on these headers, you can set it to watch these headers, and take some action (such as deleting the message) if there is a high probability of the message being spam.

For ease-of-use, the spam-score is represented in a header by a series of asterisks ("*"), with more asterisks indicating a higher probability of "spam-ness." A typical spam-score header might look like this:

As a general rule, anything scoring 5 "stars" or higher is very likely to be spam. So you might want to configure your spam-filter to "trigger" on:

Note there are five "stars" in the above example. That would match any message having a score of 5 OR HIGHER.

Initially, you might want to set your filter to 6 or 7 stars, to make sure that mail you *do* want isn't caught in the filter "by accident." Then make the filter more restrictive (fewer stars) as you become more familiar with how it affects your mail.

No. We are not rejecting/discarding it because many legitimate e-mail messages have spam characteristics and would also get rejected or discarded. We are leaving the decision of how to handle the messages in the hands of the end user. But with filters in place, you won’t have to sift through it, or even see most of it, if you don't want to.

Currently, CNS rejects outright (at the server) only the most blatant examples of Spam, in order to minimize the risk of accidentally blocking valid messages. Before you see any further benefit, you will have to set up filters in your e-mail program to read the "spam scores" and take whatever action you decide (e.g. move them to a separate folder to review later). For information on how to do this, please see the CNS Junk E-mail (spam) Filter Information Web page.

We would like to say yes to this question. However, there is no way to be certain that all spam will be caught. In testing the spam filtering system some reports showed the filters stopping close to 85 percent of spam. (http://www.geetel.net/spamfaq.html)

If a message that the user thinks is SPAM is not marked as such, they should forward the message the entire message (including the "spam report" headers) to <report-spam@ufl.edu>.

Since some of the tests are self-learning, letting the system administrators know about incorrectly classified messages is important.

We have successfully configured, tested, and provided documentation for Pine, Eudora, Netscape/Mozilla/Thunderbird, and Microsoft Outlook (but NOT Outlook Express).

It undoubtedly works with many others , but there are too many e-mail clients in use for us to give a comprehensive list. These are some of the more common that we know to have this ability.

CNS has developed a series of "cookbook" documents to help you configure some of the more common clients. See links in previous question.

Outlook Express does not include the ability to selectively filter messages based on the special headers added by our spam-scoring software ("SpamAssassin"). It can only filter messages based on "standard" fields such as the "To:", "From:", "Subject:", "Priority:", message body, etc.

In order to make SpamAssassin's spam-scores work with Outlook Express, we would need to modify one of these fields. We have chosen NOT to modify any existing fields, because we don't want to do anything to alter the content of the message as originally transmitted by the sender.

12 May 2004: We are planning (within the next 6 months) to offer an application that lets users upload the equivalent of a procmail script to the GatorLink IMAP server. Once implemented, this service will let users specify custom filtering rules to be executed on the server (before the mail gets to the user mailbox). This should compensate for Outlook Express' lack of flexibility in filtering.

This is why, at least at first, we recommend that you have your filter move offending messages to a folder which you can review later.

However, we also encourage you to report the error, to help us better mark similar messages in the future. If a message marked by the system as SPAM in error, the user should forward the entire message (including the report) to <report-ham@ufl.edu>. If a message that the user thinks is SPAM is not marked as such, they should forward the message to <report-spam@ufl.edu>.

Since some of the tests are self-learning, letting the system administrators know about incorrectly classified messages is important.

Our best advice is, if you have any doubt, send a copy of the message to yourself first. Then examine the headers of the message you receive. Look for the header:

If X-Spam-Level shows 4 or more asterisks, there is a good chance your message may be flagged as spam by some recipients.

If your message has a high spam-score, look at the header:

The (somewhat cryptic) notations in this header will provide you with some clues as to why your message received the score. SpamAssassin.org maintains a Web page listing all the tests, along with explanatory information, which you can use to decode the "X-Spam-Status:" header.