Outbound Index FAQ
Outbound Index FEO (Frequently Expressed Objections)
Our format for the FEO section is as follows:
What happens when I join?
You will be asked what your email domain is. If you have an existing Web SSL certificate from a trusted Certification Authority, you will simply submit the https address and you can become a member. If you do not have an SSL certificate, you will initiate a payment transaction of $1.50, to establish a connection between you and a verified identity (credit card or bank account).
Once you have identified yourself in either of these two ways, you will be able to move on to the process of verifying control of your domain, and listing your domains and their associated authorized servers (and those not authorized) in the Outbound Index. As few as three simple pieces of information will be entered by you into the Outbound Index to include the domains under your control.
After authenticating ownership and/or control of your domain, you will gain immediate access to—and be able to test—the live reporting tools associated with the Outbound Index.
Back to Top
How does the Outbound Index work?
The Outbound Index has three basic functions for email and network administrators:
- Announce facts about your outbound servers to the rest of the world
- Monitor unauthorized email-based abuse of your domain names and networks
- Reduce server loads and increase accuracy by splitting inbound email into three streams: Acceptable, Rejectable, Unknown/Suspicious
Each of these functions can be implemented independently of the others, for smooth integration into your existing network or to make it convenient to test certain functions individually.
Announce:
Using a web interface, you can manage the Outbound Index listings for the networks and domains you control. Click here for screen shot. You can choose to define netblocks under your control that are forbidden to operate mail servers, using FQDN patterns or slash notation IP addresses. If you wish, you may also announce authorized outbound servers for domains under your control.
You may also qualify your organization for a global whitelist if your organization has verifiable identifiability, longevity, and stability. The Outbound Index will verify and announce information in your record about the identifiability, longevity, and stability of your organization, including the existence of an SSL certificate on your website, if issued by a trusted Certification Authority. (Other verification methods are being added: TBA)
The facts about your authorized or not-authorized outbound servers are announced worldwide by the Outbound Index, to members who query the Outbound Index, either by DNSBL / DNSWL or using an SIQ protocol query client.
Monitor:
Through the web interface of the Outbound Index Dashboard, you can monitor email from your networks which is being rejected by inbound servers around the world as they query the Outbound Index. If you have specified authorized servers for your domains, you can also monitor a list of incidents where your domain was forged, found by inbound servers around the world that query the Outbound Index.
Your data on these reports show up instantaneously, as the spammers or viruses are attempting connections to inbound servers around the world that query the Outbound Index.
Click here for screen shots of the Dashboard, Outbound from My Networks, and Forgeries of My Domains monitoring report examples.
On each live or historical report, you can drill down for detail to the specific source of each message. You can see the IP address, PTR record if any, and click to look up information leading to the party responsible for that domain and/or IP address. When unauthorized outbound email servers on your own networks are being used to send spam or viruses, you can quickly pinpoint the unauthorized customer or machine.
Reduce server load and increase accuracy:
By configuring your inbound email servers to query the Outbound Index before accepting email, you can:
- Protect clearly authorized mail from false positives and reduce server load by skipping content filtering on that stream
- Avoid wasting CPU cycles, bandwidth and storage space on clearly unauthorized mail by rejecting it prior to accepting the message body.
Only mail from unknown sources needs to be subjected to additional scrutiny using scoring/flagging content filtering, grey listing, and/or challenge-response.
Monitoring reports are available that summarize your inbound server queries to the Outbound Index, allowing you to trace the responsible party for that netblock or domain. Click here for screen shots (scroll down to My Inbound Queries and Drill Down Inbound Detail).
Back to Top
Where does the Outbound Index obtain facts about outbound servers?
Facts stored in the Outbound Index come from the owners of the sender-domains and third-party verifiable sources such as Certificate Authorities, payment processors, ARIN / RIPE / APNIC/ *NIC, whois, DNS lookups, public records, publications, and mail delivery connections.
Unlike traditional DNSBL blacklists, domain owners and ISPs can directly add and edit listings pertaining to the domains they control, including—optionally—the servers they authorize to send mail using their domain name, and the FQDN (Fully Qualified Domain Name) naming convention for outbound email servers "forbidden for use as outbound mail servers" IP addresses. Some of the optional checks available in the Outbound Index use data collected from publicly available sources.
The Outbound Index is designed to look up, collect, cache, and correlate data related to outbound email servers, domains and their owners, which has been verified by other highly visible, known, and trusted sources.
The Outbound Index can store facts about server X or domain X such as:
- The matching forward and reverse DNS of server X identifies it as a home cable modem account for a broadband ISP whose AUP (Acceptable Use Policy) forbids the operation of any type of server on home accounts.
- Domain X whois records indicate a Florida address.
- Domain X has been seen actively in the email system for X days / has been registered for X days / has no website / website has no phone number or does not match whois info.
- Domain X owner is / is not a corporation making its longevity and reliable "contactability" known by one of the following methods: inclusion on a well-known published list of legitimate long-term corporations, SSL certificate from a trusted Certification Authority, and so on.
- Domain X shares a name server host or registrar or whois contact information with other domains fitting the same profile of low-level identifiability and/or a pattern of frequent moving and changing.
The Outbound Index is not a "collective judgement" system. The community guides the creation of checks that are useful in differentiating identifiable and responsible senders from those who hide and move. These checks must work when equally applied to all—rather than subjectively applied to email "I don't happen to want." The Outbound Index policy is not to store subjective judgements or opinions. For example, if I think server X "is a spammer"—because I received email from it that I wasn't interested in, that would be a subjective judgement. The Outbound Index is not the place for that kind of information.
Back to Top
How does the Outbound Index complement other anti-spam methods?
Content Filters. The Outbound Index can reduce the load on content filtering/scoring/flagging systems such as SpamAssassin by cutting out both known-acceptable and known-rejectable email messages from the group that needs to be scrutinized by the content filter. The Outbound Index offers a dependable additional source of data to exempt messages from content filtering—thus reducing the false positive rate of the content filter and potentially allowing the content filter to run with a less-forgiving configuration on the remaining suspicious/unknown email.
Content filters complement the Outbound Index by taking care of the unknown/suspicious category that the Outbound Index has insufficient data on.
Blacklists. The Outbound Index can incorporate black or whitelist results into the composite query response returned to the query client. For example, if the Habeus Infringers List agreed to let the Outbound Index offer an HIL_allowed check, an email administrator could optionally choose to have that check included in the processing the Outbound Index does internally on their queries. The email administrator could also choose how he wants the pass or fail of the HIL_allowed check to influence the query response result.
Local blacklists and whitelists complement the Outbound Index by allowing the email administrator to add further control of the treatment of mail in his system, beyond the scope of what the Outbound Index can do from a legal/liability standpoint. For example, an email administrator in a corporation may be instructed by his employer to block mail from certain domains because they have made a judgement about those domains as a matter of policy. So long as a domain meets the standards of identifiability, stability, and longevity (not moving around and hiding), the Outbound Index may be reporting that domain's email as acceptable, in the whitelisted stream. A well-placed RHSBL blacklist (RHSBL = Right Hand Side of the @ symbol—the domain part of the address) can knock out any "opt-out" operations which are willing to be identified and keep using the same domain.
Challenge/Response. The Outbound Index complements challenge/response systems by reducing the number of queue-filling "never gonna be deliverable" messages generated by attempting challenge/response on fraudulent return addresses, and by eliminating the need to challenge the majority of well-known email sources.
Challenge/response systems complement the Outbound Index by taking care of the grey, suspicious/unknown remainder email which may contain some wanted mail.
Seal / Key Object Methods. In general, these methods stamp each outgoing email with a unique token at the outbound server, and then use various methods later at the inbound server and/or recipient client software to determine if the message is valid. The Outbound Index may complement these methods by helping to prevent replay attacks.
Designated Sender. The Outbound Index can store information about outbound servers that are authorized by the domain owner to send mail—using their domain in the return address, for use by the optional domain_allowed check. The Outbound Index complements other designated-sender systems by sharing many of the same goals, supporting the concept that domain owners have the right to specify authorized servers, and working to find solutions to common issues such as forwarding and return path. We are also working on ways to accurately include data from domains with SPF records in an SPF_allowed check.
Back to Top
What personal information does the Outbound Index store?
Could the Outbound Index be abused to invade my privacy?
The queries the Outbound Index receives from inbound servers only contain the domain name part of the return address and the IP address of the outbound server. Query information sent to the Outbound Index never contains any part of the email recipient's address, nor does it ever contain the content of the message.
The Outbound Index does not contain information about individuals who send email to each other. If law enforcement or other parties wish to trace information about individuals, they would not bother using the Outbound Index, because nothing about individuals is contained in the Outbound Index. Far more detailed info can easily be obtained by tapping into the recipient and sender ISP mail system directly.
The Outbound Index never knows the user name of the person who typed an email message. The Outbound Index DOES receive and store the IP address of the outbound server which is attempting to deliver mail, as do all inbound servers. That information is stamped into the headers of every email. Usually, this IP address belongs to the sender's ISP outbound mail server—NOT the sender's home computer or workstation. If a home user or workstation pretends to be an outbound mail server, the Outbound Index would have the individual's IP address—a series of numbers like 201.199.18.47—which cannot be converted to an email address.
Even if the data in the Outbound Index were stolen, the data contain no email addresses and could not be used to find out email addresses. In other words, it is not possible for anyone to abuse the data in the Outbound Index for the purpose of sending you more spam, or even email of any kind.
Virus-infected or security-comprimised computers often do pretend to be outbound servers. In those cases it is the lack of security on your home computer or workstation that has caused any breach of privacy.
Back to Top
Objection: Ineffective without critical mass of participation
Response: The effectiveness of the Outbound Index given minimal participation
The Outbound Index works effectively with as few customers as one, as we've experienced for ourselves since July 2003 using it in an email system of 800 users. It has allowed us to:
- Reject spam and viruses claiming to be "from" one of our own domains. We couldn't do this in a workable way without the Outbound Index, because we'd wind up rejecting a variety of non-malicious forgeries. The Outbound Index gave us the flexibility to allow consensual forgeries from domains such as Ebay, PayPal, and AmericanGreetings.com, and to include, for example, Blackberry outbound servers—without having to update these listings as those domains added or removed mail servers.
- Reject spam and viruses claiming to be "from" the popular-to-forge domains such as Yahoo, AOL, MSN, Juno, Excite, Hotmail et al.
- Safely whitelist all the domains listed in the Outbound Index and avoid marking mail from them as spam.
- Monitor all domains we host for forgeries, virus infected customers, or misconfigurations.
- Reject spam and viruses from every home cable modem user belonging to a particular ISP—with a single FQDN (Fully Qualified Domain Name) pattern unique to their home cable modem PTR record naming convention.
- Monitor the mix of authorized, unauthorized, and unknown mail flowing into our gateway server, with the ability to drill down and look up details about the source, with a click. Monitor the mix by net or volume. Look up records by pattern matching such as IP addresses resolving to .edu addresses, etc.
Back to Top
Objection: It won't scale
Response: Scalability factors
The Outbound Index system was conceived from the beginning as global in scale. Every platform, software, protocol, programming, management interface, and authentication decision was cast in that light.
- The number of query servers can be increased infinitely
- Query servers can be pulled out and replaced with upgraded ones without bringing the system down
- Caching of requests at the client end, and caching of answers in the query server, reduce the number of queries necessary per day to serve the world's email volume down to a manageable number
- High-volume ISPs can have updates pushed to query servers at their own locations
- Multiple locations spread around the globe have always been anticipated and are being rolled out in concert with initial growth in demand.
We are sure we have a lot to learn. We are also sure that this is not an impossible task.
Back to Top
Objection: Spammers will circumvent it
Response: Closing the exits
Spammers could join the Outbound Index, in violation of the Terms and Conditions contract. But in doing so, they would be identifying themselves and making their operations available to law enforcement and civil suits, if the goods they spamvertise are illegal, their methods fraudulent, or their emailing practices violate any laws. They'll also be making it easier for ISPs and recipients to block mail from them, using RHSBL (Right-Hand-Side Black Lists, such as @spammerdomain.com) whether local or shared. If spammers choose to use throw-away domains, and go to the trouble to list them in the Outbound Index, their usage patterns will still differentiate them from the majority of non-spamming members.
Already built into the Outbound Index design (although not yet in use) is a "scoring" mechanism, where email administrators can set thresholds for factual responsibility and identifiability factors. Combined with an effective anti-forgery system and filtering, we can imagine spam being reduced to a minor irritation instead of the $10 billion productivity drain it is today.
Another thing we think the Outbound Index will do is make it more difficult for ISPs to "look the other way" when someone is using that ISP's resources to send spam. ISP responsibility factors may include:
- My AUP (Acceptable Use Policy) forbids my customers to spam
- I do X when customers are discovered spamming
- I take X steps to prevent customers from spamming through my networks (rate limiting, monitoring)
- I take X steps to protect my networks and servers from use by non-customers (no open proxies or relays, appropriate security measures etc)
- I do/do not buy/harvest/sell lists of email addresses to others
With a record of security competence / policy enforcement, the above factors lose the convenient veil of plausible deniability exercised by many complicit spammer accomplice ISPs in the past. If you want to run your servers and networks in an incompetent or insecure way, that's your choice. Mail from you is likely to be subjected to harsher scrutiny than mail from those who run their networks in a competent and secure way, consistent with their public AUP, and who make every effort to insure compliance by technical means.
With the instant availability of factual accept/reject/scrutinize data being generated as queries are sent to the Outbound Index, it will also become more difficult for ISPs to claim that they don't know what is happening on their own networks.
While most anti-spam methods today can be bypassed by spammers re-tuning their sends, the Outbound Index will work like an ever-advancing crusher on email sources that are unwilling to identify themselves. The amount of known/authorized email will grow—and the "known/unauthorized" email will be revealed and rejected safely—as as organizations add their domains, outbound servers, and non-outbound-server IP ranges to the Outbound Index. What will be left will be the grey pile of unknown-and-refusing-to-be-identified emails.
This "unknown" pile will grow smaller and smaller as greater scrutiny is focused on it. Content filtering, challenge-response, domain white and blacklisting, and legal action, if appropriate, can be focused on that shrinking unknown remainder.
Our goal is that the Outbound Index will put the control of domains and servers back where it belongs: in the hands of those who are authorized to use them, and who are willing to be identified as the authorized users of those domains and servers.
A large percentage of spam uses forged addresses. So by exposing forgeries as they happen, we will start to choke spam out of the network. It's going to take a community-wide effort, something that anti-spam experts have been saying for years. So far, it has been difficult for the community to act efficiently and effectively. We do believe that the Outbound Index, which can serve as a world-wide window in to unauthorized email activity, is one of the methods through which spam can be eliminated over time.
Back to Top
Objection: Free alternatives will be used instead of the Outbound Index
Response: Freedom of choice
We believe that free options should never be surpressed or discouraged; likewise, it makes sense to us that for-profit service alternatives also should not be surpressed or discouraged. Customers and users have the freedom to choose the services and features that work best for them. Some will want to do it themselves; others will want it done for them.
We often choose to pay the creators of open source software, to install, customize or configure their software for us. We are willing to pay for services that are more convenient for us, require less adaptation on our part, or have special functionality that isn't available in free alternatives. We have the freedom to spend many hours of our own employees' time (which isn't free) to implement and maintain free solutions or figure them out. And, we are also free to choose what we consider to be cheaper or more attractive solutions.
When we find a great vendor whose work we love, we don't even want free service from him or her - we want to see that vendor grow and expand and enhance their services. We feel reassured when vendors have a business model that clearly can sustain their business services long-term, with plenty of support for maintenance and improvement that will in turn serve our business well.
No company will survive if it does not provide value to the customer beyond the cost of its services, and beyond what those customers can get by other means for free. Those of us maintaining the Outbound Index fully expect to be required to live by this rule.
The Outbound Index team also strives to allow those who aren't saving or gaining at all financially by the use of the Outbound Index to access at least basic functionality without spending cash. We give free usage credits upon joining. If you have an SSL certificate from a trusted Certification Authority on your website, no financial transaction is necessary to join as a member of the Outbound Index.
Back to Top
Objection: Centralized systems are inferior:
Response: Multiple locations, DDOS attacks, failure scenarios
The Outbound Index is in fact a distributed system, in that it uses DNS records to look up information about sources of email, just like anti-spam systems built into inbound mail servers and specialized designated sender query clients. For those who choose to query the Outbound Index, some advantages are gained by the quasi-centralized nature of it: caching, more complex correlation of factors than individual inbound servers could pull together independently, and acquiring new functionality/adjusting configuration with a web interface, without touching the stability of the inbound server software.
Multiple locations, which we are rolling out over the next 30 days, will provide some protection from DDOS. We will also be putting anti-DDOS hardware in place. Our policy of only answering queries from our members gives us additional ways to mitigate DDOS attacks. For example, one option is to dedicate a particular DNS service only for the purpose of reporting the locations of our query servers - and as we shift those IPs under attack - we can give different answers to segmented portions of our membership until we find the members who are leaking data to the attackers. Meanwhile members in other segments would continue to receive service.
We will also be looking at using AnyCast, looking at methods presently used to protect the DNS root servers, and seeking the counsel of more DNS / anti-DDOS gurus.
Membership terms and conditions require that any query client that interacts with the Outbound Index will leave the inbound mail server functioning normally if the Outbound Index query server does not respond for any reason.
Let's look at what occurs during even a complete failure of all Outbound Index locations due to DDOS.
- Answers cached in the query clients continue to work if the outage is less than cache timeout.
- Private line or local connections betweeen the Outbound Index and high volume customers would be unaffected by DDOS.
- Inbound email servers that query the Outbound Index continue receiving mail as usual after local cache timeout.
- When the DDOS attack is mitigated or other failure fixed, the Outbound Index resumes answering queries.
The Outbound Index system is similar in some ways to the DNS root servers. Of the thirteen root servers, attackers have so far only managed to bring two down at a time. Protection and mitigation options are improving; operating a useful DNS or Outbound Index system is not an impossible task.
Back to Top
Objection: Malicious users will be able to break into Outbound Index accounts
Response: Email-less, password-less authentication of domain control
Authenticating control of a domain to the Outbound Index is equivalent to being able to alter files and folders in the web root of that domain and/or alter the DNS for that domain. If an unauthorized person is able to do either of those things, your own security has already been fatally comprimised.
Read about How Authentication Works
Screen shots on Demo page - scroll down to "Prove Control of Domain" and "Status: Authenticated"
We will also be offering RSA SecurID as an option in the near future.
Back to Top
