Policy Database

ze-filter policy database is a central concept in ze-filter. This is the place where most static information defining ze-filter behaviour is stored.

This page contains information specific to policy database : how to add entries to it, how data is queried, …

If you're looking for information on the meaning of data stored here, they're better described elsewhere, where the feature using these data are explained.

Syntax

The syntax of this file is very simple but flexible and strong. Its idea is very similar to that one found at sendmail access database.

Entries syntax is :

Prefix:Key          Value

where,

Prefix - is the kind of information we're looking for.
Key - is the key of the information
Value - is the value associated with the Prefix and Key

An example of already implemented kind of entries follows :

CtrlChan:DEFAULT           REJECT
CtrlChan:127.0.0.1         OK
CtrlChan:194.21.16.16      OK
#
ConnRate:DEFAULT           15
ConnRate:127.0.0.1         1000
ConnRate:LOCAL             500
#
NetClass:199.199.123       DOMAIN
NetClass:192.168           LOCAL
#
BadMX:saveinternet.net     ERROR:421:4.5.1:Too busy now... Try later !

How database is looked up

Basically, query follows the rule : “if the given key exists, its associated value is returned”.

We tried to implement into ze-filter almost the same logic used by sendmail to query access database. But, still, there may be some differences.

If the key is an IP address, a domain, a hostname or an email, the database will be queried multiple times, with keys derived from the initial key. If the search isn't satisfied, a last query is usually done using the default value, in order to get the default value for the searched information.

IP address - derived keys are the network addresses. E.g. if the key is 1.2.3.4, the database will be queried with the keys 10.20.30.40, 10.20.30, 10.20, 10 and default.

domain or hostname - derived keys are upper level domains. E.g. if the key is host.example.com, the database will be queried with the keys host.example.com, example.com, com and default.

e-mail address - derived keys are the domain and user part of the address. E.g., if the key is jeff.smitt@example.com, the database will be queried with the keys jeff.smith@example.com, example.com, jeff.smith@ and default.

There is no CIDR based database queries. Network addresses are looked up based on byte frontiers, even if the concept of network classes is considered deprecated.
When the key isn't found,

Some policy decisions are made upon multiple database queries - based on triplets : SMTP client information (IP address or network address or host name or network class), sender e-mail address and recipient e-mail address. E.g., greylisting, content check, …

Database queries are sequentially done with all three informations, unless a stop condition if found. A stop condition is the suffix -QUICK added to the record value.
Connect, From or To words are added to the key before queries.
ze-policy database is queried in the following sequence :
1. SMTP client information
  1. IP address / network address
  2. network class
  3. host name / domain name
  4. default value for SMTP client information
2. Sender information
  1. sender e-mail address
  2. default value for sender information
3. Recipient information
  1. recipient e-mail address
  2. default value for recipient information
As long as all three informations are sequentially used to query policy database, an intermediate value may not be the final value, unless it correspond to a stop condition.

The easiest way to explain all this is with an example.

GreyCheckConnect:default           YES
GreyCheckConnect:10.3              NO-QUICK
GreyCheckConnect:193.22.33         NO
GreyCheckConnect:yahoo.com         NO-QUICK
GreyCheckConnect:dnsbl             YES-QUICK
GreyCheckFrom:joe@domain.com       NO
GreyCheckFrom:ze-filter.org        NO-QUICK
GreyCheckTo:postmaster@            NO
GreyCheckTo:joe@domain.com         NO
GreyCheckTo:charles@domain.com     YES

When duplicate keys are found at ze-policy.txt file, usually only the first one will be taken into account. ze-policy.db is created from ze-policy.txt, using ze-makemap (see /var/ze-filter/cdb/Makefile). The way ze-makemap handles duplicate records depends on the -m option : -m s to preserve first found keys (skip new ones) and -m u to update with new values.

Current data defined at policy database

Defining networks classes

NetClass - this prefix allows you to define known IP addresses or IP networks. The syntax is :

NetClass:key      value

Key maybe one of :

IP address
IP network prefix
hostname
domain name

Value represents the class to which you'll class the key. There are three static predefined known classes : LOCAL, DOMAIN and FRIEND, but you can define other classes if these three classes aren't enough.

Examples :

# predefined classes
NetClass:10               LOCAL 
NetClass:199.201          DOMAIN 
NetClass:1.2.3.4          FRIEND
NetClass:domain.com       DOMAIN
# locally defined classes
NetClass:10.1             DEPMATH
NetClass:10.2             DEPPHYS
NetClass:10.3             DEPCHEM

Authenticated SMTP connections fall into an special network class : AUTH. AUTH class is considered a dynamic network class to which are assigned the same rigths than LOCAL network class.

Defining Limits per SMTP client

These prefixes can be used to define limits to be applied to SMTP clients :

ConnRate - Limit on the number of connections on a 10 minutes time window
ConnOpen - Limit on the number of simultaneously open connection
RcptRate - Limit on the number of recipients on a 10 minutes time window
MsgRate - Limit on the number of connections on a 10 minutes time window
MaxMsgs - Limit on the number of messages per SMTP connection
MaxRcpt - Limit on the number of recipients per message

When the limits are evaluated on a sliding time window, the size of this window is ten minutes.

ConnRate:DEFAULT              15
ConnRate:LOCAL                300
ConnRate:DOMAIN               200
ConnRate:FRIEND               30
ConnRate:10.3                 400
ConnRate:domain.com           2
ConnRate:DEPMATH              400
 
ConnRate:1.2.3.4              20
ConnOpen:1.2.3.4              4
RcptRate:1.2.3.4              40
MaxRcpt:1.2.3.4               4
MsgRate:1.2.3.4               30
MaxMsgs:1.2.3.4               3

Greylisting

GreyCheckConnect, GreyCheckFrom and GreyCheckTo

- Default policy : ENABLED

GreyCheckConnect:default               YES
GreyCheckConnect:MATH                  NO-QUICK
GreyCheckConnect:1.2.3.4               NO-QUICK
GreyCheckFrom:john@friend.com          NO-QUICK
GreyCheckFrom:spammer.com              YES-QUICK
GreyCheckTo:postmaster@mydomain.com    NO

- Default policy : DISABLED

GreyCheckConnect:default               NO
GreyCheckFrom:spammer.com              YES-QUICK
GreyCheckTo:postmaster@mydomain.com    NO
GreyCheckTo:Alice@mydomain.com         YES
GreyCheckTo:Bob@mydomain.com           YES
GreyCheckTo:Charles@mydomain.com       YES

Greylisting is never applied to predefined known networks

GreyEquivDomain Equivalent domains - messages from one domain come from a mail server of the other domain. E.g. messages which sender is in domain domain.com comes from a SMTP client on domain domain-one.com or domain-two.com.

GreyEquivDomain:domain.com       domain-one.com, domain-two.com

As shown above, you can define more than one value for this particular kind of key. You should just separate them by spaces or commas.

Defining rules for message content checking

Rules to define which messages will have its content checked

ContentCheckConnect, ContentCheckFrom and ContentCheckTo
BayesCheckConnect, BayesCheckFrom and BayesCheckTo
XFilesCheckConnect, XFilesCheckFrom and XFilesCheckTo
VirusCheckConnect, VirusCheckFrom and VirusCheckTo
RejectShortMsgsConnect, RejectShortMsgsFrom and RejectShortMsgsTo

# sympa mailing list server
ContentCheckConnect:X.X.X.115         NO-QUICK
XFilesCheckConnect:X.X.X.115          NO-QUICK
VirusCheckConnect:X.X.X.115           NO-QUICK
BayesCheckFrom:domain.com             NO-QUICK
RejectShortMsgsTo:joe@example.com     NO

Message Archiving

Rules to define which messages shall be archived

ArchiveConnect, ArchiveFrom and ArchiveTo

ArchiveConnect:DEFAULT          NO
ArchiveConnect:10.3             YES-QUICK
ArchiveFrom:joe@spammer.com     YES-QUICK
ArchiveTo:archive@domain.com    YES

Centralized greylisting server access

This prefix is used to control which computers have access to the centralized greylisting server

GreydAccess

GreydAccess:DEFAULT                  REJECT
GreydAccess:127.0.0.1                OK
GreydAccess:10.3.4.5                 OK

ze-filter control channel access

This prefix controls which IP addresses can connect to the filter control channel.

CtrlChan

CtrlChan:DEFAULT             REJECT
CtrlChan:127.0.0.1           OK
CtrlChan:10.3.3              OK

BadMX

Use this entries to define unwanted domains or IP addresses, based on their MX.

BadMX

BadMX:10                ERROR:421:4.5.1:Sender MX in an unallocated or reserved network 
BadMX:saveinternet.net  ERROR:421:4.5.1:Unwanted or unreacheable MX... Try later !

Maintaining policy database

Policy database is usually at /var/ze-filter/cdb. As it's installed, it's the concatenation of some text files :

[martins@localhost cdb]$ pwd
/var/ze-filter/cdb
[joe@localhost cdb]$ ls -l ze-policy.*.txt
-rw-r--r-- 1 root root     0 Dec 11 21:17 ze-policy.base.txt
-rw-r--r-- 1 root root    66 Feb  9 11:47 ze-policy.local.txt
-rw-r--r-- 1 root root 16172 Feb  7 15:08 ze-policy.z-badmx.txt
-rw-r--r-- 1 root root  3745 Feb  7 15:08 ze-policy.z-defaults.txt
-rw-r--r-- 1 root root 11146 Feb  7 15:08 ze-policy.z-grey.txt
-rw-r--r-- 1 root root  5727 Feb  7 15:08 ze-policy.z-iana.txt
[joe@localhost cdb]$

In order to ease management of the policy database source files, you can have as many as you want. They will be concatenated to generate the text file ze-policy.txt, which will be read to create the hash database which will be read by the filter.
ze-policy.z-*.txt files contains default values. These files installed and updated with the filter. As long as their suffix begins with “z-“, they will be the last files to be read. Only entries which weren't yet defined will be taken into account.

You'll probably need to adapt the Makefile installed inside /var/ze-filter/cdb to match the make program in you system.

[martins@localhost cdb]$ more /var/ze-filter/cdb/Makefile
...
# POLICY_SRC variable defines which files to include in your
#      policy database
#
# Depending on your Operating System, you can use one of this options :
# - GNU make
POLICYSRC = $(wildcard ze-policy.*.txt)
# - SunOS make
#POLICYSRC:sh =ls ze-policy.*.txt
# - List text files by hand
#POLICYSRC = ze-policy.grey.txt ze-policy.badmx.txt ze-policy.base.txt

Table of Contents