Using ze-filter headers (filtering on MUA)

The content of this page can help you to explain your users how to exploit information presented on ze-filter headers.

Introduction

When ze-filter checks the content of a message it can add two kind of headers : X-ze-filter-Score and X-ze-filter-Status.

These headers can be used, by the final recipient, to create a filter on the MUA (Thunderbird, …) or a Sieve filter on a Cyrus IMAP server, to redirect the messages to some folders upon some criteria.

These headers look like below :

X-ze-filter-Score: MSGID : 45E477DA.000 on paris-2 : ze-filter score : XXXXX : R=. U=. O=# B=0.999 -> G=5
X-ze-filter-Status: HI

X-ze-filter-Score header gives detailed scores for each filtering class : Pattern Matching, URL filtering, Heuristic filter and Bayesian filter, while X-ze-filter-Status has only two possible values (HI and LO) resuming a global score assigned to the message.

As long as error probability is never zero, the probably best decision, if the score seems to indicate a spam message, is to redirect it to a quarantine folder instead of discarding, rejecting it1) . Then you can have a look on it for debugging purpose.

Simplified Filtering

Simplified filtering uses the X-ze-filter-Status header, which can have only two values HI and LO.

Usually, filter based on this header will realize following algorithm :

  • It this header contains HI, then move message to PROBABLY-SPAM folder
  • If this header contains LO, then move message to SUSPECT folder
  • Else, move message to Inbox folder

Most of the time, this filter will satisfy most users.

ze-filter sets the content of this header based on the value of two configuration options : XSTATUS_HEADER_HI_REGEX and XSTATUS_HEADER_LO_REGEX. Here is what we have at our servers presently.

XSTATUS_HEADER_HI_REGEX    (U=####|B=0.9|B=0.8|XXXX.*B=0.7)

This means : Header X-ze-filter-Status is set to HI if just added X-ze-filter-Score header matches one of below conditions :

  • URL filtering score is greater than 3
  • Bayesian filter score is greater or equal to 0.8
  • Bayesian filter score is greater or equal to 0.7 AND global score is greater than 3
XSTATUS_HEADER_LO_REGEX    (XXXX|X.*B=0.7|XX.*B=0.6|XXX.*B=0.5)

This means : Header X-ze-filter-Status is set to LO if just added X-ze-filter-Score header matches one of below conditions :

  • Global score is greater than 3
  • Bayesian filter score is greater or equal to 0.5 AND global score is greater than 2
  • Bayesian filter score is greater or equal to 0.6 AND global score is greater than 1
  • Bayesian filter score is greater or equal to 0.7 AND global score is greater than 0
Advantages of the simplified filter are :
  • easier to define by most users
  • thresholds updated on the mail server filter are automatically propagated to user filters

Advanced filtering

In order to use all scores, user should acquire some knowledge about ze-filter filtering and the meaning of each entry in the X-ze-filter-Score header :

X-ze-filter-Score: MSGID : 45E477DA.000 on paris-2 : ze-filter score : XXXXX : R=. U=. O=# B=0.999 -> G=5
                                                                         |      |   |   |   |          |
                                                        Global Score   <-|      |   |   |   |          |
                                              Pattern Matching Score   <--------|   |   |   |          |
                                                 URL Filtering Score   <------------|   |   |          |
                                           Heuristic Filtering Score   <----------------|   |          |
                                            Bayesian Filtering Score   <--------------------|          |
                                                        Global Score   <-------------------------------|

Most of the time, all these individual checks can be combined in a “near binary” classification function to decide if the message is a spam or not. Advanced users can base the initial configuration of their filter on regular expressions (see Simple Filter above) used by ze-filter to set X-ze-filter-Status header value and tweak these expressions to best fit his needs.

1)
As we are operating in before queue mode, we can reject, instead of discarding
doc/spam/using_headers.txt · Last modified: 2017/12/01 12:00 (external edit)
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0