X-Deliver-At; Message delivery postponed
Table of Contents
1 Postponed delivery of e-mail messages
Lots of services support postponing delivery of requests. E-mail services don't. They're basically a store-and-forward service until final delivery; it's just store after that.
In this product, we implement a message filter which keeps the
message queued at the MTA until the timestamp indication in the
X-Deliver-At
header field expires.
1.1 Use case
This product only implements half of the work. The person sending
the e-mail must be capable of adding the X-Deliver-At
-header
field. People using telnet mx 25
don't have a problem with that,
people using other e-mail clients may. In order to use ths product,
one may have to adapt the client program used.
2 Business case
Large batches of timed services, e.g. salary payments, may require accompanying messages to the receivers of these payments. The banking world fully supports this mechanism, but none of the e-mail message solutions do.
Of course, one may implement some calendar-event on a private account, which sends e-mails at a specific date/time. However this links the message to domains outside the context of the sender and e.g. the source of the payment. This may be against company policy and are subject to different spam-filters. Some people ignore e-mails from people in a specific context from outside that context. This is probably a good idea.
3 Risks
There are hundreds of other use cases for the X-Deliver-At
header field, including abuse of this service by planting time-bombs
for DDOS purposes.
Also, I need to buffer all header lines before flushing them out
to the sendmail
-command, sending out millions of header lines will
create huge resource consumption by the filter used in the MTA,
leading to a crash or system slow down.
As I decided to read lines (i.e. a string of characters ending in a
'\n'
), long lines may suffocate our program. GO performs well
under these circumstances and will not allow stack-overruns to
occur.
GDPR-conformance is another risk, storing data is never a good idea; this filter will not do that. The MTA's GDPR compliance is as good as it is with or without the postponed delivery filter.
4 Choices
I implement this as a filter in postfix; this MTA is widely used and supports filters which forces the MTA to retry delivery at some other time.
The GO-programming language seems to be a good fit as performance is key; when configured in a simple way, all messages will pass through this filter.
Also, upgrading a server, including upgrades of the MTA, should not invoke dependencies of the filter. GO-programs depend on libc and nothing else.
It should be safe as well, messaging services are notorious for being abused by hackers. The filter should not allow stack overflows and other nasty business to occur. GO performs very well with regards to this.
I choose for the simple filter and not the advanced filter. The simple filter processes messages, the advanced one runs through the SMTP scenario for sending messages.1
5 Filter for X-deliver-at
The expected behaviour of a filter is described in Simple content filter example.
The filter gets the message on standard input and then decides to either exit with the appropriate error code or feed the message into the next delivery step. Or as the documentation explains:
“The job of the content filter is to either bounce mail with a suitable diagnostic, or to feed the mail back into Postfix.”
The job of our filter is to inspect the X-Deliver-At
content and
match it the current time. In case the deliver-at-moment expires,
deliver the message, otherwise exit with the suitable diagnostic. An
absent X-Deliver-At
header field implies expiration, so messages not
using this feature will be delivered as normal.
This error code comes from sysexits.h
and 75 will be used to
indicate a temporary failure:
EX_TEMPFAIL 75 /* temp failure; user is invited to retry */
5.1 Why this may not work
Exiting with EX_TEMPFAIL
doesn't seem right; the concept if
failing is used to opt for a retry. The MTA may decide to
implement some high watermark for retrying and implement backing
off indefinitely after one or two hundred retries.
Postfix documents this well however, and it accepts retrying as a way to ultimately deliver a message without imposing some maximum number of retries or duration for delivery.
Some configuration variables influence the succes of the
X-Deliver-At
service:
Frequency of queue handling.
“The scan interval is controlled by the
queue_run_delay
(300s) parameter.”This default value of 5 minutes has impact on the resolution of expiration.
Backing off.
“The "cool-off" time is at least
$minimal_backoff_time
(300s) and at most$maximal_backoff_time
(4000s).”This is good news for heavily loaded MTA's as the
queue_run_delay
may bring it to it's knees. Beware of the approx. 1 hour maximum backoff time; the resolution of expiration may end up being a bit over one hour.Backing off variation.
“The next retry time is set by doubling the message's age in the queue, and adjusting up or down to lie within the limits. This means that young messages are initially retried more often than old messages.”
This doesn't help or sabotage the postponed delivery service. It's vital for spreading MTA-load, but has no other impact than that the actual delivery time after expiration will vary a lot.
5.2 structure of the program
The complete GO program can be used for testing purposes which then reads from stdin and writes to stdout. This filter then is incorporated into the Postfix configuration for fulfilling the postponed delivery service. When called from Postfix, the testing program will behave according to what Postfix expects.
5.2.1 The main part: processing a message
A for-loop reads an incoming message line-by-line.
for { <<read line>> if <<is end of headerfields>> { <<enable in body>> <<flush headerfields>> } if <<is in body>> { <<write line>> } else { <<add to headerfields>> } if <<is input done>> { break; } if <<is in body>> { continue } if <<is header field X-deliver-at>> { <<parse header field body>> if <<is expired>> { continue; } else { <<fail for retry>> } } }
While reading lines from the input, lines are copied onto the output
as well. We buffer the header fields, i.e. we only output these
until after the body starts, as the sendmail
command will
otherwise push header fields into the SMTP-server for postponed
messages too. This will result in incomplete e-mails being
delivered for every retry.
When we reach the end of the input, the program will break out of the message-reading loop.
As we interpret header fields only, it's vital to short-cut the
message-reading loop in case we're in the body. Otherwise one may
introduce postponed delivery for a message about the header field
X-Deliver-At
in which an example happens to start at the
beginning of a line.
The filter will then check for the X-Deliver-At
header field and
interpret it's contents, possibly returning a TEMFAIL
-status. In
all other cases, the program will exit with an OK-status after the
last input line was read and written.▮
5.2.1.1 Reading a line
line, readerr := reader.ReadString('\n')
Reading a line from the input is the easiest thing to do. I read lines as this is the smallest parseable unit for processing header fields. We may switch to reading bigger chunks once the header fields are processed, but for maintainability now, this is left for possible optimisation later. Note that the returned error is stored in readerr. This errorcode is used later in order to determine end-of-input.▮
5.2.1.1.1 end of message
readerr != nil && readerr == io.EOF
The predicate for end of input is to check whether the error from
the previous read is
io.EOF
. It's interesting to see that GO implements iobuf
in
such a way that EOF
may be raised while there was some data
read. It's an error condition while also providing valid
data. This is why there may be calls between the read and checking
out because of EOF
, this is also why we store the error of the
read in a seperate variable; err
is used by other calls.▮
5.2.1.1.2 Buffering lines with header fields
The sendmail
-command we pipe our message into, will start
sending SMTP-commands to the server upon receiving the first
header fields. In case a message with a X-Deliver-At
header
field does not expire, the filter exits with exit.TEMPfail
,
this terminates the sendmail
-command but leaves a valid
SMTP-session between sendmail
and the server. This results in
an incomplete message being delivered looking like:
From joost@localhost.localdomain Tue Sep 19 14:02:32 2017 Return-Path: <joost@localhost.localdomain> X-Original-To: root Delivered-To: root@localhost.localdomain Received: by localhost (Postfix, from userid 1000) id AB1F0A01055; Tue, 19 Sep 2017 12:02:32 +0000 (UTC)
This is very ugly, hence I buffer all header fields before starting
the sendmail
-command. A simple array of strings is used for
this purpose.
headerfields = append( headerfields, line )
5.2.1.2 Writing a line
if line != "" { writer.WriteString( line ) }
We read line-by-line; we also write line-by-line. Only non-empty
strings by the way. The case with an empty string can occur when
the input size is exactly a multiple of the buffer-size of iobuf
. The
last read may raise EOF and a 0-sized string.▮
5.2.1.2.1 Flushing buffered writes
<<start sendmail>> <<flush out headerfields once>>
Only after starting with the body, we know that we're going to
use sendmail
to send this message; hence we flush out the
header fields. We can only do that into a running sendmail
, so
we start that command first.▮
The code for writing the actual lines into the sendmail
-pipe
is:
for _, elt := range headerfields { writer.WriteString( elt ) } headerfields = nil if <<is logging>> { logger.Info( "flushed header fields" ) }
This code flushes all header fields out once. It makes sure this happens once by nulling the object holding all header fields.▮
5.2.1.3 State: in body
The state in body is tracked by a boolean.
inbody
With this predicate enabling it and disabling it go as:
inbody = true
inbody := false
5.2.1.3.1 Switching from header to body
The predicate which determines whether to switch uses the current line.
line == "\n" && inbody == false
According to the e-mail RFC's the body starts when the headers end. The headers-end indicator is an empty line2. Of course, empty lines may occur in the body too, but then it's not considered the end of the header fields. Hence the additional predicate inbody == false.▮
5.2.1.4 Exploring the X-Deliver-At header field
An RFC2822 header field consists of a label and a body. Their seperator is a colon.
headerfield := strings.SplitN( line, <<name body seperator>>, 2 )
SplitN is used as we expect only two items in a header
field. This avoids letting the “:”-separators in the date/time
string screw up our processing of the X-Deliver-At
-header field.▮
":"
5.2.1.4.1 Is it an X-Deliver-At label?
<<get headerfield>>; strings.ToLower( headerfield[0] ) == "x-deliver-at"
The label is lowered and compared to "x-deliver-at"
. Note
that we do not support “folding” the body of the header
field. Section 2.2.3 in RFC2822 may not allow this anyway, as
folding is only allowed at “higher-level syntactic breaks”. I
cannot see that a date/time-body can be folded anywhere without
violating this rule.▮
For converting the header field timestamp into a GO-native type
we use time.Parse from the time
-package. We expect the
format for the timestamp to be RFC1123 compliant (with TZ
code), hence the use of time.RFC1123Z.
deliverat_body := strings.TrimSpace( headerfield[1] ) deliverat, err := time.Parse( time.RFC1123Z, deliverat_body ) if err != nil { if <<is logging>> { logger.Warning( fmt.Sprintf( "Failed to parse [%s]", deliverat_body ) ) } <<fail fatal>> }
Parsing can fail, this is considered fatal. The program will
exit with an appropriate errorcode. This is a tough choice,
returning TEMPFAIL
is useless, as the parsing will fail the
next time as well. The filter can also decide to expire the
message; this will lead to a delivered message where the
intention was to postpone it. We decided to raise an error big
enough to make the message bounce. This will attract the
attention of both the sender and the administrator.
Note that we trim spaces around the date/time-text, as it
includes the trailing "\n"
. ▮
now := time.Now(); now.After( deliverat )
time.Now()
is the current moment in time;
now.After(...)
will then do exactly what we expect of
expiration3.▮
5.2.1.5 The pipe to sendmail
Preparing for starting the sendmail
-command involves some
housekeeping.
if <<is logging>> { logger.Info( fmt.Sprintf( "preparing sendmail with %s", append( []string{"-G", "-i"}, os.Args[1:]... )) ) } cmd = exec.Command( "/usr/sbin/sendmail", append( []string{"-G", "-i"}, os.Args[1:]... )... ) newstdout, err = cmd.StdinPipe() if err != nil { if <<is logging>> { logger.Err( fmt.Sprintf( "failed sendmail pipe [%s]", err ) ) } os.Exit( exit.UNAVAILABLE ) } if <<is logging>> { logger.Info( fmt.Sprintf( "pipe to sendmail [%s]", newstdout ) ) }
The simple filter interface needs the filter to post the message
itself back into Postfix using the sendmail
command.▮
Starting the command prepared earlier:
if cmd != nil { if err := cmd.Start(); err != nil { if <<is logging>> { logger.Err( fmt.Sprintf( "failed to start sendmail [%s]", err ) ) } os.Exit( exit.UNAVAILABLE ) } }
Starting the command only makes sense in case there is a command to start, hence testing for cmd.▮
5.2.1.6 Finishing
5.2.1.6.1 Postpone! aka Retry aka TEMPFAIL
if <<is logging>> { logger.Notice( fmt.Sprintf( "Postponing delivery based on header [%s]", line ) ) } os.Exit( exit.TEMPFAIL );
The Postfix API expects a TEMPFAIL
for messages which need retries.▮
5.2.1.6.2 Failed exit
Something fatal leads to an exit, in our case we raise a data-error.
if <<is logging>> { logger.Err( fmt.Sprintf( "Failing with DATAERROR for line [%s]", line ) ) } os.Exit( exit.DATAERR );
Most errors, except TEMPFAIL
, will force Postfix to log a line
and bounce the message back to it's sender. This is exactly what
we want.
5.2.1.6.3 OK
os.Exit( exit.OK );
5.2.1.6.4 sendmail
Finishing output means flushing it and closing it. The closing is
important, as the sendmail
-command will not terminate if it's
input is not closed.
if <<is logging>> { logger.Info( "flushing output" ) } writer.Flush() if <<is logging>> { logger.Info( fmt.Sprintf( "closing output %s", newstdout ) ) } if newstdout != nil { newstdout.Close() }
It's silly to
find out that closing cannot be done via the writer. That is
why we need the variable newstdout. Actually we don't know if
newstdout is used, while testing e.g. it is not. We should close
stdout in case of testing. There is no sense in that, as there
is nothing to gain there. If you think there is, plase add
else {os.Stdout.Close()}
.▮
The sendmail
command needs to complete in order te decide whether
it succeeded delivering the message.
if cmd != nil { if err := cmd.Wait(); err != nil { if <<is logging>> { logger.Err( fmt.Sprintf( "waiting for sendmail failed [%s]", err ) ) } os.Exit( exit.UNAVAILABLE ) } }
As always, exceptions are logged as well.▮
There is no need for it now, but we may want to kill the
sendmail
-process under certain circumstances. Here's how to do it.
if cmd != nil {cmd.Process.Kill()}
5.2.1.7 Logging
The filter doesn't always log. Hence a predicate for logging.
logger != nil
While testing e.g. it may not do logging.▮
5.2.1.8 GO babble
import ( "bufio" "fmt" "os" "os/exec" "log/syslog" "io" "time" "strings" "github.com/Code-Hex/exit" )
Bufio is used for fast buffered I/O. Fmt for formatting
log-messages. The github-pulled code is to import a GO-version of
sysexits.h
, defining errorcodes to be returned by
serverprograms.▮
5.2.2 The full program
package main <<imports>> // use -t for testing, other parameters will start sendmail func main() { var cmd *exec.Cmd = nil // declare cmd as it is needed for Start() later var headerfields []string = nil // buffering the header fields var newstdout io.WriteCloser // one can only close via this interface logger, err := syslog.New( syslog.LOG_MAIL, "x-deliver-at" ) if err != nil { os.Exit( exit.UNAVAILABLE ) } else { defer logger.Close() } reader := bufio.NewReader( os.Stdin ) writer := bufio.NewWriter( os.Stdout ) if len( os.Args ) > 1 && os.Args[ 1 ] != "-t" { <<feed into sendmail>> writer = bufio.NewWriter( newstdout ); } else { // we're testing logger = nil } <<disable in body>> <<filter-implementation>> <<finish output>> <<wait for sendmail to end>> <<finish OK>> }
The complete program has some GO-babble for package declaration
and import-stuff. Also we declare main()
, prepare the I/O
variables and prepare the inbody
variable.
Some variable declarations are necessary as I/O classes need symmetrical actions. After an open/creat, the close follows, but later. Later also lexicographically, hence a variable. This is the case for Cmd and newstdout.
Headerfields is used for buffering the header fields. Adding lines to it is done via append. As append supports adding a string to a nil-array, headerfields is initialized with nil to make the code for adding a simple one-liner.
The syslog-lines are prefixed with "x-deliver-at"
. Note that the
length of this prefix matches "postfix/qmgr"
and
"postfix/pipe"
in order to neatly align logging output.
Setting up the reader and writer then is simple,
except… os.Stdout
doesn't have the same interface as the
stream into the pipe of a command (which is WriteCloser
). We do this
by assigning a bufio.NewWriter()
to the writer variable. This
overwrites previous setting of writer.4
5.3 Building and deploying
Standard GO procedure: set GOPATH
and get/build/install.
export GOPATH=`pwd` go get main go build main go install main
5.4 Installing into a Postfix installation
Configuration of a simple filter gives a perfect explanation on how to make a production setup for the use of this filter.5
Please make sure that you test the setup extensively. All messages will pass through the filter, you don't want to loose e-mail!
5.5 Testing
5.5.1 functionality
We begin by testing a lot of messages in my own INBOX. None of
these should have a X-Deliver-At
header field. All diff-output
should be empty.
Then we test message 1 and 2. The second message is due for non-expiration and should exit with 75.
cnt=0 for f in `ls -tr ~/*/INBOX/cur/ | tail -1000` do cnt=$(expr $cnt + 1) echo -n "$cnt: " cat ~/*/INBOX/cur/"$f" | bin/main -t | diff - ~/*/INBOX/cur/"$f" echo done for f in `ls -tr ~/*/INBOX/cur/ | tail -1000` do cnt=$(expr $cnt + 1) echo -n "$cnt: " cat ~/*/INBOX/cur/"$f" | bin/main -t > /dev/null 2>&1 echo "$?" done tomorrow=`date --rfc-2822 -d tomorrow` yesterday=`date --rfc-2822 -d yesterday` cat test/m1 | sed "s/\(X-Deliver-At: \).*/\1$tomorrow/" | bin/main -t > /dev/null; echo $? cat test/m1 | sed "s/\(X-Deliver-At: \).*/\1$yesterday/" | bin/main -t > /dev/null; echo $?
5.5.2 performance
TBD
5.5.3 example message
From harrie.jansen@cistus.nl Return-Path: <harrie.jansen@cistus.nl> X-Original-To: gert@imap2.cistus.nl Delivered-To: gert@imap2.cistus.nl Received: from mx.cistus.nl (mx.cistus.nl [213.154.248.214]) by mail.cistus.nl (Postfix) with ESMTP id 66EFD2BE722 for <gert@imap2.cistus.nl>; Mon, 6 Feb 2018 08:04:14 +0100 (CET) Received: by mx.cistus.nl (Postfix) id 28A5A40466F2; Mon, 6 Feb 2018 08:02:57 +0100 (CET) Delivered-To: gert@cistus.nl Received: from mail.cistus.nl (imap2.cistus.nl [213.154.248.218]) by mx.cistus.nl (Postfix) with ESMTP id 1EB8940466E9; Mon, 6 Feb 2018 08:02:57 +0100 (CET) Received: from pmbp-2.local (k5124.upc-k.chello.nl [62.108.5.124]) by mail.cistus.nl (Postfix) with ESMTPSA id 1EA9F2BE721; Mon, 6 Feb 2018 08:04:14 +0100 (CET) To: Gert Helberg <gert@cistus.nl> From: Harrie Jansen <harrie.jansen@cistus.nl> Subject: Updates naar Gribbel Message-ID: <01b355d7-8153-d45a-5da2-80ef2cd8c806@cistus.nl> Date: Mon, 6 Feb 2018 08:04:21 +0100 X-Deliver-At: Mon, 18 Sep 2017 22:04:21 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Goedemorgen Gert, habbeldiebabbel Mvg, Harrie
6 About this title
The document to generate the code has the same source as the document you are reading now.
The full code, generated of course,is in tempfail-if-not-expired.go. The git repo is at: gitlab.com.
This title was written between the 18th and the 22nd of September 2017
Footnotes:
enable