AboutGDPROthersSitemap

X-Deliver-At; Message delivery postponed

1 Postponed delivery of e-mail messages

Lots of services support postponing delivery of requests. E-mail services don't. They're basically a store-and-forward service until final delivery; it's just store after that.

In this product, we implement a message filter which keeps the message queued at the MTA until the timestamp indication in the X-Deliver-At header field expires.

1.1 Use case

This product only implements half of the work. The person sending the e-mail must be capable of adding the X-Deliver-At-header field. People using telnet mx 25 don't have a problem with that, people using other e-mail clients may. In order to use ths product, one may have to adapt the client program used.

2 Business case

Large batches of timed services, e.g. salary payments, may require accompanying messages to the receivers of these payments. The banking world fully supports this mechanism, but none of the e-mail message solutions do.

Of course, one may implement some calendar-event on a private account, which sends e-mails at a specific date/time. However this links the message to domains outside the context of the sender and e.g. the source of the payment. This may be against company policy and are subject to different spam-filters. Some people ignore e-mails from people in a specific context from outside that context. This is probably a good idea.

3 Risks

There are hundreds of other use cases for the X-Deliver-At header field, including abuse of this service by planting time-bombs for DDOS purposes.

Also, I need to buffer all header lines before flushing them out to the sendmail-command, sending out millions of header lines will create huge resource consumption by the filter used in the MTA, leading to a crash or system slow down.

As I decided to read lines (i.e. a string of characters ending in a '\n'), long lines may suffocate our program. GO performs well under these circumstances and will not allow stack-overruns to occur.

GDPR-conformance is another risk, storing data is never a good idea; this filter will not do that. The MTA's GDPR compliance is as good as it is with or without the postponed delivery filter.

4 Choices

I implement this as a filter in postfix; this MTA is widely used and supports filters which forces the MTA to retry delivery at some other time.

The GO-programming language seems to be a good fit as performance is key; when configured in a simple way, all messages will pass through this filter.

Also, upgrading a server, including upgrades of the MTA, should not invoke dependencies of the filter. GO-programs depend on libc and nothing else.

It should be safe as well, messaging services are notorious for being abused by hackers. The filter should not allow stack overflows and other nasty business to occur. GO performs very well with regards to this.

I choose for the simple filter and not the advanced filter. The simple filter processes messages, the advanced one runs through the SMTP scenario for sending messages.1

5 Filter for X-deliver-at

The expected behaviour of a filter is described in Simple content filter example.

The filter gets the message on standard input and then decides to either exit with the appropriate error code or feed the message into the next delivery step. Or as the documentation explains:

“The job of the content filter is to either bounce mail with a suitable diagnostic, or to feed the mail back into Postfix.”

The job of our filter is to inspect the X-Deliver-At content and match it the current time. In case the deliver-at-moment expires, deliver the message, otherwise exit with the suitable diagnostic. An absent X-Deliver-At header field implies expiration, so messages not using this feature will be delivered as normal.

This error code comes from sysexits.h and 75 will be used to indicate a temporary failure:

EX_TEMPFAIL 75 /* temp failure; user is invited to retry */

5.1 Why this may not work

Exiting with EX_TEMPFAIL doesn't seem right; the concept if failing is used to opt for a retry. The MTA may decide to implement some high watermark for retrying and implement backing off indefinitely after one or two hundred retries.

Postfix documents this well however, and it accepts retrying as a way to ultimately deliver a message without imposing some maximum number of retries or duration for delivery.

Some configuration variables influence the succes of the X-Deliver-At service:

  • Frequency of queue handling.

    “The scan interval is controlled by the queue_run_delay (300s) parameter.”

    This default value of 5 minutes has impact on the resolution of expiration.

  • Backing off.

    “The "cool-off" time is at least $minimal_backoff_time (300s) and at most $maximal_backoff_time (4000s).”

    This is good news for heavily loaded MTA's as the queue_run_delay may bring it to it's knees. Beware of the approx. 1 hour maximum backoff time; the resolution of expiration may end up being a bit over one hour.

  • Backing off variation.

    “The next retry time is set by doubling the message's age in the queue, and adjusting up or down to lie within the limits. This means that young messages are initially retried more often than old messages.”

    This doesn't help or sabotage the postponed delivery service. It's vital for spreading MTA-load, but has no other impact than that the actual delivery time after expiration will vary a lot.

5.2 structure of the program

The complete GO program can be used for testing purposes which then reads from stdin and writes to stdout. This filter then is incorporated into the Postfix configuration for fulfilling the postponed delivery service. When called from Postfix, the testing program will behave according to what Postfix expects.

5.2.1 The main part: processing a message

A for-loop reads an incoming message line-by-line.

for {
  <<read line>>
  if <<is end of headerfields>> {
      <<enable in body>>
      <<flush headerfields>>
  }
  if <<is in body>> {
    <<write line>>
  } else {
    <<add to headerfields>>
  }
  if <<is input done>> {
     break;
  }
  if <<is in body>> {
     continue
  }
  if <<is header field X-deliver-at>> {
     <<parse header field body>>
     if <<is expired>> {
       continue;
     } else {
       <<fail for retry>>
     }
  }
}

While reading lines from the input, lines are copied onto the output as well. We buffer the header fields, i.e. we only output these until after the body starts, as the sendmail command will otherwise push header fields into the SMTP-server for postponed messages too. This will result in incomplete e-mails being delivered for every retry.

When we reach the end of the input, the program will break out of the message-reading loop.

As we interpret header fields only, it's vital to short-cut the message-reading loop in case we're in the body. Otherwise one may introduce postponed delivery for a message about the header field X-Deliver-At in which an example happens to start at the beginning of a line.

The filter will then check for the X-Deliver-At header field and interpret it's contents, possibly returning a TEMFAIL-status. In all other cases, the program will exit with an OK-status after the last input line was read and written.▮

5.2.1.1 Reading a line
line, readerr := reader.ReadString('\n')

Reading a line from the input is the easiest thing to do. I read lines as this is the smallest parseable unit for processing header fields. We may switch to reading bigger chunks once the header fields are processed, but for maintainability now, this is left for possible optimisation later. Note that the returned error is stored in readerr. This errorcode is used later in order to determine end-of-input.▮

5.2.1.1.1 end of message
readerr != nil && readerr == io.EOF

The predicate for end of input is to check whether the error from the previous read is io.EOF. It's interesting to see that GO implements iobuf in such a way that EOF may be raised while there was some data read. It's an error condition while also providing valid data. This is why there may be calls between the read and checking out because of EOF, this is also why we store the error of the read in a seperate variable; err is used by other calls.▮

5.2.1.1.2 Buffering lines with header fields

The sendmail-command we pipe our message into, will start sending SMTP-commands to the server upon receiving the first header fields. In case a message with a X-Deliver-At header field does not expire, the filter exits with exit.TEMPfail, this terminates the sendmail-command but leaves a valid SMTP-session between sendmail and the server. This results in an incomplete message being delivered looking like:

From joost@localhost.localdomain  Tue Sep 19 14:02:32 2017
Return-Path: <joost@localhost.localdomain>
X-Original-To: root
Delivered-To: root@localhost.localdomain
Received: by localhost (Postfix, from userid 1000)
    id AB1F0A01055; Tue, 19 Sep 2017 12:02:32 +0000 (UTC)

This is very ugly, hence I buffer all header fields before starting the sendmail-command. A simple array of strings is used for this purpose.

headerfields = append( headerfields, line )
5.2.1.2 Writing a line
if line != "" {
   writer.WriteString( line )
}

We read line-by-line; we also write line-by-line. Only non-empty strings by the way. The case with an empty string can occur when the input size is exactly a multiple of the buffer-size of iobuf. The last read may raise EOF and a 0-sized string.▮

5.2.1.2.1 Flushing buffered writes
<<start sendmail>>
<<flush out headerfields once>>

Only after starting with the body, we know that we're going to use sendmail to send this message; hence we flush out the header fields. We can only do that into a running sendmail, so we start that command first.▮

The code for writing the actual lines into the sendmail-pipe is:

for _, elt := range headerfields {
  writer.WriteString( elt )
}
headerfields = nil
if <<is logging>> {
  logger.Info( "flushed header fields" ) 
}

This code flushes all header fields out once. It makes sure this happens once by nulling the object holding all header fields.▮

5.2.1.3 State: in body

The state in body is tracked by a boolean.

inbody

With this predicate enabling it and disabling it go as:

inbody = true
inbody := false
5.2.1.3.1 Switching from header to body

The predicate which determines whether to switch uses the current line.

line == "\n" && inbody == false 

According to the e-mail RFC's the body starts when the headers end. The headers-end indicator is an empty line2. Of course, empty lines may occur in the body too, but then it's not considered the end of the header fields. Hence the additional predicate inbody == false.▮

5.2.1.4 Exploring the X-Deliver-At header field

An RFC2822 header field consists of a label and a body. Their seperator is a colon.

headerfield := strings.SplitN( line, <<name body seperator>>, 2 )

SplitN is used as we expect only two items in a header field. This avoids letting the “:”-separators in the date/time string screw up our processing of the X-Deliver-At-header field.▮

":"
5.2.1.4.1 Is it an X-Deliver-At label?
<<get headerfield>>; strings.ToLower( headerfield[0] ) == "x-deliver-at"

The label is lowered and compared to "x-deliver-at". Note that we do not support “folding” the body of the header field. Section 2.2.3 in RFC2822 may not allow this anyway, as folding is only allowed at “higher-level syntactic breaks”. I cannot see that a date/time-body can be folded anywhere without violating this rule.▮

5.2.1.4.1.1 Yes it is!

For converting the header field timestamp into a GO-native type we use time.Parse from the time-package. We expect the format for the timestamp to be RFC1123 compliant (with TZ code), hence the use of time.RFC1123Z.

deliverat_body := strings.TrimSpace( headerfield[1] )
deliverat, err := time.Parse( time.RFC1123Z, deliverat_body )
if err != nil {
  if <<is logging>> {
    logger.Warning( fmt.Sprintf( "Failed to parse [%s]", deliverat_body ) )
  }
  <<fail fatal>>
}

Parsing can fail, this is considered fatal. The program will exit with an appropriate errorcode. This is a tough choice, returning TEMPFAIL is useless, as the parsing will fail the next time as well. The filter can also decide to expire the message; this will lead to a delivered message where the intention was to postpone it. We decided to raise an error big enough to make the message bounce. This will attract the attention of both the sender and the administrator.

Note that we trim spaces around the date/time-text, as it includes the trailing "\n". ▮

5.2.1.4.1.2 Expiration
now := time.Now(); now.After( deliverat )

time.Now() is the current moment in time; now.After(...) will then do exactly what we expect of expiration3.▮

5.2.1.5 The pipe to sendmail

Preparing for starting the sendmail-command involves some housekeeping.

if <<is logging>> {
  logger.Info( fmt.Sprintf( "preparing sendmail with %s", 
                   append( []string{"-G", "-i"}, os.Args[1:]... )) )
}
cmd = exec.Command( "/usr/sbin/sendmail", 
                    append( []string{"-G", "-i"}, os.Args[1:]... )... )
newstdout, err = cmd.StdinPipe()
if err != nil {
  if <<is logging>> {
    logger.Err( fmt.Sprintf( "failed sendmail pipe [%s]", err ) ) 
  }
  os.Exit( exit.UNAVAILABLE )
}
if <<is logging>> {
  logger.Info( fmt.Sprintf( "pipe to sendmail [%s]", newstdout ) )
}

The simple filter interface needs the filter to post the message itself back into Postfix using the sendmail command.▮

Starting the command prepared earlier:

if cmd != nil {
  if err := cmd.Start(); err != nil {
    if <<is logging>> {
      logger.Err( fmt.Sprintf( "failed to start sendmail [%s]", err ) ) 
    }
    os.Exit( exit.UNAVAILABLE )
  }
}

Starting the command only makes sense in case there is a command to start, hence testing for cmd.▮

5.2.1.6 Finishing
5.2.1.6.1 Postpone! aka Retry aka TEMPFAIL
if <<is logging>> {
  logger.Notice( fmt.Sprintf( "Postponing delivery based on header [%s]", line ) )
}
os.Exit( exit.TEMPFAIL );

The Postfix API expects a TEMPFAIL for messages which need retries.▮

5.2.1.6.2 Failed exit

Something fatal leads to an exit, in our case we raise a data-error.

if <<is logging>> {
  logger.Err( fmt.Sprintf( "Failing with DATAERROR for line [%s]", line ) )
}
os.Exit( exit.DATAERR );

Most errors, except TEMPFAIL, will force Postfix to log a line and bounce the message back to it's sender. This is exactly what we want.

5.2.1.6.3 OK
os.Exit( exit.OK );
5.2.1.6.4 sendmail

Finishing output means flushing it and closing it. The closing is important, as the sendmail-command will not terminate if it's input is not closed.

if <<is logging>> {
  logger.Info( "flushing output" )
}
writer.Flush()
if <<is logging>> { 
  logger.Info( fmt.Sprintf( "closing output %s", newstdout ) ) 
}
if newstdout != nil { newstdout.Close() }

It's silly to find out that closing cannot be done via the writer. That is why we need the variable newstdout. Actually we don't know if newstdout is used, while testing e.g. it is not. We should close stdout in case of testing. There is no sense in that, as there is nothing to gain there. If you think there is, plase add else {os.Stdout.Close()}.▮

The sendmail command needs to complete in order te decide whether it succeeded delivering the message.

if cmd != nil {
  if err := cmd.Wait(); err != nil { 
    if <<is logging>> {
      logger.Err( fmt.Sprintf( "waiting for sendmail failed [%s]", err ) ) 
    }
    os.Exit( exit.UNAVAILABLE ) 
  }
}

As always, exceptions are logged as well.▮

There is no need for it now, but we may want to kill the sendmail-process under certain circumstances. Here's how to do it.

if cmd != nil {cmd.Process.Kill()}
5.2.1.7 Logging

The filter doesn't always log. Hence a predicate for logging.

logger != nil

While testing e.g. it may not do logging.▮

5.2.1.8 GO babble
import (
     "bufio"
     "fmt"
     "os"
     "os/exec"
     "log/syslog"
     "io"
     "time"
     "strings"
     "github.com/Code-Hex/exit"
)

Bufio is used for fast buffered I/O. Fmt for formatting log-messages. The github-pulled code is to import a GO-version of sysexits.h, defining errorcodes to be returned by serverprograms.▮

5.2.2 The full program

package main
<<imports>>
// use -t for testing, other parameters will start sendmail
func main() {
  var cmd *exec.Cmd = nil         // declare cmd as it is needed for Start() later
  var headerfields []string = nil // buffering the header fields
  var newstdout io.WriteCloser    // one can only close via this interface

  logger, err := syslog.New( syslog.LOG_MAIL, "x-deliver-at" )
  if err != nil {
    os.Exit( exit.UNAVAILABLE )
  } else {
    defer logger.Close()
  }

  reader := bufio.NewReader( os.Stdin )
  writer := bufio.NewWriter( os.Stdout )
  if len( os.Args ) > 1 && os.Args[ 1 ] != "-t"  {
    <<feed into sendmail>>
    writer = bufio.NewWriter( newstdout );
  } else { // we're testing
    logger = nil
  }

  <<disable in body>>

  <<filter-implementation>>

  <<finish output>>
  <<wait for sendmail to end>>
  <<finish OK>>
}

The complete program has some GO-babble for package declaration and import-stuff. Also we declare main(), prepare the I/O variables and prepare the inbody variable.

Some variable declarations are necessary as I/O classes need symmetrical actions. After an open/creat, the close follows, but later. Later also lexicographically, hence a variable. This is the case for Cmd and newstdout.

Headerfields is used for buffering the header fields. Adding lines to it is done via append. As append supports adding a string to a nil-array, headerfields is initialized with nil to make the code for adding a simple one-liner.

The syslog-lines are prefixed with "x-deliver-at". Note that the length of this prefix matches "postfix/qmgr" and "postfix/pipe" in order to neatly align logging output.

Setting up the reader and writer then is simple, except… os.Stdout doesn't have the same interface as the stream into the pipe of a command (which is WriteCloser). We do this by assigning a bufio.NewWriter() to the writer variable. This overwrites previous setting of writer.4

5.3 Building and deploying

Standard GO procedure: set GOPATH and get/build/install.

export GOPATH=`pwd`
go get main
go build main
go install main

5.4 Installing into a Postfix installation

Configuration of a simple filter gives a perfect explanation on how to make a production setup for the use of this filter.5

Please make sure that you test the setup extensively. All messages will pass through the filter, you don't want to loose e-mail!

5.5 Testing

5.5.1 functionality

We begin by testing a lot of messages in my own INBOX. None of these should have a X-Deliver-At header field. All diff-output should be empty.

Then we test message 1 and 2. The second message is due for non-expiration and should exit with 75.

cnt=0
for f in `ls -tr ~/*/INBOX/cur/ | tail -1000`
do
  cnt=$(expr $cnt + 1)
  echo -n "$cnt: "
  cat ~/*/INBOX/cur/"$f" | bin/main -t | diff - ~/*/INBOX/cur/"$f"
  echo
done
for f in `ls -tr ~/*/INBOX/cur/ | tail -1000`
do
  cnt=$(expr $cnt + 1)
  echo -n "$cnt: "
  cat ~/*/INBOX/cur/"$f" | bin/main -t > /dev/null 2>&1
  echo "$?"
done
tomorrow=`date --rfc-2822 -d tomorrow`
yesterday=`date --rfc-2822 -d yesterday`
cat test/m1 | sed "s/\(X-Deliver-At: \).*/\1$tomorrow/" | bin/main -t > /dev/null; echo $?
cat test/m1 | sed "s/\(X-Deliver-At: \).*/\1$yesterday/" | bin/main -t > /dev/null; echo $?

5.5.2 performance

TBD

5.5.3 example message

From harrie.jansen@cistus.nl
Return-Path: <harrie.jansen@cistus.nl>
X-Original-To: gert@imap2.cistus.nl
Delivered-To: gert@imap2.cistus.nl
Received: from mx.cistus.nl (mx.cistus.nl [213.154.248.214])
	by mail.cistus.nl (Postfix) with ESMTP id 66EFD2BE722
	for <gert@imap2.cistus.nl>; Mon,  6 Feb 2018 08:04:14 +0100 (CET)
Received: by mx.cistus.nl (Postfix)
	id 28A5A40466F2; Mon,  6 Feb 2018 08:02:57 +0100 (CET)
Delivered-To: gert@cistus.nl
Received: from mail.cistus.nl (imap2.cistus.nl [213.154.248.218])
	by mx.cistus.nl (Postfix) with ESMTP id 1EB8940466E9;
	Mon,  6 Feb 2018 08:02:57 +0100 (CET)
Received: from pmbp-2.local (k5124.upc-k.chello.nl [62.108.5.124])
	by mail.cistus.nl (Postfix) with ESMTPSA id 1EA9F2BE721;
	Mon,  6 Feb 2018 08:04:14 +0100 (CET)
To: Gert Helberg <gert@cistus.nl>
From: Harrie Jansen <harrie.jansen@cistus.nl>
Subject: Updates naar Gribbel
Message-ID: <01b355d7-8153-d45a-5da2-80ef2cd8c806@cistus.nl>
Date: Mon, 6 Feb 2018 08:04:21 +0100
X-Deliver-At: Mon, 18 Sep 2017 22:04:21 +0200
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Goedemorgen Gert,

habbeldiebabbel

Mvg,

Harrie

6 About this title

The document to generate the code has the same source as the document you are reading now.

The full code, generated of course,is in tempfail-if-not-expired.go. The git repo is at: gitlab.com.

This title was written between the 18th and the 22nd of September 2017

Footnotes:

1
I may adapt the filter to do both in the future.
2
RFC2822 section 2.1
3
note that we use the concept of predicates in this document. Read about my use of it in relation to literate programming in Predicates in no-web and please help me find something better than enable
4
there is more about this, I like GO, but it has it's nasty things. Autodeclaration and scoping is one of them.
5
a mistake I made was that no cli mail-client uses SMTP over the inet-interface, no filtering is done via the pickup transport, better use a telnet dialogue for testing.

Author: Joost Helberg

Created: 2017-10-06 Fri 16:39