%$Id: milter-protocol.txt,v 1.6 2004/08/04 16:27:50 tvierling Exp $ % % ##THE SENDMAIL MILTER PROTOCOL, VERSION 2 The Sendmail and "libmilter" implementations of the protocol described herein are: > Copyright (c) 1999-2002 [Sendmail, Inc.](http://www.sendmail.com) and its suppliers. > All rights reserved. This document is: > Copyright (c) 2002-2003, Todd Vierling > All rights reserved. Permission is granted to copy or reproduce this document in its entirety in any medium without charge, provided that the copy or reproduction is without modification and includes the above copyright notice(s). ###OVERVIEW The date of this document is contained within the "Id" symbolic CVS/RCS tag present at the top of this document. This document describes the Sendmail "milter" mail filtering and MTA-level mail manipulation protocol, version 2, based on the publicly available C-language source code to Sendmail, version 8.11.6. As of this writing, this protocol document is based on the implementation of milter in Sendmail 8.11, but has been verified compatible with Sendmail 8.12. Some Sendmail 8.12 extensions, determined by flags sent with the SMFIC_OPTNEG command, are not yet described here. Technical terms describing mail transport are used throughout. A reader should have ample understanding of RFCs 821, 822, 2821, and their successors, and (for Sendmail MTAs) a cursory understanding of Sendmail configuration procedures. ###LEGEND All integers are assumed to be in network (big-endian) byte order. Data items are aligned to a byte boundary, and are not forced to any larger alignment. This document makes use of a mnemonic representation of data structures as transmitted over a communications endpoint to and from a milter program. A structure may be represented like the following: 'W' |SMFIC_HWORLD |Hello world packet --------|---------------|------------------ uint16 len ||Length of string char str[len] ||Text value This structure contains a single byte with the ASCII representation 'W', a 16-bit network byte order integer, and a character array with the length given by the "len" integer. Character arrays described in this fashion are an exact number of bytes, and are not assumed to be NUL terminated. A special data type representation is used here to indicate strings and arrays of strings using C-language semantics of NUL termination. char str[] |String, NUL terminated ------------------------|-------------------------------- char array[][] |Array of strings, NUL terminated Here, "str" is a NUL-terminated string, and subsequent data items are assumed to be located immediately following the NUL byte. "array" is a stream of NUL-terminated strings, located immediately following each other in the stream, leading up to the end of the data structure (determined by the data packet's size). ###LINK/PACKET PROTOCOL The MTA makes a connection to a milter by connecting to an IPC endpoint (socket), via a stream-based protocol. TCPv4, TCPv6, and "Unix filesystem" sockets can be used for connection to a milter. (Configuration of Sendmail to make use of these different endpoint addressing methods is not described here.) Data is transmitted in both directions using a structured packet protocol. Each packets is comprised of: uint32 len |Size of data to follow ------------------------|-------------------------------- char cmd |Command/response code char data[len-1] |Code-specific data (may be empty) The connection can be closed at any time by either side. If closed by the MTA, the milter program should release all state information for the previously established connection. If closed by the milter program without first sending an accept or reject action message, the MTA will take the default action for any message in progress (configurable to ignore the milter program, or reject with a 4xx or 5xx error). ###A TYPICAL MILTER CONVERSATION The MTA drives the milter conversation. The milter program sends responses when (and only when) specified by the particular command code sent by the MTA. It is an error for a milter either to send a response packet when not requested, or fail to send a response packet when requested. The MTA may have limits on the time allowed for a response packet to be sent. The typical lifetime of a milter connection can be viewed as follows: MTA |Milter ------------------------|---------------------- SMFIC_OPTNEG | |SMFIC_OPTNEG SMFIC_MACRO:'C' | SMFIC_CONNECT | |Accept/reject action SMFIC_MACRO:'H' | SMFIC_HELO | |Accept/reject action SMFIC_MACRO:'M' | SMFIC_MAIL | |Accept/reject action SMFIC_MACRO:'R' | SMFIC_RCPT | |Accept/reject action SMFIC_HEADER (multiple) | |Accept/reject action (per SMFIC_HEADER) SMFIC_EOH | |Accept/reject action SMFIC_BODY (multiple) | |Accept/reject action (per SMFIC_BODY) SMFIC_BODYEOB | |Modification action (multiple, may be none) |Accept/reject action | |(Reset state to before SMFIC_MAIL and continue, | unless connection is dropped by MTA) Several of these MTA/milter steps can be skipped if requested by the SMFIC_OPTNEG response packet; see below. ###PROTOCOL NEGOTIATION Milters can perform several actions on a SMTP transaction. The following is a bitmask of possible actions, which may be set by the milter in the "actions" field of the SMFIC_OPTNEG response packet. (Any action which MAY be performed by the milter MUST be included in this field.) 0x01 |SMFIF_ADDHDRS |Add headers (SMFIR_ADDHEADER) --------|-----------------------|----------------------------------- 0x02 |SMFIF_CHGBODY |Change body chunks (SMFIR_REPLBODY) 0x04 |SMFIF_ADDRCPT |Add recipients (SMFIR_ADDRCPT) 0x08 |SMFIF_DELRCPT |Remove recipients (SMFIR_DELRCPT) 0x10 |SMFIF_CHGHDRS |Change or delete headers (SMFIR_CHGHEADER) 0x20 |SMFIF_QUARANTINE |Quarantine message (SMFIR_QUARANTINE) >XXX: SMFIF_DELRCPT has an impact on how address rewriting affects addresses sent in the SMFIC_RCPT phase. This will be described in a future revision of this document. Protocol content can contain only selected parts of the SMTP transaction. To mask out unwanted parts (saving on "over-the-wire" data churn), the following can be set in the "protocol" field of the SMFIC_OPTNEG response packet. 0x01 |SMFIP_NOCONNECT |Skip SMFIC_CONNECT --------|-----------------------|----------------- 0x02 |SMFIP_NOHELO |Skip SMFIC_HELO 0x04 |SMFIP_NOMAIL |Skip SMFIC_MAIL 0x08 |SMFIP_NORCPT |Skip SMFIC_RCPT 0x10 |SMFIP_NOBODY |Skip SMFIC_BODY 0x20 |SMFIP_NOHDRS |Skip SMFIC_HEADER 0x40 |SMFIP_NOEOH |Skip SMFIC_EOH For backwards-compatible milters, the milter should pay attention to the "actions" and "protocol" fields of the SMFIC_OPTNEG packet, and mask out any bits that are not part of the offered protocol content. The MTA may reject the milter program if any action or protocol bit appears outside the MTA's offered bitmask. ####COMMAND CODES The following are commands transmitted from the MTA to the milter program. The data structures represented occupy the "cmd" and "data" fields of the packets described above in LINK/PACKET PROTOCOL. (In other words, the data structures below take up exactly "len" bytes, including the "cmd" byte.) 'A' |SMFIC_ABORT |Abort current filter checks --------|---------------|--------------------------- | |Expected response: NONE >Resets internal state of milter program to before SMFIC_HELO, but keeps the connection open. 'B' |SMFIC_BODY |Body chunk --------|---------------|--------------------------- ||Expected response: Accept/reject action char buf[] ||Up to MILTER_CHUNK_SIZE (65535) bytes >These body chunks can be buffered by the milter for later replacement via SMFIR_REPLBODY during the SMFIC_BODYEOB phase. 'C' |SMFIC_CONNECT |SMTP connection information --------|---------------|--------------------------- ||Expected response: Accept/reject action char hostname[] ||Hostname, NUL terminated char family ||Protocol family (see below) uint16 port ||Port number (SMFIA_INET or SMFIA_INET6 only) char address[] ||IP address (ASCII) or unix socket path, NUL terminated >Sendmail invoked via the command line or via "-bs" will report the connection as the "Unknown" protocol family. >Protocol families used with SMFIC_CONNECT in the "family" field: >'U' |SMFIA_UNKNOWN |Unknown (NOTE: Omits "port" and "host" fields entirely) >--------|---------------|--------------------------- >'L' |SMFIA_UNIX |Unix (AF_UNIX/AF_LOCAL) socket ("port" is 0) >'4' |SMFIA_INET |TCPv4 connection >'6' |SMFIA_INET6 |TCPv6 connection
'D' |SMFIC_MACRO |Define macros --------|---------------|--------------------------- ||Expected response: NONE char cmdcode ||Command for which these macros apply char nameval[][] ||Array of NUL-terminated strings, alternating ||between name of macro and value of macro. >SMFIC_MACRO appears as a packet just before the corresponding "cmdcode" (here), which is the same identifier as the following command. The names correspond to Sendmail macros, omitting the "$" identifier character. >Types of macros, and some commonly supplied macro names, used with SMFIC_MACRO are as follows, organized by "cmdcode" value. Implementations SHOULD NOT assume that any of these macros will be present on a given connection. In particular, communications protocol information may not be present on the "Unknown" protocol type. >'C' |SMFIC_CONNECT |$_ $j ${daemon_name} ${if_name} ${if_addr} --------|---------------|--------------------------- 'H' |SMFIC_HELO |${tls_version} ${cipher} ${cipher_bits} ||${cert_subject} ${cert_issuer} 'M' |SMFIC_MAIL |$i ${auth_type} ${auth_authen} ${auth_ssf} ||${auth_author} ${mail_mailer} ${mail_host} ||${mail_addr} 'R' |SMFIC_RCPT |${rcpt_mailer} ${rcpt_host} ${rcpt_addr} >For future compatibility, implementations MUST allow SMFIC_MACRO at any time, but the handling of unspecified command codes, or SMFIC_MACRO not appearing before its specified command, is currently undefined. 'E' |SMFIC_BODYEOB |Final body chunk --------|---------------|--------------------------- ||Expected response: Zero or more modification ||actions, then accept/reject action
'H' |SMFIC_HELO |HELO/EHLO name --------|---------------|--------------------------- ||Expected response: Accept/reject action char helo[] ||HELO string, NUL terminated
'L' |SMFIC_HEADER |Mail header --------|---------------|--------------------------- ||Expected response: Accept/reject action char name[] ||Name of header, NUL terminated char value[] ||Value of header, NUL terminated
'M' |SMFIC_MAIL |MAIL FROM: information --------|---------------|--------------------------- ||Expected response: Accept/reject action char args[][] ||Array of strings, NUL terminated (address at index 0). ||args[0] is sender, with <> qualification. ||args[1] and beyond are ESMTP arguments, if any.
'N' |SMFIC_EOH |End of headers marker --------|---------------|--------------------------- ||Expected response: Accept/reject action
'O' |SMFIC_OPTNEG |Option negotiation --------|---------------|--------------------------- ||Expected response: SMFIC_OPTNEG packet uint32 version |SMFI_VERSION|(2) uint32 actions ||Bitmask of allowed actions from SMFIF_* uint32 protocol ||Bitmask of possible protocol content from SMFIP_*
'R' |SMFIC_RCPT |RCPT TO: information --------|---------------|--------------------------- ||Expected response: Accept/reject action char args[][] ||Array of strings, NUL terminated (address at index 0). ||args[0] is recipient, with <> qualification. ||args[1] and beyond are ESMTP arguments, if any.
'Q' |SMFIC_QUIT |Quit milter communication --------|---------------|--------------------------- ||Expected response: Close milter connection ####RESPONSE CODES The following are commands transmitted from the milter program to the MTA, in response to the appropriate type of command packet. The data structures represented occupy the "cmd" and "data" fields of the packets described above in LINK/PACKET PROTOCOL. (In other words, the data structures below take up exactly "len" bytes, including the "cmd" byte.) Response codes: '+' |SMFIR_ADDRCPT |Add recipient (modification action) --------|---------------|--------------------------- char rcpt[] ||New recipient, NUL terminated
'-' |SMFIR_DELRCPT |Remove recipient (modification action) --------|---------------|--------------------------- char rcpt[] ||Recipient to remove, NUL terminated (string must match the one in SMFIC_RCPT exactly)
'a' |SMFIR_ACCEPT |Accept message completely (accept/reject action) --------|---------------|--------------------------- >This will skip to the end of the milter sequence, and recycle back to the state before SMFIC_MAIL. The MTA may, instead, close the connection at that point. 'b' |SMFIR_REPLBODY |Replace body (modification action) --------|---------------|--------------------------- char buf[] ||Full body, as a single packet
'c' |SMFIR_CONTINUE |Accept and keep processing (accept/reject action) --------|---------------|--------------------------- >If issued at the end of the milter conversation, functions the same as SMFIR_ACCEPT. 'd' |SMFIR_DISCARD |Set discard flag for entire message (accept/reject action) --------|---------------|--------------------------- >Note that message processing MAY continue afterwards, but the mail will not be delivered even if accepted with SMFIR_ACCEPT. 'h' |SMFIR_ADDHEADER|Add header (modification action) --------|---------------|--------------------------- char name[] ||Name of header, NUL terminated char value[] ||Value of header, NUL terminated
'm' |SMFIR_CHGHEADER|Change header (modification action) --------|---------------|--------------------------- uint32 index ||Index of the occurrence of this header char name[] ||Name of header, NUL terminated char value[] ||Value of header, NUL terminated >Note that the "index" above is per-name--i.e. a 3 in this field indicates that the modification is to be applied to the third such header matching the supplied "name" field. A zero length string for "value", leaving only a single NUL byte, indicates that the header should be deleted entirely.
'p' |SMFIR_PROGRESS |Progress (asynchronous action) --------|---------------|--------------------------- This is an asynchronous response which is sent to the MTA to reset the communications timer during long operations. The MTA should consume as many of these responses as are sent, waiting for the real response for the issued command.
'q' |SMFIR_QUARANTINE|Quarantine message (modification action) --------|---------------|--------------------------- char reason[] ||Reason for quarantine, NUL terminated This quarantines the message into a holding pool defined by the MTA. (First implemented in Sendmail in version 8.13; offered to the milter by the SMFIF_QUARANTINE flag in "actions" of SMFIC_OPTNEG.)
'r' |SMFIR_REJECT |Reject command/recipient with a 5xx (accept/reject action) --------|---------------|---------------------------
't' |SMFIR_TEMPFAIL |Reject command/recipient with a 4xx (accept/reject action) --------|---------------|---------------------------
'y' |SMFIR_REPLYCODE|Send specific Nxx reply message (accept/reject action) --------|---------------|--------------------------- char smtpcode[3] ||Nxx code (ASCII), not NUL terminated char space ||' ' char text[] ||Text of reply message, NUL terminated >'%' characters present in "text" must be doubled to prevent problems with printf-style formatting that may be used by the MTA. 'O' |SMFIC_OPTNEG |Option negotiation (in response to SMFIC_OPTNEG) --------|---------------|--------------------------- uint32 version ||SMFI_VERSION (2) uint32 actions ||Bitmask of requested actions from SMFIF_* uint32 protocol ||Bitmask of undesired protocol content from SMFIP_* ###CREDITS Sendmail, Inc. - for the Sendmail program itself The anti-spam community - for making e-mail a usable medium again The spam community - for convincing me that it's time to really do somthing to quell the inflow of their crap