How to override "@localhost.localdomain" in email message-id?

cluster.mailers is configured in local.json with “type”:“smtp”. It uses an accout on a remote SMTP server to send emails. This works fine. The only problem is that the Message-ID header in all emails sent by Phabricator contains @localhost.localdomain as the FQDN part of the message id. To avoid problems with spam filters, I would like to fix the message id in outgoing emails to some real FQDN that I own.

Can I set something in my

  • local.json,
  • php.ini,
  • database,
  • preamble.php,
  • nginx configuration,
  • or operating system

to replace this @localhost.localdomain part with a real domain name that I control?

I have already tried to add
“message-id”: false
to the SMTP options, but it did not have the desired effect.

I’m following the stable branch, on Linux.

My Phabricator instance runs fine on php-fpm behind an nginx instance that forwards based on hostname to different services.

Thanks for any insights.

I believe the answer is straightforward and prominently documented, so there might be an issue with the documentation not being clear enough if you aren’t able to find it.

What steps have you taken so far to try to find the answer? (Which pieces of documentation have you read, what have you searched for, etc?) Knowing this may help me fix the documentation so the next person has an easier time figuring this out.

Thanks for caring, even if I seem to not find obvious documentation.

I have read the “configuring outbound email” documentation. I know it has a section on Message-IDs, but this does not help me (see below). I have also searched for

  • phabricator localhost.localdomain
  • phabricator message-id
  • All three above keywords together
  • phabricator fqdn

My searches find mainly the section titled “Message-IDs” in https://secure.phabricator.com/book/phabricator/article/configuring_outbound_email/.

This section documents how one can set “message-id”: false in the SMTP options. It does not tell me how to influence the FQDN part of the message ID, or what information ends up being used as the FQDN part. Configuring phabricator to not use a message-ID at all and let the SMTP server add one would also solve my problem, so I have tried to do that, too:

My cluster.mailers configuration is inside my local.json file. I have changed my local.json file to read like this:

{
  "metamta.default-address": "noreply@phabricator.EXAMPLE.ORG",
  "cluster.mailers": [
    {
      "key": "university-smtp",
      "type": "smtp",
      "options": {
        "host": "smtp.EXAMPLE.ORG",
        "port": 25,
        "user": "EXAMPLE",
        "password": "EXAMPLE",
        "message-id": false
      }
    }
  ],
  "storage.local-disk.path": "/var/phabricator_storage/",
  "phabricator.base-uri": "https://phabricator.EXAMPLE.ORG/",
  "security.alternate-file-domain": "https://phabricator.EXAMPLE.COM/",
  "mysql.pass": "EXAMPLE",
  "mysql.user": "EXAMPLE",
  "mysql.host": "db.EXAMPLE.ORG"
}

(replaced some data with EXAMPLEs)

While adding the “message-id”:false option, I have also changed the “metamta.default-address” so that I notice whether this local.json file has an effect by looking at the From: address of emails sent by phabricator.

After restarting phabricator, emails sent by it have the new From: email address, but still have the @localhost.localdomain message ID.

I have also checked the SMTP server that I use if this server is responsible for adding the @localhost.localdomain message-ids: I have submitted a test email without message-id through netcat. It adds a valid message-id with its own hostname as the FQDN part.

Please tell me what I have overlooked. Where can I set the part of the message-id that comes after the @. Or why does my message-id: false have no effect.

Ah, sorry! I assumed you’d missed metamta.default-address since you didn’t mention it, but it looks like you do already have it configured.

I think this is actually a bug in Phabricator. The Phabricator-level code specifies a “Message-ID” only when it sends the first message in a thread and the “message-id” setting is not disabled. However, the SMTP client layer (“PHPMailer”) has code which generates a “message-id” if one isn’t provided, and does so with default settings. This ignores all Phabricator configuration, and can generate localhost.localdomain message IDs.

A possible fix is to make this change:

diff --git a/externals/phpmailer/class.phpmailer.php b/externals/phpmailer/class.phpmailer.php
index 5ddbddc144..69f9c45ba5 100644
--- a/externals/phpmailer/class.phpmailer.php
+++ b/externals/phpmailer/class.phpmailer.php
@@ -1110,8 +1110,6 @@ class PHPMailer {
 
     if($this->MessageID != '') {
       $result .= $this->HeaderLine('Message-ID',$this->MessageID);
-    } else {
-      $result .= sprintf("Message-ID: <%s@%s>%s", $uniq_id, $this->ServerHostname(), $this->LE);
     }
     $result .= $this->HeaderLine('X-Priority', $this->Priority);
     $result .= $this->HeaderLine('X-Mailer', 'PHPMailer '.$this->Version.' (phpmailer.sourceforge.net)');

I’m not entirely sure this fixes things since this depends on your environment and SMTP mailer behavior to some degree. If you’re able to test and confirm the behavior of this patch, I’m happy to bring it upstream. If you have time to give that a shot, the steps are:

  • apply the patch to phabricator/;
  • restart the daemons with bin/phd restart;
  • send first-in-thread and not-first-in-thread mail (for example: create a test task, then reply to it) and verify that both messages arrive with sensible Message-ID headers.

The intent of this change is to disable PHPMailer’s automatic generation of message IDs. Once this behavior is disabled, your SMTP server will presumably fill in a reasonable message ID in cases where Phabricator does not provide one.

It’s possible that your SMTP server (or some other SMTP server) will have some other behavior when a message it is passed has no Message-ID (for example: rejecting the message, or generating an unsuitable Message-ID). If so, a slightly larger patch in Phabricator could make us always provide a reasonable Message-ID, but this patch is a little bit simpler if it works as-is.

(If you don’t have time to test that patch, I’ll set up some kind of similar environment locally and see if I can convince myself that it’s correct, but may not get to this for a bit.)

I have time to test this and will do so soon. One question:

The second line that your patch removes seems to use
$this->ServerHostname()
as the FQDN part of the message ID. Is there any chance that I can influence what this function returns via php.ini, environment variables, nginx settings, …, any other configuration mechanisms?

I don’t think so. I believe it’s coming from here:

https://secure.phabricator.com/source/phabricator/browse/master/externals/phpmailer/class.phpmailer.php$1936-1951

  /**
   * Returns the server hostname or 'localhost.localdomain' if unknown.
   * @access private
   * @return string
   */
  private function ServerHostname() {
    if (!empty($this->Hostname)) {
      $result = $this->Hostname;
    } elseif (isset($_SERVER['SERVER_NAME'])) {
      $result = $_SERVER['SERVER_NAME'];
    } else {
      $result = 'localhost.localdomain';
    }

    return $result;
  }

No codepath in Phabricator can set $this->Hostname.

In some cases, you could influence $_SERVER['SERVER_NAME'], but mail is sent by the daemons, and the process evaluating this code will be a daemon process. As far as I know, there is no way to make PHP or any other language-level layer populate $_SERVER['SERVER_NAME'] in a CLI process.

I have tested the patch, and after removing option “message-id”:false again from my local.json, I could see that:

  • First email with [Created] in subject has a sensible message ID with the FQDN of the phabricator instance.
    • They always had this before I changed to “message-id”:false, but I had never noticed before today.
  • Followup emails with [Commented] in subject have a sensible message ID with the FQDN of the SMTP server.

As you said, whether this works or not, will depend on the SMTP server. For me it works.

I have learned that phabricator only creates message-ids when it wants to reference these IDs in later messages. It would not hurt if phabricator generated a message-id for every email that it sends, whether that ID is later referenced or not. IMO this would be the easiest fix for this situation, without having to mess with phpmailer.

I have also noticed that phpmailer in its current version on github also tries to use the gethostname() function when the other options are empty, which is usually not a valid FQDN, but it can be hacked to be one at least on Linux.

I landed this change upstream in https://secure.phabricator.com/D21272.