Webhooks that bypass the outbound-blacklist?


#1

Hello and welcome to another installation of my regular series “James is trying to convert stuff from feed http hooks to firehose webhook”.

We have a security.outbound-blacklist set up to prevent connections to RFC1918 addresses to prevent our users from messing with our infrastructure by sending arbitrary requests to arbitrary ports on local machines. This is good and proper.

However, as part of migrating from Feeds to Webhooks, we need to set up something inside our infrastructure to receive the webhooks and ingest them where the feed worker used to ingest them.

Is there any way to create a webhook which is allowed to bypass the outbound-blacklist? Is there some flag I can set with a command-line utility to “bless” a webhook thusly? Is this use just unsupported and should we assume that there’s no intersection (besides me) of (people who use feed events to ingest into other local infrastructure) and (people who don’t want their users to be able to arbitrarily frob local infrastructure)? If worst comes to worst I’m sure I can put some patch in our local fork to skip the blacklist check for some specific webhook PHID but that seems unspeakably dirty.

I tried disabling the blacklist, creating the webhook, and re-enabling the blacklist, but the filter are well-implemented and check against the blacklist at send time, not at webhook creation time.


#2

There’s currently no way to whitelist addresses.

You can create a “hole” in the list with clever CIDR specification. For example, if your server is at 172.16.13.13 and the rule causing problems is 172.16.0.0/12, you can replace that rule with these rules:

172.16.0.0/21
172.16.8.0/22
172.16.12.0/24
172.16.13.0/29
172.16.13.8/30
172.16.13.12/32
172.16.13.14/31
172.16.13.16/28
172.16.13.32/27
172.16.13.64/26
172.16.13.128/25
172.16.14.0/23
172.16.16.0/20
172.16.32.0/19
172.16.64.0/18
172.16.128.0/17

This list of rules means “all of 172.16.0.0/12 except 172.16.13.13”.

This tool (or probably other tools) can help automate this conversion:

You can use a script like this in phabricator/ to make sure your list is correct:

<?php

require_once 'scripts/init/init-script.php';

$list = <<<EOLIST
172.16.0.0/21
172.16.8.0/22
172.16.12.0/24
172.16.13.0/29
172.16.13.8/30
172.16.13.12/32
172.16.13.14/31
172.16.13.16/28
172.16.13.32/27
172.16.13.64/26
172.16.13.128/25
172.16.14.0/23
172.16.16.0/20
172.16.32.0/19
172.16.64.0/18
172.16.128.0/17
EOLIST;

$list = explode("\n", trim($list));
$cidr_list = PhutilCIDRList::newList($list);

for ($ii = 0; $ii <= 255; $ii++) {
  for ($jj = 0; $jj <= 255; $jj++) {
    $address = "172.16.{$ii}.{$jj}";
    if (!$cidr_list->containsAddress($address)) {
      echo "Allowed: {$address}\n";
    }
  }
}

echo "Done.\n";

This produces:

$ php -f test.php 
Allowed: 172.16.13.13
Done.

#3

I actually don’t want non-administrative-defined actions (webhooks, harbormaster, etc) to be able to hit this host, either, though…