Classic differential.creatediff cURL/56

Observed Behavior:
I have created diff. Than update some file and try to update revision with “arc diff”. As result I have curl 56 error =(

The problem in one file, even 1 char change produce error. File is a python about 500 lines 21445 bytes. In the repo I have much more bigger files, but error cause only its change.

Seems like configuration problem, but I can’t find a reason. Help to troubleshoot why it happens or workaround the problem.

Phabricator Version:
phabricator d9b41d3a0f878d91b7485007d53998516d30731d (Tue, May 28)
arcanist bbbd1502f07888ee574353f96d77e1b418657755 (Tue, May 28)
phutil 0d206344cfd3db3fd4ea7c12ef0ec054e7898528 (Tue, May 28)
php 5.3.3
diff 2.8.1 at /usr/bin/diff
git 2.16.2 at /usr/bin/git
hg 1.4 at /usr/bin/hg
pygmentize 1.6 at /usr/bin/pygmentize
svn 1.6.11 at /usr/bin/svn

$ arc version
arcanist bbbd1502f07888ee574353f96d77e1b418657755 (28 May 2019)
libphutil b33ac46f6acb5021c83064a90a55e8dbde08c8c3 (6 Dec 2016)

Reproduce:
arc diff --trace ARGV '/Users/some_user/distr/arcanist/bin/../scripts/arcanist.php' 'diff' '--trace' LOAD Loaded "phutil" from "/Users/some_user/distr/libphutil/src". LOAD Loaded "arcanist" from "/Users/some_user/distr/arcanist/src". Config: Reading user configuration file "/Users/some_user/.arcrc"... Config: Did not find system configuration at "/etc/arcconfig". Working Copy: Reading .arcconfig from "/Users/some_user/some_project/src/.arcconfig". Working Copy: Path "/Users/some_user/some_project/src" is part of `git` working copy "/Users/some_user/some_project/src". Working Copy: Project root is at "/Users/some_user/some_project/src". Config: Did not find local configuration at "/Users/some_user/some_project/src/.git/arc/config". Loading phutil library from '/Users/some_user/some_project/src/contrib/arcanist-eslint'... >>> [0] <conduit> user.whoami() <bytes = 117> >>> [1] <http> http://phabricator.some_domain.org/api/user.whoami <<< [1] <http> 295,761 us <<< [0] <conduit> 299,070 us >>> [2] <exec> git --version
<<< [2] 9,286 us
>>> [3] $ git status --porcelain=2 -z
<<< [3] 216,024 us
You have untracked files in this working copy.

  Working copy: /Users/some_user/some_project/src/

  Untracked changes in working copy:
  (To ignore these changes, add them to ".git/info/exclude".)
    celerybeat-schedule
    contrib/bin/postsave.sh
    pdd/common/checkers.py
    pdd/contacts/
    pdd/healthchecks/
    pdd/mpop/migrations/
    pdd/mpop/views.py
    pdd/statsd/
    testapp/

    Ignore these untracked files and continue? [y/N] y

>>> [4] <event> diff.didCollectChanges <listeners = 0>
<<< [4] <event> 155 us
>>> [5] <exec> $ git rev-parse --verify HEAD^
<<< [5] <exec> 12,736 us
>>> [6] <exec> $ git merge-base 'origin/master' HEAD
<<< [6] <exec> 11,168 us
>>> [7] <exec> $ git rev-parse 'HEAD'
<<< [7] <exec> 11,294 us
>>> [8] <exec> $ git log --first-parent --format=medium 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1'..'8527e6c80692ab59f62b7fe28d03632084b0a3b7'
<<< [8] <exec> 10,043 us
>>> [9] <conduit> differential.query() <bytes = 238>
>>> [10] <http> http://phabricator.some_domain.org/api/differential.query
<<< [10] <http> 752,624 us
<<< [9] <conduit> 752,726 us
>>> [11] <conduit> differential.query() <bytes = 148>
>>> [12] <http> http://phabricator.some_domain.org/api/differential.query
<<< [12] <http> 174,012 us
<<< [11] <conduit> 174,110 us
>>> [13] <conduit> differential.getcommitmessage() <bytes = 171>
>>> [14] <http> http://phabricator.some_domain.org/api/differential.getcommitmessage
<<< [14] <http> 274,943 us
<<< [13] <conduit> 275,025 us
>>> [15] <conduit> differential.parsecommitmessage() <bytes = 459>
>>> [16] <http> http://phabricator.some_domain.org/api/differential.parsecommitmessage
<<< [16] <http> 258,054 us
<<< [15] <conduit> 258,137 us
>>> [17] <event> diff.didBuildMessage <listeners = 0>
<<< [17] <event> 15 us
>>> [18] <exec> $ git rev-parse --git-dir
<<< [18] <exec> 8,685 us
>>> [19] <exec> $ subl -w  '/var/folders/k9/d5mhk1s52g310pyg49vs9ds40000gp/T/edit.67ry63v83j0gggc4/differential-update-comments'
<<< [19] <exec> 2,697,093 us
Linting...
>>> [20] <exec> $ git rev-parse 'HEAD'
<<< [20] <exec> 20,336 us
>>> [21] <exec> $ git merge-base 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1' '8527e6c80692ab59f62b7fe28d03632084b0a3b7'
<<< [21] <exec> 14,634 us
>>> [22] <exec> $ git diff --no-ext-diff --no-textconv --submodule=short --raw 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1' HEAD --
<<< [22] <exec> 18,276 us
>>> [23] <exec> $ git diff --no-ext-diff --no-textconv --submodule=short --no-color --src-prefix=a/ --dst-prefix=b/ -U32767 -M -C 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1' --
<<< [23] <exec> 80,994 us
Examining paths for linter 'flake8'.
Found 5 matching paths for linter 'flake8'.
>>> [24] <exec> $ which 'node_modules/.bin/eslint'
<<< [24] <exec> 6,192 us
Examining paths for linter 'eslint'.
Found 0 matching paths for linter 'eslint'.
>>> [25] <exec> $ which 'flake8'
<<< [25] <exec> 5,081 us
>>> [26] <exec> $ which 'flake8'
<<< [26] <exec> 4,553 us
>>> [27] <exec> $ 'flake8' --version
<<< [27] <exec> 715,969 us
>>> [28] <exec> $ which 'node_modules/.bin/eslint'
<<< [28] <exec> 4,462 us
>>> [29] <exec> $ which 'node_modules/.bin/eslint'
<<< [29] <exec> 4,090 us
>>> [30] <exec> $ 'node_modules/.bin/eslint' --version
<<< [30] <exec> 623,281 us
>>> [31] <lint> Python Flake8 multi-linter <paths = 5>
>>> [32] <exec> $ which 'flake8'
<<< [32] <exec> 4,876 us
>>> [33] <exec> $ 'flake8'  '/Users/some_user/some_project/src/pdd/models/account.py'
>>> [34] <exec> $ 'flake8'  '/Users/some_user/some_project/src/pdd/models/role.py'
>>> [35] <exec> $ 'flake8'  '/Users/some_user/some_project/src/pdd/mpop/clients/client.py'
>>> [36] <exec> $ 'flake8'  '/Users/some_user/some_project/src/pdd/mpop/gates/client.py'
>>> [37] <exec> $ 'flake8'  '/Users/some_user/some_project/src/tests/pdd/models/test_account.py'
<<< [31] <lint> 10,006 us
>>> [38] <lint> Python Flake8 multi-linter <paths = 5>
<<< [36] <exec> 1,010,491 us
<<< [35] <exec> 1,060,926 us
<<< [34] <exec> 1,093,561 us
<<< [37] <exec> 1,250,380 us
<<< [33] <exec> 1,274,240 us
<<< [38] <lint> 1,268,754 us
 LINT OKAY  No lint problems.
Running unit tests...
No unit test engine is configured for this project.
>>> [39] <exec> $ git rev-parse 'HEAD'
<<< [39] <exec> 8,074 us
>>> [40] <exec> $ git merge-base 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1' '8527e6c80692ab59f62b7fe28d03632084b0a3b7'
<<< [40] <exec> 9,024 us
>>> [41] <exec> $ git diff --no-ext-diff --no-textconv --submodule=short --no-color --src-prefix=a/ --dst-prefix=b/ -U32767 -M -C 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1' '8527e6c80692ab59f62b7fe28d03632084b0a3b7' --
<<< [41] <exec> 13,887 us
>>> [42] <exec> $ git rev-parse 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1'
<<< [42] <exec> 7,764 us
>>> [43] <exec> $ git symbolic-ref --quiet HEAD
<<< [43] <exec> 7,716 us
>>> [44] <exec> $ git log --format=medium -n16 'ec23e04ef176e2a9e7fe755cfef4247d570ae0a1'
<<< [44] <exec> 11,406 us
>>> [45] <exec> $ git symbolic-ref --quiet HEAD
<<< [45] <exec> 7,766 us
>>> [46] <exec> $ git rev-parse --symbolic-full-name 'BIZ-5907_MPOP_PHONES'@{upstream}
<<< [46] <exec> 9,356 us
>>> [47] <exec> $ git ls-remote --get-url 'origin'
<<< [47] <exec> 7,843 us
>>> [48] <conduit> repository.query() <bytes = 195>
>>> [49] <http> http://phabricator.some_domain.org/api/repository.query
<<< [49] <http> 188,606 us
<<< [48] <conduit> 188,720 us
>>> [50] <conduit> differential.creatediff() <bytes = 85,195>
>>> [51] <http> http://phabricator.some_domain.org/api/differential.creatediff
<<< [51] <http> 137,050 us
<<< [50] <conduit> 137,140 us

[2019-06-18 09:32:15] EXCEPTION: (HTTPFutureCURLResponseStatus) [cURL/56] (http://phabricator.some_domain.org/api/differential.creatediff) <CURLE_RECV_ERROR> The cURL library raised an error while making a request. You may be able to find more information about this error (error code: 56) on the cURL site: http://curl.haxx.se/libcurl/c/libcurl-errors.html#CURLERECVERROR at [<phutil>/src/future/http/HTTPSFuture.php:408]
arcanist(head=stable, ref.master=fad85844314b, ref.stable=bbbd1502f078), arcanist-eslint(), phutil(head=master, ref.master=b33ac46f6acb)
  #0 HTTPSFuture::isReady() called at [<phutil>/src/future/Future.php:37]
  #1 Future::resolve(NULL) called at [<phutil>/src/future/FutureProxy.php:34]
  #2 FutureProxy::resolve() called at [<phutil>/src/conduit/ConduitClient.php:64]
  #3 ConduitClient::callMethodSynchronous(string, array) called at [<arcanist>/src/workflow/ArcanistDiffWorkflow.php:519]
  #4 ArcanistDiffWorkflow::run() called at [<arcanist>/scripts/arcanist.php:394]

/var/log/nginx/phabricator.access.log

185.6.245.156 - - [18/Jun/2019:12:31:47 +0300] "POST /api/user.whoami HTTP/1.1" 200 412 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.query HTTP/1.1" 200 1246 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.query HTTP/1.1" 200 1246 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.getcommitmessage HTTP/1.1" 200 285 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.parsecommitmessage HTTP/1.1" 200 793 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:15 +0300] "POST /api/repository.query HTTP/1.1" 200 463 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:15 +0300] "POST /api/differential.creatediff HTTP/1.1" 400 0 "-" "-"
185.6.245.156 - - [18/Jun/2019:12:32:15 +0300] "POST /api/differential.creatediff HTTP/1.1" 400 0 "-" "-"

/var/log/nginx/phabricator.error.log

2019/06/18 12:32:15 [warn] 14406#0: *15540 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000078, client: 185.6.245.156, server: phabricator.some_domain.org, request: "POST /api/differential.creatediff HTTP/1.1", host: "phabricator.some_domain.org"
2019/06/18 12:32:15 [info] 14406#0: *15540 recv() failed (104: Connection reset by peer), client: 185.6.245.156, server: phabricator.some_domain.org, request: "POST /api/differential.creatediff HTTP/1.1", host: "phabricator.some_domain.org"
2019/06/18 12:32:15 [warn] 14406#0: *15574 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000079, client: 185.6.245.156, server: phabricator.some_domain.org, request: "POST /api/differential.creatediff HTTP/1.1", host: "phabricator.some_domain.org"
2019/06/18 12:32:15 [info] 14406#0: *15574 recv() failed (104: Connection reset by peer), client: 185.6.245.156, server: phabricator.some_domain.org, request: "POST /api/differential.creatediff HTTP/1.1", host: "phabricator.some_domain.org"
2019/06/18 12:32:33 [info] 14406#0: *15579 client 5.61.236.245 closed keepalive connection

/etc/nginx/conf.d/phabricator.conf

server {
    listen 80;
    server_name phabricator.some_domain.org;
    access_log /var/log/nginx/phabricator.access.log;
    error_log /var/log/nginx/phabricator.error.log debug;
    client_max_body_size 64m;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:8080;
    }
}

/var/log/httpd/phabricator.access.log

185.6.245.156 - - [18/Jun/2019:12:31:46 +0300] "POST /api/user.whoami HTTP/1.0" 200 412
185.6.245.156 - - [18/Jun/2019:12:32:08 +0300] "POST /api/differential.query HTTP/1.0" 200 1246
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.query HTTP/1.0" 200 1246
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.getcommitmessage HTTP/1.0" 200 285
185.6.245.156 - - [18/Jun/2019:12:32:09 +0300] "POST /api/differential.parsecommitmessage HTTP/1.0" 200 793
185.6.245.156 - - [18/Jun/2019:12:32:15 +0300] "POST /api/repository.query HTTP/1.0" 200 463

Update

We have figured out that problem was related to corporate network firewall. It drops some http requests due security reasons. So we just switched out phabricator installation to https and problem fade away.

This report does not include reproduction instructions which allow someone without access to your environment to reproduce the issue.