Importing an observed svn repository fails for all commits


#1

I am currently trying to get Diffusion to observe our repository. However, it is stuck on importing and 0.00%. It’s been running since yesterday afternoon.

Here’s the status:

Update Frequency 15 s · Learn More Status
Repository Active
Found Binary svn /usr/bin/svn
Pull Daemon Running
Task Daemon Running
Updates OK Last updated Wed, Jun 6, 9:08 AM (9 s ago).
Importing 0.00% Complete

If I run:
./bin/repository reparse --message --trace R4:1

gives the following error:

[sjm@vauxhall]~/phabricator/phabricator% ./bin/repository reparse --message --trace R4:1

[2] phabricator_repository
<<< [2] 1,052 us

[3] SELECT r., r. FROM repository r WHERE ((r.id IN (4))) ORDER BY r.id DESC
<<< [3] 521 us

[4] SELECT commit.* FROM repository_commit commit WHERE (((commit.repositoryID = 4 AND commit.commitIdentifier = ‘1’))) ORDER BY commit.id DESC
<<< [4] 445 us

[5] SELECT r., r. FROM repository r WHERE (r.id IN (4)) ORDER BY r.id DESC
<<< [5] 426 us

[6] SELECT commit.* FROM repository_commit commit WHERE (commit.id IN (6935)) ORDER BY commit.id DESC
<<< [6] 303 us

[7] SELECT r., r. FROM repository r WHERE (r.id IN (4)) ORDER BY r.id DESC
<<< [7] 407 us

[8] diffusion.querycommits()
[9] SELECT r., r. FROM repository r WHERE (r.phid IN (‘PHID-REPO-6er37kejvez7r4wimupf’)) ORDER BY r.id DESC
<<< [9] 1,065 us

[10] SELECT commit.* FROM repository_commit commit WHERE (commit.phid IN (‘PHID-CMIT-j5oe2zmkjcy4o7m6kd6b’)) AND (commit.repositoryID IN (4)) ORDER BY commit.id DESC LIMIT 101
<<< [10] 426 us

[11] SELECT r., r. FROM repository r WHERE (r.id IN (4)) ORDER BY r.id DESC
<<< [11] 1,221 us

[12] SELECT * FROM repository_commitdata WHERE commitID in (6935)
<<< [12] 397 us

[13] phabricator_passphrase
<<< [13] 416 us

[14] SELECT c.* FROM passphrase_credential c WHERE (c.phid IN (‘PHID-CDTL-rwguqkt2to4kmeszlbsq’)) ORDER BY c.id DESC
<<< [14] 686 us

[15] SELECT * FROM passphrase_secret WHERE id IN (1)
<<< [15] 496 us

[16] $ svn --non-interactive --no-auth-cache --username ‘’ --password '’ log --xml --limit 1 ‘svn://svn.uk.cyberscience.com/cqcs@1’
<<< [16] 75,156 us
<<< [8] 98,260 us

[17] SELECT repository_identity.* FROM repository_identity repository_identity WHERE (repository_identity.identityNameHash IN (‘qVzKuHbdOl_L’)) ORDER BY repository_identity.id DESC
<<< [17] 1,142 us

[18] diffusion.lookupUser <listeners = 1>
<<< [18] 458 us

[19] phabricator_repository
<<< [19] 1,653 us

THIS IS THE DATA…
‘PHID-CMIT-j5oe2zmkjcy4o7m6kd6b’, ‘qVzKuHbdOl_L’, NULL, ‘utf8’, NULL, NULL, NULL, ‘PHID-RIDT-nnus7yjatlrzera5ccek’, ‘1528272512’, ‘1528272512’

[20] INSERT INTO repository_identity (authorPHID, identityNameHash, identityNameRaw, identityNameEncoding, automaticGuessedUserPHID, manuallySetUserPHID, currentEffectiveUserPHID, phid, dateCreated, dateModified) VALUES (‘PHID-CMIT-j5oe2zmkjcy4o7m6kd6b’, ‘qVzKuHbdOl_L’, NULL, ‘utf8’, NULL, NULL, NULL, ‘PHID-RIDT-nnus7yjatlrzera5ccek’, ‘1528272512’, ‘1528272512’)
<<< [20] 277 us
[2018-06-06 09:08:32] EXCEPTION: (AphrontQueryException) #1048: Column ‘identityNameRaw’ cannot be null at [/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:355]
arcanist(head=master, ref.master=d581c453b83c), phabricator(head=master, ref.master=dbe72df55726), phutil(head=master, ref.master=20eff1c8d14f)
#0 AphrontBaseMySQLDatabaseConnection::throwQueryCodeException(integer, string) called at [/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:289]
#1 AphrontBaseMySQLDatabaseConnection::throwQueryException(mysqli) called at [/src/aphront/storage/connection/mysql/AphrontBaseMySQLDatabaseConnection.php:185]
#2 AphrontBaseMySQLDatabaseConnection::executeRawQuery(string) called at [/src/xsprintf/queryfx.php:8]
#3 queryfx(AphrontMySQLiDatabaseConnection, string, string, string, array, string) called at [/src/aphront/storage/connection/AphrontDatabaseConnection.php:58]
#4 AphrontDatabaseConnection::query(string, string, string, array, string) called at [/src/infrastructure/storage/lisk/LiskDAO.php:1263]
#5 LiskDAO::insertRecordIntoDatabase(string) called at [/src/infrastructure/storage/lisk/LiskDAO.php:1106]
#6 LiskDAO::insert() called at [/src/infrastructure/storage/lisk/LiskDAO.php:1075]
#7 LiskDAO::save() called at [/src/applications/repository/storage/PhabricatorRepositoryIdentity.php:73]
#8 PhabricatorRepositoryIdentity::save() called at [/src/applications/repository/worker/commitmessageparser/PhabricatorRepositoryCommitMessageParserWorker.php:80]
#9 PhabricatorRepositoryCommitMessageParserWorker::updateCommitData(DiffusionCommitRef) called at [/src/applications/repository/worker/commitmessageparser/PhabricatorRepositoryCommitMessageParserWorker.php:42]
#10 PhabricatorRepositoryCommitMessageParserWorker::parseCommit(PhabricatorRepository, PhabricatorRepositoryCommit) called at [/src/applications/repository/worker/PhabricatorRepositoryCommitParserWorker.php:51]
#11 PhabricatorRepositoryCommitParserWorker::doWork() called at [/src/infrastructure/daemon/workers/PhabricatorWorker.php:123]
#12 PhabricatorWorker::executeTask() called at [/src/applications/repository/management/PhabricatorRepositoryManagementReparseWorkflow.php:325]
#13 PhabricatorRepositoryManagementReparseWorkflow::execute(PhutilArgumentParser) called at [/src/parser/argument/PhutilArgumentParser.php:441]
#14 PhutilArgumentParser::parseWorkflowsFull(array) called at [/src/parser/argument/PhutilArgumentParser.php:333]
#15 PhutilArgumentParser::parseWorkflows(array) called at [/scripts/repository/manage_repositories.php:22]

You can see I’ve added a print statement (“THIS IS THE DATA…”) just before the error to show what the data is. Lots of variables seem to be NULL so not sure what is going on.

Does anyone have an idea what is happening, or any tips for trying to debug this?

Cheers!


#2

I dealt with this yesterday.

It can be resolved by running bin/repository rebuild-identities REPO for each REPO that is affected.

Source: comments on https://secure.phabricator.com/rPf191a66490b194785fae28c062b71be99bb14584


Importing an observed svn database fails to import any commits
#3

Great, thanks @jsonr, just trying that now. Does the command normally give any indication it is doing something? I’ve started it off, and get no output in the terminal.


#4

Ok, whilst the rebuild-identities command doesn’t give any output, I can see the import is now working! Thanks!