Subversion revision corruption

Say that three times fast...

We ran into an interesting issue earlier this week with Subversion. When attempting to perform any sort of operation on one of our repositories, the operation failed with an error stating "Final line in revision file longer than 64 characters svn: PROPFIND of '/svn/myRepository/myProject'."

When a checkin is made to Subversion, the file differences are compressed and saved to disk under \myRepository\db\revs\. Each checkin will be in a separate file named to match the revision number. Thus revision 1200 is saved to a file named "1200" (no extension.) If you open one of these files in a text editor, you will mostly see unreadable binary. One noticeable feature in all of the files is a pair of numbers on the last line - "10550 10745" perhaps. In our particular case the final revision contained two additional lines following the number pair:

100.1.1.10 - myUser [16/Mar/2009:10:12:06 -0500] "PROPFIND /svn/myRepository/!svn/bln/2914 HTTP/1.1" 207 464
100.1.1.10 - myUser [16/Mar/2009:10:12:06 -0500] "PROPFIND /svn/myRepository/myProject HTTP/1.1" 207 714

These lines are normally found in the Apache log file - not a good sign. A web search verified that it is a known issue with no resolution. The good news is that the page links to a Python script that attempts to repair corrupt revisions. (More information on using the script can be found here.)

The bad news? The script didn't entirely work for us - some of the header info on the compressed diffs had been lost. Fortunately, it did remove the Apache log info from the tail of the file, so other developers could again work. The only section now corrupt was the project where the revision error occurred. This project started returning an error stating "Checksum mismatch while reading representation."

With the issue isolated, the next step was to remove the corrupt revision. Unfortunately, the only way to do that in Subversion is to export the good revisions and then build a new repository using those exports. Say revision 120 is bad and we have a total of 150 revisions. We start by dumping revisions 1 through 119 to a file:

svnadmin dump c:\myRepository\ -r1:119 > c:\DumpPart1.dmp

We then need an incremental dump for changes 121 through 150:

svnadmin dump c:\myRepository\ -r121:150 -- incremental > c:\DumpPart2.dmp

Rename the folder "myRepository" to something else as a quick backup, then create a new database:

svnadmin create c:\myRepository

Copy any config files from the backup repository (such as user accounts) then load the dumps into the new database:

svnadmin load c:\myRepository < c:\DumpPart1.dmp
svnadmin load c:\myRepository < c:\DumpPart2.dmp

One item of note: Loading a Subversion dump effectively creates new checkins for each item in that dump. All items in the second dump above will have a revision number one lower than previously - checkin 121 in the old database becomes checkin 120 in the new database. Though I've not seen it, some developers may receive an error if their local code contains a newer revision number than the database. As I ran a few test checkins after creating the new database, this wasn't a problem.