RogueForge

Redux

How a version is rebuilt

From a file in a 1981 BSD distribution tree to C source that compiles back into it, and how we know when it does.

  1. Find the binary. Take the executable or object file from a dated distribution, such as Rogue 3.4 from 2.8BSD or Rogue 5.3 from 4.2BSD.
  2. Read the machine code. Disassemble and decompile it with Ghidra and radare2, after fixing their PDP-11 and VAX support well enough to trust.
  3. Write the C. Start from the closest surviving source, then change it function by function until each one compiles to the instructions in the binary.
  4. Prove it. Build with the original compiler, assembler and linker, ported to modern Linux, and compare the whole file. Done means the checksums match.

Finding the binary

Most of the targets come from Disc 1 of the CSRG Archive CD-ROMs, which preserve Berkeley's own distribution trees. The 4.1BSD copy of Rogue 3.6 comes from The Unix Heritage Society's reconstruction of that release.

The binaries differ in how much help they give. Rogue 3.4 is fully stripped: 70,364 bytes of PDP-11 code with no symbol names at all. The 3.6, 4.22 and 5.2 binaries keep between 1,600 and 2,000 symbols, so every function and global variable arrives with its name. Rogue 5.4 is an object file with its symbols stripped, which keeps the relocations that say where each call goes.

Reading the machine code

Three disassemblers were used side by side: the GNU objdump for the PDP-11, radare2, and Ghidra running headless. None of them handled these machines well at first, and each needed fixing:

  • radare2's PDP-11 decoder read instructions through a four-byte buffer, too small for the longest PDP-11 instructions, which take six.
  • Ghidra's PDP-11 description did not understand the csv and cret routines that every compiled C function calls on entry and exit, nor the floating-point instruction set, nor several addressing modes. After the fixes, clean decompiles of Rogue 3.4 went from none of its 145 functions to 144.
  • Ghidra's VAX description needed four corrections to its instruction semantics, and a script that repaired 2,145 relocations in the 5.4 object file. That recovered 234 functions and named 270 globals.

The decompiled C is a guide, never an answer. Ghidra regularly got argument counts wrong, misread stack frames, and moved code out of loops in ways the 1981 compiler never would.

Writing the C

Rebuilding starts from the closest of the three source trees that survive near their versions: the site copies of Rogue 3.6, the Illinois copy of Rogue 5.2, and Ken Arnold's Rogue 5.4. The versions in between start from a neighbour; Rogue 3.4, for example, began as the 3.6 source cut down and moved back. Then each function is compiled on its own and compared with the binary, instruction by instruction, with addresses masked out so that one function can match before its neighbours do.

That is where period compilers show their personality. The PDP-11 compiler generates different code for while (1) than for for (;;). The VAX compiler picks registers in an order that depends on how variables are declared. A chained assignment, or the order of two declarations, can move a byte. Matching a function often means finding the one way of writing it that the original author happened to choose.

Proving it

The test is the whole file. The rebuilt source is compiled and linked with the original toolchain, and the result is compared with the binary from the tape by checksum. Two details are allowed for and stated wherever they apply:

  • Some tape copies are missing the zero padding that the linker adds at the end of the file, so the comparison covers the length on the tape.
  • Rogue 5.4's main.obj survives with its symbols stripped, so the rebuild is stripped the same way before comparing.

The toolchains themselves are tested too. The 2.8BSD compiler was proven by running the original 1981 compiler, assembler and linker under a PDP-11 simulator and checking that they produce the same output as the ported versions. More on that under the toolchains and the simulators.

A matching file is not quite the end. The rebuilt Rogue 3.4 was also played against the original with the random number generator seeded the same way: 28 of 28 opening dungeons were identical, and more than 350 turns of play ran without any difference.

Results

Binary Result
Rogue 3.4, 2.8BSD PDP-11 Byte-identical
Rogue 3.6, 4.1BSD VAX Byte-identical
Rogue 3.6, 2.9BSD PDP-11 Byte-identical
Rogue 3.6, 2.9BSD PDP-11, overlaid Code and data identical; 49 symbol table entries differ
Rogue 4.22, 4.1aBSD VAX Byte-identical
Rogue 5.2, 4.1BSD VAX Byte-identical
Rogue 5.3, 4.2BSD VAX, object file Byte-identical, including symbol and string tables
Rogue 5.3, 4.2BSD VAX, installed game Byte-identical
Rogue 5.4, 4.3BSD VAX, main.obj Identical after stripping, with two small patches applied after compiling

The remaining differences are reported as they are. For Rogue 5.4, two spots still need patching by hand after the compiler runs: the exit from do_move() and six bytes of register choice in command(). Closing them probably needs a slightly different build of the 1985 compiler than the one we have.