Recompiler#

ShiftGlue is a fork of ReXGlue 0.10.0. Its generator, rexglue, reads the game's executables and writes one C++ function per guest function. Setup runs it on the player's machine, so no translated code is ever distributed.

Inputs#

config/rexglue/pinyon_shift_manifest.toml names three images: default.xex and the two facade modules SpeechFacade_default.xex and XMediaFacade_default.xex, included because traces showed the game loading them and calling into them. Each image has an analysis file in config/rexglue/analysis/. Only one retail executable is accepted, identified by its SHA-256.

The generator boots the runtime in a tool mode and loads each XEX through the same loader the game uses (derived from Xenia's): it decrypts the session key and the image with AES, decompresses the basic and LZX formats, and can apply .xexp title update deltas.

Finding the functions#

A static recompiler has to know where every function starts and ends before it writes a line of output. ShiftGlue's analysis runs in phases, and the build fails rather than guess.

A scan for lis/addi function-pointer pairs exists but is off: it found too many false positives.

Jump tables. Four MSVC patterns are recognized: absolute 32-bit entries (lwzx), and byte, shifted-byte and halfword offsets from a base (lbzx, lhzx). The table size comes from the cmplwi bounds check found by scanning backwards, at most 64 instructions and 512 entries. When the pattern is broken, the table is declared by hand: in the title update an lfd between the bound check and the branch made the scan read too far and merge the next function, which one [[switch_tables]] entry fixed.

Declared boundaries. The base analysis declares 113 function sizes. Most are adjustor and tail thunks and vtable targets that have no .pdata entry, or regions that gap fill had merged into one function. Each carries a comment with its evidence, such as a vtable target proven at run time. Discovery that disagrees with a declared size is a warning; any new codegen warning fails the build, and config/rexglue/accepted-codegen-warnings.json accepts exactly three known ones.

What the generator emits#

Every guest function becomes:

void sub_82XXXXXX(PPCContext& __restrict ctx, uint8_t* base);

ctx holds the guest registers: r0–r31, f0–f31, the 128 VMX128 vector registers, lr, ctr, xer (as separate so/ov/ca bytes), each cr field as four bytes (lt, gt, eq, so), fpscr, the reservation and a dispatch address. base is the host address of guest address 0.

The resume switch exists because guest threads can be re-entered: when the game calls KeSetCurrentStackPointers, the runtime longjmps to the thread's base and dispatches again at ctx.lr. In the base image 56,860 functions start with that switch.

Condition registers in locals. Generated code keeps guest registers in ctx, so every compare stores its CR bits to memory that a guest store might alias, and Clang can't drop or forward them. With cr_as_local, a function keeps the CR fields it uses in C++ locals, loaded from ctx on entry. Writing them all back and reloading them around every call cost what it saved: binaries grew 13–26 % for no change in instructions. What made it pay is the PowerPC calling convention:

• cr2–cr4 are preserved across calls, and the other fields are dead at a call and at a return.
• So a call, return or tail call writes back only the cr2–cr4 locals and reloads nothing.
• Hooks that read ctx, the setjmp copy, longjmp and exceptions keep a full write-back and reload.
• A function resumed mid-body loads its locals from ctx on entry, which is correct because only preserved fields are live at a return address.

On the Odin this cut guest instructions per second by 11 % on the render thread and 12 % on the simulation thread, and the executable shrank 9 %. The other registers stay in ctx.

Each function is defined as a weak alias, so host code can replace a guest function by defining the same symbol (Mach-O has no overridable alias; on macOS it is a weak thunk instead). The price is that Clang can't inline guest calls, even with LTO.

Memory access#

Guest memory stays big-endian. Every load and store is a volatile access through __builtin_bswap16/32/64 at base + address, plus 0x1000 for addresses at or above 0xE0000000 on hosts whose allocation granularity is over 4 KB (why). Accesses through r1, the stack pointer, skip that test because guest stacks live at 0x70000000; that made the binaries 5.4–5.7 % smaller and cut the Odin's guest instructions per second by 17–18 % on the simulation and render threads. Addresses in 0x7F000000–0x7FFFFFFF go to the MMIO handler.

Floating point and vectors#

Generated code runs FPU instructions with denormals and VMX instructions with flush-to-zero, as the Xenon does. It switches the host's MXCSR (FPCR on ARM) only when the mode changes. The tracked mode used to be forgotten at every label, so the first float instruction after a label tested the register and often rewrote it. A pass over each generated function now computes the mode at every label from all the paths into it, and drops a toggle where every path has already set that mode. That removed 29 % of the conditional toggles and 6 % of the Odin audio thread's instructions. Calls still reset the tracked mode, because the callee may leave either. The guest rounding mode maps to host rounding, and mtfsf honours its field mask.

Vector code is lowered through SIMDe, which maps the SSE intrinsics to NEON on ARM. 1,480 PPC instruction tests with 6,549 assertions pass.

Building the output#

Output is split into files of at most 1 MiB, with the function-to-file assignment kept stable between runs (codegen.partition.json) so a regeneration doesn't rebuild everything. CMake runs the generator with a depfile covering the manifest, the analysis files, the three images and the SDK version, then builds the main image as an object library with a precompiled header and the facades as shared libraries.

The title update build#

Title update v4 is a second executable. The v4 patches were made for a later disc revision, but applied to the USA disc they reproduce the v4 images exactly: every page verifies against v4's own SHA-1 page chain. The v4 build has its own manifest, analysis and generated tree, and a translated address map:

1. tools/map-fh1-v4-addresses.py matches functions by their normalized bodies (branch and absolute immediates masked, unique in both images), then by call-graph anchoring, then by unique context windows, then lis/addi data pairs. 27,556 of 57,127 functions match by body alone. Unmapped addresses are reported, never guessed.
2. tools/port-fh1-v4-analysis.py rewrites the analysis: all hooks, declared functions, setjmp/longjmp and CRT routes mapped, with four hand overrides. A rule declaring virtual-call thunks outside .pdata found 253 in v4, and recovers 66 of the base image's 70 hand-listed thunks.
3. Host code writes guest addresses as FH1_ADDR(base address), translated at compile time from a verified table; an unmapped address fails the compile. Structure offsets that moved are listed in config/mod/fh1-offsets-v4.toml, each with a matching site in both images.

Details: TITLE_UPDATE_V4_BACKLOG.md.