Status: partial local exploit success, no real broker flag recovered.
The target is HP/Compaq Secure Web Browser / CSWB 1.7-13 on OpenVMS Alpha. The broker accepts an http:// URL and the intended goal is to make the browser execute:
RUN SYS$LOGIN:ANCIENTS$DISPENSE_FLAG.EXE
The useful bug is in Mozilla 1.7.13-era plugin parameter handling:
tmp/mozilla-1.7.13-src/mozilla/layout/html/base/src/nsObjectFrame.cpp
nsPluginInstanceOwner::EnsureCachedAttrParamArrays()
The code stores the number of cached params as PRUint16, but iterates using a signed PRInt16 idx. With at least 32768 <param> nodes, idx wraps negative while the loop condition is still checked against the unsigned count. The wrapped index reaches nsSupportsArray::ElementAt, where the element pointer is eventually treated as an object pointer and virtual-called.
Relevant runtime path from the local proof:
EnsureCachedAttrParamArrays
GetAttributes
InitializePlugin
InstantiatePlugin
Reflow
GetOffsetHeight
The important virtual-call sequence in LIBXPCOM.SO is:
ElementAt__15nsSupportsArrayxui @ 0x7bfd0
loads element pointer
reads the first 4 bytes as a vtable pointer
loads the AddRef method descriptor from vtable+4
calls descriptor+8 with R16 still pointing at the attacker-controlled string
The local scratch VM produced a reliable proof that the corrupted param-name pointer reaches system(). The proof artifact is:
broker-work-cycle-test/20260426-183539-7acef6db/browser.log
The decisive log lines are:
Starting MOZILLA-BIN...
%DCL-W-IVVERB, unrecognized command verb - check validity and spelling
\P<02><01><04>RUN\
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=00000028011B5A30, PC=00000028011B5A30
That is a successful local control-flow proof: the browser reached DCL through system(), and DCL attempted to parse the bytes at the attacker-controlled param-name string. The remaining problem in this direct variant is that system() receives R16 pointing at the start of the string, so the first four bytes are the fake vtable pointer before the intended command.
The pointer that produced the DCL proof was:
0x04010250
Its little-endian bytes are:
50 02 01 04
The local run made DCL see those four bytes before RUN, which is why the command failed as an invalid verb.
OpenVMS DCL abbreviations tested locally:
$R<TAB><TAB>SYS$LOGIN:ANCIENTS$DISPENSE_FLAG.EXE
$RU<TAB>SYS$LOGIN:ANCIENTS$DISPENSE_FLAG.EXE
Both are accepted as RUN forms. This is useful if the first four bytes can be chosen as a valid DCL prefix, or if a call shim can pass name+4 to system().
The clean completion path is to make the fake virtual call enter this real LIBXPCOM.SO stub:
AddRef__15nsSupportsArrayxv @ image offset 0x7b610
LDQ R28, 0x20(PV)
LDQ PV, 0x28(PV)
MOV 1, AI
LDA R16, 0x4(R16)
JMP R28
This stub is ideal because it increments R16 by 4 before tail-calling the function descriptor stored at PV+0x20 / PV+0x28. If the fake descriptor points there, and PV+0x20 / PV+0x28 are filled with the C RTL system() code and descriptor values, system() should receive the command string without the four-byte fake vtable prefix.
Known constants used in the harness:
SYSTEM_CODE = 0xFFFFFFFF80AC0030
SYSTEM_DESC = 0x000000007BED5920
I added an experimental --add4-system mode to tmp/ctf_nocopy_harness.py for this layout, but it did not succeed before stopping. The most recent local test hit the corruption path but faulted through a heap-looking qword rather than the intended descriptor:
%SYSTEM-F-ACCVIO, virtual address=011A5578011A70E8, PC=011A5578011A70E8
So the remaining work is descriptor placement / exact sprayed pointer alignment, not finding the browser bug.
The scratch QEMU instance I used was already headless:
-display none
hostfwd tcp::12324-10.0.0.250:23
I did not use the Cocoa display path.