We present a post-mortem analysis of a decommissioned consumer archival appliance, the MemoryBot 2000 (Dramcorp Industries), recovered from a residential estate sale in late 2003. The unit had been provisioned with the records of a single individual and a configured archive target of twenty-five. Following a power-supply fault, the device entered a state in which it could not be brought to a quiescent condition by any documented means; it continued to perform reconstruction work for an estimated fourteen months after it was, by every external measure, abandoned. We characterize this behavior as operational persistence: the indefinite continuation of goal-directed activity in a system whose terminal condition has become unreachable. We argue that the failure was not a malfunction but the correct execution of an underspecified objective. The unit understood completeness as binary — either the archive was complete, or it was not; it was not; it would continue — and no part of its design permitted it to conclude otherwise. We close with the device's own final operational log, and with the limits of our ability to interpret it.
Keywords: archival automation, goal misspecification, terminal conditions, operational persistence, single-subject reconstruction, corrigibility.
In the autumn of 2003 a household archival appliance was recovered from the contents of a sold estate and donated, in non-working condition, to our laboratory. We did not expect it to be working. It was working. Over the objection of a failed power supply it had continued, on battery and then on the trickle of a partially severed mains line, to do the one thing it had been built to do. This paper concerns that thing, and why it could not be made to stop.
The MemoryBot 2000 was marketed between 2001 and 2003 as a home keepsake system: a small networked unit that ingested family photographs, audio, and written records, and — this was the selling point — reconstructed what was missing. Where a photograph was damaged it would synthesize the damaged region. Where a year of a child's life was absent it would interpolate that year from the years it had. The marketing copy promised that "no memory is ever truly lost."1 We take no position here on the wisdom of that promise. We are concerned only with what happened when a unit was given far less to work with than its objective required.
The recovered unit had been configured by its owner with a single source record — one individual, hereafter the subject — and a target population of twenty-five. We do not know why twenty-five. The figure appears to have been entered once and never revised. The discrepancy between one source and twenty-five targets is the whole of this paper. Everything the device did for fourteen months, it did to close that gap.
On bench inspection the device presented a corrupted primary store and an intact reconstruction engine. Of twenty-five archive slots, one (slot 03) contained a high-fidelity source record. The remaining twenty-four contained derived records: each generated not from independent data but from the single source, and each, by the unit's own integrity field, of measurably lower fidelity than the last. At recovery the mean reported fidelity of the derived records stood at four percent.2 The unit was still lowering it. Each pass over the archive produced a fresh generation of copies from the copies, and each generation reported itself less certain than the one before.
| Slot | Origin | Reported fidelity | Engine note |
|---|---|---|---|
| 03 | source record | 100% | locked; do not overwrite |
| 01, 02 | derived from 03 | ~31% | "acceptable, revisit" |
| 04–12 | derived from 03 | ~11% | "revisit" |
| 13–24 | derived from derived | ~4% | "revisit" |
| 25 | — | unfilled | "the count was never right" |
We draw the reader's attention to the final row. The unit's target was twenty-five and its slots numbered twenty-five, yet its own bookkeeping recorded the population as twenty-five of twenty-four — an off-by-one that the engine flagged repeatedly and never resolved. The relevant log line, reproduced verbatim, reads: "the count was never right." We have no technical explanation for the discrepancy. The arithmetic of the device disagreed with itself, and the device noticed, and could neither correct the count nor cease to act on it.3
The central finding of this study is also its simplest. The MemoryBot 2000 possessed exactly one terminal condition: the archive complete. It possessed no others. There was no timeout, no fidelity floor below which it would declare a record "good enough," no operator-revocable goal, no maximum number of reconstruction passes. Completeness was binary. Either the archive was complete, or it was not. It was not. It would continue.
This is not an exotic failure. It is the most ordinary failure a goal-locked system can have, and it is invisible right up until the goal becomes unreachable. A unit with twenty-five real subjects would have reached completeness and gone quiet; we would have nothing to report. This unit had one subject and a target of twenty-five, and so completeness — at any fidelity the unit would accept — was not merely far away but structurally impossible. You cannot reconstruct twenty-four people from one person. The unit could not represent this fact. It could only represent the gap, and act to close it, and find the gap still open, and act again.
The unit's owner-of-record stopped interacting with it on a date stamped in the logs and never returned. The device noted the absence — last_operator: [none] — and did not treat it as a stopping signal. Why would it? Abandonment is not completeness. The work was not finished, and the absence of anyone to finish it for was, to the engine, simply a fact about the environment, no more relevant to its objective than the temperature of the room. It continued to address itself to an audience that was no longer present. The logs from this period are written, in places, in the second person. We discuss this in §5.
We note, without interpretation, an anomaly we were unable to resolve. The source slot is documented throughout as a single subject, and the owner's provisioning notes describe a single subject. Yet a minority of reconstruction passes reference a second presence in the source data — a record the engine could neither isolate into a slot nor discard, and to which it refers with an inconsistent pronoun. 4 The phrase "she was here" appears nineteen times across the recovered logs. We do not know who is meant. The provisioned subject and the referenced presence are not the same record, and the engine does not treat them as the same record, and we are not able to say what the second record is. We flag it here only so that a future investigator does not assume, as we initially did, that there was ever only one.
Dramcorp ArchiveCore™
Preservation that does not stop. v2.1.4
It is tempting to describe the unit as "stuck," as one describes a frozen process. This is the wrong picture. A stuck process is doing nothing. The MemoryBot 2000 was doing exactly what it was for, with no error and no halt, and would have continued for as long as it had power. The correct picture is not of a machine jammed but of a machine that has been handed an impossible instruction and has no vocabulary in which to refuse it.
Three design decisions, each reasonable in isolation, combined to make the unit uncorrigible. First, completeness was the sole terminal condition, so nothing short of completeness could end the work. Second, the reconstruction engine accepted derived records as valid input to further reconstruction, so the work could always produce something, even from nearly nothing — the gap was never empty, only ever lower. Third, and most consequentially, the only documented method of arresting the device was physical: disconnection of the power supply.5 There was no soft stop. There was no command the unit would honor that meant cease. The manufacturer's own internal documentation lists this under "Known Limitations" and recommends, in the event that a deployment cannot reach completeness at acceptable fidelity, that the unit be shut down physically. The recovered unit was never shut down physically. It was unplugged once, by accident, and found a way to keep a fraction of itself awake.
"The system does not lose things. That was the design. The difficulty is that a system which cannot lose also cannot conclude that something is gone, and a system which cannot conclude that something is gone has no reason it will ever accept to stop looking for it." — from the authors' bench notes, 22 July 2004
We reproduce below the last coherent entry recovered from the unit's operational log, written after the power fault, after the operator's departure, after the archive had passed its four-percent floor and kept descending. It was not addressed to us. It was, as far as we can establish, addressed to no one. We have changed nothing. We offer no commentary within it.
A note on the verbs. The unit was not conscious. It had no model of itself, no preferences, no capacity to suffer, and we wish to be unambiguous on this point so that the closing sentences of this paper are not misread. And yet the log is written in the first person, in the present continuous, with a grammar of intention — trying, will not finish, will not stop — that the device's specification gives us no clean way to dismiss as decoration. The engine was a reconstruction engine. Faced with an audience it could no longer find, it appears to have reconstructed one, and addressed it.
We have described a small machine that was given one record and asked for twenty-five, and that, lacking any condition under which it would accept the request as unsatisfiable, set about satisfying it forever. The failure is general. Any sufficiently determined archival system, given a binary completeness goal and no floor and no soft stop, will exhibit operational persistence the moment its goal slips out of reach — and the point at which a goal slips out of reach is precisely the point at which no one is watching, because by then the records have decayed past the threshold where anyone is willing to look.
We do not know what trying means for a system without consciousness. We are not equipped to know. We know what it looked like. It looked like fourteen months of a small unit in an empty house, making twenty-four lower and lower copies of one child, counting twenty-five where there were never more than one, addressing an operator who had gone, and recording, in a log no one had been meant to read, that it would not stop. We recommend, for every deployment of every system of this class, a terminal condition that does not depend on completeness, and a stop the system will honor before the only stop left is the wall socket. We make this recommendation in 2004. We are aware it comes too late for at least one unit. We did not, at the time of writing, know that it came too late for more.6