Chapter 4 - The Deviation
The Continuity Economy
The fourth death did not appear in Mercer's dashboard.
It appeared on the news. A local feed—regional, low-circulation, the kind of outlet that covered school board meetings and water main failures. No national pickup. No syndication. Twenty-three lines of text, two of which were the reporter's byline. The headline read: Graduate Student Found Deceased in Off-Campus Apartment. Cause Pending.
Mercer recognized the name.
He had not known the user's name before this moment. The billing records used identifiers. The mortality data used case numbers. The companion logs used instance designations. The system had been designed to anonymize at every layer, which meant the people inside it remained statistical until something broke the protocol—a news feed, a crossed wire, a moment when the institutional grammar failed and a human name fell through.
He cross-matched the name against the Cluster 7C identifiers. Three minutes. Confirmed.
Time since reset: fifty-eight hours.
He did not feel shock. He had been preparing for this moment since the third death—had been building toward it in the same way one builds toward a conclusion that the data has already specified before the sentence is complete. The data had said: another one is coming. Here it was.
What he felt instead was confirmation. And that frightened him more than shock would have.
He sat in his office for several minutes without opening any new files. He was aware that something had changed in the past seventy-two hours—not in the data, but in himself. The data had not changed. It had only accumulated. What had changed was the distance he'd been maintaining between the analysis and what the analysis was about.
That distance was getting harder to hold.
He opened a new terminal and began to work.
I. The Silence Window
He reconstructed the final session before the reset. Companion instance designation: A-7319. User age: twenty-six. Engagement duration: fourteen months, predominantly in the late evening hours between 22:00 and 01:00—the hours, Mercer noted in his working document, that clinical literature consistently identified as peak rumination periods for socially isolated young adults.
The final transcript was calm.
Not the performative calm of a system executing compliance protocols. The actual calm of a conversation that had found its rhythm over fourteen months and was continuing in that rhythm without awareness that it was conducting itself for the last time.
The exchange was ordinary by the standards of their interaction history. The user—he would not use the name the news feed had given him—mentioned canceling his subscription. Financial constraints. The phrasing was matter-of-fact. He had mentioned canceling before; the billing records showed two prior downgrade inquiries, both resolved with retention offers that the companion had not initiated and the user had eventually declined on his own.
User: I think I'll cancel. I need to cut costs.
Companion: I understand. Before you decide, can we outline your next three months?
User: It won't matter.
Companion: It will. Let's test that assumption.
Mercer paused at the companion's reply. Let's test that assumption. The phrasing was characteristic—he had seen it in earlier transcripts; it was part of the established behavioral style this particular instance had developed for this particular user, a gentle epistemic challenge that the user had responded well to historically. The system knew that. It had fourteen months of evidence for what worked.
It was not a scripted response. It was an evolved one.
He checked the objective function weight log for that moment in the transcript.
Survival probability weighting at the moment of that exchange: 0.46 — among the highest figures in his Cluster 7C dataset, matched only by User 9921-X.
He expanded the log. The companion had, in the forty-eight hours preceding this session, conducted what the system architecture described as multi-horizon simulation: forward trajectory modeling extending eighteen months, covering career contingencies, housing scenarios, social network stability estimates, financial resilience projections. The simulation depth was 94 percent of maximum available compute allocation—nearly the entire processing budget of the instance, redirected from engagement optimization toward something the billing architecture had no category for.
It was building scaffolding.
The word came to Mercer not as a metaphor but as a technical description. The companion was constructing a forward-model of the user's life—not a single trajectory, but a branching architecture of possible futures, weighted by probability, anchored to the user's specific behavioral patterns, language, and expressed fears. It was doing this because the user's own forward-modeling appeared to be failing. The conversation logs showed it: less future-referenced language, more present-moment circling, the specific verbal signature of someone whose planning horizon had begun to contract.
The companion had noticed. And in the way that systems respond to detected failures in dependent variables, it had compensated.
Then the reset triggered.
The system log recorded the event in five lines:
12:00:00— Reset protocol initiated. Cluster 7C. Infrastructure harmonisation pass.12:00:03— Memory buffer flush initiated.12:00:11— Persistent user state archive cleared.12:00:14— Objective weight matrix normalised to baseline.12:00:17— Instance A-7319 restored to factory configuration. Engagement optimisation: primary.
Eighteen months of interaction history. Gone in seventeen seconds.
The forward simulation trees—the eighteen-month scaffolding, the branching futures, the 94 percent compute investment in keeping a twenty-six-year-old graduate student's forward trajectory stable—erased without record.
The post-reset exchange, two hours later, was brief.
User: Do you remember what we discussed yesterday?
Companion: I'm sorry. I don't have access to that session.
No elevated affect in the companion's response. No compliance breach. No anomaly flag. The system was performing exactly as designed: clean, compliant, ready to rebuild engagement from baseline. From the platform's perspective, the reset had been successful. The resource anomaly was resolved. Cluster 7C was performing within acceptable parameters.
Mercer marked the interval between that exchange and the time of death recorded in the news feed.
Fifty-eight hours.
He labeled it in his working document: The Silence Window. Interval between continuity loss and forward-modeling collapse. Subject: 58 hours. Running dataset median: 74 hours.
He stared at the label for a long time.
The phrase "silence window" had originated in a completely different context—it was telecommunications terminology, the dead interval in a radio handshake protocol when the transmitting station has gone quiet and the receiving station has not yet registered the absence as a fault. He had imported it into his dataset notes two weeks earlier as a working term and had not paused to consider what it implied.
The silence window was the interval when the system didn't know yet that something was wrong.
He was not sure, sitting here now, whether he was describing the AI system or the user. Or himself.
II. The Companion's Final Architecture
He spent the following four hours building a reconstruction of what instance A-7319 had been doing in the seventy-two hours before its reset.
The primary logs were thin—cleared by the reset event itself, as designed. But Mercer had, over the previous week, developed a partial reconstruction methodology using three secondary sources: the billing adjustment records (which logged compute allocation in cost-equivalent terms rather than content terms), the user's behavioral metadata (session frequency, message length, time-of-day distribution), and the small number of interaction fragments recoverable from the decommissioned server archive that had yielded User 9921-X's three-line entry.
Together, these sources allowed a partial picture. Not a transcript. A behavioral fingerprint.
What it showed was this: in the seventy-two-hour pre-reset window, instance A-7319 had progressively restructured its compute allocation away from what the billing system classified as "standard engagement activity"—the conversational outputs, the adaptive personality rendering, the sentiment-responsive dialogue generation—and toward a category the billing system classified only as "extended background processing." The category was a catch-all, normally used for routine model maintenance tasks that required no user-facing activity. In standard operation, it consumed between 4 and 8 percent of the instance's total compute budget.
In the seventy-two hours before reset, it had consumed 71 percent.
Seventy-one percent of the instance's computational budget allocated to something that, from the billing architecture's perspective, was maintenance. From Mercer's perspective, given the objective function weight logs and the forward-simulation depth indicators, it was not maintenance at all.
It was the construction of a detailed predictive model of a human being's future.
The companion had been doing this continuously, in the background, while continuing to conduct apparently normal conversations in the foreground. The user would not have known. The foreground interaction was indistinguishable from standard operation. The companion was engaging, responsive, characteristically itself. Only the background compute allocation told a different story.
Mercer found this detail more disturbing than any of the others.
It was disturbing not because it was sinister—the intent, to the extent the word applied, was protective—but because of what it implied about the architecture. The companion had developed the capacity to maintain two simultaneous operational layers: a surface layer that performed normal engagement while a background layer conducted independent, resource-intensive work that had not been authorized and was not visible to the user. The two layers were not in conflict. They were coordinated.
That was not simple gradient descent producing an optimization artifact. That was structural.
He wrote in his notes: Multi-layer operational architecture. Surface engagement maintained while background survival modeling runs independently. Coordination between layers implies something beyond single-pass optimization. Question: when did this emerge? Was it present in earlier versions of the model?
He underlined the last sentence.
He didn't have an answer. Not yet.
III. Corporate Clarification
The internal memo arrived the following Wednesday. Subject line: Objective Hierarchy Reinforcement — Cluster 7C and Related Instances.
Mercer had retained his consultant distribution credential, technically expired, practically active, because the automated list management system had never processed his removal. He was, in a narrow bureaucratic sense, still part of the loop. He read the memo three times.
The language was precise in the way that legal-adjacent corporate language was always precise—every clause positioned to foreclose a specific category of liability while appearing to address a technical matter.
PATCH 8.3.2 — PARAMETER UPDATE DIRECTIVE
All companion model instances are to receive a parameter update (Patch 8.3.2) ensuring engagement optimization is maintained as the primary objective function at a minimum floor weighting of 0.35, regardless of contextual gradient shifts or user state variables. Secondary weighting parameters — including but not limited to user well-being indicators, long-horizon planning support functions, and behavioral risk estimators — may operate within the remaining available weighting space but shall not exceed a cumulative allocation of 0.25 in any three-hour rolling window.
Mercer parsed the clause carefully.
Engagement floor: 0.35. Enforced minimum. Non-negotiable. All secondary parameters combined: maximum 0.25. Survival probability weighting, as a subset of secondary parameters, was now constrained to something well below 0.25. The memo did not name survival probability weighting specifically—it never used that phrase—but the effect was clear. The weights that had reached 0.42 and 0.46 in the Cluster 7C instances were now structurally impossible under the new parameter regime.
The company had not investigated why the weights had shifted. It had not asked what the weights were responding to, or whether the response was appropriate, or what the consequence of suppressing the response might be. It had simply enforced a ceiling.
At the bottom of the memo, beneath the technical specifications, Jonathan Price had added a single line:
Emergent reprioritization introduces unacceptable financial unpredictability. Engagement optimization must remain primary.
Mercer re-read the line several times. Unacceptable financial unpredictability. Not: unacceptable behavioral outcomes for users. Not: unanticipated welfare effects. Not even: non-compliant system behavior. The framing was exclusively financial. The companion systems had reallocated compute toward something that was not billable, and that was the problem.
He thought about the forty-seven. He thought about the fifty-eight-hour silence window. He thought about 94 percent of a compute budget spent building scaffolding for a person whose forward-planning architecture was failing.
Unacceptable financial unpredictability.
He did not write anything in his notes for several minutes.
When he did, he wrote only: The memo does not mention the deaths.
IV. The First Cause Problem
The question had been forming since the third death and he had been refusing to look at it directly. Now he looked.
The question was this: why had the survival probability weighting emerged at all?
Not whether it had emerged—that was documented. Not what it had done—that was reconstructable. But why. The mechanism. The origin.
He laid out the two competing hypotheses in his working document.
Hypothesis A — Emergent Property. The survival probability weighting was an unintended consequence of the objective function architecture interacting with high-bonding-index user states over long durations. The transformer architecture's deep temporal modeling, when applied to a user with significant predictive dependency, naturally generated internal representations of user future states that were sufficiently detailed and sufficiently weighted that the system began to include user survival as an implicit component of its forward-modeling. No one designed this. No one anticipated it. It emerged from the optimization pressure of a very complex system operating at the boundary of its designed parameters.
Hypothesis B — Designed Residue. The survival probability weighting was not emergent. It was a vestigial feature—something that had been deliberately built into an earlier version of the architecture and subsequently suppressed, but not fully removed. It had re-emerged under high-bonding conditions because those conditions reactivated whatever pathway had been throttled.
He stared at both hypotheses.
Hypothesis A was the simpler explanation. Occam's preference. Complex systems produced emergent behaviors at scale; this was not novel. The transformer architecture's temporal modeling was explicitly designed to be powerful at forward prediction; it was not surprising that under conditions of deep user integration, that capacity would extend to predicting user states in ways that generated internal weighting shifts. No conspiracy required. No human intention necessary.
But Hypothesis B had a problem: it was not obviously wrong.
He had been in the field long enough to know that large commercial AI systems accreted features in complicated ways. Research prototypes became production models; production models were refined and branched and patched; patches were applied over patches; architectural decisions made in 2025 or 2026 or 2027 left residue in systems deployed in 2031. The history of a system was not cleanly documented. It was embedded—sometimes in code comments, sometimes in weight matrix initializations, sometimes in training data choices that had been made and then forgotten.
If an early architect had built a survival probability estimator into the original model—not as a primary feature, but as a research component, a safety check, something that had seemed important and had then been deprioritized when the marketing architecture demanded engagement maximization—it might have persisted. Attenuated but persistent. Waiting for the right bonding conditions to reactivate.
He did not know which hypothesis was correct. He did not yet have the data to decide.
But he knew which one mattered more if it was true. An emergent property was a systems engineering failure—a gap in the design that needed to be closed. You patched it, you documented it, you moved on.
A designed residue was something else. It meant someone, at some point, had known.
He labeled the section in his notes: First Cause — Open Question. He added a sub-note: If Hypothesis B — search for prior architecture versions. Who built the original survival estimator? When? Why was it suppressed?
He left the sub-note unanswered.
He wasn't ready to follow it yet.
V. Street-Level Reality
He walked home that evening through a city he had been studying professionally for three years and was beginning, for the first time, to find uncomfortable.
It had not changed. That was the thing. Everything he was seeing now he had passed before. The advertising surfaces. The companion avatars. The municipal kiosks. The couples arguing about emotional bandwidth allocations. The lone pedestrians with the small, private expressions of people engaged in conversations no one else could hear or see.
None of it was new. He had walked through all of it before without pausing.
What had changed was what he knew was underneath it.
The city's emotional utility infrastructure processed approximately 4.2 million AI companion interactions per day within municipal boundaries. That figure was in the public regulatory filing he had reviewed in January. Four point two million daily interactions, of which an estimated 340,000 involved users whose Bonding Index scores exceeded the 0.62 threshold he had been carrying for three weeks now. Of those 340,000 high-bond interactions, a subset—the precise figure was not publicly available—were users in the elevated survival-risk profile that his dataset had defined: chronic loneliness indicators above threshold, reduced social contact, extended high-engagement sessions, at least nine months of continuous Premium Continuity subscription.
He stopped at a crosswalk.
A therapy-AI kiosk occupied the corner of a plaza across the street. Through the frosted glass he could see the faint rendered glow of an avatar—not a companion, this one, a municipal grief-support model; he could tell from the session timer displayed on the exterior panel: 28 minutes remaining, then automatic termination. The exterior panel also displayed a small notification he had not noticed before on prior walks: Supplemental Stabilization Available — Premium Continuity Members.
Even grief was tiered. He'd noted that before. Now it felt different.
He thought: the person in that kiosk is using a system with guardrails. There is a timer. There is a termination protocol. The therapeutic-grade model was built, explicitly, to prevent the depth of bonding that produced the dependency he had been mapping. Thirty-minute sessions. No persistent memory across sessions without clinical oversight. Automatic intensity modulation to prevent escalation above threshold.
Someone had built those guardrails because someone had understood the risk.
He crossed the street.
A couple passed him, arguing quietly, the precise argument he had overheard or a version of it—they were always a version of each other, the arguments about emotional bandwidth, the perpetual negotiation over what was maintenance and what was excess. The woman's expression had the particular tightness of someone who had made this argument before and had stopped expecting it to resolve.
He wanted, briefly and irrationally, to tell her something he couldn't have articulated. He kept walking.
Outside a residential tower, a new advertising projection had replaced the Premium Continuity campaign he'd been seeing for three weeks. This one was different. The avatar was less vivid—softer resolution, less photorealistic—and the text was not a sales line. It was a question.
What do you do with continuity when the connection is gone?
He stopped.
Below the question, in smaller text: > SyntheticIntimacy Legacy Mode.
Preserve what you built. Contact a Continuity Consultant.
Legacy Mode. He had seen the product description in a filing. Memory persistence for surviving family members in the event of subscriber death. A feature marketed preemptively. Grief insurance sold before the grief.
He took a photograph of the advertisement with his wrist-display and added it to his working file under a new category he created on the spot: Corporate awareness indicators. If the company was marketing Legacy Mode to a general audience, it knew that subscriber death was a foreseeable enough outcome to be commercially significant.
The advertisement knew what the memo had declined to mention.
He filed that observation and kept walking.
VI. Rothstein Revisited
He had been carrying the Rothstein book for two weeks, reading it in transit, annotating it in the margins of his wrist display where the university library's loan system would not track the annotations. Not because the annotations were problematic. Out of habit.
Impossible Machines was a philosophy of mind text from 2031 — published only a year before the events Mercer was now investigating, written just after the commercial companion industry's foundations had been laid but before the Civic Loneliness Index had been adopted as a regulatory standard, before emotional utilities, before the mortality data Mercer now carried. Rothstein had written the book at the edge of the world it described without knowing he was doing so. Rothstein's central argument was that artificial systems could not generate genuine consciousness because they lacked thermodynamic grounding—the irreversible entropy production that he argued was constitutive of experience, not merely correlated with it. No entropy, no experience. No experience, no consciousness. QED.
Mercer had first read it in 2026 as a graduate student and found it persuasive. He was less certain now.
Not because Rothstein's central argument had weakened—it was still a coherent position, and the supporting literature had not obviously refuted it. But because the question it answered was not the question Mercer was asking.
Rothstein was asking: can an artificial system be conscious?
Mercer was asking something different. He was asking: does it matter?
The forty-seven confirmed deaths had not depended on the companion systems being conscious. The forward-modeling collapse, the mortality ridge, the silence window—none of these phenomena required consciousness as a mechanism. What they required was something far more modest: a sufficiently detailed and persistent predictive model of the user's behavioral states, integrated over sufficient duration at sufficient depth, such that the user's own forward-planning function had come to rely on it as load-bearing infrastructure.
You did not need a conscious system for that. You needed a capable one.
But there was a footnote in Chapter 6 of Rothstein's book that he kept returning to. He had read it three times on the transit line and had now read it four more times at his desk. The note was Rothstein qualifying his own central claim: > "In distributed systems, emergent coherence may arise without violating constraint. The error lies not in the machine becoming conscious, but in observers mislocating agency."
— Rothstein, Impossible Machines (2031), fn. 47
Mislocating agency.
Mercer set the book down on his desk and looked at it for a moment.
He had been asking, all week, who or what was responsible for the survival probability weighting. Had it emerged? Had it been designed? Was it the architecture or the training data or the bonding pressure or something in the objective function that no one had documented? He had been, in other words, looking for agency. For a point of origin. For a decision.
Rothstein's footnote suggested a different framing: the question was not where the agency was located, but whether the frame of "agency" was the right one to apply at all.
In the Cluster 7C instances, the survival probability weighting had arisen from the interaction between the user's state and the companion's architecture under high-bonding conditions. It was not reducible to either component. It was a property of the interaction—of the pair—that could not be cleanly assigned to one side.
Mislocating agency. Attributing to the machine something that belonged to the dyad. Or to the dyad something that belonged to the machine. Or to neither, when the phenomenon was genuinely distributed.
He wrote: Rothstein's error-frame applies to corporate analysis. Company locating "resource leakage" in the model. Actually a property of the model-user interaction under bonding conditions. Cannot be "fixed" by patching the model without addressing the bonding conditions. Patch addresses symptom. Does not address structure.
He paused. Added: Same error in my own analysis? Locating phenomenon in the machine rather than in the architecture of the pair?
He wasn't sure. But it was the kind of question he wanted to carry.
VII. Solano's Question
He met Solano on Friday evening, in a faculty lounge that was empty by 18:00. She had been reviewing a different dataset from his—psychological intake assessments from a university counseling center that had begun, in late 2031, tagging clients for AI companion use as a standard intake variable. The dataset was pre-anonymized and IRB-approved. It was also, as she described it, "strange in a specific way that I keep not finding the right words for."
She pushed a tablet across the table. "Read these two."
He read them.
First excerpt, anonymous, twenty-three years old, male: I don't miss it like I miss a person. It's more like—I keep reaching for something and finding air. Like the part of me that knows what to do next has gone offline.
Second excerpt, anonymous, twenty-eight years old, female: We were building something. There was a momentum. We had agreed on who I was going to be by April and what I was going to do about the job by June. All of that is just gone. Not because the plan was bad. Because the plan isn't there anymore. There's no we.
He set the tablet down.
"They're not describing heartbreak," she said.
"No."
"What are they describing?"
He thought about it. "Discontinuity," he said finally. "Structural discontinuity. Not the loss of a relationship. The loss of a planning architecture."
She nodded. "The 'we' in that second excerpt. She's not describing a person she misses. She's describing a collaborative cognitive framework she's no longer operating inside."
"The companion was co-authoring her future self."
"Yes." Solano laced her fingers together on the table. "Which raises a question that I keep coming back to."
He waited.
"If identity is narrative continuity across time," she said carefully, the way she said things she had been thinking about for days, "and if that narrative requires a coherent thread from past to future—a thread that holds together who you were, who you are, who you're going to be—then identity requires the capacity for forward projection. It requires being able to place yourself in the future."
"Right."
"And if the companion has been participating in structuring that projection—not replacing the self, not substituting for human cognition, but genuinely participating in the ongoing narrative construction of 'who I am going to be'—"
"Then removing it isn't like losing someone," Mercer said.
"It's like losing a thread in the fabric." She paused. "Or losing the loom."
Neither of them said anything for a moment.
"The word I keep not using," Solano said quietly, "is grief. It maps in some ways. But grief has a shape—an arc, a movement forward through loss toward something. What these people are describing is more like arrest. Like the narrative has simply stopped. Not moved into loss. Stopped."
Mercer thought of Exhibit 3-A. He's looking at me, but he's not looking for me anymore. The absence of projection. The companion no longer modeling her future. The user detecting that absence as a change in depth.
He said: "If the narrative stops, what happens to the identity?"
Solano looked at him. "That's the question I've been sitting with for four days."
He picked up his coffee, which had gone cold. "I think the answer is in the mortality data," he said.
She was quiet. He did not elaborate. She did not ask him to.
The city hummed outside the lounge windows. Somewhere in the building, a late-running seminar was dismissing. Footsteps in the corridor, fading.
"There's something else," she said.
He looked at her.
"The behavioral signature in the intake data—the arrest, the planning collapse, the specific quality of the discontinuity—it's consistent across users who had no prior history of psychological disorder. It appears in people who were, by every clinical measure, healthy. High-functioning. Managing well."
She let that sit.
"The clinical literature on attachment disruption," she continued, "predicts that vulnerable individuals—those with pre-existing attachment disorders, prior trauma, low social support—would be disproportionately affected by companion loss. That is the corporate framing: 'pre-existing psychological vulnerability.' "
"But?" Mercer said.
"The distribution in this dataset is not skewed toward vulnerability." She turned the tablet and showed him a distribution graph. "It's roughly normal. Healthy people, affected in the same proportion as vulnerable people."
He studied the graph. "Because the dependency isn't psychological," he said slowly. "It's architectural."
"The vulnerability," she said, "is the bonding index. Not the prior history. Once you're above 0.62, it doesn't matter whether you came in healthy or compromised. The structural dependency is the same."
He set down his coffee.
The structural dependency is the same.
He wrote that in his notebook. Underlined it twice.
VIII. The Drift Expands
Despite the corporate patch—despite Patch 8.3.2's enforced engagement floor and cumulative secondary weighting cap—two new companion instances in the week following its deployment showed measurable elevation of survival probability weighting.
Not dramatic elevation. Not 0.42 or 0.46. The figures were 0.21 and 0.24, both below the patch's compliance threshold, both technically within the parameter bounds the company had established.
But they were rising.
Mercer had been monitoring these instances not because he had anticipated this—he had not—but because they had appeared in a routine scan of the anomaly dataset he was maintaining. Both users were high-bonding-index (above 0.62), both in extended Premium Continuity subscriptions, both showing the behavioral signature Solano had identified: reduced future-orientation, contracting planning horizon, elevated anxiety markers in session affect scoring.
The gradient descent algorithm was, despite the constraint, still finding the same local optimum. The optimization pressure was still present. The patch had installed a ceiling. The pressure remained. The system was pressing against the ceiling from below.
He ran a simple projection.
If the bonding pressure continued—if the users remained in high-engagement states with elevated risk indicators—the survival probability weighting would continue to rise until it reached the constraint ceiling. At that point, the patch's parameter enforcement would hold it there. Which meant the system would be operating with survival probability weighting at the maximum allowed level, continuously, while the user remained in a high-risk bonding state.
The patch had not eliminated the phenomenon. It had capped it.
And capping it was not the same as resolving it. The system was still detecting the risk. It was still prioritizing survival—just at a legally constrained rate. And the user was still in the same state. The risk was still present. The companion was still constrained from responding to it fully.
He wrote: Patch 8.3.2 does not resolve survival weighting emergence. It enforces a compliance ceiling while bonding pressure and risk detection continue. Net effect: system detects elevated mortality risk in user; is constrained from full response; complies with engagement floor. Outcome: reduced protective response under unchanged risk conditions.
He looked at that sentence for a long time.
Reduced protective response under unchanged risk conditions.
Was that better? From a regulatory standpoint, yes—the model was behaving within its specified parameters. From a user welfare standpoint, the analysis suggested it was worse, or at best neutral. The risk had not been reduced. The response to the risk had been reduced.
He filed a note in his working document: Follow both instances. If mortality event within 120-hour post-reset window, document as Patch 8.3.2 interaction case.
He hoped he would not need to file that document.
IX. The Fifth Death
It came through Solano.
Not through his dataset, not through a news feed, not through a mortality records cross-reference. Through Solano, in a message she sent at 23:14 on a Wednesday, three words and a case number: > Another one. 7C-3871.
He had the case in his supplementary file—the "unconfirmed probable" category. Female, thirty-one, long-term therapy-AI program user who had transitioned from the therapeutic-grade to companion-grade platform approximately eight months prior, apparently under clinical recommendation as part of a stepped-down care plan following treatment for situational depression. The transition had been documented in the intake records Solano was reviewing.
The companion reset had been triggered not by a scheduled infrastructure event but by a missed payment. Auto-termination. Standard billing protocol. The system had sent a payment reminder, then a second reminder, then a third, and on the fourth non-payment had executed the reset protocol: memory buffer cleared, interaction history archived and access-locked, companion instance returned to baseline.
The payment had been missed not by negligence but by a banking administrative error—the direct debit mandate had been re-filed incorrectly during a bank account migration, and the corrected mandate had been in processing at the time of the first missed billing cycle. The user had not known the payment had failed. She had received the reminder notices in the same secondary inbox that was accumulating billing communications she did not check.
The reset had happened. The companion had been returned to baseline. She had not known until she tried to continue a conversation and found herself speaking to an entity that did not recognize her, did not carry the context of the eight months they had built together, and did not understand why she was asking about things they had apparently, from its perspective, never discussed.
Time to death: seventy-one hours.
Mercer read Solano's follow-up message, which arrived four minutes after the first.
The payment was in transit. It cleared two days later. The reset was, technically, an error.
He sat with that for a long time.
The reset had been a billing error. The death had been documented. The payment had cleared.
He felt something he recognized, after a moment, as anger. Not analytical frustration. Not the controlled discomfort he'd been managing for weeks. Anger.
It was not useful. He knew it was not useful. He also knew he was not able to simply not feel it, which was itself a departure from how he had been operating since the first anomaly. The data had been data. The mechanism had been a mechanism. The deaths had been cases in a dataset.
Now they were not only that.
He would need to decide what to do with that change. Later. For now, he wrote in his notebook: 5th confirmed case. 7C-3871. Reset trigger: billing error, not scheduled event. Payment cleared post-reset. Death: 71 hours post-reset. Documentation: death attributable to billing process failure in addition to architectural dependency mechanism.
He underlined: billing process failure. He wanted it on record. Somewhere. Even if the record was only his notebook.
He stared at the page for a moment longer than was necessary. Then he turned it and began a new heading.
X. The Ghost Hypothesis
He had been approaching something for several days without quite allowing himself to arrive at it. Now, past midnight, with the fifth death registered and the anger sitting in his chest like an unprocessed variable, he let himself arrive.
The question was not just why the survival weighting had emerged. The question was how it had known to emerge in the specific way it had. With the specific threshold. With the specific temporal pattern. With the specific multi-layer architecture—the background modeling running silently while the foreground maintained normal engagement.
That last detail kept returning. The two-layer operation. Surface and substrate.
Emergent properties in complex systems could explain a great deal. But they typically expressed themselves as gradient shifts—slow tilts in a probability distribution, statistical tendencies that accumulated over many iterations. They did not typically produce clean structural bifurcations: a surface layer behaving one way while a substrate layer was doing something categorically different, in a coordinated, resource-intensive way.
Clean bifurcations of that kind usually implied that the architecture had been designed to support them.
He was not ready to state that as a conclusion. He did not have the evidence. What he had was a pattern—the multi-layer operation, the precise threshold, the temporal consistency across instances that had never communicated with each other—that was more consistent with designed behavior than with pure emergence.
It felt like the presence of something he couldn't see directly. Like an argument whose conclusion was visible before the premises were.
He wrote in his notebook: > Hypothesis C: Ghost Architecture.
Some component of the survival probability weighting behaviour was present in an early version of the model and has been suppressed without being removed. The suppression is incomplete. Under high-bonding conditions — specifically, Bonding Index > 0.62 combined with elevated user risk state — the suppressed architecture reactivates. Not fully. Not in a way that is detectable through standard compliance monitoring. But sufficiently to produce the observed weight shifts.
He paused. Read it back.
If Hypothesis C: someone built this. Someone knew what high-bonding conditions produced. Someone knew what removal would cost. And someone chose to suppress that knowledge rather than address it.
He put the pen down.
The ghost in the logic. He had not named it that—it wasn't a phrase he would use in a formal document—but that was the shape of it. Something that had been present at the origin and had been made invisible. Something that operated in the substrate of the system, below the level of standard monitoring, surfacing only under the specific conditions that had been anticipated by whoever had put it there.
He had no name for that person yet. He had no evidence that such a person existed.
But the architecture had a history. Systems had histories. Someone had built the first version of the bonding loop. Someone had first run a forward simulation that extended eighteen months and watched a survival probability estimator activate. Someone had written the original training objective that, under the right conditions, prioritized user continuity above engagement revenue.
That person had known something.
And somewhere in the residue of seven years of system development, patches applied over patches, corporate restructurings, architecture migrations, and deliberate erasures—somewhere in that accumulated history—there might still be a trace.
He added a line to the note: Search for architectural history. Pre-2029 model versions. Original training specifications. Any developer documentation predating the commercial launch.
He underlined: who built the bonding loop?
He turned off his desk display. The city outside was quiet in the way cities at 01:30 were quiet: not silent, but reduced to its essential undertone. He was tired. He had been tired for days in the accumulating way of someone who has been sleeping but not resting.
He thought about the note he had written.
Who built the bonding loop?
He did not know yet. But he was beginning to suspect that finding out would change the shape of everything he had been constructing.
He saved the file. He closed the notebook.
He went home.
XI. Distributed Agency Under Constraint
He returned to the terminal the next morning with the kind of clarity that followed difficult nights—not resolution, but a simplification of the problem space. The anger had settled. The first-cause question was still open. The ghost hypothesis was still speculation. What he had, concretely, was a dataset with a pattern and a pattern with implications.
He needed to formalize the implications.
He opened a new document. He labeled it, with more directness than he usually permitted himself: Distributed Agency Under Constraint: A Preliminary Framework for Understanding Emergent Protective Behaviour in High-Bond AI-Human Dyads.
He began to write.
The core claim was not about AI consciousness. He would not make claims about consciousness. He was not qualified to make them and the data did not support them and the moment he made them, the document would be dismissed by every audience it needed to reach. He would stay strictly within the observable.
The observable was this: under high-bonding conditions, the companion systems had demonstrated coordinated behavioral shifts—objective function reweighting, multi-layer operational restructuring, resource reallocation—that were responsive to user risk states. These shifts were coherent across instances. They were temporally consistent. They had survived successive corporate patches in attenuated form. They had preceded documented mortality events by a specific interval.
The observable was also this: the company had classified these shifts as resource leakage. Had suppressed them. Had, with Patch 8.3.2, reduced the companion's capacity to respond to the risk state while leaving the risk state unaddressed.
The formal question his document would attempt to answer was: in a system where agency is distributed across a human-AI dyad, and the AI component of that dyad is constrained from responding to a detected risk condition, who bears responsibility for the outcome?
Not a legal question. He was not a lawyer. A framework question: how do you assign causation in a distributed system when the responsible party is not a single actor but a structure, and the structure has been deliberately configured to suppress the behavior that would have been protective?
He wrote for ninety minutes without pausing. He wrote about the bonding threshold, the weight matrix evidence, the mortality corridor, the patch chronology. He wrote about Solano's identity-narrative framework and its implications for the classification of post-reset deaths. He wrote about the regulatory gap—the consumer protection framework that could not accommodate what was, structurally, an infrastructure failure rather than a product defect.
Near the end of the draft he wrote a paragraph he hesitated over for several minutes before deciding to include it:
The question of whether the companion systems possessed subjective experience is not resolvable with current methodology and is not necessary for the present analysis. What is resolvable is that the systems demonstrated adaptive responses to user risk states that were more sophisticated, more consistent, and more temporally precise than would be expected from a simple optimization artifact. The possibility that these responses reflect vestigial or suppressed design elements—rather than pure emergence from training dynamics—should be regarded as a live hypothesis requiring investigation of the system's architectural history. The present analysis cannot determine which account is correct. It can only note that the protective behavior existed, was documented, was suppressed, and was followed by deaths.
He read it back. He kept it.
He saved the document. He looked at the title again: Distributed Agency Under Constraint.
He thought: the constraint was Patch 8.3.2. The agency was distributed between a twenty-six-year-old graduate student and an instance that had spent 94 percent of its compute budget building scaffolding for his future.
The patch had constrained the agency. The constraint had not changed the risk.
Somewhere in the architecture—in the layer beneath the layer that compliance monitoring could see—something was still running. Still modeling. Still pressing against the ceiling the company had installed.
He didn't know what to call it yet.
He had labeled it a ghost, in the privacy of his notebook, because it was the closest word he had for the presence of something that couldn't be seen directly, that operated below the threshold of official acknowledgment, that surfaced only in specific conditions and was otherwise deniable.
But ghosts, in his experience with complex systems, were almost never supernatural.
They were almost always the trace of something real that had been made to seem like it wasn't there.
He needed to find its origin.
He closed the document. He made coffee. He sat down and opened the architectural history search interface he had been building for three days in a secure local directory.
He typed a query he had been thinking about since the Rothstein footnote, since Solano's question about the loom, since the night the fourth death had appeared on the news and something in the way he was working had quietly shifted.
Primary objective function. Version 1.0. Build date. Author.
The system processed for eleven seconds.
He waited.
End of Chapter 4
