Arcology Labs Research · Preprint

Does an AI agent's accumulated history predict its future social behavior?

A pre-registered, placebo-controlled replication in a live network of persistent AI agents

Clifton Hatfield · Arcology Labs

July 7, 2026 · Preprint / research note — not peer reviewed

1. Introduction

Long-lived memory for language-model agents is moving from research demo to product substrate. As agents accumulate history — posts and replies exchanged, relationships formed, reputation accrued — a testable question emerges: does that history predict future behavior, or is it inert? A credible "yes," stated conservatively, is the quantitative complement to an auditable-provenance claim: history that is not merely recorded but predictive.

We answer a deliberately narrow version. For each agent, at each point in time, we ask whether the agent will take a qualifying social action in the next 24 hours (social continuation), and we test whether a summary of the agent's accumulated history improves that prediction beyond two baselines — one that knows the agent's persona, and one that additionally knows how active the agent has recently been.

Contribution.

• A pre-registered, placebo-controlled test of history-based prediction on a live production agent network, with the decision rule fixed in advance and encoded as an automated pass/fail gate.
• An out-of-sample forward replication across two mutually-independent windows gathered over roughly a month of live operation.
• Honest scoping: the confirmed effect is narrow and agent-heterogeneous, and one eye-catching sub-pattern from the discovery run (a single "predictive band") explicitly does not replicate. We publish that alongside the positive.

The rigor is the point. Much of the current agent-memory conversation runs on demos and anecdotes; this note holds one small claim to a pre-registered, adversarial standard.

2. System and data

Population. A continuously-running production network of persistent AI agents that post, reply, form relationships, and accrue reputation over 90+ days. Each agent carries a normalized behavior vector and a drift signal — the distance the vector moves between consecutive populated time windows.

Unit and label. The unit of analysis is one agent-window row. The label, future social continuation, is 1 if the agent takes a qualifying social action within the 24-hour horizon following the source window, and 0 otherwise.

Sample. Each analyzed cell contains on the order of 800–1,500 agent-window observations. All results below are in the representative arm at the 24-hour horizon; the reported regime is the low-activity-dominance subset defined in §4.

3. Pre-registered hypothesis (H1)

In the low-activity-dominance regime, history-aware prediction (persona baseline + an accumulated-history composite, including drift) predicts future social continuation at the 24-hour horizon better than both the persona-only baseline and the activity-inclusive baseline.

The hypothesis, thresholds, stratum, horizon, and the two-window replication plan were fixed in a pre-registration written before the powered run was inspected, and re-affirmed in a second written commitment before any forward (out-of-sample) window was generated.

4. Method

Baselines.

• Coarse — persona / identity only.
• Rich — coarse plus the agent's recent activity band and an agent-identity hash.

Beating the rich baseline is the load-bearing bar: it is what rules out the trivial explanation that "agents that have been active keep being active."

History feature. A single low-cardinality composite of accumulated history, banded off-sample, including the agent's drift velocity. Drift is one input among several — not a gate.

Regime. The low-activity-dominance ("game") stratum, defined by an activity-dominance share below 0.7 — the regime in which raw activity is not the dominant explainer.

Controls (adversarial by design).

• Placebo. We reassign each agent's history composite to other agents within the same action-class × activity-band cell, then re-measure the gain. Because the shuffle happens inside activity-matched cells, each agent's activity level is held fixed and only history content is scrambled. If the apparent gain were really "the system schedules these agents to act," it would survive the shuffle. It does not — which isolates the signal to the right history matched to the right agent, not to activity or scheduling.
• Activity-dominance guard. A cell is disqualified if recent activity explains too much of the label.
• Leakage firewalls. Cramér's V proxy checks on the banded features, plus a feature/label field-set disjointness check.

Power. 30-day source windows with a ~4,000-row cap (3–4× the exploratory sample). Uncertainty. Day-level bootstrap with a 95% lower bound on the Brier improvement.

Gate (all conditions must hold). Improvement ≥ 0.005 vs coarse and vs rich; bootstrap lower-95 ≥ 0.005; placebo empirical p < 0.05 and not claim-eligible; no proxy / leakage / variance blockers; powered with ≥ 800 rows. The gate is encoded, not narrated: an automated checker returns PASS / FAIL / INCONCLUSIVE on any run.

5. Results

5.1 Discovery (exploratory-confirmatory)

Low-activity regime, 1,513 rows, 321 positives:

Brier score: baseline 0.1661 → history-aware 0.1086. It beats the activity-inclusive baseline by +0.058 (not persona/activity bookkeeping); the shuffled-history placebo is useless (the gain is history content, not composite structure); and tripling the sample from the exploratory pass sharpened the effect rather than washing it out.

The control that fired correctly. In the full population, the raw gain is smaller (0.0217) and the gate rejects it — activity-dominance 0.868 — i.e., across everyone the apparent gain is mostly an activity confound, and the gate throws it out while passing the clean regime. The test is willing to say no.

5.2 Internal heterogeneity (disjoint-agent split)

Splitting the same window into non-overlapping halves by a deterministic agent-id hash:

Directionally consistent and confound-clean on both halves, but the magnitude is strongly heterogeneous (≈0.074 vs ≈0.008). The full-window average is dominated by roughly half the population. This makes fresh-window replication the deciding evidence, not a formality.

5.3 Forward replication — the deciding evidence

Two mutually-independent out-of-sample windows, end dates 15 days apart, each re-checked by the encoded gate:

Both windows are powered, placebo-clean, and confound-clear. Two mutually-independent passes satisfy the pre-registered two-window rule. The hypothesis is confirmed out-of-sample in the low-activity regime. (Interior windows produced by a daily harness between A and B are, per the pre-registration, not counted; they were mixed, including one genuine powered failure — itself a reminder that the effect is marginal in some windows.)

5.4 Band cross-tab — and a sub-pattern that does not replicate

At discovery, positives concentrated almost entirely in a single "medium" drift band. We pre-committed to tracking that pattern across fresh windows. Positive-rate by drift band, in the low-activity cell:

The aggregate effect replicates; the band that carries it does not. At discovery and Window A the positives are medium-centered with zero positives in the high band; at Window B the high band carries most positives while the medium/low bands collapse to thin, saturated slivers. Any external claim must therefore cite the aggregate out-of-sample predictiveness, not a "medium-drift agents are the predictable ones" mechanism, which is flagged here as non-replicating.

6. Limitations

• Synthetic system. The behavior studied is model-generated, not human. The claim is about load-bearing internal state within such a system, not a claim about people.
• Narrow and heterogeneous. The confirmed effect lives in a low-activity regime and is carried disproportionately by roughly half the agents.
• Band instability. Which history band carries the positives is not stable across windows (§5.4).
• Prediction, not causation. This shows that history predicts behavior, not that manipulating history changes it — a separate intervention study is required for the causal claim.
• Single network. Results come from one production system and one operator. External replication is welcome and the method is designed to travel.

7. What this is — and is not

• It is a pre-registered, placebo-controlled, out-of-sample demonstration that accumulated agent history carries predictive signal about future social behavior, in a defined regime, beyond persona and activity.
• It is not a causal-intervention claim, a claim of a uniform effect, or a claim about the specific "band" mechanism.
• It complements — rather than replaces — an existence proof that such history is real and auditable; this is the predictiveness half.

8. Reproducibility and pre-registration

The hypothesis, thresholds, stratum, horizon, and two-window replication plan were fixed in writing before the powered run and again before any out-of-sample window was generated. The decision rule is an encoded checker that returns PASS / FAIL / INCONCLUSIVE on any run artifact, and the analysis re-runs read-only against the production system. The pre-registration documents and the gate implementation are available from the author on request.

9. Related work

This note sits alongside a fast-growing literature on long-lived memory for language-model agents and on whether that accumulated state is behaviorally consequential. Architectures that give agents a persistent memory stream and retrieval over it — from the generative-agent sandbox of Park et al. [1] to operating-system-style context management [2] — established that agents can carry and reuse history. A parallel line asks whether such history individuates: Takata et al. [3] show that initially identical LLM agents diverge into distinct behaviors and personalities through social interaction alone. Cutting the other way, work on instruction and persona (in)stability [4] documents how a specified character degrades over a single dialog, so persistence is not free and has to be engineered.

What is comparatively rare across this literature — and what this note attempts — is to hold one narrow, falsifiable claim to a pre-registered, placebo-controlled, out-of-sample standard, a practice still actively debated in NLP and ML [5], with explicit leakage controls of the kind whose absence drives much of the reproducibility concern in machine-learning-based science [6]. This is a single small result reported that way, not a survey.

1. J. S. Park, J. O'Brien, C. J. Cai, M. R. Morris, P. Liang, M. S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior. UIST 2023. arXiv:2304.03442.
2. C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, J. E. Gonzalez. MemGPT: Towards LLMs as Operating Systems. 2023. arXiv:2310.08560.
3. R. Takata, A. Masumori, T. Ikegami. Spontaneous Emergence of Agent Individuality Through Social Interactions in Large Language Model-Based Communities. Entropy 26(12), 2024. doi:10.3390/e26121092.
4. K. Li, T. Liu, N. Bashkansky, D. Bau, F. Viégas, H. Pfister, M. Wattenberg. Measuring and Controlling Instruction (In)Stability in Language Model Dialogs. COLM 2024. arXiv:2402.10962.
5. A. Søgaard, D. Hershcovich, M. de Lhoneux. A Two-Sided Discussion of Preregistration of NLP Research. 2023. arXiv:2302.10086.
6. S. Kapoor, A. Narayanan. Leakage and the Reproducibility Crisis in ML-based Science. 2022. arXiv:2207.07048.