SIM507 Week 1: The research simulator: run, artefact, manifest

Lecture 0 — First, the plain picture

This is the opening week of SIM507, and it starts from scratch. The only things assumed are comfort with Python, JSON, and the command line. No prior simulation experience is needed. Before we drive any single tool, we settle one question that governs the whole course: what is a research simulator, and what has to be true of its output before that output counts as scientific evidence?

Here is the one-sentence answer to keep in your head:

A research simulator is a computer program that runs an experiment you could not safely or cheaply run in the real world — and records every decision it made, so that someone else can run the exact same experiment and get the exact same answer.

Think of a careful chemistry lab. A good experiment writes down every step, every amount, and even the brand of every chemical, so that a stranger across the world can repeat it and reach the same result. A research simulator is the software version of that discipline: it runs the experiment and it records how, so the result is repeatable.

The words you need. Six terms carry this whole week. Each gets a one-line plain meaning here and comes back in context later — you do not have to memorise them now.

Simulator — a program that imitates a piece of reality so you can test ideas safely.
Run — one execution of a simulator with one specific set of choices.
Artefact — a file a run leaves behind: data, a summary, or a figure.
Manifest — a packing list that names every artefact a run should produce.
Seed — a single number that pins down all the "random" choices a run makes.
Campaign — a research question, plus the runs gathered to answer it.

Why bother? Three reasons, all real. Safety: you can test a fire-drill or a packed-platform crowd without ever putting real people at risk. Cost: you can run a thousand scenarios overnight for the price of electricity, which is impossible with real volunteers. Honesty: when a reviewer asks "how do you know?", you can hand them the experiment to re-run themselves. That last point matters more than it sounds. Science has a well-known reproducibility problem — many published results cannot be repeated, and some are eventually retracted. A simulator that records every choice is one honest answer to that problem.

The throughline of the whole course, stated once: a simulation is a scientific instrument, and an instrument you cannot reproduce, version, and validate is not evidence — it is an opinion with a plot attached.

What a research simulator is

Before any symbols, a homely picture. Bake a cake with the same recipe, the same ingredients, and the same model of oven, and you get the same cake every single time. Change any one of the three and the cake changes. A simulator is exactly that: ingredients plus method plus version produce a result. Strip away the domain and every research simulator is the same object — a function from a set-up to a fixed set of output files.

We can write that recipe tidily, one piece at a time.

The ingredients (parameters). What you set up: how big the room, how many people, which radio frequency. We bundle these as \theta ("theta").
The one random knob (seed). A single number that fixes every "random" choice. We call it s.
The exact version (code). Which exact build of the program ran. We call it c.
The result (artefacts). A fixed set of files left behind. We call them a_1, \dots, a_n.

Put together:

\text{simulator}(\,\theta,\; s,\; c\,) \;\longmapsto\; \{\,a_1,\, a_2,\, \dots,\, a_n\,\}

Read it as a sentence: feed in ingredients \theta, the random knob s, and the exact version c — and get out a fixed set of files a_i. Notice that nothing in the recipe mentions pedestrians, radio, or ray tracing. That is the point: the workflow generalises, and this course teaches the workflow first and the individual tools second.

Two properties of this function are what make its output evidence rather than decoration:

Determinism given the inputs. Fix (\theta, s, c) and the files come back the same. If they do not, the simulator is not yet an instrument — it is a generator of anecdotes.
Self-description. The files arrive with a manifest — a packing list declaring what each file is, what shape it must obey, and which are required. A re-runner reads the packing list, not your memory.

We unpack both. First, though, the four kinds of simulator that \theta and the artefact set come in.

Four kinds of simulator

Across sensing and crowd research, simulators recur in four kinds, told apart entirely by what they take in and what they give out. The everyday version first, before any names:

The crowd mover — like a tiny, realistic video game of people walking. You set the floor and the headcount; it reports where everyone is, frame by frame.
The measurer — takes someone else's walking paths and works out the numbers: how crowded, how fast. It moves nobody; it only measures.
The radio tracer — traces how a wireless signal bounces around a room full of people, the way a ray-traced game traces light.
The notebook — a worksheet of code run from top to bottom; the finished worksheet, figures and all, is the result.

The four simulator classes: a research simulator maps a parameterised input to versioned artefacts. Reproduce in Workbook Cell 2.

Now the precise definitions, one at a time.

The agent-based engine moves many interacting people forward in time. In: a floor geometry and an agent count. Out: per-frame positions — a "trajectory". Picture an ant farm in fast-forward: lots of little agents, each following simple rules, and you record every position.

The measurement library takes walking paths it did not create and computes physical numbers from them — density, speed, and the like. It is a referee with a stopwatch and a tape measure: it produces no motion of its own, it only measures motion that already happened. In: a trajectory. Out: metrics.

The ray-traced channel sends a radio signal through a room full of walls and bodies and reports what the wireless channel looks like. It is a video game tracing light bouncing off walls — only here it traces radio bouncing off walls and people, because human bodies (mostly water) reflect radio. In: a scene plus the crowd's motion. Out: the radio channel — the signal's fingerprint.

The notebook-as-simulator is a worksheet of code, run from top to bottom with chosen settings; the finished worksheet, with its text, numbers, and figures, is itself the result. It is a lab notebook that fills itself in: set the inputs at the top, press run, and every page below writes out automatically. In: parameters. Out: an executed notebook plus its figures.

The four kinds are not islands. The arrow in the figure is load-bearing: the crowd mover's trajectory is the input to both the measurer and the radio tracer. That single coupling — crowd feeds measurement, crowd feeds radio — is the spine of the lab's research and of Weeks 2, 3, and 5. The lab happens to run one carefully maintained instance of each kind, but those instances are examples; the four kinds are the lesson.

The four kinds grounded in concrete instances, with the experiments each drives. Reproduce in Workbook Cell 5.

The manifest: a run's packing list

Before the formal idea, a picture. Every box that leaves a warehouse carries a packing list: exactly what is inside, how many items, and which are essential versus optional. Open the box, check it against the list — nothing missing, nothing wrong. A manifest is a run's packing list.

A run is not its console output; it is its artefacts and the manifest that describes them. Each simulator ships a manifest — a fixed list of artefact descriptors. Each descriptor names the file, what format it is, what role it plays, whether it is required, and optionally a validation contract — a check that the file's contents have the expected shape (in the platform this is a reference to a Python class the file is validated against; you will see the mechanism in the workbook, but the idea is simply "checked against a contract").

The role vocabulary is the heart of it. Six roles recur. The first three:

scalar-summary — a small file of headline numbers; required, and checked against its contract.
trajectory — the large per-frame table of where everyone was; required, and the heavy file two other kinds consume.
per-bin-summary — a reduced table, one row per slice of time.

And three more:

domain-specific — a result meaningful only to this field, such as the flow-balance check we meet below.
upstream-input — a file produced by one simulator and re-used by the next. This is the paper trail: it records, in the packing list itself, that the measurer's input came straight from the crowd mover's output. It is exactly how the platform writes down that one kind fed another.
figure — a rendered picture: a chart saved as an image.

Artefact-role inventory of the real lab run: eight artefacts grouped by manifest role. Reproduce in Workbook Cell 3.

In words: most of a run's files are derived — two summaries and two figures — resting on a single heavy trajectory and the one upstream-input re-use that records the crowd-feeds-measurement coupling. The required flag and the validation contract together let the platform refuse a run whose files do not match the packing list. That refusal has a name: the gate. A run that delivers everything the manifest promised, in the right shape, passes the gate; one that does not is rejected. A passed gate is the difference between "a folder of files" and "evidence".

A real run, concretely

Abstractions earn their keep against a real example. The lab ran the crowd-mover and measurer pair on a small sandbox experiment, and we will inspect that run for the rest of the lecture. In plain terms: the crowd mover put 12 people through a 180.0-second scenario, recording 3601 frames of positions; the measurer then reported a mean crowd density of 0.125 ped/m² and a mean walking speed of 1.147 m/s, together with a mean flow-balance error of 0.0769. The run passed its gate.

A few of the field-specific terms in those numbers, defined the first time they appear. ped/m² is pedestrians per square metre, the natural unit of how crowded a patch of floor is. The "mean crowd density" is computed by a standard method called Voronoi density: each person is assigned the patch of floor closer to them than to anyone else, and density is one person divided by the size of that patch — a fair way to share out the floor. The "flow-balance error" is the continuity residual, which we define properly in the next section; for now, read it as "how badly the bookkeeping of people-in versus people-out fails to add up," where zero would be perfect.

These six numbers — 3601, 12, 180.0, 0.125, 1.147, 0.0769 — are the run's quantitative content. They appear identically in the slides and the workbook, and they all trace to this one run. That is the consistency discipline the course enforces: a number with no traceable run behind it does not belong in a result.

What a seed actually pins

Start with cards, not symbols. Shuffle a deck, then cut it at exactly the same spot every time, and the order comes out identical, run after run. The shuffle looks random — but pinned to the same starting cut, it repeats perfectly. A seed is that starting cut: a single number that pins every random choice a run makes. Same seed, same choices, same result.

Reproducibility of the random part reduces to one mechanical fact: a pseudo-random number generator is a deterministic function of its seed. Call the generator g; for a seed s,

g(s) = g(s) \quad\text{always}, \qquad g(s) \neq g(s') \;\text{ for }\; s \neq s'

In words: feed the generator the same seed and you get byte-identical numbers every time; feed it a different seed and you almost surely get different numbers. There is nothing probabilistic about re-running with a fixed seed — only about choosing the seed in the first place.

The workbook demonstrates this on a trivial synthetic simulator — a one-dimensional random walk of 12 agents over 3601 steps, scaled to the real run's agent and frame counts — and asserts the invariant rather than merely claiming it.

Same seed reproduces identically; a different seed diverges. A toy walk scaled to 12 agents over 3601 steps. Reproduce in Workbook Cell 4.

A subtlety worth a sentence, because the real run shows it. The run carries two seeds, for two different purposes: one pins the campaign's run-selection, the other pins the crowd mover's per-scenario choices. These are not a contradiction — they pin different random streams. Picture two dice on the table, each with its own starting cut: knowing one tells you nothing about the other, so you must write down both. The lesson is not "which number is right" but that a seed is meaningful only relative to the stream it pins — so a reproducible run records every seed it depends on. The determinism figure deliberately compares the two seeds to make the divergence visible.

A seed alone is still not enough. A recipe is no use without knowing which printing of the cookbook you followed; a book's ISBN, or a wax seal on an envelope, names one exact frozen edition and proves it has not changed. The same goes for code. Reproducing a run requires the full triple from the definition: the parameters \theta, the seed(s) s, and the code version c — and c here is a frozen edition, fixed by a unique fingerprint of the exact source code together with a unique fingerprint of the exact packaged program. (Both are sha256 hashes; a moving "latest" tag would not pin a version, which is exactly the point.) Pin all three and the 0.125 ped/m² re-appears; change any one and you should expect a different number — and that difference is information.

From runs to evidence: the campaign workflow

A single reproducible run is necessary but not sufficient. Research produces many runs, and an undisciplined heap of them is no more citable than a shoebox of loose photos is an album. The platform imposes three layers of order so that a sprawl of runs becomes a body of evidence a colleague can navigate and cite.

The run. One reproducible execution — everything from the sections above. Every run is addressable by a stable address that resolves, forever, to its manifest, its metrics, and its figures, and it is synced into the vault as a note.
The campaign. A campaign is a research question with a written brief and explicit success criteria, recorded as a vault note. Campaigns are how a study states, up front, what would count as success or as a negative result — so that a negative result is data, not silence.
Brief → sessions → runs → synthesis. A campaign brief opens one or more sessions (a supervised batch of related runs); each session attaches its runs; a closing synthesis reads across the runs and writes the finding back into the vault. This is the chain that converts execution into argument.

The academic framing matters more than the names of the verbs. The reason to put a run behind a stable address, a manifest, and a campaign brief is not tidiness — it is citability under reproduction. When a reviewer asks "how do you know?", the answer is a run they can re-execute from the manifest alone, at the pinned recipe, seeds, and frozen edition, and watch the same 0.125 ped/m² come back. That is the standard this course holds every result to.

Summary

A research simulator is a function from a set-up to a fixed set of output files; it becomes an instrument only when it is deterministic given its inputs and self-describing through a manifest. Simulators fall into four kinds — crowd mover, measurer, radio tracer, and notebook — and the crowd mover's trajectory feeds both the measurer and the radio tracer. A real lab run made the contract concrete: eight files across six roles, pinned to a frozen code version, gated and reproducible, reporting 0.125 ped/m² and 1.147 m/s over 12 agents in a 180-second scenario. A seed pins one random stream and nothing more, so a reproducible run records every seed, parameter, and code version it depends on. And the run / campaign / synthesis workflow is what turns a heap of runs into evidence a colleague can re-execute and cite.

Next week makes the crowd-mover and measurer pair concrete end-to-end, rebuilding the speed-versus-density curve from this very run.