Concept

Reproducible survey research

A survey result is reproducible when someone else — or you, eighteen months later — can say exactly which instrument produced the data, exactly which code produced the table, and exactly what the codebook meant. Most of the time at least one of those three is missing, and the answer becomes an email thread.

Three things

What has to be pinned

  1. The instrument

    The questionnaire is a Python module in a git repository, so any state of it is a commit. “The wording respondents saw in wave 1” is a specific object, not a recollection. That mechanism is version control for surveys.

  2. The analysis, and the commit it ran against

    When a run is queued, the branch head is resolved to a concrete commit SHA and the job checks out that commit. The run record keeps the SHA, the log and the outputs, so a table in a report traces back to the exact code that built it.

  3. The environment

    siamang.yaml declares the Python version and pinned package versions, and scripts execute in an ephemeral sandbox with no network access by default, against a curated allowlist. The environment travels with the project instead of depending on whose machine ran the script.

Precisely

Provenance, not replay

The distinction matters if you plan around it. A run tells you exactly which commit produced it — that is provenance, and it is the hard part. It is not a time machine.

The Re-run button runs the current head of the branch, not the commit the old run used. To reproduce an old run you check that commit out yourself. Deployments work the same way: they take a branch and use its head, so fielding an exact past state means branching from it.

We would rather you read this here than discover it in month four.

How an analysis run records its commit When a run is queued, the branch head is resolved to a concrete commit and the run record stores that SHA. Later commits move the branch head on; pressing Re-run uses the new head, not the commit the original run used. The record is provenance, not replay. c1 c2 c3 head when the run was queued c4 head now run record commit = c3 Kept: the exact commit, the log, the outputs Re-run does not replay c3 It resolves the branch head again — so it runs c4. Provenance, not replay. Check c3 out to reproduce it.
The last step

The codebook has to survive the export

Reproducibility fails quietly at the end, when a labelled dataset becomes a table of integers. Because each variable declares its label, its value labels and which codes are missing and why, an export can carry that metadata rather than discarding it — which is exactly the difference between the two kinds of SPSS export. A refusal coded 99 stays a refusal, instead of becoming a category with 99 respondents in it.

And the data

Somewhere you control

A result nobody else can reach is not reproducible either. The same repository declares where responses go — your warehouse, bucket, database or REDCap — so the path from response to table is written down instead of being a thing someone did once.

Scope

What we are not claiming

Reproducibility is a property of a workflow, not a feature you buy. This platform makes the specific claims above and no more: it does not mint DOIs, export citations, generate a data-management plan, or manage ethics approvals beyond carrying an approval statement on the survey page. If a page elsewhere promises you those, it is wrong.

FAQ

Questions

What exactly is recorded about an analysis run?

The commit SHA it ran against, the script, who started it, when, its log, and the files it produced. The branch head is resolved to a concrete commit when the job is queued, so the record points at immutable code rather than at a moving branch.

Can I re-run a past analysis exactly as it ran then?

Not with one click today. The recorded commit tells you precisely what ran, and you can check that commit out to reproduce it, but the Re-run button runs the current head of the branch rather than replaying the old commit. Worth knowing before you rely on it.

Can I deploy a specific tag or commit?

Deployments take a branch and use its current head. If you need to field an exact state, branch from it and deploy that branch — tags are useful for marking a state, not for deploying one.

How is the analysis environment kept stable?

siamang.yaml declares the Python version and pinned package versions, and scripts run in an ephemeral sandbox with no network by default, against a curated package allowlist. The environment is part of the committed project, not a property of whoever's laptop ran it.

Does this make my study citable?

It makes it specific, which is the hard part — you can name the commit an instrument and a result came from. There is no DOI minting or citation export in the product; if you need a DOI, deposit the repository with a service that issues one.

See the run history

The demo project has real analysis runs, each recording the commit it used. No account needed.

Open the demo → See pricing