Data provenance for research: from downloaded files to traceable evidence
Record sources, versions, processing scripts, and licenses so later checks do not depend on memory or a temporary folder.
Lab NotesA data file is not complete evidence by itself. A traceable data item should state its source, version or release date, download time, license, processing script, and validation method.
Do not put this information only in the filename. Use a manifest or metadata file to connect raw data, processed data, script version, and creation time. Keep raw data read-only, and make cleaning and transformation executable through a repeatable script.
For data from an API or a changing website, also save request time, query conditions, pagination rules, and returned fields. When the source changes, create a new version and change note instead of silently overwriting the old result.
This adds a little recording work, but lets a team answer where a number came from, what a rerun would do, and whether the data can be shared or reused.
Licensing, privacy, and institutional policy come before convenience. Publicly downloadable does not mean freely redistributable or suitable for every research use.