Know where every training file came from

Provenance is the history of a piece of data: where it came from, under what terms, what changed it, and who handled it along the way. For a training set, that history used to be an afterthought. Now it’s a question that buyers, auditors, procurement reviewers, and research peer reviewers ask directly — what was this model trained on, and did you have the right to use it? A folder of contracts and a vendor’s word don’t answer it.

The trouble is that training data passes through many hands. A file might be captured by one contractor, cleaned by a script, pre-labeled by a model, annotated in a third-party tool, and reviewed by a different team. Each hand-off is a place where the record breaks. Good provenance closes those gaps: the history is attached to each file, it’s written as the work happens rather than reconstructed later, and it follows the data through every version you deliver.

MLtwist records it in a Data ID Card. When a file enters the platform, its origin, owner, and usage terms are captured. Each Twist that transforms or pre-labels it adds an entry, including the model and prompt behind any draft label. Labelers, reviewers, vendors, and tools are recorded along with how each met your requirements. Every delivery is a version you can diff, roll back, and cite. Stanford researchers used the Data ID Card to audit their data’s path for two published studies, and regulated programs rely on the same record for chain of custody.

Use it when you need to show where your training data came from — for a procurement review, a customer’s security questionnaire, a published paper, or your own confidence before a model ships. It comes with every MLtwist delivery, whether the data was collected, generated, or brought by you, and it’s central to our public sector work.