Pilot 7: Agentic data standardization¶
Pilot 7 uses publicly available pilot studies and synthetic clinical trial data to demonstrate and evaluate agentic workflows for clinical data standardization. AI agents draft derivation specifications in YAML, the yamaa engine executes them, and automated checks compare the outputs with the published submissions and the study documents. Every result on this site is a draft pending human review.
Start here¶
Explore the results, then review one finding against its source documents, specification and data. No installation is needed; one evidence-backed recommendation is a complete contribution.
-
Results
Compare ADaM dataset agreement and consistency findings across the pilot studies.
-
Review guide
Check the evidence and submit your assessment on GitHub. Start with critical findings or those marked Needs review.
Pilot study review pages:
Pilot studies¶
AI agents draft specifications from each submission package, and yamaa derives ADaM datasets from the staged SDTM data. Two measures answer different questions:
- Generated ADaM dataset agreement: does the derived dataset reproduce the submission's ADaM dataset under the study's comparison rules?
- Consistency review: do the documents, derivation rules and data agree with one another? Findings stay open until a person decides them, even when generated ADaM dataset agreement is 100%.
| Study | Generated ADaM dataset agreement | Consistency review |
|---|---|---|
| Overall | Datasets matching generated ADaM datasets: 20 / 20 (100%) Columns in matching datasets: 847 / 847 Cells in matching datasets: 13.7M / 13.7M |
153 findings 77 critical; 0 non-critical; 76 needing assessment |
| CDISC Pilot 01 | Datasets matching generated ADaM datasets: 10 / 10 (100%) Columns in matching datasets: 415 / 415 Cells in matching datasets: 9.1M / 9.1M |
83 findings 48 critical; 0 non-critical; 35 needing assessment |
| Submission Pilot 3 | Datasets matching generated ADaM datasets: 5 / 5 (100%) Columns in matching datasets: 216 / 216 Cells in matching datasets: 2.3M / 2.3M |
34 findings 15 critical; 0 non-critical; 19 needing assessment |
| Submission Pilot 5 | Datasets matching generated ADaM datasets: 5 / 5 (100%) Columns in matching datasets: 216 / 216 Cells in matching datasets: 2.3M / 2.3M |
36 findings 14 critical; 0 non-critical; 22 needing assessment |
Agreement values are matching / generated ADaM total. Column and cell counts come from datasets that pass the comparison with generated ADaM datasets; M means one million. Select a findings count to open its evidence. The pilot study report compares the packages dataset by dataset.
Synthetic studies¶
Synthetic studies exercise the complete workflow:
EDC data extraction -> SDTM mapping -> ADaM derivation -> define.xml generation.
These studies have no independent ADaM reference datasets. Their dashboards support source exploration and analysis review for eCTD submission package preparation; they do not establish reference agreement.
| Study | Scope | Review |
|---|---|---|
| KN189 | Protocol/SAP-based SDTM and ADaM development with submission metadata | Analysis coverage and open findings |
| KN564 | Protocol/SAP-based SDTM and ADaM development with submission metadata | Analysis coverage and open findings |
| CAR-T for ALL | OpenClinica ODM export standardized to SDTM, exploratory ADaM and draft Define-XML | Linked specification and data workspace |
The synthetic data report inventories the staged sources and workflow readiness of all three studies.
Agentic SDLC workflow¶
We use yamaa within four stages of the software development lifecycle (SDLC):
- Plan: define specifications, derivation rules, and acceptance criteria. Record decisions.
- Develop: implement mappings, derivations, metadata, and custom functions.
- Evaluate: run specification checks and dataset comparisons. Review discrepancies and evaluation reports.
- Deploy: release reviewed programs, specifications, and outputs.
Human reviewers work with AI agents to refine plans and evaluate results throughout the workflow.
Responsibilities¶
| Role | Responsibility |
|---|---|
| Human reviewers | Define clinical and statistical requirements and acceptance criteria. Review and approve specifications, code, results, and releases. |
| AI agents | Draft specifications, custom functions, evaluation reports, and GitHub Actions workflows. Propose fixes for human review. |
Tools¶
| Tool | Purpose |
|---|---|
| yamaa | Check and execute YAML specifications to generate datasets and define.xml. |
| GitHub Issues and Discussions | Record requests, review assessments, and decisions. |
| GitHub Actions | Run automated checks and dataset comparisons. Build and deploy the project website through CI/CD. |