Skip to content

Pilot 7: Agentic data standardization

Pilot 7 uses publicly available pilot studies and synthetic clinical trial data to demonstrate and evaluate agentic workflows for clinical data standardization. AI agents draft derivation specifications in YAML, the yamaa engine executes them, and automated checks compare the outputs with the published submissions and the study documents. Every result on this site is a draft pending human review.

Start here

Explore the results, then review one finding against its source documents, specification and data. No installation is needed; one evidence-backed recommendation is a complete contribution.

Pilot studies

AI agents draft specifications from each submission package, and yamaa derives ADaM datasets from the staged SDTM data. Two measures answer different questions:

  • Generated ADaM dataset agreement: does the derived dataset reproduce the submission's ADaM dataset under the study's comparison rules?
  • Consistency review: do the documents, derivation rules and data agree with one another? Findings stay open until a person decides them, even when generated ADaM dataset agreement is 100%.
Study Generated ADaM dataset agreement Consistency review
Overall Datasets matching generated ADaM datasets: 20 / 20 (100%)
Columns in matching datasets: 847 / 847
Cells in matching datasets: 13.7M / 13.7M
153 findings
77 critical; 0 non-critical; 76 needing assessment
CDISC Pilot 01 Datasets matching generated ADaM datasets: 10 / 10 (100%)
Columns in matching datasets: 415 / 415
Cells in matching datasets: 9.1M / 9.1M
83 findings
48 critical; 0 non-critical; 35 needing assessment
Submission Pilot 3 Datasets matching generated ADaM datasets: 5 / 5 (100%)
Columns in matching datasets: 216 / 216
Cells in matching datasets: 2.3M / 2.3M
34 findings
15 critical; 0 non-critical; 19 needing assessment
Submission Pilot 5 Datasets matching generated ADaM datasets: 5 / 5 (100%)
Columns in matching datasets: 216 / 216
Cells in matching datasets: 2.3M / 2.3M
36 findings
14 critical; 0 non-critical; 22 needing assessment

Agreement values are matching / generated ADaM total. Column and cell counts come from datasets that pass the comparison with generated ADaM datasets; M means one million. Select a findings count to open its evidence. The pilot study report compares the packages dataset by dataset.

Synthetic studies

Synthetic studies exercise the complete workflow:

EDC data extraction -> SDTM mapping -> ADaM derivation -> define.xml generation.

These studies have no independent ADaM reference datasets. Their dashboards support source exploration and analysis review for eCTD submission package preparation; they do not establish reference agreement.

Study Scope Review
KN189 Protocol/SAP-based SDTM and ADaM development with submission metadata Analysis coverage and open findings
KN564 Protocol/SAP-based SDTM and ADaM development with submission metadata Analysis coverage and open findings
CAR-T for ALL OpenClinica ODM export standardized to SDTM, exploratory ADaM and draft Define-XML Linked specification and data workspace

The synthetic data report inventories the staged sources and workflow readiness of all three studies.

Agentic SDLC workflow

We use yamaa within four stages of the software development lifecycle (SDLC):

  1. Plan: define specifications, derivation rules, and acceptance criteria. Record decisions.
  2. Develop: implement mappings, derivations, metadata, and custom functions.
  3. Evaluate: run specification checks and dataset comparisons. Review discrepancies and evaluation reports.
  4. Deploy: release reviewed programs, specifications, and outputs.

Human reviewers work with AI agents to refine plans and evaluate results throughout the workflow.

Responsibilities

Role Responsibility
Human reviewers Define clinical and statistical requirements and acceptance criteria. Review and approve specifications, code, results, and releases.
AI agents Draft specifications, custom functions, evaluation reports, and GitHub Actions workflows. Propose fixes for human review.

Tools

Tool Purpose
yamaa Check and execute YAML specifications to generate datasets and define.xml.
GitHub Issues and Discussions Record requests, review assessments, and decisions.
GitHub Actions Run automated checks and dataset comparisons. Build and deploy the project website through CI/CD.