Pilot study report¶
ADaM derivation¶
100% of ADaM datasets matched by using yamaa framework
- Datasets matching generated ADaM datasets
- 20 / 20
- Columns in matching datasets
- 847 / 847
- Cells in matching datasets
- 13.7M / 13.7M
Consistency review¶
153 findings awaiting review
77 critical; 0 non-critical; 76 needing assessment
Method¶
- AI agents use yamaa to reproduce derivation specifications. Executing these specifications on the SDTM data produces ADaM datasets that fully match those submitted in each pilot.
- AI agents compare the specifications in the submitted Define-XML with the reproduced yamaa specifications to identify differences.
- Generated ADaM dataset agreement measures reproduction of the generated ADaM datasets using yamaa framework.
- Consistency review checks agreement between ADaM dataset and Define-XML.
- Findings can remain open even when reproduced ADaM dataset agreement is 100%.
Studies¶
| Study | CDISC Pilot 01 | Submission Pilot 3 | Submission Pilot 5 |
|---|---|---|---|
| Study information | |||
| Description | The original CDISC SDTM/ADaM pilot study, with ten active derivations and generated ADaM datasets. | Five ADaM datasets from the R Consortium's Pilot 3 FDA submission, derived from the staged SDTM data. | The Dataset-JSON pilot: SDTM converted from Dataset-JSON 1.1 and five generated ADaM datasets. |
| Source | cdisc-org/sdtm-adam-pilot-project | RConsortium/submissions-pilot3-adam | RConsortium/submissions-pilot5-datasetjson |
| Documents | define-adam.xml, define-sdtm.xml, SAP.pdf, dataguide.pdf | define-adam.xml, define-sdtm.xml, adam-pilot-3.xlsx, adrg.pdf | define-adam.xml, define-sdtm.xml, adam-pilot-5.xlsx, adrg.pdf |
| Subjects | 306 | 306 | 306 |
| SDTM | |||
| SDTM inputs | 10 of 22 staged domainsDM, EX, VS, SV, DS, QS, SC, MH, AE, LB | 10 of 22 staged domainsDM, EX, VS, SV, DS, QS, SC, MH, LB, AE | 10 of 21 staged domainsDM, EX, VS, SV, DS, QS, SC, MH, LB, AE |
| Define-XML version | 1.0.0 | 1.0.0 | 1.0.0 |
| Generated ADaM dataset agreement | |||
| ADaM datasets matched | 10 / 10 (100%) | 5 / 5 (100%) | 5 / 5 (100%) |
| Columns in datasets matching generated ADaM data | 415 / 415 (100%) | 216 / 216 (100%) | 216 / 216 (100%) |
| Cells in datasets matching generated ADaM data | 9.1M / 9.1M (100%) | 2.3M / 2.3M (100%) | 2.3M / 2.3M (100%) |
| Define-XML version | 1.0.0Generated: 2.0.0 | 2.0.0 | 2.0.0 |
| Define-XML match yamaa specification | Different | Matched | Matched |
| Consistency review | |||
| Findings awaiting review | 83 findings48 critical; 0 non-critical; 35 needing assessment | 34 findings15 critical; 0 non-critical; 19 needing assessment | 36 findings14 critical; 0 non-critical; 22 needing assessment |
Versions refer to the published Define-XML. ADaM matching compares generated metadata with the published source. Not compared means the study has no Define-XML comparison result.
Dataset results¶
| Dataset | Description | CDISC Pilot 01 | Submission Pilot 3 | Submission Pilot 5 |
|---|---|---|---|---|
| ADSL | Subject-Level Analysis Dataset | 254 × 489 issues | 254 × 498 issues | 254 × 498 issues |
| ADAE | Adverse Events Analysis Dataset | 1,191 × 554 issues | 1,191 × 554 issues | 1,191 × 553 issues |
| ADLBC | Analysis Dataset Lab Blood Chemistry | 74,264 × 469 issues | 37,132 × 4613 issues | 37,132 × 4613 issues |
| ADTTE | AE Time To 1st Derm. Event Analysis | 254 × 261 issue | 254 × 262 issues | 254 × 265 issues |
| ADADAS | ADAS-Cog Analysis | - | 12,463 × 407 issues | 12,463 × 407 issues |
| ADLBH | Analysis Dataset Lab Hematology | 49,932 × 4614 issues | - | - |
| ADLBHY | Analysis Dataset Lab Hy's Law | 9,954 × 437 issues | - | - |
| ADQSADAS | ADAS-Cog Analysis | 12,463 × 4012 issues | - | - |
| ADQSCIBC | CIBIC+ Analysis | 730 × 366 issues | - | - |
| ADQSNPIX | NPI-X Item Analysis Data | 31,140 × 4115 issues | - | - |
| ADVS | Vital Signs Analysis Dataset | 32,139 × 346 issues | - | - |
Rows × columns of the yamaa output. A dash means the dataset is not part of that study.
Issue badges count distinct consistency review items for each study and dataset, including documentation gaps and questions awaiting human review. Each issue counts once even when it has multiple supporting checks. Issue counts are separate from generated ADaM dataset agreement. Select a nonzero badge to view the issues.