Minutes 2025-11-21

Author

Eric Nantz

Published

November 21, 2025

  • Ben Straub (GSK)
  • Brandon Theodorou (Keiji AI)
  • HyeSoo Cho (FDA)
  • Keaven Anderson (Merck)
  • Kevin Lee (Clinvia)
  • Lluis Revilla Sancho (Roche)
  • Nicholas Masel (Johnson & Johnson)
  • Ning Leng (Roche/Genentech)
  • Paul Schuette (FDA)
  • Robert Devine (Johnson & Johnson)
  • Sam Parmar (Pfizer)
  • Stephen Cieply
  • Trinath Panda
  • Yilong Zhang (Meta)
  • Yiwen Luo (Merck)
  • Zifeng Wang (Keiji AI)

Pilot 4 Review Status

  • Hye Soo shared that their group is continuing to test the Docker container and WebAssembly versions in multiple systems. Their team will share test results once available.
  • If there are minor issues identified that do not require immediate resolution, they will be documented in the review summary.
  • Assuming everything goes smoothly, the team hopes to have the Pilot 4 evaluation completed by end of the year!

Pilot 5 Review

  • FDA reviewers successfully received the eCTD transfer of the Pilot 5 submission bundle earlier in November.
  • The team will provide an updated on their progress at the next Working Group meeting in December.

Future Work in 2026

AI/LLM Generation of ADaMS/TLFs & Synthetic Data

  • Earlier this month, members of the pilot 5 team began discussions on potential uses of AI/LLMs to generate R programs for the creation of new ADaM data sets and TLF outputs to reduce the burdon of creating these scripts manually.
  • There is excitement to see how various solutions could help in this project, which will be officially known as Pilot 6. In the most recent meeting, Dimitry Kolosov and Kevin Lee showcased different AI solutions such as GitHub Copilot inside Visual Studio Code and CodeZen.
  • While there is not an expectation that this would require a formal transfer of a submission bundle to FDA, the hope is that these new data and outputs will serve as a foundation for new Pilot submissions in 2026 and beyond.
  • Any future work involving the use of AI tools to generate programming and/or other artifacts will not compromise the current “ethos” of our working group in which all of our materials and the development processes to create key outputs/data is all created within the “open.”
  • Ben will create a new Slack channel for Pilot 6 to hold group communication.
  • In the WG call, Brandon Theodorou and Zifeng Wang from Keiji AI shared a demonstration of using their custom platform called TrialMind involving tailor-made agents to produce common statistical analysis outputs related to demographic summaries.
  • One desired workflow would be to create ADaM data sets from the specifications file (such as define.xml).
  • Brandon mentioned that the platform also has the ability to access Databases, which could be useful for the large-scale synthetic data generation.
  • While it is possible other external groups (such as CDISC and Phuse) might also be investigating this possibility of a larger data set, our group would still like to initiate this project to prioritize our group’s needs for future Pilot submissions.
  • Eric will create a separate Slack channel for the synthetic data generation project. More details will be shared at the next WG call.

Validation / Risk Assessment of future submissions

  • Lluis Revilla Sancho: They are focused on public CRAN packages but also to create repositories of packages
  • Companies measure risk differently
  • Teams can make different decisions, it is not up to the validation hub to make those decisions for sponsors
  • A good first test would be to assess the packages used in Pilot 5 and generate risk reports, and sharing that report in the public
  • It would be possible to share this report with FDA reviewers, and they could provide feedback on what would be useful for a statistician reviewer
  • FDA would like to have the synthetic data sets available to benchmark methodologies, and others that are plug-ins on top of R such as Jags, Stan, and Nimble. Those could be key inputs for the future transparency of risk assessments.

Action Items

  • Investigate the root cause of the extra XML file that was produced in the Pilot 4 eCTD transfer. Eric will follow up with Ning and other Roche colleagues to gather further information.
  • Invite Doug Kelkhoff to a future Submissions WG meeting to discuss potential new work on validating the R package suite used in Pilot 5.
  • Brandon will assess the possibility of creating a custom organization within their TrialMind platform and how many members could be added for prototyping solutions.