
Prizes awarded under this Challenge will be paid by electronic funds transfer and may be subject to federal income taxes. HHS/NIH will comply with Internal Revenue Service withholding and reporting requirements, where applicable. Entities participating in this Challenge are encouraged, but not required, to obtain a free Unique Entity ID (UEI) via SAM.gov, as this will expedite prize payment. Additional information is available at sam.gov/content/entity-registration.
NIH/NLM reserves the right, in its sole discretion, to (a) cancel, suspend, or modify the Challenge, or any part of it, for any reason, and/or (b) not award any prizes if no submissions are deemed worthy.
NIH/NLM also reserves the right to validate submissions based on the Docker Images, repositories, and/or documentation provided by participants. Evidence of gaming the system will result in disqualification.
Use of Prize Funds
Participants are reminded that under this challenge announcement, NIH/NLM is awarding unrestricted cash prizes, not grants. The purpose of this Challenge is to reward innovation, not provide financial assistance, and NIH/NLM does not limit how winners may use cash prizes awarded to them.
For each individual task, a total pool of $125,000 will be split among the top two performing participants (whether an individual, team, or entity) using fixed percentage proportions:
Task 2
First Place (Winner): 80%--$100,000
Second Place: 20%----$25,000
Participants are solely responsible for ensuring their submissions and activities comply with all applicable laws, regulations, and policies. NIH cannot provide legal advice to third parties. Participants should consult their own legal counsel as necessary and appropriate.
Task 2: Conversational Cohort Feasibility & Discovery
Participants will build a privacy-preserving conversational interface over public dbGaP metadata and aggregate statistics, enabling researchers to assess the feasibility and potential fit of a cohort before submitting a formal Data Access Request.
An evaluation set comprising approximately 1,000 natural-language queries will be provided to participants shortly before the end of the challenge period. Participants will apply their systems to the evaluation set and submit a file containing a ranked list of predicted relevant studies for each query.
Phase 2 will use the following evaluation criteria:
•NDCG: Measures the relevance and ranking quality of retrieved studies against the expert-verified reference set.
•Ease of Use: Assesses the effort required to deploy, configure, operate, and use the system in the standardized NIH cloud environment.
•Compute Time: Measures the computational time required to generate responses.
•Compute Effort: Measures the computational effort required to generate responses, including the resources and processing required by the system.
•Memory Footprint: Measures the memory and other computational resources required to operate the system and generate responses.
NDCG will serve as the primary metric for determining final system rankings and identifying the overall winning systems. In the event of a tie in this metric, the additional evaluation criteria listed above may be used to differentiate systems and determine the final ranking.
NDCG: Normalized Discounted Cumulative Gain of the system's ranked study recommendations against an expert-verified gold answer key.
Normalized Discounted Cumulative Gain of the system's ranked study recommendations against an expert-verified gold answer key. It answers: How well does the system rank the most relevant studies near the top of its recommendations?