ASA24® Evaluation & Validation

The ASA24 system is based upon the United States Department of Agriculture's (USDA) Automated Multiple-Pass Method (AMPM), which has been validated and shown to estimate mean total energy and protein intakes that are somewhat lower than those estimated from recovery biomarkers (Subar et al, 2003External Web Site Policy; Moshfegh et al, 2008External Web Site Policy).

The ASA24 development team conducted numerous small-scale cognitive and usability tests on the ASA24-Beta, ASA24-2011, and ASA24-2014 Respondent Websites. Data from validation and evaluation studies indicate close agreement between the ASA24 system and standardized interviewer-administered 24-hour recalls.

All evaluation studies of ASA24 recalls were conducted using ASA24-2011. Changes between ASA24-2011 and ASA24-2014 were intended to improve the usability of the search functions, in addition to adding new features. ASA24-2016 was further modified to allow users to enter data using mobile devices smartphones, and to allow for use as a food record. The methods by which data are collected for 24-hour recalls in ASA24-2016, ASA24-2014 and ASA24-2011/12 versions are similar in terms of asking Respondents to report intake for the past day. NCI has no reason to believe that previous validation findings based on ASA24-2011 would be meaningfully different for collecting 24-hour recalls using later versions of the ASA24 system. There has been no validation research on the use of ASA24-2016 as a food record. No formal evaluation or validation efforts are planned by NCI for the ASA24-2016 system.

Comparison of Interviewer-Administered to Self-Administered Recalls

To formally evaluate the impact of the change in mode of administration from the interviewer-administered AMPM 24-hour recall to a self-administered web-based recall, two studies were conducted using ASA24-2011:

  • One large study of healthy individuals in a variety of geographic regions compared the nutrient, food group, and supplement intake estimates from Respondents completing ASA24 to those completing a standardized AMPM interviewer-administered recall. Findings indicate comparability between the ASA24 system and AMPM in reported intakes and response rates, and that Respondents prefer the ASA24 system over AMPM.
    View Citation

    Thompson FE, Dixit-Joshi S, Potischman N, Dodd KW, Kirkpatrick SI, Kushi LH, Alexander GL, Coleman LA, Zimmerman TP, Sundaram ME, Clancy HA, Groesbeck M, Douglass D, George SM, Schap TE, Subar AF. Comparison of interviewer-administered and automated self-administered 24-hour dietary recalls (ASA24) in three diverse integrated health systemsExternal Web Site Policy. Am J Epidemiol. 2015 Jun 15; 181(12):970-8.

  • Another manuscript from this study that assesses comparability of reported dietary supplement usage between AMPM and the ASA24 system is in press.
    View Citation

    Schap TE, Thompson FE, Bailey R, Dodd KW, Dixit-Joshi S, Potischman N, Kirkpatrick SI, Alexander GL, Coleman LA, Kushi LH, Groesbeck M, Sundaram ME, Clancy HA, Zimmerman, TP, Douglass D, Mittl B, George SM, Gahche JJ, Kahle L, Subar AF. Comparing reported dietary supplement intakes between two 24-hour recall methods: The Automated Self-administered 24-hour recall (ASA24) and the interview-administered Automated Multiple Pass Method (AMPM). In Press: J Acad Nutr Dietet.

  • These data were also analyzed to assess the impact of default coding of free text options in the ASA24 system and found that, in large studies, default coding led to findings similar to those when free text was edited to fix inconsistencies.
    View Citation

    Zimmerman TP, Potischman N, Douglass D, Dixit-Joshia S, Kirkpatrick SI, Subar AF, McNutt S, Coleman LA, Alexander GL, Kushi LH, Thompson FE. The effect of editing open-ended text responses on nutrient and food group estimates from the Automated Self-Administered 24-Hour Dietary Recall (ASA24)External Web Site Policy. Procedia Food Sci. 2015; 4: 160-72.

  • In a second smaller study, investigators unobtrusively documented food intakes of Respondents randomly assigned to one of two groups. One group completed a recall using the ASA24 system and the second group completed a standardized AMPM interviewer-administered recall. Analyses considered how each of these recalls performed relative to true intake. The AMPM performed slightly better than the ASA24 system relative to true intake for matches (the proportion of items consumed that were reported), exclusions (foods consumed but not reported), and intrusions (food reported but not consumed). There was little evidence of differences between the AMPM and the ASA24 system in true and reported energy, nutrient, and food group intakes or portion sizes. Overall, the ASA24 system performed well and is comparable to the AMPM as an instrument for collecting dietary intake data from large samples.

    View Citation

    Kirkpatrick SI, Subar AF, Douglass D, Zimmerman TP, Thompson FE, Kahle LL, George SM, Dodd KW, Potischman N. Performance of the Automated Self-Administered 24-hour Recall relative to a measure of true intakes and to an interviewer-administered 24-h recallExternal Web Site Policy. Am J Clin Nutr. 2014; 100:233-40.

  • A study was conducted to assess the accuracy of portion size reporting using ASA24 compared with interviewer-administered recalls. Results indicated that digital images tailored to different types and formats of foods may facilitate improved estimation of amounts eaten but highlight the need for continued work in this aspect of dietary assessment.

    View Citation

    Kirkpatrick SI, Potischman N, Dodd KW, Douglass D, Zimmerman TP, Kahle LL, Thompson FE, George SM, Subar AF. The use of digital images in 24-hour recalls may lead to less misestimation of portion size compared to traditional interviewer-administered recallsExternal Web Site Policy. Am J Clin Nutr. 2016 Nov 2 [Epub ahead of print]

Assessment of Measurement Error in Self-Report Instruments

ASA24-2011 is being evaluated within the Multi-Cohort Eating and Activity Study for Understanding Reporting Error (MEASURE), which is designed to assess the structure of measurement error in self-report diet and physical activity instruments in multiple large cohorts. Data collection for MEASURE, which includes sub-studies from three cohorts (the Harvard Nurses' Health Study, the Harvard Health Professionals Follow-Up Study, and the National Institutes of Health-AARP Diet and Health Study), began in late 2010. The Nurses' Health study used the Beta version of the ASA24 system; the other two cohorts used ASA24-2011. Data collection has finished for all three studies. Results are expected in 2017.

Collaboration for New Evaluation Studies

Collaboration with Researchers who are interested in evaluating ASA24-2016 as part of their research is welcome. Please contact Amy Subar for further information.

Return to Top

ASA24 is a registered trademark of HHS.