Examining Plausibility of Self-Reported Energy Intake Data: Considerations for Method Selection
Jinan C. Banna
Megan A. McCrory
Marie Kainoa Fialkowski
Carol Boushey- 1University of Hawaii at Manoa, Honolulu, HI, United States
- 2Boston University, Boston, MA, United States
- 3University of Hawaii Cancer Center, Honolulu, HI, United States
Self-reported dietary intake data contain valuable information and have long been used in the development of nutrition programs and policy. Some degree of measurement error is always present in such data. Biological plausibility, assessed by determining whether self-reported energy intake (rEI) reflects physiological status and physical activity level, must be examined and accounted for before drawing conclusions about intake. Methods that may be used to account for plausibility of rEI include crude methods such as excluding participants reporting EIs at the extremes of a range of intake and individualized methods such as statistical adjustment and applying cutoffs that account for the errors associated with within-participant variation in EI and total energy expenditure (TEE). These approaches allow researchers to determine how accounting for under- and overreporting affects study results and to appropriately address misreporting in drawing conclusions with data collected and in interpreting reported research. In selecting a procedure to assess and account for plausibility of intake, there are a number of key considerations, such as resources available, the dietary-report instrument, as well as the advantages and disadvantages of each method. While additional studies are warranted to recommend one procedure as superior to another, researchers should apply one of the available methods to address the issue of implausible rEI. If no method is applied, then at minimum, mean TEE or rEI/TEE should be reported to allow readers to ascertain the degree of misreporting at a gross level and better interpret the data and results provided.
Introduction
Self-reported dietary intake data contain valuable information and have long been used in the development of nutrition programs and policy. Some degree of measurement error is always present in such data (1). In a recent article, Subar et al. point to the importance of acknowledging measurement error and developing methods to mitigate error in self-report intake data (2). The authors provide the following recommendation: acknowledge the limitations and analyze and interpret self-report dietary data appropriately. Biological plausibility, assessed by determining whether self-reported energy intake (rEI) reflects physiological status and physical activity level (PAL), must be examined and accounted for before drawing conclusions about intake. Key terms related to plausibility of rEI data are presented in Table 1. In this study, the primary objectives are to: (1) provide a guide to key terms that will allow readers to better interpret study results related to assessment of plausibility of rEI data; (2) briefly introduce methods that may be used to account for plausibility of rEI, and (3) present and discuss studies that have compared different methods used to account for plausibility of rEI data.
Table 1. Definition of key words related to topic of plausibility of self-report energy intake data.
Accounting for Plausibility of rEI
In any study involving analysis or interpretation of rEI data, researchers should apply a method to address the issue of implausible rEI. The different methods currently available are presented below. Regardless of the method used, the first step is to select an assessment method for energy requirement. Energy expenditure determined by the doubly labeled water (DLW) method is widely accepted as a gold standard for energy requirement to which rEI data may be compared (6). Energy expenditure serves as a biomarker of energy intake when body weight is relatively stable (7) or when there is weight change and the energy cost of weight change is estimated (8). Validation studies have shown that the DLW technique has an accuracy of 1–3% and a precision of 2–8%, when the method is compared with respirometry (9). While DLW is preferred over respirometry given the difficulty of use of respirometry in community dwelling conditions (6), the use of DLW is often not practical given the high costs of isotopes and equipment for isotope analysis, as well as the expertise required for analysis (6). Energy requirement may therefore be predicted by other methods, such as estimated energy requirement (EER) equations, or by multiplying basal metabolic rate (BMR) or resting metabolic rate by an activity factor, where the activity factor is either assumed or assessed by using a method such as a validated questionnaire or an objective method such as a validated accelerometer (10).
Specific Methods to Account for rEI Plausibility
Crude methods of accounting for plausibility such as excluding participants reporting EIs at the extremes of a range of intake and individualized methods such as statistical adjustment and applying cutoffs that account for the errors associated with within-participant variation in EI and total energy expenditure (TEE) will be briefly presented. A full description of each method is beyond the scope of this study, and readers should refer to the original references to learn more about each method. The different approaches described in this study are summarized in Table 2.
Table 2. Overview of methods used to account for plausibility of rEI.
Exclusion of Participants with High or Low rEI (Non-Individualized Method)
A crude method to account for plausibility of rEI involves excluding from the analysis participants reporting EIs at the extreme low and high ends of rEI (13). While the specific values used differ among studies, common values used to exclude participants include fewer than an average of 500 and greater than 3,500 kcal per day over time for women (14–19) and fewer than an average of 800 and greater than 4,200 kcal per day for men (14, 19). Willett notes that the range of 500–3,500 kcal/day may be applied to data from women, and an allowable range of 800–4,000 kcal/day for men may be used, as intakes of more than 4,000 kcal/day are unlikely to be true for even relatively active men (13). An advantage of this method is that it provides a consistent protocol when the dietary-report instrument does not allow for an accurate estimation of energy intake, as is the case with food frequency questionnaires (FFQs), whether they focus on intake of specific nutrients or foods (20, 21) or attempt to comprehensively assess diet. However, a drawback of this crude approach is that it is not individualized and does not capture all implausible reports. This is illustrated by observed TEE values greater than 3,500 kcal/day in obese and extremely obese men and women with low or normal PALs, and in normal weight, very active individuals (22–24). Therefore, using a blanket assumption that all rEIs exceeding a given value are implausible may be inappropriate depending on the characteristics of participants within a dataset.
Use of Cutoffs for Under- and Overreporting of rEI (Individualized Method)
When assessing plausibility of rEI, cutoffs for rEI may be used to distinguish and group misreporters, or implausible reporters (i.e., under- or overreporters), separately from acceptable reporters, or plausible reporters. Goldberg et al. derived one such cutoff limit, known as CUT-OFF 2 (11). This method uses the number of days of self-reported intake, sample size and within-day coefficients of variation for rEI, estimated BMR, and PAL to determine low and high cutoff values [e.g., 95% confidence limits (CL)], as follows:
where CVwEI is the within-subject coefficient of variation in EI across days, d is the number of days of diet assessment, CVwB is the coefficient of variation of repeated BMR measurements of the precision of estimated compared with measured BMR, and CVtP is the total variation in PAL and takes into account both between and within-subject variation as well as methodological errors in PAL. We refer the reader to the original reference (11) for a full explanation of the use of this equation.
Strengths of the method include accounting for the within-subject errors in TEE and rEI, including measurement error and normal day to day variation. This method represents a simple individualized approach to assessing plausibility of rEI. Several limitations of this approach have also been noted, such as the failure to account for the error in assigning PAL and the lack of ability to identify misreporting that is only mildly inaccurate (12). In a study seeking to quantify the accuracy of the Goldberg method for categorizing misreporters, Tooze et al. (25) demonstrated that the Goldberg method was able to adequately discriminate between underreporters and acceptable reporters. For these sensitivity and specificity analyses, the authors excluded overreporters due to the small number of participants classified as such.
An additional method was introduced by McCrory et al. (12) and updated in a subsequent paper by the same group (3). Like the Goldberg method, this method takes into account the within-subject errors in TEE and rEI, including measurement error and normal day to day variation, and is a simple and individualized approach to assessing plausibility of rEI (3). Using this method, 1 SD cutoffs for rEI are calculated as a percentage of predicted energy requirement (pER) specific to sex and age and weight status as follows:
where CVrEI is the within-subject coefficient of variation in rEI across days, d is the number of days of intake, CVpER is the coefficient of variation of pER (taken as the root mean squared error of the prediction equation), and CVmTEE is the within-person CV of measured TEE and takes into account the measurement error and day to day biological variation in TEE [taken as 8.2%, from Black et al. (26)]. We refer the reader to the original reference (3) for a full explanation of the use of this equation.
However, the method could be adapted to use TEE measured by DLW, predicted by any prediction equation (Table 1), or estimated using a questionnaire such as the Minnesota Leisure Time Physical Activity Questionnaire (27). A limitation of the method, as with Goldberg CUT-OFF 2, is that the error in assigning PAL during calculation of EER, if EER is used as the method to estimate energy requirement, is not considered.
Statistical Adjustment Using rEI:TEE or rEI:pER (Individualized Method)
Conclusions put forth by Tooze et al. (25) were that the use of cutoffs to exclude implausible reporters may lead to loss of statistical power and biased estimates of associations of both underreporting with personal characteristics and dietary variables with outcomes. Another problem in applying cutoffs is that body weight is included in both the calculation of implausible rEI and the outcome variable in subsequent analyses of how dietary variables are associated with the outcome (13), which could artificially elevate the association. These drawbacks must be considered when applying either of these cutoff methods. One approach that has been used that avoids these potential pitfalls is to calculate the ratio of rEI:TEE or rEI:pER and include one of these ratios in regression models of dietary variables predicting health outcomes such as risk for overweight or obesity, thereby statistically adjusting for energy intake misreporting. Murakami and Livingstone recently used this method in a study examining associations of eating frequency, meal frequency, and snack frequency with adiposity and found positive associations between eating frequency and overweight/obesity that were inverse or null before EI:EER was taken into account (28). Mendez et al. also adjusted for misreporting in a study examining associations between dietary factors and BMI and found that adjusting for the ratio versus excluding implausible reporters yielded qualitatively similar results (29). The authors suggest that adjusting is a viable alternative to omitting a substantial proportion of the study sample. In examining misreporting, however, other studies have shown that individuals tend to underreport macronutrients disproportionately, with carbohydrates, fat, and alcohol underreported to a greater degree than protein (30–32). This does pose a problem when the assumption underlying the adjustment for misreporting is that macronutrients are reported proportionately.
Research Comparing Different Methods to Account for Plausibility of rEI
A small number of studies have compared different methods used to account for plausibility of rEI data. In a study seeking to elucidate the methods to best identify and account for misreporting, Mendez et al. identified misreporters on the basis of disparities between rEI and TEE (29). The authors performed the calculation using the Goldberg CUT-OFF 2 method and two alternatives: one that substituted BMR equations that are more valid at higher BMIs (11, 33) and the method of Huang et al. described earlier (3, 12). Results indicated that levels of underreporting were lower and levels of overreporting higher using both of the alternative methods compared to results using the Goldberg CUT-OFF 2 method, with the two alternative methods yielding concordant results in their subsequent examination of the relationship between dietary factors and weight status. Mendez et al. also applied the crude cutoff method of excluding participants consuming fewer than 500 and greater than 3,500 kcal per day and found that results differed from when using alternative methods. When these participants were excluded rather than using EERs to identify implausible reporters, coefficients in all models were similar to baseline multivariate models (29).
In another study, Rhee et al. presented a comparison of three methods to account for implausible reporting of intake in epidemiologic studies (34). In addition to excluding participants according to rEI (<500 and >3,500 kcal/day), the authors also used the Goldberg CUT-OFF 2 method and predicted total energy expenditure (pTEE) (3) methods to classify under- and overreporters. All results concerning the relationships between the dietary variables and outcome were qualitatively similar with the application of each method. However, the two latter methods estimated a higher prevalence of under- and overreporters than the first, leading the authors to conclude that using either of the latter individualized methods did not provide a major advantage in detecting diet–BMI associations over the crude method. In the carotenoids and retinol sample, for example, 98.9% of participants were classified as plausible reporters using the crude method, while only 71.1 and 69.6% were classified as such using the Goldberg CUT-OFF 2 and pTEE methods, respectively (34). This study included only women; thus, additional studies in men and children would be needed before applying these results more broadly.
Jessri et al. evaluated several methods for accounting for implausible reporting among a nationally representative sample in Canada as part of an examination of the association between dietary factors and obesity (35). Included in this analysis was a propensity score, which is a statistical technique used to reduce bias by equating groups based on variables associated with misreporting (36, 37). The methods examined were as follows: (1) statistically adjusting for variables related to misreporting; (2) excluding misreported recalls; (3) statistically adjusting for reporting groups (underreporters, plausible reporters, and overreporters); (4) statistically adjusting for propensity score; and (5) stratifying the analyses by reporting groups (35). Results indicated that exclusion of misreporters, adjusting for the reporting groups, and stratification yielded risk estimates that were more consistent than the other methods with expectations regarding the relationship between dietary factors and obesity. There was no benefit of adjusting for the propensity score over adjusting for the reporting group with regard to improving the association between dietary factors and obesity.
Inappropriate Methods to Account for Plausibility of EI
In addition to the aforementioned procedures that may be used to account for plausibility of EI, there are several other methods that are inappropriate for use, but are still being employed. The first is another cutoff limit derived by Goldberg et al., known as CUT-OFF 1 (11). Using this method, an absolute value of 1.35 for EI:BMR is used, assuming that BMR had been measured rather than predicted and that the rEI represented habitual intake. Black notes that this strategy ignores the errors associated with within-subject variation in EI and TEE and should not be employed in accounting for plausibility of EI (26). The use of a single cutoff for EI:BMR for all participants fails to identify underreporters among those with high energy requirements and is inappropriate for use (26). Further, it does not take overreporting into account. A second inappropriate method is that a value of rEI/pER < 1.0 is classified as underreporting, and a value of rEI/pER > 1.0 is classified as overreporting. When this is done, measurement error and day to day variation in EI and TEE are not taken into account and perfection is assumed. While it is inappropriate to classify participants as under- or overreporters based on these values, researchers may wish to report the number of participants falling above and below rEI/pER = 1.0 to describe the study sample. Researchers may also use this calculation as a covariate as described earlier.
Conclusion
As misreporting of dietary intake is a well-documented problem across population groups, it is imperative that plausibility of rEI be addressed in studies examining dietary intake. Any of the methods described in this study may be applied to data collected in children, adolescents, adults, and elderly individuals provided an estimate of energy requirement is available. In this study, both crude methods of accounting for plausibility such as excluding participants reporting EIs at the extremes of a range of intake and individualized methods such as adjustment and cutoffs that account for the errors associated with within-participant variation in EI and TEE have been described. These approaches allow researchers to determine how accounting for under- and overreporting affects study results and to appropriately address misreporting in drawing conclusions with data collected and in interpreting reported research.
In selecting a procedure to assess and account for plausibility of intake, there are a number of key considerations, such as resources available, the dietary-report instrument, as well as the advantages and disadvantages of each method. DLW may be used or another technique to assess energy requirement when resources are limited. If the dietary-report instrument is not able to yield an accurate estimation of EI, as is the case with FFQs, then a crude method to account for misreporting may be the best option. However, further study is needed in this area to determine the most appropriate method to account for misreporting when using such dietary-report instruments. Regardless of which method is used, for the time being, analyses in the total sample without exclusion of participants should also be conducted and reported. This will allow the researcher to examine the impact of using each method, and will also address the potential bias that may be introduced when participants are excluded from the analysis. While additional studies are warranted to recommend one procedure as superior to another, researchers should apply one of the available methods to address the issue of implausible rEI. If no method is applied, then at minimum, mean TEE or rEI/TEE should be reported to allow readers to ascertain the degree of misreporting at a gross level and better interpret the data and results provided.
Author Contributions
All authors jointly conceived of the content of the manuscript. JB and MF drafted the manuscript. All the authors participated in critical revision of the manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1. Murakami K, Livingstone MB. Prevalence and characteristics of misreporting of energy intake in US adults: NHANES 2003–2012. Br J Nutr (2015) 114(8):1294–303. doi:10.1017/s0007114515002706
2. Subar AF, Freedman LS, Tooze JA, Kirkpatrick SI, Boushey C, Neuhouser ML, et al. Addressing current criticism regarding the value of self-report dietary data. J Nutr (2015) 145(12):2639–45. doi:10.3945/jn.115.219634
3. Huang TT, Roberts SB, Howarth NC, McCrory MA. Effect of screening out implausible energy intake reports on relationships between diet and BMI. Obes Res (2005) 13(7):1205–17. doi:10.1038/oby.2005.143
4. Poslusna K, Ruprich J, de Vries JH, Jakubikova M, van’t Veer P. Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice. Br J Nutr (2009) 101(Suppl 2):S73–85. doi:10.1017/s0007114509990602
5. Institute of Medicine (US). Dietary Reference Intakes for Energy, Carbohydrate, Fiber, Fat, Fatty Acids, Cholesterol, Protein and Amino Acids. Washington, DC: The National Academies Press (2005).
6. Park J, Kazuko IT, Kim E, Kim J, Yoon J. Estimating free-living human energy expenditure: practical aspects of the doubly labeled water method and its applications. Nutr Res Pract (2014) 8(3):241–8. doi:10.4162/nrp.2014.8.3.241
7. Bingham SA. Biomarkers in nutritional epidemiology. Public Health Nutr (2002) 5(6a):821–7. doi:10.1079/PHN2002368
8. Gilmore LA, Ravussin E, Bray GA, Han H, Redman LM. An objective estimate of energy intake during weight gain using the intake-balance method. Am J Clin Nutr (2014) 100(3):806–12. doi:10.3945/ajcn.114.087122
9. Schoeller DA, Hnilicka JM. Reliability of the doubly labeled water method for the measurement of total daily energy expenditure in free-living subjects. J Nutr (1996) 126(1):348s–54s.
10. Westerterp KR. Physical activity and physical activity induced energy expenditure in humans: measurement, determinants, and effects. Front Physiol (2013) 4:90. doi:10.3389/fphys.2013.00090
11. Goldberg GR, Black AE, Jebb SA, Cole TJ, Murgatroyd PR, Coward WA, et al. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1. Derivation of cut-off limits to identify under-recording. Eur J Clin Nutr (1991) 45(12):569–81.
12. McCrory MA, McCrory MA, Hajduk CL, Roberts SB. Procedures for screening out inaccurate reports of dietary energy intake. Public Health Nutr (2002) 5(6A):873–82. doi:10.1079/phn2002387
14. Michels KB, Edward G, Joshipura KJ, Rosner BA, Stampfer MJ, Fuchs CS, et al. Prospective study of fruit and vegetable consumption and incidence of colon and rectal cancers. J Natl Cancer Inst (2000) 92(21):1740–52. doi:10.1093/jnci/92.21.1740
15. Schulze MB, Manson JE, Ludwig DS, Colditz GA, Stampfer MJ, Willett WC, et al. Sugar-sweetened beverages, weight gain, and incidence of type 2 diabetes in young and middle-aged women. JAMA (2004) 292(8):927–34. doi:10.1001/jama.292.8.927
16. Fung TT, Malik V, Rexrode KM, Manson JE, Willett WC, Hu FB. Sweetened beverage consumption and risk of coronary heart disease in women. Am J Clin Nutr (2009) 89(4):1037–42. doi:10.3945/ajcn.2008.27140
17. Salmeron J, Hu FB, Manson JE, Stampfer MJ, Colditz GA, Rimm EB, et al. Dietary fat intake and risk of type 2 diabetes in women. Am J Clin Nutr (2001) 73(6):1019–26.
18. Fung TT, Rexrode KM, Mantzoros CS, Manson JE, Willett WC, Hu FB. Mediterranean diet and incidence of and mortality from coronary heart disease and stroke in women. Circulation (2009) 119(8):1093–100. doi:10.1161/circulationaha.108.816736
19. Turner-McGrievy GM, Davidson CR, Wilcox S. Does the type of weight loss diet affect who participates in a behavioral weight loss intervention? A comparison of participants for a plant-based diet versus a standard diet trial. Appetite (2014) 73:156–62. doi:10.1016/j.appet.2013.11.008
20. Son SM, Huh GY, Lee HS. Development and evaluation of validity of dish frequency questionnaire (DFQ) and short DFQ using Na index for estimation of habitual sodium intake. Korean J Community Nutr (2005) 10:677–92.
21. Park YK, Kim Y, Park EJ, Kim JS, Kang MH. Estimated flavonoids intake in Korean adults using semiquantitative food-frequency questionnaire. Korean J Nutr (2002) 35:1081–8.
22. DeLany JP, Kelley DE, Hames KC, Jakicic JM, Goodpaster BH. High energy expenditure masks low physical activity in obesity. Int J Obes (2013) 37(7):1006–11. doi:10.1038/ijo.2012.172
23. Prentice AM, Black AE, Coward WA, Cole TJ. Energy expenditure in overweight and obese adults in affluent societies: an analysis of 319 doubly-labelled water measurements. Eur J Clin Nutr (1996) 50(2):93–7.
24. Santos DA, Silva AM, Matias CN, Magalhaes JP, Fields DA, Minderico CS, et al. Validity of a combined heart rate and motion sensor for the measurement of free-living energy expenditure in very active individuals. J Sci Med Sport (2014) 17(4):387–93. doi:10.1016/j.jsams.2013.09.006
25. Tooze JA, Krebs-Smith SM, Troiano RP, Subar AF. The accuracy of the Goldberg method for classifying misreporters of energy intake on a food frequency questionnaire and 24-h recalls: comparison with doubly labeled water. Eur J Clin Nutr (2012) 66(5):569–76. doi:10.1038/ejcn.2011.198
26. Black AE. Critical evaluation of energy intake using the Goldberg cut-off for energy intake: basal metabolic rate. A practical guide to its calculation, use and limitations. Int J Obes Relat Metab Disord (2000) 24(9):1119–30. doi:10.1038/sj.ijo.0801376
27. Richardson MT, Leon AS, Jacobs DR Jr, Ainsworth BE, Serfass R. Comprehensive evaluation of the Minnesota leisure time physical activity questionnaire. J Clin Epidemiol (1994) 47(3):271–81. doi:10.1016/0895-4356(94)90008-6
28. Murakami K, Livingstone MB. Eating frequency is positively associated with overweight and central obesity in US adults. J Nutr (2015) 145(12):2715–24. doi:10.3945/jn.115.219808
29. Mendez MA, Popkin BM, Buckland G, Schroder H, Amiano P, Barricarte A, et al. Alternative methods of accounting for underreporting and overreporting when measuring dietary intake-obesity relations. Am J Epidemiol (2011) 173(4):448–58. doi:10.1093/aje/kwq380
30. Heitmann BL, Lissner L, Osler M. Do we eat less fat, or just report so? Int J Obes Relat Metab Disord (2000) 24(4):435–42. doi:10.1038/sj.ijo.0801176
31. Livingstone MB, Black AE. Markers of the validity of reported energy intake. J Nutr (2003) 133(Suppl 3):895s–920s.
32. Subar AF, Kipnis V, Troiano RP, Midthune D, Schoeller DA, Bingham S, et al. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study. Am J Epidemiol (2003) 158(1):1–13. doi:10.1093/aje/kwg092
33. Black AE. The sensitivity and specificity of the Goldberg cut-off for EI:BMR for identifying diet reports of poor validity. Eur J Clin Nutr (2000) 54(5):395–404. doi:10.1038/sj.ejcn.1600971
34. Rhee JJ, Sampson L, Cho E, Hughes MD, Hu FB, Willett WC. Comparison of methods to account for implausible reporting of energy intake in epidemiologic studies. Am J Epidemiol (2015) 181(4):225–33. doi:10.1093/aje/kwu308
35. Jessri M, Lou WY, L’Abbe MR. Evaluation of different methods to handle misreporting in obesity research: evidence from the Canadian national nutrition survey. Br J Nutr (2016) 115(1):147–59. doi:10.1017/s0007114515004237
36. Bornhorst C, Huybrechts I, Hebestreit A, Vanaelst B, Molnar D, Bel-Serrat S, et al. Diet-obesity associations in children: approaches to counteract attenuation caused by misreporting. Public Health Nutr (2013) 16(2):256–66. doi:10.1017/s1368980012004491
Keywords: misreporting, plausibility, reported energy intake, Goldberg cutoff, estimated energy requirement
Citation: Banna JC, McCrory MA, Fialkowski MK and Boushey C (2017) Examining Plausibility of Self-Reported Energy Intake Data: Considerations for Method Selection. Front. Nutr. 4:45. doi: 10.3389/fnut.2017.00045
Received: 12 January 2017; Accepted: 11 September 2017;
Published: 25 September 2017
Edited by:
Emily Jane Dhurandhar, Texas Tech University, United StatesReviewed by:
James Shikany, University of Alabama at Birmingham, United StatesEileen R. Gibney, University College Dublin, Ireland
Copyright: © 2017 Banna, McCrory, Fialkowski and Boushey. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jinan C. Banna, jcbanna@hawaii.edu