From measurement to analysis reporting: grand challenges in nutritional methodology
Tapan Mehta
David B. Allison- 1Department of Physical Therapy, Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA
- 2Department of Biostatistics, Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA
Nutrition has been central to the progress and well-being of mankind. Nutritional treatments and interventions, nutritional public policy and guidelines, issues related to agriculture and the food supply, nutritional genomics for individualized treatments, and other such topics remain vital areas of scientific inquiry. The foundation of progress in science is rigorous methods. Many of our methods have improved substantially over the years, yet there are always healthy calls for ever greater rigor in nutrition research (1–3). Ongoing advancement in methodological aspects of nutrition, energetics, and obesity studies include at least three components: (a) observations and measurements, (b) experimental design, and (c) statistical analysis and interpretation. We challenge ourselves and the field to expand the frontiers of nutritional knowledge by advancing the breadth, the rigor, and the quality of use of scientific methods in nutrition-related research.
The beginnings of modern medicine and nutritional science can be dated to the early seventeenth century, when the old ideas of the Four Humors Theory (4) were finally being questioned. Back then it was customary to attribute any disease to one of the four body humors without careful measurements or analysis. Jean-Baptiste van Helmont was one of the first investigators who chose to take detailed measurements. Incidentally, this was the same era in which Galileo questioned old Greek ideas and developed devices such as the thermometer, which van Helmont used to measure temperatures. Sanctorius further improvised this device to develop a clinical thermometer for examining sick individuals (4). It was also during the seventeenth century when medically oriented scientists focused on developing microscopes, an advancement from the glass lens that had been developed since the thirteenth century. The invention of the microscope was critical to advancing our understanding of general medical science, including nutritional research. Indeed, microscopes were crucial in our understanding the processes of energy production and chemical interaction and in the study of animal fluids (5). Analogously, in modern times, advances like the Google Glass (6) may set the stage for us to develop next-generation data collection methods and protocols.
Our first challenge is to advance data collection methods by using an ensemble of cutting-edge technological advances from fields such as biophysics, bioengineering, psychometrics, nanotechnology, and biomaterials (7–10). Studies to estimate measurement errors and biases in data collection techniques will also aid in advancing the reliability and accuracy of data collection methods (9). This challenge includes reducing some of the problems with or supplanting some outmoded existing data collection methods such as self-reported dietary intake (11, 12).
The maturing of nutritional research from measurements and observations to intervention study design can be dated to James Lind’s scurvy trial in the eighteenth century (13). Lind divided 12 sailors affected with scurvy into 6 pairs and treated each pair with a different nutritional intervention. These 12 sailors were selected such that they were similar in terms of general characteristics such as diet as well as symptoms of scurvy. Lind discovered that the two patients who had lemons and oranges clearly showed the best results. This nutrition trial by Lind can be viewed as the first controlled, but not randomized, comparative clinical trial (4). Later in the early twentieth century, Fisher championed the idea of randomization when he was assigned to study and improve wheat yields and in a series of papers proposed ideas about randomization and blocking with respect to agricultural experimentation (13, 14).
Randomized controlled experiments using animal models or model organisms also provide us with a unique opportunity to understand the biological, behavioral, and other processes in play when evaluating interventions. The value and importance of conducting RCEs in humans including randomized controlled trials (RCTs) in nutrition and obesity research is profound. However, conducting these RCEs in humans for clinically relevant traits for clinically relevant durations can be demanding. As a result the field has relied heavily on observational study designs (cross-sectional or longitudinal) where the individuals are not assigned to levels of independent variables by the experimenter or at random. Findings from these studies are then based on ordinary tests of associations (OATs) where the only or primary means of controlling for potential confounding is by including selected measured variables in the analysis as covariates, matching on selected measured variables, or by restricting the analysis to certain factors. While OATs are useful and necessary at times, in nutrition research, OATs have a poor track record for identifying effects that hold up in RCTs. Therefore, it has been difficult to affirm the clinical or public health value of the findings from OATs in epidemiological studies (15–19). Because of these factors, it is important that, whenever possible, we go beyond sole reliance on OATs from epidemiological studies, even when RCTs are not viable. Evidence of causation exists on a continuum, and a variety of designs are available, such as co-twin control studies, quasi-experiments, and natural experiments, that are intermediate between OATs and RCTs (20). Our next challenge is to be creative and bold in designing studies that utilize the full range of this continuum instead of reflexively settling with OATs. This is especially important given that one of the central underlying themes of research related to nutrition has been to identify causal effects of interventions.
The quality of the statistical analysis, interpretation of the results, and accurate reporting of findings are key aspects of nutritional research. The quality of an analytic approach of a study is influenced by factors such as: (1) validity and performance of the chosen statistical method in realistic settings; (2) intelligent use of the method; (3) availability of software; and (4) fair and transparent reporting and interpretation of results. Validity of the statistical methods applied in any scientific inquiry is a necessary aspect of its sound epistemological foundations and it is important to assess the validity rigorously (21, 22). Simulation and plasmode studies allow assessment of the performance of statistical methods better than ever before (23). Too often, the choice of statistical analysis seems driven by tradition or convention, rather than a thoughtful consideration of available options (24, 25). Novel and valid analytic approaches need to be proposed to address statistical issues to overcome the limitations of existing statistical approaches (26). Unavailability of software that provides valid and specialized statistical approaches can also limit the analytic choices. Part of improving the standards of statistical analyses in nutritional research would also be developing software that disseminates valid statistical methods applicable to the field and supports reproducible research.
Finally, nutrition studies, perhaps especially observational studies, which are widely used in nutritional research, have been subject to publication and reporting biases (27, 28). Hence, a need also exists to improve standards of reporting and interpreting the results, which are sometimes misreported (2, 27, 29). Studies also report varying and sometimes conflicting results, at least in part, due to analytic and design differences (30–32). Rigorous meta-methods, which include systematic reviews, meta-analyses, meta-regression, and pooled analyses of individual-level data, offer opportunities to summarize and integrate the evidence across studies in a valid manner. These meta-method studies, which have beginnings in early 1900s but remained under-utilized in the field of epidemiology, have been gaining popularity (33, 34). Meta-analyses or systematic reviews that rely primarily on the published literature are also subject to publication and search bias (35). A well-designed individual-level pooled data analysis offers more flexibility in synthesizing evidence, when the individual-level data are available (35). Recent advances in explicit guidelines and requirements, to conduct meta-analyses and systematic reviews, also help document and alleviate some of these issues related to synthesizing the evidence (36, 37). However, reporting studies of meta-methods is not consistently met up to these standards prescribed by the guidelines. There remains a need and opportunity to further the role of these meta-methods in reporting the evidence across studies, improving the standards of transparency, and improving methods in conducting meta-analyses (38). Thus, our final grand challenge is to elevate the overall standards of statistical analyses and reporting in nutritional research through: the development of novel methods and intelligent use of existing methods including meta-methods; evaluation of methods in realistic settings; evaluating standards of reporting and interpretation in the literature; and improving the standards of transparency for meta-method studies.
The history of science shows that advances in knowledge frequently preceded and were made possible by advances in methods that allow us to see what we could not see before. We look forward to working with all of you to see further than we have.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This project was supported in part by the National Institutes of Health grant P30DK056336 and by the National Institute on Disability and Rehabilitation Research grant H133E120005.
References
1. Meyskens FL, Szabo E. Diet and cancer: the disconnect between epidemiology and randomized clinical trials. Cancer Epidemiol Biomarkers Prev (2005) 14(6):1366–9. doi: 10.1158/1055-9965.EPI-04-0666
2. Brown AW, Bohan Brown MM, Allison DB. Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence. Am J Clin Nutr (2013) 98(5):1298–308. doi:10.3945/ajcn.113.064410
3. Casazza K, Fontaine KR, Astrup A, Birch LL, Brown AW, Bohan Brown MM, et al. Myths, presumptions, and facts about obesity. N Engl J Med (2013) 368(5):446–54. doi:10.1056/NEJMsa1208051
4. Solomon J. Discovering the Cure for Scurvy. Hatfield: Association for Science and Education (1989).
6. Google Inc. GLASS FAQ. Available from: https://sites.google.com/site/glasscomms/faqs.
7. Bartoshuk LM, Duffy VB, Hayes JE, Moskowitz HR, Snyder DJ. Psychophysics of sweet and fat perception in obesity: problems, solutions and new perspectives. Philos Trans R Soc Lond B Biol Sci (2006) 361(1471):1137–48. doi:10.1098/rstb.2006.1853
8. Cha BH, Kim JS, Ahn JC, Kim HC, Kim BS, Han DK, et al. The role of tauroursodeoxycholic acid on adipogenesis of human adipose-derived stem cells by modulation of ER stress. Biomaterials (2014) 35(9):2851–8. doi:10.1016/j.biomaterials.2013.12.067
9. Toyooka T, Hiyama S, Yamada Y. A prototype portable breath acetone analyzer for monitoring fat loss. J Breath Res (2013) 7(3):036005. doi:10.1088/1752-7155/7/3/036005
10. Cristescu SM, Marchenko D, Mandon J, Hebelstrup K, Griffith GW, Mur LAJ, et al. Spectroscopic monitoring of NO traces in plants and human breath: applications and perspectives. Appl Phys B (2013) 110(2):203–11. doi:10.1007/s00340-012-5050-5
11. Burrows TL, Martin RJ, Collins CE. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. J Am Diet Assoc (2010) 110(10):1501–10. doi:10.1016/j.jada.2010.07.008
12. Poslusna K, Ruprich J, de Vries JH, Jakubikova M, van’t Veer P. Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice. Br J Nutr (2009) 101(Suppl 2):S73–85. doi:10.1017/S0007114509990602
13. Bhatt A. Evolution of clinical research: a history before and beyond James Lind. Perspect Clin Res (2010) 1(1):6–10.
14. Box J. R. A. Fisher and the design of experiments, 1922-1926. Am Stat (1980) 34(1):1–7. doi:10.1080/00031305.1980.10482701
15. Albert RK. “Lies, damned lies.” and observational studies in comparative effectiveness research. Am J Respir Crit Care Med (2013) 187(11):1173–7. doi:10.1164/rccm.201212-2187OE
16. Barba G, Russo P. Dairy foods, dietary calcium and obesity: a short review of the evidence. Nutr Metab Cardiovasc Dis (2006) 16(6):445–51. doi:10.1016/j.numecd.2006.04.004
17. Hennekens CH, Gaziano JM, Manson JE, Buring JE. Antioxidant vitamin-cardiovascular disease hypothesis is still promising, but still unproven: the need for randomized trials. Am J Clin Nutr (1995) 62(6 Suppl):1377S–80S.
18. Lawlor DA, Davey Smith G, Kundu D, Bruckdorfer KR, Ebrahim S. Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet (2004) 363(9422):1724–7. doi:10.1016/S0140-6736(04)16260-0
19. Taubes G. Epidemiology faces its limits. Science (1995) 269(5221):164–9. doi:10.1126/science.7618077
20. Majumdar S, Soumerai S. The unhealthy state of health policy research. Health Aff (2009) 28(5):900–8. doi:10.1377/hlthaff.28.5.w900
21. Mehta T, Tanik M, Allison DB. Towards sound epistemological foundations of statistical methods for high-dimensional biology. Nat Genet (2004) 36(9):943–7. doi:10.1038/ng1422
22. Mehta TS, Zakharkin SO, Gadbury GL, Allison DB. Epistemological issues in omics and high-dimensional biology: give the people what they want. Physiol Genomics (2006) 28(1):24–32. doi:10.1152/physiolgenomics.00095.2006
23. Gadbury GL, Xiang Q, Yang L, Barnes S, Page GP, Allison DB. Evaluating statistical methods using plasmode data sets in the age of massive public databases: an illustration using false discovery rates. PLoS Genet (2008) 4(6):e1000098. doi:10.1371/journal.pgen.1000098
25. Robertson HT, de los Campos G, Allison DB. Turning the analysis of obesity-mortality associations upside down: modeling years of life lost through conditional distributions. Obesity (2013) 21(2):398–404. doi:10.1002/oby.20019
26. Coffey CS, Gadbury GL, Fontaine KR, Wang C, Weindruch R, Allison DB. The effects of intentional weight loss as a latent variable problem. Stat Med (2005) 24(6):941–54. doi:10.1002/sim.1964
27. Elizabeth L, Trish G, Domhnall M. Registration of observational studies. BMJ (2010) 340:c950. doi:10.1136/bmj.c950
28. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: is it time? CMAJ (2010) 182(15):1638–42. doi:10.1503/cmaj.092252
29. Schoenfeld JD, Ioannidis JP. Is everything we eat associated with cancer? A systematic cookbook review. Am J Clin Nutr (2013) 97(1):127–34. doi:10.3945/ajcn.112.047142
30. Flegal K, Graubard B, Williamson D, Gail M. Excess deaths associated with underweight, overweight, and obesity. JAMA (2005) 293(15):1861–7. doi:10.1001/jama.293.15.1861
31. Calle EE, Teras LR, Thun MJ. Obesity and mortality. N Engl J Med (2005) 353(20):2197–9. doi:10.1056/NEJM200511173532020
32. Ioannidis JP. Implausible results in human nutrition research. BMJ (2013) 347:f6698. doi:10.1136/bmj.f6698
33. Bracken MB. Commentary: toward systematic reviews in epidemiology. Int J Epidemiol (2001) 30(5):954–7. doi:10.1093/ije/30.5.954
34. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med (2007) 4(3):e78. doi:10.1371/journal.pmed.0040078
35. Walker E, Hernandez AV, Kattan MW. Meta-analysis: its strengths and limitations. Cleve Clin J Med (2008) 75(6):431–9. doi:10.3949/ccjm.75.6.431
36. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg (2012) 10(1):28–55. doi:10.1016/j.ijsu.2011.10.001
37. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA (2000) 283(15):2008–12. doi:10.1001/jama.283.15.2008
Keywords: observations, measurements, designs, analyses, interpretation
Citation: Mehta T and Allison DB (2014) From measurement to analysis reporting: grand challenges in nutritional methodology. Front. Nutr. 1:6. doi: 10.3389/fnut.2014.00006
Received: 07 April 2014; Accepted: 30 May 2014;
Published online: 26 June 2014.
Edited and reviewed by: John L. Sievenpiper, St. Michael’s Hospital, Canada
Dominik D. Alexander, EpidStat Institute, USA
Copyright: © 2014 Mehta and Allison. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: tapan@uab.edu; dallison@uab.edu