Specialty Grand Challenge ARTICLE

Front. Nutr., 26 June 2014 | http://dx.doi.org/10.3389/fnut.2014.00006

From measurement to analysis reporting: grand challenges in nutritional methodology

  • 1Department of Physical Therapy, Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA
  • 2Department of Biostatistics, Office of Energetics and Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA

Nutrition has been central to the progress and well-being of mankind. Nutritional treatments and interventions, nutritional public policy and guidelines, issues related to agriculture and the food supply, nutritional genomics for individualized treatments, and other such topics remain vital areas of scientific inquiry. The foundation of progress in science is rigorous methods. Many of our methods have improved substantially over the years, yet there are always healthy calls for ever greater rigor in nutrition research (13). Ongoing advancement in methodological aspects of nutrition, energetics, and obesity studies include at least three components: (a) observations and measurements, (b) experimental design, and (c) statistical analysis and interpretation. We challenge ourselves and the field to expand the frontiers of nutritional knowledge by advancing the breadth, the rigor, and the quality of use of scientific methods in nutrition-related research.

The beginnings of modern medicine and nutritional science can be dated to the early seventeenth century, when the old ideas of the Four Humors Theory (4) were finally being questioned. Back then it was customary to attribute any disease to one of the four body humors without careful measurements or analysis. Jean-Baptiste van Helmont was one of the first investigators who chose to take detailed measurements. Incidentally, this was the same era in which Galileo questioned old Greek ideas and developed devices such as the thermometer, which van Helmont used to measure temperatures. Sanctorius further improvised this device to develop a clinical thermometer for examining sick individuals (4). It was also during the seventeenth century when medically oriented scientists focused on developing microscopes, an advancement from the glass lens that had been developed since the thirteenth century. The invention of the microscope was critical to advancing our understanding of general medical science, including nutritional research. Indeed, microscopes were crucial in our understanding the processes of energy production and chemical interaction and in the study of animal fluids (5). Analogously, in modern times, advances like the Google Glass (6) may set the stage for us to develop next-generation data collection methods and protocols.

Our first challenge is to advance data collection methods by using an ensemble of cutting-edge technological advances from fields such as biophysics, bioengineering, psychometrics, nanotechnology, and biomaterials (710). Studies to estimate measurement errors and biases in data collection techniques will also aid in advancing the reliability and accuracy of data collection methods (9). This challenge includes reducing some of the problems with or supplanting some outmoded existing data collection methods such as self-reported dietary intake (11, 12).

The maturing of nutritional research from measurements and observations to intervention study design can be dated to James Lind’s scurvy trial in the eighteenth century (13). Lind divided 12 sailors affected with scurvy into 6 pairs and treated each pair with a different nutritional intervention. These 12 sailors were selected such that they were similar in terms of general characteristics such as diet as well as symptoms of scurvy. Lind discovered that the two patients who had lemons and oranges clearly showed the best results. This nutrition trial by Lind can be viewed as the first controlled, but not randomized, comparative clinical trial (4). Later in the early twentieth century, Fisher championed the idea of randomization when he was assigned to study and improve wheat yields and in a series of papers proposed ideas about randomization and blocking with respect to agricultural experimentation (13, 14).

Randomized controlled experiments using animal models or model organisms also provide us with a unique opportunity to understand the biological, behavioral, and other processes in play when evaluating interventions. The value and importance of conducting RCEs in humans including randomized controlled trials (RCTs) in nutrition and obesity research is profound. However, conducting these RCEs in humans for clinically relevant traits for clinically relevant durations can be demanding. As a result the field has relied heavily on observational study designs (cross-sectional or longitudinal) where the individuals are not assigned to levels of independent variables by the experimenter or at random. Findings from these studies are then based on ordinary tests of associations (OATs) where the only or primary means of controlling for potential confounding is by including selected measured variables in the analysis as covariates, matching on selected measured variables, or by restricting the analysis to certain factors. While OATs are useful and necessary at times, in nutrition research, OATs have a poor track record for identifying effects that hold up in RCTs. Therefore, it has been difficult to affirm the clinical or public health value of the findings from OATs in epidemiological studies (1519). Because of these factors, it is important that, whenever possible, we go beyond sole reliance on OATs from epidemiological studies, even when RCTs are not viable. Evidence of causation exists on a continuum, and a variety of designs are available, such as co-twin control studies, quasi-experiments, and natural experiments, that are intermediate between OATs and RCTs (20). Our next challenge is to be creative and bold in designing studies that utilize the full range of this continuum instead of reflexively settling with OATs. This is especially important given that one of the central underlying themes of research related to nutrition has been to identify causal effects of interventions.

The quality of the statistical analysis, interpretation of the results, and accurate reporting of findings are key aspects of nutritional research. The quality of an analytic approach of a study is influenced by factors such as: (1) validity and performance of the chosen statistical method in realistic settings; (2) intelligent use of the method; (3) availability of software; and (4) fair and transparent reporting and interpretation of results. Validity of the statistical methods applied in any scientific inquiry is a necessary aspect of its sound epistemological foundations and it is important to assess the validity rigorously (21, 22). Simulation and plasmode studies allow assessment of the performance of statistical methods better than ever before (23). Too often, the choice of statistical analysis seems driven by tradition or convention, rather than a thoughtful consideration of available options (24, 25). Novel and valid analytic approaches need to be proposed to address statistical issues to overcome the limitations of existing statistical approaches (26). Unavailability of software that provides valid and specialized statistical approaches can also limit the analytic choices. Part of improving the standards of statistical analyses in nutritional research would also be developing software that disseminates valid statistical methods applicable to the field and supports reproducible research.

Finally, nutrition studies, perhaps especially observational studies, which are widely used in nutritional research, have been subject to publication and reporting biases (27, 28). Hence, a need also exists to improve standards of reporting and interpreting the results, which are sometimes misreported (2, 27, 29). Studies also report varying and sometimes conflicting results, at least in part, due to analytic and design differences (3032). Rigorous meta-methods, which include systematic reviews, meta-analyses, meta-regression, and pooled analyses of individual-level data, offer opportunities to summarize and integrate the evidence across studies in a valid manner. These meta-method studies, which have beginnings in early 1900s but remained under-utilized in the field of epidemiology, have been gaining popularity (33, 34). Meta-analyses or systematic reviews that rely primarily on the published literature are also subject to publication and search bias (35). A well-designed individual-level pooled data analysis offers more flexibility in synthesizing evidence, when the individual-level data are available (35). Recent advances in explicit guidelines and requirements, to conduct meta-analyses and systematic reviews, also help document and alleviate some of these issues related to synthesizing the evidence (36, 37). However, reporting studies of meta-methods is not consistently met up to these standards prescribed by the guidelines. There remains a need and opportunity to further the role of these meta-methods in reporting the evidence across studies, improving the standards of transparency, and improving methods in conducting meta-analyses (38). Thus, our final grand challenge is to elevate the overall standards of statistical analyses and reporting in nutritional research through: the development of novel methods and intelligent use of existing methods including meta-methods; evaluation of methods in realistic settings; evaluating standards of reporting and interpretation in the literature; and improving the standards of transparency for meta-method studies.

The history of science shows that advances in knowledge frequently preceded and were made possible by advances in methods that allow us to see what we could not see before. We look forward to working with all of you to see further than we have.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This project was supported in part by the National Institutes of Health grant P30DK056336 and by the National Institute on Disability and Rehabilitation Research grant H133E120005.

References

1. Meyskens FL, Szabo E. Diet and cancer: the disconnect between epidemiology and randomized clinical trials. Cancer Epidemiol Biomarkers Prev (2005) 14(6):1366–9. doi: 10.1158/1055-9965.EPI-04-0666

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

2. Brown AW, Bohan Brown MM, Allison DB. Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence. Am J Clin Nutr (2013) 98(5):1298–308. doi:10.3945/ajcn.113.064410

CrossRef Full Text

3. Casazza K, Fontaine KR, Astrup A, Birch LL, Brown AW, Bohan Brown MM, et al. Myths, presumptions, and facts about obesity. N Engl J Med (2013) 368(5):446–54. doi:10.1056/NEJMsa1208051

CrossRef Full Text

4. Solomon J. Discovering the Cure for Scurvy. Hatfield: Association for Science and Education (1989).

5. Doak RS. The Telescope and Microscope. Gareth Stevens Publishing (2005).

6. Google Inc. GLASS FAQ. Available from: https://sites.google.com/site/glasscomms/faqs.

7. Bartoshuk LM, Duffy VB, Hayes JE, Moskowitz HR, Snyder DJ. Psychophysics of sweet and fat perception in obesity: problems, solutions and new perspectives. Philos Trans R Soc Lond B Biol Sci (2006) 361(1471):1137–48. doi:10.1098/rstb.2006.1853

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

8. Cha BH, Kim JS, Ahn JC, Kim HC, Kim BS, Han DK, et al. The role of tauroursodeoxycholic acid on adipogenesis of human adipose-derived stem cells by modulation of ER stress. Biomaterials (2014) 35(9):2851–8. doi:10.1016/j.biomaterials.2013.12.067

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

9. Toyooka T, Hiyama S, Yamada Y. A prototype portable breath acetone analyzer for monitoring fat loss. J Breath Res (2013) 7(3):036005. doi:10.1088/1752-7155/7/3/036005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

10. Cristescu SM, Marchenko D, Mandon J, Hebelstrup K, Griffith GW, Mur LAJ, et al. Spectroscopic monitoring of NO traces in plants and human breath: applications and perspectives. Appl Phys B (2013) 110(2):203–11. doi:10.1007/s00340-012-5050-5

CrossRef Full Text

11. Burrows TL, Martin RJ, Collins CE. A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. J Am Diet Assoc (2010) 110(10):1501–10. doi:10.1016/j.jada.2010.07.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

12. Poslusna K, Ruprich J, de Vries JH, Jakubikova M, van’t Veer P. Misreporting of energy and micronutrient intake estimated by food records and 24 hour recalls, control and adjustment methods in practice. Br J Nutr (2009) 101(Suppl 2):S73–85. doi:10.1017/S0007114509990602

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

13. Bhatt A. Evolution of clinical research: a history before and beyond James Lind. Perspect Clin Res (2010) 1(1):6–10.

Pubmed Abstract | Pubmed Full Text

14. Box J. R. A. Fisher and the design of experiments, 1922-1926. Am Stat (1980) 34(1):1–7. doi:10.1080/00031305.1980.10482701

CrossRef Full Text

15. Albert RK. “Lies, damned lies.” and observational studies in comparative effectiveness research. Am J Respir Crit Care Med (2013) 187(11):1173–7. doi:10.1164/rccm.201212-2187OE

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

16. Barba G, Russo P. Dairy foods, dietary calcium and obesity: a short review of the evidence. Nutr Metab Cardiovasc Dis (2006) 16(6):445–51. doi:10.1016/j.numecd.2006.04.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

17. Hennekens CH, Gaziano JM, Manson JE, Buring JE. Antioxidant vitamin-cardiovascular disease hypothesis is still promising, but still unproven: the need for randomized trials. Am J Clin Nutr (1995) 62(6 Suppl):1377S–80S.

Pubmed Abstract | Pubmed Full Text

18. Lawlor DA, Davey Smith G, Kundu D, Bruckdorfer KR, Ebrahim S. Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet (2004) 363(9422):1724–7. doi:10.1016/S0140-6736(04)16260-0

CrossRef Full Text

19. Taubes G. Epidemiology faces its limits. Science (1995) 269(5221):164–9. doi:10.1126/science.7618077

CrossRef Full Text

20. Majumdar S, Soumerai S. The unhealthy state of health policy research. Health Aff (2009) 28(5):900–8. doi:10.1377/hlthaff.28.5.w900

CrossRef Full Text

21. Mehta T, Tanik M, Allison DB. Towards sound epistemological foundations of statistical methods for high-dimensional biology. Nat Genet (2004) 36(9):943–7. doi:10.1038/ng1422

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

22. Mehta TS, Zakharkin SO, Gadbury GL, Allison DB. Epistemological issues in omics and high-dimensional biology: give the people what they want. Physiol Genomics (2006) 28(1):24–32. doi:10.1152/physiolgenomics.00095.2006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

23. Gadbury GL, Xiang Q, Yang L, Barnes S, Page GP, Allison DB. Evaluating statistical methods using plasmode data sets in the age of massive public databases: an illustration using false discovery rates. PLoS Genet (2008) 4(6):e1000098. doi:10.1371/journal.pgen.1000098

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

24. Reid N. A conversation with Sir David Cox. Stat Sci (1994) 9:3.

25. Robertson HT, de los Campos G, Allison DB. Turning the analysis of obesity-mortality associations upside down: modeling years of life lost through conditional distributions. Obesity (2013) 21(2):398–404. doi:10.1002/oby.20019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

26. Coffey CS, Gadbury GL, Fontaine KR, Wang C, Weindruch R, Allison DB. The effects of intentional weight loss as a latent variable problem. Stat Med (2005) 24(6):941–54. doi:10.1002/sim.1964

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

27. Elizabeth L, Trish G, Domhnall M. Registration of observational studies. BMJ (2010) 340:c950. doi:10.1136/bmj.c950

CrossRef Full Text

28. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: is it time? CMAJ (2010) 182(15):1638–42. doi:10.1503/cmaj.092252

CrossRef Full Text

29. Schoenfeld JD, Ioannidis JP. Is everything we eat associated with cancer? A systematic cookbook review. Am J Clin Nutr (2013) 97(1):127–34. doi:10.3945/ajcn.112.047142

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

30. Flegal K, Graubard B, Williamson D, Gail M. Excess deaths associated with underweight, overweight, and obesity. JAMA (2005) 293(15):1861–7. doi:10.1001/jama.293.15.1861

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

31. Calle EE, Teras LR, Thun MJ. Obesity and mortality. N Engl J Med (2005) 353(20):2197–9. doi:10.1056/NEJM200511173532020

CrossRef Full Text

32. Ioannidis JP. Implausible results in human nutrition research. BMJ (2013) 347:f6698. doi:10.1136/bmj.f6698

CrossRef Full Text

33. Bracken MB. Commentary: toward systematic reviews in epidemiology. Int J Epidemiol (2001) 30(5):954–7. doi:10.1093/ije/30.5.954

CrossRef Full Text

34. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med (2007) 4(3):e78. doi:10.1371/journal.pmed.0040078

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

35. Walker E, Hernandez AV, Kattan MW. Meta-analysis: its strengths and limitations. Cleve Clin J Med (2008) 75(6):431–9. doi:10.3949/ccjm.75.6.431

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

36. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg (2012) 10(1):28–55. doi:10.1016/j.ijsu.2011.10.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

37. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA (2000) 283(15):2008–12. doi:10.1001/jama.283.15.2008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

38. Anderson LM, Oliver SR, Michie S, Rehfuess E, Noyes J, Shemilt I. Investigating complexity in systematic reviews of interventions by using a spectrum of methods. J Clin Epidemiol (2013) 66(11):1223–9. doi:10.1016/j.jclinepi.2013.06.014

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Keywords: observations, measurements, designs, analyses, interpretation

Citation: Mehta T and Allison DB (2014) From measurement to analysis reporting: grand challenges in nutritional methodology. Front. Nutr. 1:6. doi: 10.3389/fnut.2014.00006

Received: 07 April 2014; Accepted: 30 May 2014;
Published online: 26 June 2014.

Edited and reviewed by: John L. Sievenpiper, St. Michael’s Hospital, Canada
Dominik D. Alexander, EpidStat Institute, USA

Copyright: © 2014 Mehta and Allison. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: tapan@uab.edu; dallison@uab.edu