Criticism of the study "Low Protein Intake is Associated with a Major Reduction in IGF-1, Cancer, and Overall Mortality in the 65 and Younger but Not Older Population"

[Mikii]

I don't think this has been posted here yet. This is in reference to Longo's epi study. A group of protein researchers wrote a letter to the editor that was never published, but they posted it online. 

Quote

The Contrived Association of Dietary Protein with Mortality

To the Editor:

We applaud efforts to improve human health by asking insightful questions that explore existing nutrition paradigms. Unfortunately, the paper by Levine et al. (Cell Metab 2014;19:407) is a flawed attempt to link health risks of a single nutrient, protein, to chronic disease states of cancer, CVD and diabetes. The study design and analyses are inappropriate; key contradictory data are neglected; and conclusions are not justified by the data. As scientists with decades of experience studying the impact of protein on health, we are concerned that translation of these flawed data and exaggerated conclusions to the public could have serious negative health consequences for adults seeking to maintain muscle health and avoid sarcopenia.

The optimum dietary intake of protein for adults remains a topic of scientific debate; however, research has established that balanced diets with protein intakes moderately above the RDA value of 0.8 g/kg/d (1) are beneficial for weight management, sarcopenia, diabetes and physical activity (2,3). While these reviews overwhelmingly demonstrate short-term benefits of moderate protein intake on metabolic status and body composition, the long-term impact of protein on disease risk or mortality is more difficult to assess and requires expert interpretation of large data sets such as the National Health and Nutrition Examination Survey (NHANES).

In their study, Levine et al. indicate (Figure 1, Table S1, and Discussion) that “…the level of protein is … not associated with differences in all-cause, cancer, or CVD mortality.” In fact the data demonstrate that cancer mortality was actually ~10% higher in the low protein group compared with the higher protein group (ie. 9.8% versus 9.0% deaths). We would argue that the obvious findings are the most important.

Subsequent subdivision and reanalysis of the data raise serious questions about the validity of the authors’ approach and conclusions. First, the NHANES Linked Mortality Files contain information for almost 12,000 adults, however, without justification, the investigators eliminated almost one-half of the data and only reported results for 6,381 over the age of 50 yr. Second the investigators arbitrarily created ill-defined protein groups of low (LP: <10% of kcal), medium (MP: 10 – 19.9%) and high (HP: 20%+). As defined by the Institute of Medicine (IOM) the Acceptable Macronutrient Distribution Range for protein is 10% to 35% of daily energy intake (1); thus, the LP group (<10% of kcal) consuming ~41 g/d (Table S1) should be designated as protein inadequate. Applying the authors’ unusual protein categories resulted in only 437 individuals remaining in the LP group. A third major problem is use of only a single 24-h recall to derive dietary data to represent food intake over the 18-yr period of life. The limitations of this approach are hard to overstate. NHANES surveys contain additional dietary recall data allowing for calculation of more meaningful “estimated usual” food intakes as utilized by other investigators (4,5). A fourth major problem is failure to report body weight or BMI for the groups. Energy balance and body fat are major risk factors for mortality from diabetes, cancer and CVD. The footnote for Table S1 contains the definition for the BMI abbreviation but the table omits the data.

The investigators also looked at the diabetes mortality data. They report a trend for increased risk of diabetes mortality for adults with higher protein intakes (Fig. 1). However, these findings are derived from severely limited data. For diabetes mortality, they report 1.0% deaths (Table S1) or a total of 68 deaths in the entire population with only a single death in the LP group. Further, 47 of the individuals who died from diabetes had diabetes at baseline, before the first dietary measurements. The very low occurrence frequency increases the probability of statistical errors resulting in differences that are not biologically true; nonetheless, the investigators used these limited data to conclude “high protein was associated with … a 5-fold increase in mortality.” Further, the authors used Hazard Ratio (HR) analysis and concluded that the HP group had a 73-fold increased risk of dying from diabetes. The HR and confidence interval (CI) were reported as 73.52 (4.47 – 1,209.70). To our knowledge, that is the highest HR ever reported for any dietary component and certainly for one within dietary guidelines of the IOM. The CI with a 400-fold range and an upper value of 1,209.70 with 6-significant figures of accuracy is not credible.

Hazard Ratio analysis is a standard method for clinical studies with equal treatment groups and survival as a primary outcome, but have important a priori criteria for their use: 1) equal size groups, 2) no evidence of selection or group bias, and 3) linear outcomes over time. The present study fails to meet all three criteria. There are recent high quality papers using the same NHANES data focused on sugar and sodium/potassium using acceptable methods (4,5). The authors should justify not following establish procedures.

Our overall assessment of this paper is that the conclusions and analyses are biased, and flawed. While there is growing consensus that a moderate protein intake between 1.0 and 1.5 g/kg/d may confer health benefits beyond those afforded by the current RDA for protein, we also recognize there are gaps in the current knowledge base and encourage discussion of important contradictory evidence/data. Future research must be well designed, rigorously reviewed, and credibility communicated. Unfortunately, the article by Levine et al. presents conclusions not supported by their own analyses or the greater literature.

Donald K. Layman, Arne Astrup, Peter M. Clifton, Heather J. Leidy, Richard D. Mattes, Douglas Paddon-Jones, Stuart M. Phillips, Nancy R. Rodriguez, Robert R. Wolfe

References:

1. Institute of Medicine, Food and Nutrition Board. Dietary Reference Intakes for energy, carbohydrates, fiber, fat, fatty acids, cholesterol, protein, and amino acids. Washington DC: National Academy Press; 2002.

2. Westerterp-Plantenga MS, Luscombe-Marsh N, Lejeune MPGM, et al. Dietary protein, metabolism, and body weight regulation: dose-response effects. Int J Obes 2006;30:S16-S23.

3. Bauer J, Biolo G, Cederholm T, et al. Evidence-based recommendations for optimal dietary protein intake in older people: A position paper from the PROT-AGE study group. JAMDA 2013;14:542-559.

4. Yang Q, Liu T, Kuklina E, et al. Sodium and potassium intake and mortality among US adults. Arch Intern Med 2011;171:1183-1191.

5. Yang Q, Zhang Z, Gregg EW, et al. Added sugar intake and cardiovascular diseases mortality among US adults. JAMA Intern Med 2014; (doi: 10.1001/jamainternmed.2013.13563.

"To our knowledge, that is the highest HR ever reported for any dietary component and certainly for one within dietary guidelines of the IOM. The CI with a 400-fold range and an upper value of 1,209.70 with 6-significant figures of accuracy is not credible." Ouch!

[Ron Put]

Well, maybe the letter wasn't published for a reason. And part of the reason may have been that some of the signatories appear to be cozy with the milk or meat industries. It doesn't automatically mean that their argument is flawed, but presumably the editors made a reasoned decision not to publish the letter.

Of course, it's far from settled that protein intake is the end all for longevity, but there are quite a few recent studies which support the idea, especially where animal protein is consumed.
 

[mccoy]

I concur that many of the authors of the letter are supporters of a high-protein diet and are involved in consultancies with food industries.

Nevertheless, some of the above objections are pretty sensible. First of all the observational study, the food questionnaire, they are methods notoriously subject to errors, so the Levien et al. article starts with some inherent bias.

What I find the single, most powerful point of the whole critique is the huge HR confidence interval of  4 to 1210. That makes no sense in data analysis. Anyone involved in data analysis would find that hugely unusual and a probable indicator of some procedural mistake in the analysis. Also, such anomalously large data/intervals should be, if accepted, fully explained at a length, to explain the really monstrous variability, which is always a bad sign in a statistical study. Some possible sources may be very few data, extremely scattered data, inhomogeneous data and so on.

In conclusion, putting aside the affiliations of the authors of the critical letter, many of their observations appear to be very legitimate.