Review of "Compositional differences in soybeans on the market: Glyphosate accumulates in Roundup Ready GM soybeans"

Soy bean field with irrigation system.
Wikimedia Commons. Image by Dwight Burdette.

A friend asked me to review the paper entitled "Compositional differences in soybeans on the market" (free available here), so I thought I'd make my comments publicly available. I'm going to read the paper first, provide comments as I go along, and then find reviews online (if any exist).

The paper starts by outlining the principle of substantial equivalence, meaning that GMOs have the same nutritional content than their unmodified counterparts. The authors outline that studies examining substantial equivalence for Round-Up Ready soybean were performed early on, but not when treated with Round-Up. A follow-up study found substantial equivalence when the soy was sprayed with Round-Up, but didn't examine how much glyphosate accumulated in the plant. It is the authors' hypothesis that this is a flaw and their study is designed to examine this question (i.e. is GMO Round-Up Ready soy substantially equivalent to its non-GMO counterpart when sprayed with Round-Up, and how much Round-Up accumulates). They hypothesize that high levels of glyphosate may affect plant metabolism. The authors point out that USDA data highlights that glyphosate use is increasing, so this question is all the more important.

I pause here to note that the authors do not clarify if this is glyphosate use per acre, or total glyphosate use. If I owned a house with 0.5 acre backyard ten years ago and now owned a house with 10 acre yard, and I used weed killers on both properties, then of course the total amount of weed killer I used would show an increase, simply because I have more land. But that may not mean that I've been using more weed killer per acre.

In their study, they examine 31 samples of soybeans grown in the state of Iowa to examine their two questions. They examined 3 different types of soy:
1) Round-Up Ready Soy
2) Conventional Soy possibly sprayed with other pesticides
3) Organic Soy bean which would have no glyphosate residues

The authors then list the variety of soy bean and how they were grown. They collected 3 kilos of soy from 31 different farmers, and it seems very odd to me that they'd select different varieties of soy and different methods of treatment for each category. Why wouldn't they try to find 31 farmers that used the same type of seed? The authors don't specify if the conventional soy is the isogenic variety to the Round-Up Ready soy. This is a key issue given the question they're trying to answer: for example, if I did a study on apples and collected apples from 31 different farmers, you'd want them all to be of the same variety rather then having some Red Delicious, some Fuji, etc.  Otherwise, it's not really an apples to apples comparison. Ha!! Get it???

So, they did a bunch of analyses on the soy beans. They found residues of glyphosate and one of the compounds it breaks down into in all the GM-soy, but not the conventional or organic (oddly enough, their graph doesn't have error bars...). Then, they did statistics on nutritional content and the authors highlight some of the differences (it's also worth nothing that they highlight measurements where the organic category had higher measurements, but not where GM/conventional had higher measurements). The authors then cluster the soy samples based on the results and find that the three different categories tend to group together.

Then the authors go on to reject the null-hypothesis of substantial equivalence. They highlight that their paper identified glyphosate residues in the crops at higher levels than had been hypothesized. They highlight the "toxicity" of Round-Up by citing Seralini (minus 200 points for citing Seralini).

My main comment, as I've mentioned in the past, is that substantial equivalence does not mean identical: "Substantial equivalence is often confused for identicality, however, the Food and Agricultural Organization of the United Nations states that substantial equivalence “is established by a demonstration that the characteristics assessed for the genetically modified organism, or the specific food product derived therefrom, are equivalent to the same characteristics of the conventional comparator. The levels and variation for characteristics in the genetically modified organism must be within the natural range of variation for those characteristics considered in the comparator and be based upon an appropriate analysis of data”" (emphasis has been added)

Also, the levels of glyphosate are provided and the authors point out that these are below the maximum permissible levels. So I'm not sure what their problemo is.

So, I searched for reviews of the paper and found one on GMOanswers, written by someone at Monsanto. They too, made the same criticism as me about the seeds: "When the authors collected the soybean varieties for this study, they separated them into the three groups — organic, conventional and GM. Unfortunately, each group contained different soybean varieties, with no overlap (with one exception) of varieties between the three groups, so each group was already inherently different from the others. Even the authors acknowledge that different varieties can have widely different seed composition. Therefore, concluding that any differences between the groups in this study were due to the way the soybeans were grown (organically or not), or the presence/absence of a glyphosate-tolerance gene in the GM varieties, is simply not possible. The three groups are expected to produce different results because they started out with different genetics."

GMOanswers also notes the fact that they were grown in different farms: "Since the plants in this study were not grown together but rather taken from separate fields across a region spanning a 200 km radius, any real differences between the three groups can’t be separated from the variation caused by location, and no reliable conclusion about their nutritional quality can be made. To put this in context, we are looking at satellite imagery to help farmers make more-informed decisions by increments of meters, not miles." I agree with this point, but having a bunch of crops that were grown in the same area could have provided some information if they were the same type of seed. But since they weren't, the different farms makes matters only worse.

GMOanswers puts the findings within the context of natural variation, which I appreciated: "Finally, when we take into account the effects of genetics and location, we see that the compositional differences the authors observed in this design are not unexpected. Protein levels in soybeans generally average ~40 percent dry weight (dwt) but have been shown to range naturally from 34.1 to 56.8 percent dwt (Wilson, 2004). This natural variability can be due to variety, location or environment, and it means that people are already consuming soybeans with larger variability than the differences in soybeans reported here."

The reviewer also notes that glyphosate residues were within permissible levels and then there's a blurb about the safety assessment of pesticides, yada, yada, yada... He also noted the slant that the authors have in not really discussing what pesticide residues in organic food means.

Amelia Jordan, whom I've interviewed in my series "Better Know a Scientist", did a review on Skepti-forum, and highlighted several other issues: the fact that there's no information on how the organic crops were treated, how the soil was treated in any of the farms, and very importantly, the
fact that 31 samples for a study with this much variability in a number of factors is very, very low.

All in all, I don't think the study is conducted well enough to draw any meaningful conclusion, especially not the one that they're trying to draw which is that organic food is "better" than GM. Their finding about glyphosate is interesting and I think that future studies that examine compositional differences between pesticide/herbicide tolerant crops and controls should do similar analyses, but the measurements in this study have been deemed to be safe.