Ever notice how fracking opponents tend to focus on quantity rather than quality when touting studies claiming shale development harms public health? The following 2015 comments by Food & Water Watch’s Emily Wurth are just one example,

“In 2014 alone there were 154 peer-reviewed studies that came out on the impacts of shale gas development, many of which found serious concerns. So that’s about three studies a week. I mean, those of us who work on this issue thought to ourselves, wow, it seems like there’s a new study on the problems of fracking every other day.”

A comprehensive report released earlier this month by environmental research group Resources for the Future (RFF) — certainly no shill for oil and gas — reveals why anti-fracking activists are focusing on quantity rather than quality. RFF reviewed 32 of the more prominent shale-focused studies on birth outcomes, cancers, asthma, and other health effects, including migraines and hospitalization.

Cumulatively, none of those major categories of studies were deemed “high quality,” while studies on birth defects, hospitalizations and multiple symptoms were cumulatively deemed to be of “low quality,” as the following matrix from the report shows.

 

Report authors Alan J. Krupnick and Isabel Echarte of RFF were rather blunt in their critiques of the shale-focused health studies they evaluated. As EID has pointed out many times before, a vast majority of studies released linking fracking to adverse health outcomes fail to prove causation, and the RFF report also notes this prevalent, glaring flaw,

“Overall, we find that the literature does not provide strong evidence regarding specific health impacts and is largely unable to establish mechanisms for any potential health effects.”

“Due to the nature of the data and research methodologies, the studies are unable to assess the mechanisms of any health impacts (i.e., whether a certain impact is caused by air pollution, stress, water pollution, or another burden). Even where good evidence is offered for a link between unconventional oil and gas development and health, the causal factor(s) driving this association are unclear.”

Krupnick and Echarte took particular issue with epidemiological studies, which represented a vast majority of the literature reviewed, including several studies EID has debunked (more on those in a bit). Epidemiological studies are inherently limited in that they can only determine associations rather than proof of causation. Notably, such studies have been employed prominently since a 2012 activist memo was released detailing a strategy to use scientific research to drive opposition to fracking and expand regulations. And although such studies have proven to be an ideal vehicle to generate alarmist headlines despite the continued trend of failing to prove causation, RFF notes that the epidemiological fracking health studies “all had shortcomings that were most often significant,” adding:

“These studies furthermore reported contradictory results for each impact. Some studies, for example, found increases in preterm birth, while others found decreases or no association. As is illustrated by the Community Risk-Benefit Matrix, all impacts had inconsistent findings across the literature for that outcome. Where the results of these studies did not contradict each other, the impact was only analyzed by a single study.”

Though the report does not directly identify exactly which studies it categorized as being “lower quality” — which is defines as having “multiple, serious flaws” that invalidate its conclusions entirely — it is not difficult to glean which studies are considered “lower quality” by RFF, based on the following comments made about the following epidemiological studies EID has debunked.

• Casey et al. (2016). Authored by researchers affiliated with the Post Carbon Institute, this study, attempted to link fracking to premature births in Pennsylvania. EID pointed out soon after the study was released that its data not only failed to establish such a link, the researchers failed to use available baseline data, take measurements and didn’t factor in genetics and socioeconomic factors. The study also drew sharp criticism from Dr. Tony Cox, a clinical professor of biostatistics and informatics at the University of Colorado-Boulder. RFF notes many of the flaws flagged by Cox and EID in its report:

RFF: “The authors found no correlation of unconventional natural gas activity with Apgar score, SGA or term birth weight. Cox (2016) critiques the study, primarily on the issue of using proxies to estimate exposure (an issue present in most studies). The problem with the methodology of the study is that Casey et al. (2016) have addressed only for mothers in 2010 and 2013 yet analyze 2009 through 2013. Between those two years, almost 80 percent had the same address, with 6 percent moving within 1,500 meters (m) and another 10 percent moving 1,500 m to 16 km from their original addresses. This discrepancy has the potential to introduce some error into the estimates — particularly those from 2009.”

Jamielita et al. (2015). This University of Pennsylvania and Columbia University study asserts correlation between natural gas development and an increase in hospitalizations based on zip codes in three Pennsylvania counties: Bradford, Susquehanna, and Wayne. But among a myriad of flaws highlighted by EID — including the fact that the county with the highest number of wells (Bradford) actually had the lowest overall inpatient occurrences, are two reoccurring themes repeatedly flagged by RFF in its report: failure to prove causation and failure to determine other contributing factors.

RFF: “This issue with this analysis — simply comparing zip codes with and without wells — is that a number of unobservable differences may bias the results, meaning zip codes that have drilling might be different than zip codes that do not have drilling in a way that affects the prevalence of inpatient cardiology or neurology rates. Additionally, this level of analysis likely is not able to address within-zip code variation in exposure.”

• Hill (2013a, 2013b) Despite the fact that these papers by then-Cornell doctoral candidate Elaine Hill purporting that there is a “causal” relationship between natural gas development and low birth rates, hadn’t (and apparently still haven’t) been subject to peer review, they were used to justify New York’s fracking ban. Outside experts interviewed by the New York Times said the paper was “devoid of meaningful data” and a “badly suspect piece of work,” and the RFF report also noted that criticism:

RFF: “[T]hese working papers have received critical reviews and are not yet published in peer-reviewed journals… The most important is that they relied on the assumption that mothers who live near a permitted well and mothers who live near a drilled or producing well have similar characteristics — that their socioeconomic characteristics, which may also influence birth outcomes, do not change between the time a well is permitted and when that well begins producing… This is an important issue, as mothers who are more well-off may move away from oil and gas development, or mothers who are less well-off may take advantage of decreased home prices.

• McKenzie et al. (2014) This study, which suggests a link between fracking and birth defects, was authored by a team of researchers from the Colorado School of Public Health (CSPH) led by Lisa McKenzie. The researchers were actually disavowedby the Colorado Department of Public Health and Environment (CDPHE), which provided the state birth records used for the paper. It was so poorly researched, and its findings were so alarmist, that the CDPHE demanded the inclusion of a disclaimer in the paper itself: “CDPHE specifically disclaims responsibility for any analyses, interpretations, or conclusions.” Considering it is the only birth defects study mentioned in the RFF report, it is clear this study alone led to RFF’s “lower quality” label for such studies, and the report notes further criticism of the 2014 paper:

RFF: “Fedak et al. (2014), in a published critique of the study, also took issue with the study’s hypothesis that benzene is the mechanism through which the defects occur, as the study provides little evidence to support this claim.”

McKenzie et al. (2017) This study, claiming to find an association between oil and gas development and childhood cancer, was immediately disavowed by the Colorado Department of Public Health (CDPHE) for a myriad of reasons, including its small sample size, failure to measure individual exposure and failure to address additional or alternate risk factors. The RFF report kept its criticism of this universally panned study short, but to the point:

RFF: “There are many issues that might bias the results of the study.”

• Rasmussen et al. (2016) This well known study, the second of a series of three Johns Hopkins University study spearheaded by researchers affiliated with the Post Carbon Institute, claimed to establish a link between proximity to natural gas development and asthma flare-ups. EID exposed several of this study’s flaws, most notably the all-too-common theme of lack of proof of causation and other factors among the population that could contribute to asthma exacerbations. RFF notes the latter flaw in its report:

RFF: “[T]he study, however, did not report the characteristics of populations within each exposure quartile, and it is therefore difficult to assess whether the results are credible.”

Stacy et al. (2015). This study, which was funded by the anti-fracking Heinz-Endowment and published in a journal that does not require peer-review, claimed to find a link between fracking and low birth weights. The study (as usual) failed to prove causation and included contradictory data. None of the average birthweights found were actually considered “low” by the medical definition of the term, and all regions studied had average birthweights above the national average. Furthermore, the study’s data actually showed that the average birth weight in study area farthest away from shale gas wells was 3,343.9 grams, much lower than those in the second region (3,370.4) and third region (3,345.4), which were closer to shale gas wells. RFF noted the latter in its report:

RFF: “The study did not find any significant effects of well density on premature births, except for a higher average birth weight and a lower share of premature infants born to mothers living in the second exposure quartile — an odd result. The study also uses a 10-mile radius, larger than many other studies for analysis, which makes it difficult to control for variation within that area of interest. This large radius means the results could reflect differences in health outcomes due to socioeconomic status, for example, rather than proximity to natural gas development.”

Tustin et al. (2017). The third in a series of Johns Hopkins University studies led by a researcher who is a fellow at the Post Carbon Institute, this study trumpeted a link between migraine headaches, chronic nasal and sinus symptoms and severe fatigue. But as the RFF report notes, one of its numerous flaws was failing to take into account the myriad of factors that can contribute to these ailments and — once again — failure to even remotely prove causation.

RFF: “Each of these symptoms can be caused by multiple factors, such as stress, sleep deprivation, noise, odors, hormonal issues, toxins, allergens, and more. Though too much weight should not be placed on the results, because of the aforementioned issues with bias in self-reporting symptoms and with exposure proxies, it is suggestive of a relationship between natural gas activity and health effects. Like many of the studies discussed in this section, however, the mechanism for such effects is not able to be ascertained.”

Conclusion

This RFF report confirms the prevailing flaws of fracking health studies EID has been highlighting for years: lack of proof of causation, contradictory findings, faulty methodology and classic “more studies needed” mantra.

And the fact remains that fracking opponents’ preferred scientific study of choice — epidemiological studies — aren’t designed to prove causation in the first place, which means we can expect more of the same going forward, so long as the media continues to give the movement the alarmist headlines it truly desires.