USGS Denies Data Quality Complaint On Highlands Study
USGS Paves Way for Gov. Christie’s DEP Rollback of Highlands Water Protections
US Dept. off Interior – home of USGS
In NJ’s first official taste of the Trump administration’s environmental science policy, on Friday (2/10/17), the United States Geological Survey (USGS) by letter denied a federal Data Quality Act complaint filed by NJ PEER almost a year ago, back on May 18, 2016. (read the PEER complaint)
While noting that the NJ DEP “third party” data USGS relied upon did not meet federal USGS standards, USGS nonetheless found that efforts to make this fact transparent in the text of the Report was an adequate response.
With respect to the flawed DEP data – although inconsistent with USGS’s own data quality standards – USGS “found the data to be of sufficient quality to meet the needs of the study.”
The USGS decision removes a key scientific challenge to the Christie DEP’s regulatory proposal and thereby effectively green-lights the Christie DEP’s proposed rollback of core clean water protections mandated by the 2004 NJ Highlands Act.
In light of the USGS decision, we can expect the DEP proposal to be adopted soon. Highlands defenders must renew their pressure on DEP and demand that the proposal be abandoned.
The USGS decision is presented below. The decision was months overdue. The USGS provided no explanation for the lengthy delay.
The timing of the USGS decision – after many months of delay and just weeks into the Trump administration – is troubling.
During the Presidential campaign, candidate Trump repeatedly criticized environmental regulations and designated NJ Gov. Christie as head of his Transition Team.
In Trump’s first weeks in office – just like his advisor Gov. Christie – Trump issued sweeping Executive Orders to rollback environmental regulatory protections. He directed the US Army Corps of Engineers to reverse the Obama Dakota Access pipeline environmental review process and ordered the State Department to reconsider the Keystone XL pipeline. Trump has pledged to abandon the Paris Climate Accords and withdraw EPA’s Clean Power Plan. He nominated Scott Pruitt, who has sued EPA 14 times and advocated for the oil and gas industry, to head EPA. The head of Trump’s EPA Transition Team threatened to abolish EPA.
All of Trump’s radical anti-environmental policies have virtually identical precedents in NJ under Gov. Christie and his DEP Commissioner Bob Martin.
Potential Grounds For Appeal
The USGS allows for appeals – here are some flaws in the decision that we may appeal:
1. USGS directly contradicts NJ DEP about data limitations and lack of QA/QC
The NJ DEP issued 3 technical Reports that noted significant limitations and flaws with the Private Well Testing Act data upon which the USGS study is based.
Specially, regarding QA/QC, a DEP Division of Science and Research report “The New Jersey Private Well Testing Act: An Overview (April 2009) stated this conclusion: (emphases mine)
The quality of NJDEP’s PWTA database is adversely affected to an unknown extent by several factors. There is no agency responsible for verifying that the data from all real estate transactions (sales and leases) subject to the PWTA are reported to the NJDEP. Therefore, some data is likely missing. Some data that were initially rejected by the E2/COMPASS quality control system were not resubmitted, despite NJDEP efforts to have these data resubmitted. One laboratory failed to submit data over a 3-year period, although this is believed to be an isolated case.
There are errors in the reported data as well. The PWTA relies on the sampling and testing laboratories for proper conduct of sampling, testing, and data accuracy. As previously stated, all laboratories performing PWTA analysis must be certified. That is, they are required to successfully complete periodic performance evaluations. Certification presumably reduces sampling and analysis errors. Nevertheless, there is no ongoing quality control of the data, either following sample collection, during or after testing, or in reporting results to the client or during electronic entry to the NJDEP database. It is not known how many errors exist in the non-location aspects of the data.
The USGS decision failed to even mention the DEP data limitations and flat out contradicted DEP’s QA/QC conclusion, by stating:
The PWTA data is subjected to QA/QC control and review.
DEP says there was no QA/QC – USGS says there was. Which is accurate?
2. The USGS decision responded to criticism by pointing the finger at DEP and/or the Highlands Council.
USGS fails to take responsibility for fatal flaws in the study design that create conflicts with existing law, regulation and policy.
Specifically, we noted several flaws with the USGS study design and methodology that created bias and conflicted with NJ law.
USGS rejected that by blaming NJ DEP, the Highlands Council. For example, in responding to criticism about using non-representative data outside the Preservation Area, USGS blamed DEP:
The area of study was defined by our agreement with DEP.
In response to criticism of bias in the statistical method, USGS again pointed the finger:
Selection of median values for use in the study was a decision made by the Highlands Council, not the USGS.
Furthermore, USGS claimed (correctly) that USGS is not subject to state law, but that surely is no excuse or justification for conducting a study that knowing conflicts with State law and ignores the legislative standard to prevent degradation of groundwater and base decisions on natural background conditions:
USGS, as a Federal Agency, is not subject to State regulations. USGS’ research standards are independent of, and not subject to, New Jersey state laws.
In addition to the flat out contradiction about QA/QC, USGS completely failed to note multiple flaws, errors and limitations that DEP found with the PWTA data (see this post for a summary of how the data were “riddled with errors”).
Below is the full text of the USGS decision.
More to follow after we have time for an in depth review and to consider our options and whether to appeal the decision.
Attachment: USGS Response to IQA Request for Information Correction
(http://www.usgs.gov/info qual/med nitrate concentrations groundwater nj highlands region.html)
The following provide the U.S. Geological Survey (USGS) final responses that specifically correspond to the “numbered” sections outlined in the Public Employees for Environmental Responsibility (PEER) request for information correction.
I. CHALLENGED INFORMATION DOES NOT COMPLY WITH THE INFORMATION QUALITY GUIDELINES
A) DataQuality-Lack of QualityAssurance/QualityControl (QA/QC)
Third party data, such as the Private Well Testing Act (PWTA) data, have different quality assurance standards than data collected and analyzed by the USGS. As required in the USGS’s Information Quality Guidelines (https://www2.usgs.gov/info qual/ section 111.5), the source and limitations of the PWTA data were made transparent in the report. Data used for the study was collected pursuant to the PWTA (N.J.S.A. 58:12A-26 et seq.) as implemented through the regulations at N.J.A.C. 7:9E. The act states that all water tests conducted under the law must be done by certified laboratories. The PWTA data is subjected to QA/QC control and review. The DEP regularly examines submitted data, and if a value appears to be anomalously high, the laboratory is asked to resubmit the value. Also, the submitted latitude and longitude is compared to the mapped lot and block location of the properties from which samples are taken; if they do not match, the location is corrected to the lot and block. The USGS conducted its own review of the data upon receipt and found no obvious indicators of problems. The USGS, NJDEP, and peer reviewers found the data to be of sufficient quality to meet the needs of the study. Furthermore, the use of the extensive data available from the PWTA database materially aided the investigation by improving the spatial data coverage.
In summary, the USGS considers that the PWTA data used in this report are appropriate in quality and geographical distribution to be used for estimating nitrate concentrations in the study area .
B) Data Reliability
1. Basic hydrogeological gaps violate Information Quality Guidelines
Where hydrogeological data were needed for identifying the explanatory variables. only NWIS data, which had the needed information, was used (page 11 of the report). Analysis of the NWIS data did not find well depth to be an important predictor, most likely because most wells are of similar depth and sampling recently recharged water. Although the construction details, including the depth, of the wells in the PWTA data base are not explicitly known, homeowner wells can be reasonably assumed to be shallow. As such, the water samples from these wells can also be reasonably assumed to represent the water quality of the surficial aquifer and can be related to land-surface characteristics. The NJDEP provided the 2000 ft by 2000 ft grid in which each well resides, and its use was determined by appropriate USGS officials to be sufficient for the needs of this study. The absence of hydrogeologic information for the private wells in the PWTA data base did not prevent the USGS from accurately quantifying and validating statistical relationships between land-surface characteristics and nitrate concentrations.
2. “Method Detection Limit” and non-detect values are not credible or reliable
As stated in the report 23 percent of the samples were nondetects and MDLs ranged from 0.05 to 10 mg/1 (page 9). However, less than one percent of the wells (152 wells, page 9 of report) had MDL values that were greater than 0.5 mg/1. More importantly, only five non-detect samples had MDLs above 1.0 mg/1. Thus almost 99.9 percent of the non-detects had MDLs of 1.0 mg/1, or less, which is well within the acceptable range required by the PWTA, as well as the EPA methods cited by PEER. This is well within the normal range of values seen in numerous studies conducted by the USGS. Based on the above, and other QA criteria required by DEP, the data were found to be acceptable for use in the analysis.
Concerns about how different methods of estimating non-detects would impact results were addressed appropriately in the report (page 21) by the use of four different methods for analyzing non-detects. No method produced median nitrate concentrations that were substantially different from any other.
C) Spatial Bias
The spatial bias of well locations is due to the fact that all the wells in PWTA data base are from areas containing homes and therefore are rarely located in forested areas. This was acknowledged in the report. To minimize bias, logistic regression was used to make predictions of median nitrate concentration. In other words, there is bias in the data, but the method used to estimate median nitrate concentrations was designed to minimize bias (see citation by Greene and others, page 25).
1. NJ PWTA data correlate with land use and anthropogenic loadings
The stated objective of the report was to quantify median nitrate concentration in the highlands region, and selected subregions. Although this information was to be used for regulatory purposes USGS did not cite DEPs methodology for developing septic standards because it was not relevant to the report objectives.
This investigation was designed to minimize the acknowledged geographical bias of well locations. A higher groundwater-nitrate concentration in areas with more urban and agricultural land use is a common finding throughout the scientific literature, and is clearly true from the raw data from the NJ Highlands regardless of what dataset is used.
The section of the report entitled “Median of measured nitrate concentrations in the NJ Highlands Region” (page 14) described in detail the median concentrations that would result if this bias was not removed. Methods used to minimize this bias were adequately addressed in this investigation and in the report (see section of the report entitled “Comparison of Median Measured Nitrate Concentrations and Estimated Median Nitrate Concentrations” on page 15).
2. Data sources located outside Highlands Preservation Area
The area of study was defined by our agreement with DEP. The objective of the study was to develop a method that could be used to define the median nitrate concentrations in administratively-defined areas of the entire Highlands Region. The Preservation Area is one such region. Each administrative region has different regulations. Median nitrate concentrations were estimated for all six sub-regions to address the needs of stakeholders interested in a particular area within the Highlands. Any spatial bias in the data set was addressed by use of the regression modeling technique.
Regarding concern that the regional analysis may not be valid for the Preservation Area – A s discussed in the validation and comparison sections of the report (pages 16 and 19 respectively) errors in the estimated median values are acceptable for this type of analysis. However, as indicated in the report, estimation errors will be greater in areas with low nitrate concentrations, such as forest. As is shown in Table 8, the percent difference between the measured and estimated median concentrations is greater in the Preservation and Conservation areas than in more developed areas. This result is expected due to the lower nitrate concentrations commonly found in these areas, but the percent difference is still quite low (4 to 6 percent), and is considered an acceptable result. Therefore there should be no concern about the validity of the model in the Preservation Area. It should also be noted that the median values in the report are for the entire Preservation Area, which contains a mix of land uses (not just forest).
D) StatisticalBias-Selection of Median
Selection of median values for use in the study was a decision made by the Highlands Council, not the USGS. However, use of the median did not bias the data or analysis. The study also addresses the deeper aquifers in that protection of the shallow aquifer recharge area would also protect groundwater that might reach deeper regional flow systems.
The USGS was aware that data could be used to develop standards in the Highlands Region, and was very careful to conduct a thorough, accurate, and impartial analysis. However, questions regarding legislative intent or use of data in developing standards should be forwarded to the appropriate legislative or regulatory authority. The regulatory intended use, or other use by any cooperator or stakeholder, does not determine the methods of scientific investigation that are used by the USGS. The data and report were peer reviewed in accordance with USGS requirements in Survey Manual (SM) chapter SM 502.3 at https://www2.usgs.gov/usgs-manual/500/502-3.html.
E) Administrativeclassificationofwelldatalacksscientificsupportandisinconsistent with law
The administrative areas used in the report are defined in the Highlands Master Plan. Regional analyses used in the report were designed to meet the needs of the Highlands Council and the NJ DEP. The publication is meant to add to our scientific knowledge of the region. The USGS consulted with the various parties mentioned in your request to maximize the usability of information in the report. Thus, median nitrate concentrations are given for each area, zone, and area:zone combination as different stakeholders are interested in nitrate concentrations for different administratively defined portions of the Highlands (including the Preservation Area).
The land use percentages are from values published by the NJDEP and were not provided by the Highlands Council or obtained from the Regional Master Plan.
USGS, as a Federal Agency, is not subject to State regulations. USGS’ research standards are independent of, and not subject to, New Jersey state laws. Although USGS is not subject to State regulations our work is held in high regard, and all States, and numerous agencies, use information we provide. Responsibility for adhering to legislative standards or intent is the responsibility of the user.
II. CHALLENGED INFORMATION IS INFLUENTIAL AND MUST MEET HIGHER STANDARDS
This report is typical of reports that are written to satisfy the needs of a cooperator, many of whom are federal, state or regional regulatory bodies. These reports are often used for regulatory purposes, yet they are not categorized as influential, or highly influential. That being said, this report received a stringent review and approval process (two colleague or peer reviews, supervisory review, specialist review, editorial review, and final review and approval by the Bureau Approving Official), and all review comments were addressed in writing at each review step. The approval process for USGS products is described in SM 502.18 at https://www2.usgs.gov/usgs-manual/200/205-18.html.
Ill. PEER IS AFFECTED BY THE INFORMATION ERRORS No comment.
IV. RECOMMENDATIONS FOR CORRECTION OF THE INFORMATION CHALLENGED BY THIS COMPLAINT
The USGS contends that the authors, peer reviewers, and approving officials followed the appropriate USGS requirements prior to release of the report and therefore stands by the methods, analysis, conclusions, and stated limitations of the report. No corrective action to modify or retract the report will be taken.