By Ana Zivanovic-Nenadovic

Whether I like it or not, the nature of my job and current developments in N.C. coastal policy have forced me to understand the basics of the computer modeling used in permitting decisions. While I am not now an expert, that education has not left me feeling very comfortable about the basis for some of the permitting that is taking place with four small jetties, or groins, that are proposed along the N.C. coast. Let me first provide some introduction.

Four permit applications for construction of terminal groins have been submitted to the Army Corps of Engineers’ district office in Wilmington: Figure Eight Island, Ocean Isle Beach, Holden Beach and Bald Head Island. The purpose for each groin is to protect (a few) oceanfront private properties. These applications were filed after the N.C. General Assembly passed a law last year to allow as many as four groins to be built at inlets along the coast. Legislators chose to ignore the 30-year accumulated wisdom of the state Coastal Resources Commission, which had long banned hardened structures like groins on the beachfront.

So how does this permit process work? Well, the applications go through a fairly complex public process dictated by a federal law called the National Environmental Policy Act, or NEPA. The federal agency taking action, in these cases the Corps issuing the permits, is required to produce an Environmental Impact Statement, or EIS. The EIS is intended to be a comprehensive study of the effects the proposed projects will have on the environment. Various alternative approaches to the proposed project need to be analyzed, compared and then the best alternative is supposed to be identified.

Because the projects haven’t been built yet, the various expected effects of these alternatives are anticipated by using computer models. And because agencies are often understaffed or underfunded or both, the permit applicants often hire “independent” consultants to produce the EIS documents. After reviewing them, the Corps will later adopt them as official documents and open them for public comment. By any measure of common sense, not to mention the requirements of law, you would expect that before adopting the EIS, the Corps would request and review all supporting materials, referenced studies and results of any experiments or modeling performed. Apparently, that is not the case.

I attended most of the public meetings of the ongoing NEPA process for all four applicants. Based on what I saw there, computer modeling of the likely effects of the proposed groins carried the major weight in decision-making, or at least in suggesting best alternatives.

 

What are these “models” and how reliable are their predictions of likely effects? Basically, you take a computer program designed for a particular predictive purpose, input theoretical and observed data to the model, make a bunch of simplifying assumptions along the way and then ask the computer to project numeric results, in this case expected shoreline changes and effects with the proposed groin in place, among others. Such models are used all the time and for many similar legitimate purposes.

The key word with models, though, is “assumption.” Those underlying assumptions are the crux of the validity of the model’s predictions. Incorrect assumptions result in invalid model results. In the case of terminal groins, making assumptions about highly dynamic and complex systems of coastal currents and ever-shifting sands affected by devastating and unpredictable storms is not an easy task and should always raise reasonable caution with respect to the validity of the models’ results.

How accurate are these models, you ask? Well, ironically, there is no way to represent their accuracy in numerical terms. Why? Because they are not statistical models, there is no numerical or statistical measurement that can demonstrate how well the results represent the actual reality or future projection for that matter. When I asked the modelers to tell me what the accuracy of the model they used was, the answer back was: “pretty good.” To me, “pretty good” means a lot when I ask a friend who had a cold how she was feeling, but it doesn’t engender much trust in a model packed with numbers, assumptions and uncertainties.

Luckily, we have some excellent geologists and model connoisseurs in the state who I was able to talk to and get some further insight into how these models work and how the accuracy of their predicted effects can be assessed. Basically, they taught me that the accuracy, also known as the “robustness,” of the model is determined by sensitivity analyses. In other words, analyses can be done that show how sensitive the results or outputs are to any change in inputs or assumptions. Such sensitivity runs are standard practice in any modeling exercise.

Two stages must precede the sensitivity analyses:

  1. Calibration – This process compares how accurately the model works with some already known, historic data. In this stage those known data are used as an input, the model is run and the model’s results are compared to the known real world results, for example, the actual shoreline position in one known, historic period in time. If the results do not accurately represent the given shoreline, then the inputs are tweaked, sort of like a turning a knob little by little, until the model’s shoreline matches the historic shoreline. It is said that model is then “calibrated.
  2. Verification – this step verifies the calibration step. The now-calibrated model is run again using a different set of known, historic data. The results are compared to a real historic shoreline position. If the model is calibrated well, the results should generally match up.

The final stage is a sensitivity analysis, which requires a time-consuming and costly set of model runs. During this stage, the calibrated and verified model is run by using a wide variety of different data options for the same variable. For example, if the variable is groin permeability, the model is run many times using different permeability values. This is repeated for as many variables as necessary. The goal of running these sensitivity analyses is to understand how sensitive the model is to changes of various variables. The more sensitive it is to a particular variable change, the more important it is to focus on the actual inputs being used for that variable.

So, let me get back to the main point –- the Corps’ review of the contractor’s modeling results for the projects being permitted. I requested the modeling results for all three stages from the Corps for Figure Eight Island, whose draft EIS is under Corps’ review, and for Holden Beach, whose draft EIS is still in process. To my surprise, the agency responded that it did not have any modeling results. In fact, it appears that the Corps did not even have access to any data regarding these analyses.

Then, I contacted the contractors, Coastal Planning & Engineering Inc. for Figure Eight Island and Applied Technology Management for Holden Beach, who did the actual modeling and asked them for the same analyses and data. More than two months have passed and I am yet to receive an answer to my request.

So, what does this all mean? Well first of all, I’m not attacking the use of models. Models are useful tools for making decisions and can help the permitting agency and the public understand coastal processes and advance our knowledge in complex and dynamic systems and barrier islands geology. However, given the uncertainties of their accuracy in projecting the actual effects of a particular project on the coast, a model’s outputs should not comprise the major factor for policy making and decisions to permit a project.

And, secondly, the Corps’ failure to verify the contractor’s model or its outputs on the effects of the various alternatives being considered in their environment review raises a number of fundamental and troubling questions for me that go to the heart of the public NEPA process and the future social and ecological wellbeing of coastal communities:

  1. Why are the models and the data on which they are based not available to the public? What happened to NEPA transparency and its public disclosure requirement? While NEPA may not be the strongest environmental law in terms of its potential to stop environmentally destructive projects, it is certainly known for its transparency requirement.
  2. What is the Corps’ commitment to accuracy of one of the major factors it relied on in making its decisions for permitting terminal groins? How can it legally rely on the models developed by an economically interested third party for its decision-making without reviewing the crucial data? And if the Corps isn’t doing its job of verifying these analyses, how can the public then rely on the Corps?
  3. Why can’t the public obtain the modeling results? I don’t know and we are still waiting for answers to that question.
  4. What do we really know about the accuracy and the reliability of the models that are used to project the future of the inlets and shorelines and their effects on the environment? Are best practices actually being followed by applicants and the reviewing agencies, particularly when the institutional wisdom of the coastal commission is being ignored?

Some wise person once said: “Trusting is good, but not trusting is better.” I think that is a good motto when so much is at stake.