Generating visual reports is now automatic for many InVEST models

This piece was originally published on the Natural Capital Review, NatCap's Substack.

At NatCap we help people better understand how changes to land- and sea-scapes translate to changes in the benefits people get from nature, and ultimately to changes in human well-being. Building that understanding is a process that includes engaging with stakeholders, collecting data, mapping, modeling, interpreting data and communicating results, and repeating that process as many times as needed. NatCap’s software team builds tools to support this process, and has mostly focused on the data processing and modeling steps (InVEST®  includes 26 different models for mapping and quantifying different ecosystem services and doing other related computational tasks). Now, we are also working on tools to help people better understand and communicate the results of those models. We hope this will help with the “last mile” of a project, which involves translating raw scientific data into meaningful, decision-relevant information.

InVEST Reports is a feature we first released with InVEST 3.18.0, where we started to include visual summaries of results for several InVEST models – something that up until now, users had to create for themselves. These reports offer maps, plots, and summary tables of key outputs and help make connections between the input data and the output data. We first introduced InVEST Reports in this earlier Substack post, but here we detail what we have created and why, and discuss the value of data visualization in the modeling process. Reports are now available for nine models and we are continuing to build and release reports for more InVEST models over time.

The Problem

People use InVEST to do complex data analysis, and “looking at the data” is a critical step in any data analysis project. But sometimes “looking at the data” is easier said than done. We recently surveyed and interviewed InVEST users, asking them, “What is the first thing you do after running an InVEST model?” We heard that everyone needs to perform some type of quality control on their results, and most people do this using desktop Geographic Information Systems (GIS) software. GIS and data visualization are specialized skills, and running an InVEST model can easily generate a workspace filled with dozens of geospatial datasets. InVEST models generally require between 2 and 10 different input datasets and generate a comparable number of outputs. On top of that, all the computation that happens to get from “input” to “output” is captured in “intermediate output” datasets. The modeler needs to sort through those files, often visualizing them one by one, in order to interrogate the results.

Generally, people have a few goals in mind after running an InVEST model:

1. Check if the results appear valid. Are there swaths of missing data in a particular map? Are the numerical outputs the correct order of magnitude and sign?
2. Interpret patterns in the results. Are the spatial patterns what I would expect given my knowledge of the study area? How do these results compare to a previous model run?
3. Communicate the results to others and create a shared understanding of how the results were generated. What do my colleagues think of the latest iteration of results? Which maps or tables best communicate the information to a group of stakeholders?

Accomplishing these objectives can be difficult, especially when the datasets are large. Visualizing them can be time-consuming and repetitive. More often than not, the modeler will determine that they need to run the model again, using slightly different input data or parameters. Each successive iteration can represent higher quality results, but each set of results takes time to interpret.

The Solution

We want to encourage people to do as many model iterations as they need to get the highest quality, most accurate results they can. The InVEST Reports feature aims to facilitate this iterative process by

• making it easier to quickly assess the validity of model results,
• making it clear to beginners what the primary model outputs are and how to interpret them,
• reducing the tedium for people who do lots of model iterations and need to visualize the results each time, and
• providing examples of how to visualize outputs, including figures people can re-use or take inspiration from.

Each report is model-specific, designed in consultation with expert users of that model who provide insight into the visualizations that are appropriate for that model. Here are some key elements of reports that demonstrate how we meet these goals.

Start with the headline

The report builds a narrative about the model run. It starts with the high-level results. If I just ran the InVEST Carbon Storage and Sequestration Model, I want to know how much carbon is being stored under each of my scenarios; this is the “headline” of the report.

Then, in order to make sense of those numbers, I might want to see the spatial patterns of the results:

These types of figures make it easy to quickly check the validity of results. They can reveal areas of missing data within the modelled area, display the range of values present in each dataset, and the spatial patterns in these output maps can be visually compared with patterns in input maps displayed elsewhere in the report. Additional metadata like the units of measurement and descriptive captions help beginners get acquainted with the type of information the model produces, and how to interpret it.

Follow conventional wisdom

InVEST models span a lot of different scientific domains, and sometimes a particular domain, like hydrology, has a conventional way of visualizing information. We want to follow that convention in certain models so that, in this case, hydrologists can quickly get oriented to the information, and also so that non-hydrologists can see an example of how to visualize and interpret their data. Here is an example of a hydrograph from the Seasonal Water Yield model’s report:

This chart displays the monthly combined average baseflow plus quickflow for each feature within the area of interest, as well as the monthly average precipitation. Users can select a watershed on the map to plot the hydrograph for that watershed. This plot synthesizes data from 25 different raster datasets¹!

Encourage understanding of the model processes

There are many aphorisms about scientific modeling, such as:

• “All models are wrong, but some are useful” (Box²).
• "As a model of a complex system becomes more complete, it becomes less understandable […] as a model grows more realistic, it also becomes just as difficult to understand as the real-world processes it represents." (Dutton & Starbuck³; Bonini’s paradox⁴)
• “Garbage in, garbage out.” 

How does one begin to understand if their model is useful, understandable, and not generating garbage? Looking at the data is one of the best ways to do this. In addition to looking at the primary results of the model, it is critical to also look at the data going in, as well as many of the intermediate data products that are stepping stones to the final outputs. For example, some InVEST models do hydrological routing over a Digital Elevation Model (DEM). If the stream network produced by the routing algorithm is unrealistic, then we should be very skeptical of the subsequent results of the model, such as sediment export or nutrient retention. InVEST Reports include maps and summary tables that encourage users to interrogate these intermediate data products and model processes.

Final thoughts

Scientific modeling requires a range of skills, including domain knowledge, data wrangling, visualization, and effective communication. We hope Reports will make InVEST models more approachable and reduce some of the friction in the modeling process. As always, the NatCap software team loves hearing from users about your experience with new features. Please find us on the NatCap Community Forum, where you can also learn how to share your reports with others.

InVEST 3.20.0 generates reports for nine models: Annual Water Yield, Carbon Storage and Sequestration, Coastal Vulnerability, Nutrient Delivery Ratio, Seasonal Water Yield, Sediment Delivery Ratio, Urban Cooling, Urban Mental Health, and Urban Nature Access. You can get InVEST from the InVEST downloads page. You can also check out some sample InVEST Reports. More reports are coming soon!

¹ 12 monthly datasets each for precipitation and quickflow, plus one baseflow dataset. 
² Box, George E. P. (1976), "Science and statistics" (PDF), Journal of the American Statistical Association, 71 (356): 791–799, doi:10.1080/01621459.1976.10480949.  
³ Dutton, John M; Starbuck, William H (1971). Computer Simulation of Human Behavior. Wiley. ISBN 0471228508.  
⁴  Bonini’s paradox (Wikipedia)

Dave is a Senior Software Engineer at the Natural Capital Alliance (based out of Stanford University) where he has contributed to InVEST and related software projects since 2018. Prior to joining the software team, he was a GIS Analyst for NatCap. He started learning about data visualization, and the importance of “looking at the data”, while earning a M.S. from the Department of Geography at the University of Oregon. InVEST Reports have been authored by many members of the NatCap software team in addition to Dave, including Emily Davis, Megan Nissel, and Claire Simpson.