Validating VAST models using simulation-based residuals

Tuesday April 7th at 2:00 PM (PST)

Andrea Havron and Cole Monnahan

Spatiotemporal delta-models are becoming increasingly popular for a wide variety of uses in ecology and fisheries science. These models rely on a hierarchical variance structure (eg., process and observation error) making standard validation measures, such as Pearson residuals, difficult to interpret. Quantile residuals provide a solution where simulations from the model are compared against observations using the cumulative distribution function (cdf) of a given distribution. This method is complicated, however, when the cdf function is not easily specified, as is this case in delta-models which consist of a mixture of zeros and positive catches. Here, we demonstrate model validation on VAST models (conditioned on EBS pollock) using the “empirical cdf” approach of Hartig (2020). We demonstrate how to validate both the process and observation components of the model, and contrast the advantages of this new approach with existing methods for model validation, and model selection tools. We conclude with practical advice for VAST users.

Challenges to estimating maturity in stock assessment: a case study of Pacific herring in Prince William Sound, AK

Tuesday April, 21st at 2:00 PM (PST)

John T. Trochta & Trevor A. Branch

Mature proportions at age or length, known as the maturity ogive, are important parameters in stock assessment. Information on the maturity ogive is consequential for fisheries management because it determines the final spawning biomass estimated by models and used for decision making (e.g. via harvest control rules). However, obtaining an accurate maturity ogive is particularly challenging for fish populations such as Pacific herring, in which the surveyed aggregations are not representative of the true population. For Pacific herring in Prince William Sound, Alaska, there is evidence not all fish are available to sampling of pre-spawning and spawning aggregations which are otherwise assumed to represent all spawning fish in the age-structured assessment (ASA). We investigated these and other maturity-related issues in the Prince William Sound herring ASA by conducting a sensitivity analysis with the ASA. We make different assumptions about the true maturity ogive and availability of herring to surveys to develop a suite of 11 models that bound the range of effects from mis-specifying maturity in the ASA. These effects are represented by key model outputs we compare across models. We also use Bayesian model selection to rank the most likely models. In this talk, I describe survey and modeling challenges regarding maturity, methods, and results from this analysis, preliminarily concluding the herring ASA is robust to different assumptions on maturity.

Developing SSMSE, an R package for Management Strategy Evaluation with Stock Synthesis

Tuesday May 12th, 2020: 2:00 PM PST

Kathryn L. Doering (1), Nathan R. Vaughan (2) , John F. Walter (3), Richard D. Methot (4), Skyler R. Sagarese (3), Matthew Smith (3), Nancie Cummings (3), Nicholas A. Farmer (5), Shannon Calay (3), Kelli Johnson (6), Kristin Marshall (6), Cassidy Peterson (7), Ian Taylor (6), and Chantel Wetzel (6)

1. Caelum Research Corporation in support of Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

2. Vaughan Analytics in support of Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

3. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

4. NOAA Senior Scientist for Stock Assessments, National Marine Fisheries Service, National Oceanic and Atmospheric Administration, Seattle, WA

5. Southeast Regional Office, National Oceanic and Atmospheric Administration, St. Petersburg, FL

6. Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

7. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Beaufort, NC

Management Strategy Evaluation (MSE) is designed to holistically evaluate alternative management strategies, data collection approaches, and modeling structures. While MSE is becoming routine, the task of conducting it is currently limited by the challenge of creating realistic operating models (OMs) for the population and fishery processes. The stock assessment software package Stock Synthesis (SS) represents one of the most complete, generalizable population assessment models available, which is why it provides the necessary architecture for developing complex MSE OMs, while also formalizing the parameterization, fitting, and evaluation of data fit to a model. Pre-existing SS assessment models therefore represent a potential pool of rich OMs, however attempts to use this existing resource for MSE often result in simplified versions of the original SS model itself. We propose an alternative path to MSE functionality of using stock assessments implemented in SS as OMs. The goal of this project is to build the capacity to facilitate converting any SS model into an OM with as much of the OM engine as possible coded within SS, thereby making OM output an innate capacity of SS. Users will be able to access this capability and conduct MSE analyses through an R package we are developing called SSMSE (https://github.com/nmfs-fish-tools/SSMSE). In this presentation, the features of SSMSE will be described and demonstrated. Further, we discuss future directions to expand the functionality of the SSMSE package. We welcome your suggestions and comments on this in-progress project throughout the presentation. This approach should allow the complexity of the MSE OMs to grow with SS, leveraging the future development of SS and innately linking the MSE and stock assessment processes.

Presentation Slides