Validating VAST models using simulation-based residuals

Tuesday April 7th at 2:00 PM (PST)

Andrea Havron and Cole Monnahan

Spatiotemporal delta-models are becoming increasingly popular for a wide variety of uses in ecology and fisheries science. These models rely on a hierarchical variance structure (eg., process and observation error) making standard validation measures, such as Pearson residuals, difficult to interpret. Quantile residuals provide a solution where simulations from the model are compared against observations using the cumulative distribution function (cdf) of a given distribution. This method is complicated, however, when the cdf function is not easily specified, as is this case in delta-models which consist of a mixture of zeros and positive catches. Here, we demonstrate model validation on VAST models (conditioned on EBS pollock) using the “empirical cdf” approach of Hartig (2020). We demonstrate how to validate both the process and observation components of the model, and contrast the advantages of this new approach with existing methods for model validation, and model selection tools. We conclude with practical advice for VAST users.

Challenges to estimating maturity in stock assessment: a case study of Pacific herring in Prince William Sound, AK

Tuesday April, 21st at 2:00 PM (PST)

John T. Trochta & Trevor A. Branch

Mature proportions at age or length, known as the maturity ogive, are important parameters in stock assessment. Information on the maturity ogive is consequential for fisheries management because it determines the final spawning biomass estimated by models and used for decision making (e.g. via harvest control rules). However, obtaining an accurate maturity ogive is particularly challenging for fish populations such as Pacific herring, in which the surveyed aggregations are not representative of the true population. For Pacific herring in Prince William Sound, Alaska, there is evidence not all fish are available to sampling of pre-spawning and spawning aggregations which are otherwise assumed to represent all spawning fish in the age-structured assessment (ASA). We investigated these and other maturity-related issues in the Prince William Sound herring ASA by conducting a sensitivity analysis with the ASA. We make different assumptions about the true maturity ogive and availability of herring to surveys to develop a suite of 11 models that bound the range of effects from mis-specifying maturity in the ASA. These effects are represented by key model outputs we compare across models. We also use Bayesian model selection to rank the most likely models. In this talk, I describe survey and modeling challenges regarding maturity, methods, and results from this analysis, preliminarily concluding the herring ASA is robust to different assumptions on maturity.

Developing SSMSE, an R package for Management Strategy Evaluation with Stock Synthesis

Tuesday May 12th, 2020: 2:00 PM PST

Kathryn L. Doering (1), Nathan R. Vaughan (2) , John F. Walter (3), Richard D. Methot (4), Skyler R. Sagarese (3), Matthew Smith (3), Nancie Cummings (3), Nicholas A. Farmer (5), Shannon Calay (3), Kelli Johnson (6), Kristin Marshall (6), Cassidy Peterson (7), Ian Taylor (6), and Chantel Wetzel (6)

1. Caelum Research Corporation in support of Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

2. Vaughan Analytics in support of Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

3. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

4. NOAA Senior Scientist for Stock Assessments, National Marine Fisheries Service, National Oceanic and Atmospheric Administration, Seattle, WA

5. Southeast Regional Office, National Oceanic and Atmospheric Administration, St. Petersburg, FL

6. Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

7. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Beaufort, NC

Management Strategy Evaluation (MSE) is designed to holistically evaluate alternative management strategies, data collection approaches, and modeling structures. While MSE is becoming routine, the task of conducting it is currently limited by the challenge of creating realistic operating models (OMs) for the population and fishery processes. The stock assessment software package Stock Synthesis (SS) represents one of the most complete, generalizable population assessment models available, which is why it provides the necessary architecture for developing complex MSE OMs, while also formalizing the parameterization, fitting, and evaluation of data fit to a model. Pre-existing SS assessment models therefore represent a potential pool of rich OMs, however attempts to use this existing resource for MSE often result in simplified versions of the original SS model itself. We propose an alternative path to MSE functionality of using stock assessments implemented in SS as OMs. The goal of this project is to build the capacity to facilitate converting any SS model into an OM with as much of the OM engine as possible coded within SS, thereby making OM output an innate capacity of SS. Users will be able to access this capability and conduct MSE analyses through an R package we are developing called SSMSE (https://github.com/nmfs-fish-tools/SSMSE). In this presentation, the features of SSMSE will be described and demonstrated. Further, we discuss future directions to expand the functionality of the SSMSE package. We welcome your suggestions and comments on this in-progress project throughout the presentation. This approach should allow the complexity of the MSE OMs to grow with SS, leveraging the future development of SS and innately linking the MSE and stock assessment processes.

Presentation Slides

Studying population dynamics and research strategies for endangered Cook Inlet Belugas

Tuesday, February 25th at 2:00 PM (PST) in FSH 203

Stephanie Thurner and Amanda Warlick

Cook Inlet belugas (Delphinapterus leucas, CIB) are a small, geographically isolated population in Cook Inlet, Alaska. In the 1990s, the CIB population experienced a notable decline and despite the cessation of hunting in 2005, there is limited evidence of recovery to date, with most recent surveys estimating further decline. Consequently, CIB are listed as endangered under the U.S Endangered Species Act. To examine population viability, identify factors limiting recovery, and further evaluate the potential contribution of research activities designed to monitor the CIB population status we are (1) developing an integrated population model and (2) analyzing research strategies. Today, we will talk about our work to date and hope to garner feedback as we continue to engage on these projects.

Integrated population modeling to examine population dynamics and viability – Amanda Warlick

Substantial monitoring effort and resources have been invested to conduct aerial and mark-resight surveys of the CIB population, yet considerable uncertainty still exists about demographic rates, population abundance, and potential factors limiting recovery. One way to improve our understanding of population dynamics and future viability is through integrated population modeling, where multiple sources of information are combined to reduce bias and improve precision in life-history parameter estimates. Here we build on a recent integrated population model to estimate time-varying adult survival, fecundity, and abundance using aerial and mark-resight survey data collected from 2004-2018. We outline a framework for conducting a population viability analysis (PVA) to quantify the magnitude of change in extinction probabilities across a range of demographic rates estimated in the integrated model. The PVA will be used in the future to examine the potential effects of anthropogenic mortality, decreased fecundity, or reduced carrying capacity due to the unquantified and unknown effects of stressors such as underwater noise, prey depletion, or habitat range contraction. This information will help identify factors limiting recovery and what, if any, management actions could ameliorate the effects of the most impactful anthropogenic activities.

Research strategy analysis – Stephanie Thurner

We are conducting a Research Strategy Analysis to evaluate the potential contribution of research actions designed to monitor CIB population status, as required by the CIB Recovery Plan. Aerial surveys of marine wildlife species that aggregate in groups are generally susceptible to four major sources of bias that could lead to underestimation of population abundance: (1) group availability bias, (2) group perception bias, (3) individual availability bias, and (4) individual perception bias. We evaluate the potential for accurate and precise estimation of CIB abundance and trends using aerial surveys involving a new aerial survey design that is standardized to ensure consistent coverage across years and provides spatially and temporally replicated counts. Preliminary results show that both linear regression and Multivariate Auto-Regressive State-Space models were able to distinguish stable (0% annual growth), increasing (~2% annual growth), and decreasing (~2% annual decrease) trends, despite various types of availability and detection bias as long as there were no long-term trends in bias. A Bayesian N-mixture-type model was able to estimate unbiased annual abundance under various types of availability and detection bias acting together; but performance deteriorated when sources of bias occurred individually.