Validating VAST models using simulation-based residuals

Tuesday April 7th at 2:00 PM (PST)

Andrea Havron and Cole Monnahan

Spatiotemporal delta-models are becoming increasingly popular for a wide variety of uses in ecology and fisheries science. These models rely on a hierarchical variance structure (eg., process and observation error) making standard validation measures, such as Pearson residuals, difficult to interpret. Quantile residuals provide a solution where simulations from the model are compared against observations using the cumulative distribution function (cdf) of a given distribution. This method is complicated, however, when the cdf function is not easily specified, as is this case in delta-models which consist of a mixture of zeros and positive catches. Here, we demonstrate model validation on VAST models (conditioned on EBS pollock) using the “empirical cdf” approach of Hartig (2020). We demonstrate how to validate both the process and observation components of the model, and contrast the advantages of this new approach with existing methods for model validation, and model selection tools. We conclude with practical advice for VAST users.

Challenges to estimating maturity in stock assessment: a case study of Pacific herring in Prince William Sound, AK

Tuesday April, 21st at 2:00 PM (PST)

John T. Trochta & Trevor A. Branch

Mature proportions at age or length, known as the maturity ogive, are important parameters in stock assessment. Information on the maturity ogive is consequential for fisheries management because it determines the final spawning biomass estimated by models and used for decision making (e.g. via harvest control rules). However, obtaining an accurate maturity ogive is particularly challenging for fish populations such as Pacific herring, in which the surveyed aggregations are not representative of the true population. For Pacific herring in Prince William Sound, Alaska, there is evidence not all fish are available to sampling of pre-spawning and spawning aggregations which are otherwise assumed to represent all spawning fish in the age-structured assessment (ASA). We investigated these and other maturity-related issues in the Prince William Sound herring ASA by conducting a sensitivity analysis with the ASA. We make different assumptions about the true maturity ogive and availability of herring to surveys to develop a suite of 11 models that bound the range of effects from mis-specifying maturity in the ASA. These effects are represented by key model outputs we compare across models. We also use Bayesian model selection to rank the most likely models. In this talk, I describe survey and modeling challenges regarding maturity, methods, and results from this analysis, preliminarily concluding the herring ASA is robust to different assumptions on maturity.

Developing SSMSE, an R package for Management Strategy Evaluation with Stock Synthesis

Tuesday May 12th, 2020: 2:00 PM PST

Kathryn L. Doering (1), Nathan R. Vaughan (2) , John F. Walter (3), Richard D. Methot (4), Skyler R. Sagarese (3), Matthew Smith (3), Nancie Cummings (3), Nicholas A. Farmer (5), Shannon Calay (3), Kelli Johnson (6), Kristin Marshall (6), Cassidy Peterson (7), Ian Taylor (6), and Chantel Wetzel (6)

1. Caelum Research Corporation in support of Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

2. Vaughan Analytics in support of Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

3. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Miami, FL

4. NOAA Senior Scientist for Stock Assessments, National Marine Fisheries Service, National Oceanic and Atmospheric Administration, Seattle, WA

5. Southeast Regional Office, National Oceanic and Atmospheric Administration, St. Petersburg, FL

6. Northwest Fisheries Science Center, National Oceanic and Atmospheric Administration, Seattle, WA

7. Southeast Fisheries Science Center, National Oceanic and Atmospheric Administration, Beaufort, NC

Management Strategy Evaluation (MSE) is designed to holistically evaluate alternative management strategies, data collection approaches, and modeling structures. While MSE is becoming routine, the task of conducting it is currently limited by the challenge of creating realistic operating models (OMs) for the population and fishery processes. The stock assessment software package Stock Synthesis (SS) represents one of the most complete, generalizable population assessment models available, which is why it provides the necessary architecture for developing complex MSE OMs, while also formalizing the parameterization, fitting, and evaluation of data fit to a model. Pre-existing SS assessment models therefore represent a potential pool of rich OMs, however attempts to use this existing resource for MSE often result in simplified versions of the original SS model itself. We propose an alternative path to MSE functionality of using stock assessments implemented in SS as OMs. The goal of this project is to build the capacity to facilitate converting any SS model into an OM with as much of the OM engine as possible coded within SS, thereby making OM output an innate capacity of SS. Users will be able to access this capability and conduct MSE analyses through an R package we are developing called SSMSE (https://github.com/nmfs-fish-tools/SSMSE). In this presentation, the features of SSMSE will be described and demonstrated. Further, we discuss future directions to expand the functionality of the SSMSE package. We welcome your suggestions and comments on this in-progress project throughout the presentation. This approach should allow the complexity of the MSE OMs to grow with SS, leveraging the future development of SS and innately linking the MSE and stock assessment processes.

Presentation Slides

Studying population dynamics and research strategies for endangered Cook Inlet Belugas

Tuesday, February 25th at 2:00 PM (PST) in FSH 203

Stephanie Thurner and Amanda Warlick

Cook Inlet belugas (Delphinapterus leucas, CIB) are a small, geographically isolated population in Cook Inlet, Alaska. In the 1990s, the CIB population experienced a notable decline and despite the cessation of hunting in 2005, there is limited evidence of recovery to date, with most recent surveys estimating further decline. Consequently, CIB are listed as endangered under the U.S Endangered Species Act. To examine population viability, identify factors limiting recovery, and further evaluate the potential contribution of research activities designed to monitor the CIB population status we are (1) developing an integrated population model and (2) analyzing research strategies. Today, we will talk about our work to date and hope to garner feedback as we continue to engage on these projects.

Integrated population modeling to examine population dynamics and viability – Amanda Warlick

Substantial monitoring effort and resources have been invested to conduct aerial and mark-resight surveys of the CIB population, yet considerable uncertainty still exists about demographic rates, population abundance, and potential factors limiting recovery. One way to improve our understanding of population dynamics and future viability is through integrated population modeling, where multiple sources of information are combined to reduce bias and improve precision in life-history parameter estimates. Here we build on a recent integrated population model to estimate time-varying adult survival, fecundity, and abundance using aerial and mark-resight survey data collected from 2004-2018. We outline a framework for conducting a population viability analysis (PVA) to quantify the magnitude of change in extinction probabilities across a range of demographic rates estimated in the integrated model. The PVA will be used in the future to examine the potential effects of anthropogenic mortality, decreased fecundity, or reduced carrying capacity due to the unquantified and unknown effects of stressors such as underwater noise, prey depletion, or habitat range contraction. This information will help identify factors limiting recovery and what, if any, management actions could ameliorate the effects of the most impactful anthropogenic activities.

Research strategy analysis – Stephanie Thurner

We are conducting a Research Strategy Analysis to evaluate the potential contribution of research actions designed to monitor CIB population status, as required by the CIB Recovery Plan. Aerial surveys of marine wildlife species that aggregate in groups are generally susceptible to four major sources of bias that could lead to underestimation of population abundance: (1) group availability bias, (2) group perception bias, (3) individual availability bias, and (4) individual perception bias. We evaluate the potential for accurate and precise estimation of CIB abundance and trends using aerial surveys involving a new aerial survey design that is standardized to ensure consistent coverage across years and provides spatially and temporally replicated counts. Preliminary results show that both linear regression and Multivariate Auto-Regressive State-Space models were able to distinguish stable (0% annual growth), increasing (~2% annual growth), and decreasing (~2% annual decrease) trends, despite various types of availability and detection bias as long as there were no long-term trends in bias. A Bayesian N-mixture-type model was able to estimate unbiased annual abundance under various types of availability and detection bias acting together; but performance deteriorated when sources of bias occurred individually.

There is no best method for constructing size-transition matrices for size-structured models

Tuesday, February 18th at 2:00 PM (PST) in FSH 106

Lee Cronin-Fine and Andrè E. Punt

Stock assessment methods for many invertebrate stocks, including crab stocks in the Bering Sea and Aleutian Islands region of Alaska rely on size-structured population dynamics models. A key component of these models is the size-transition matrix, which specifies the probability of growing from one size-class to another after a certain period of time. Size-transition matrices can be defined using three parameters, the growth rate (k), the asymptotic height (L_∞), and the variability in the size increment. Most assessments assume that all individuals follow the same growth curve. Unfortunately, not accounting for individual variation in growth can result in biased estimates of growth parameters. It is also unrealistic to assume that every individual has the same k and L_∞. We developed a new method for constructing size-transition matrices that allows k and L_∞ to vary among individuals. We compared this technique to two other methods each with different assumptions about individual variation in growth. The first assumes all individuals follow the same growth curve. The second assumes individuals follow one of three growth curves through the “platoon” method. This method divides the population into separate platoons, each with their own growth curve and size transition matrix. My talk shall describe the methods and results from this research and provide suggestions on how best to model individual variation in growth in size structured models.

Is ignoring predation mortality leading to an inability to achieve management goals in Alaska?

Tuesday, January 28th at 2:00 PM (PST) in FSH 203

Grant Adams

Changes in the dynamics of exploited marine resources have long been understood to coincide with, or be the result of, trophic interactions. However, tactical management of exploited marine resources almost universally effectively ignores trophic interactions despite copious calls for Ecosystem Based Fisheries Management (EBFM). In particular, predation is thought to represent a large proportion of mortality of groundfish in Alaska, and research continues to be needed to identify the relevance of predation to management performance. Management strategy evaluation is used to examine the impact of ignoring predation mortality on the performance of the current management strategy for walleye pollock, pacific cod, and arrowtooth flounder in the Gulf of Alaska and Bering Sea. Multi-species age-structured operating models fitted to available data are used both systems. The current management approach of using single-species assessment models linked to single-species harvest control rules is evaluated under alternative predation and fishing scenarios. Performance for each scenario is evaluated by comparing the ability of the current management strategy to achieve single-species biological reference points, maximize catch, and reduce bias in biomass estimates.

Thinking about the methods & climate-dependencies in time-varying biological reference points: a case study with summer flounder.

Cecilia O’Leary

December 3rd, 2019 9:00 AM (PST): FSH 203

Fisheries managers use biological reference points (BRPs) as targets or limits on fishing and biomass to maintain productive levels of fish stock biomass. There are multiple ways to calculate BRPs when biological parameters are time-varying. Using summer flounder (Paralichthys dentatus) as a case study, we investigated time-varying approaches in concert with climate-linked population models to understand the impact of environmentally-driven variability in natural mortality, recruitment, and size-at-age on two commonly-used BRPs (B_0(t) and F_35%(t) ). We used two approaches to calculate time-varying BRPs: dynamic-BRP and moving-average-BRP. We quantified the variability and uncertainty of different climate dependencies and estimation approaches, attributed BRP variation to variation in life-history processes, and evaluated how using different approaches impacts estimates of stock status. Results indicate that the dynamic BRP approach using the climate-linked natural mortality model produced the least variable reference points compared to others calculated. Summer flounder stock status depended on the estimation approach and climate model used. These results emphasize that understanding climate dependencies is important for summer flounder reference points and perhaps other species, and careful consideration is warranted when considering what time-varying approach to use, ideally based upon simulation studies within a proposed set of management procedures.

Evaluating the impacts of fixing or estimating natural mortality, across life histories and data availability.

Claudio A. Castillo Jordan

November 12th, 2019 9:00 AM (PST): FSH 203

In integrated, age-structured stock assessment model, common practice gives us two alternatives on how to use natural mortality. First, to use a value coming from an indirect method and not estimate it. Second, natural mortality is estimated in the model simultaneously with other parameters. We used simulation testing with models simplified from real stock assessments to generate simulated data, using the ss3sim framework. The operating model “truth” included various levels of complexity, such as sex- and age-specific, and time-varying M.  Simulated data sets were used to investigate the bias and uncertainty in estimates of natural mortality and management quantities when assuming and estimating various levels of complexity of M in stock assessments, as well as the data sources that inform natural mortality by varying the type and amount of data available to the model. Given a large number of assessment models available and the range of data, life histories, and other characteristics that they cover, the results were evaluated to determine what characteristics are essential to be able to reliably estimate natural mortality. The uncertainty in the estimates of M was contrasted with the variation in the different indirect methods to evaluate in what context estimation of M inside the stock assessment model is an improvement over the indirect methods and in which cases it is worse. This study provides information about the overall uncertainty in management quantities given the uncertainty in M.

Combining fishery-independent and -dependent catch observations in a single index of abundance by allowing for spatially varying catchability 

John Best, Jim Thorson, Rick Methot, Andre Punt

October 29th, 2019 9:00 AM (PST): FSH 203

For fisheries where both fishery-independent and -dependent catch observations are available, spatiotemporally standardized indices of abundance currently use only the fishery-independent data because it is difficult to reliably standardize fishery-dependent observations. There may be an order of magnitude more fishery-dependent observations than fishery-independent observations, representing a currently unused source of information. Integrating these two types of data into a single index of abundance may increase precision, but the fishery-dependent observations would need to be standardized carefully to avoid unacceptable levels of additional bias. One approach to standardizing fishery-dependent observations in a spatiotemporal standardization model would include spatial and/or spatiotemporal random effects on catchability. A model that allows this will be presented along with preliminary simulation study results.