Moving beyond Q-Q plots for spatio-temporal models

Kelli F. Johnson,1 Dr. Andre E. Punt2 and Dr. James T. Thorson3

April 25, 2017 9:00 (PST): FSH 203

Moving beyond Q-Q plots for spatio-temporal models

The assessment of models, which are always wrong, largely consists of two components, assessing the relative performance of candidate models and characterizing differences between observed data and model predictions. Residuals and relative error are traditionally used to identify the presence of outliers, quantify the relevance of modelled and excluded factors, and assess overall model fit. With linear models, the Q-Q plot is a standard a metric of model fit, e.g., the fat-pen test. Unfortunately, Q-Q plots can be misleading when used to assess models that include spatially-explicit parameters, where spatial patterns in residuals can indicate model misspecification. Here, we present the results of a simulation experiment to quantify the performance of the structural similarity index as a metric of model fit. Simulations were conducted using a vector-autoregressive spatio-temporal model that estimated spatio-temporal variation in density from simulated survey data for a hypothetical species off the US West coast. The structural similarity index provided spatially-explicit information regarding lack of model fit not uncovered by traditional model assessment methods. Future work will focus on how well the structural similarity index can identify misspecification, rather than lack of fit in a properly specified model, as was done here.

Posted in Fisheries Think Tank.

Leave a Reply

Your email address will not be published.