Kelli F. Johnson,1 Dr. Andre E. Punt2 and Dr. James T. Thorson3
1SAFS
2SAFS
3NWFSC
April 25, 2017 9:00 (PST): FSH 203
Moving beyond Q-Q plots for spatio-temporal models
The assessment of models, which are always wrong, largely consists of two components, assessing the relative performance of candidate models and characterizing differences between observed data and model predictions. Residuals and relative error are traditionally used to identify the presence of outliers, quantify the relevance of modelled and excluded factors, and assess overall model fit. With linear models, the Q-Q plot is a standard a metric of model fit, e.g., the fat-pen test. Unfortunately, Q-Q plots can be misleading when used to assess models that include spatially-explicit parameters, where spatial patterns in residuals can indicate model misspecification. Here, we present the results of a simulation experiment to quantify the performance of the structural similarity index as a metric of model fit. Simulations were conducted using a vector-autoregressive spatio-temporal model that estimated spatio-temporal variation in density from simulated survey data for a hypothetical species off the US West coast. The structural similarity index provided spatially-explicit information regarding lack of model fit not uncovered by traditional model assessment methods. Future work will focus on how well the structural similarity index can identify misspecification, rather than lack of fit in a properly specified model, as was done here.