A framework of map comparison methods to evaluate geosimulation models from a geospatial perspective

Abstract

Geosimulation is a form of microsimulation that seeks to understand geographical patterns and dynamics as the outcome of micro level geographical processes. Geosimulation has been applied to understand such diverse systems as lake ecology, traffic congestion and urban growth. A crucial task common to these applications is to express the agreement between model and reality and hence the confidence one can have in the model results. Such evaluation requires a geospatial perspective; it is not sufficient if the micro-level interactions are realistic. Importantly the interactions should be such that the meso and macro level patterns that emerge from the model are realistic. In recent years, a host of map comparison methods have been developed that address different aspects of the agreement between model and reality. This paper places such methods in a framework to systematically assess the breadth and width of model performance. The framework expresses agreement at the continuum of spatial scales ranging from local to the whole landscape and separately addresses agreement in structure and presence. A common reference level makes different performance metrics mutually comparable and guides the interpretation of results. The framework is applied for the evaluation of a constrained cellular automata model of the Netherlands. The case demonstrates that a performance assessment lacking either a multi-criteria and multi-scale perspective or a reference level would result in an unbalanced account and ultimately false conclusions.