Disclaimer
This is the one of (what should be) a few posts which aim to connect some basic puzzles in the philosophy and methodology of science to the practice of mathematical and computational modelling. They are not intended to be particularly deep philosophically or to be (directly) practical scientifically. Nor are they fully complete expositions. Still, I find thinking about these puzzles in this context to be an interesting exercise which might provide a conceptual guide for better understanding (and perhaps improving?) the practice of mathematical and computational modelling. These are written by a mathematical modeller grappling with philosophical questions, rather than by a philosopher, so bear that in mind! Comments, criticisms and feedback of course welcome! [Current version: 2.0]

Overview
I. Model closure and formalism in economics
II. A topological metaphor for model closure – manifolds, charts and atlases

I. Model closure and formalism in economics
Lars P. Syll gives a nice quote here from Shelia Dow, an expert in economic methodology (who I haven’t encountered before), on what is required for obtaining model closure in the context of economics:

…structures with fixed (or at least predictably random) interrelations between separable parts (e.g., economic agents) and predictable (or at least predictably random) outside influences…

…Any formal model is a closed system. Variables are specified and identified as endogenous or exogenous, and relations are specified between them. This is a mechanism for separating off some aspect of an open-system reality for analysis. But, for consistency with the subject matter, any analytical closure needs to be justified on the grounds that, for the purposes of the analysis, it is not unreasonable to treat the variables as having a stable identity, for them to have stable interrelations and not to be subject to unanticipated influences from outside … But in applying such an analysis it is important then to consider what has been assumed away…

I take the quote to make quite similar points to my previous post about the need not just for all formal models to have a closure but also where these closure assumptions come in. This post tried to connect the issue of model closure to debates about ‘catchall’ hypotheses in Bayesian inference.

The point I argued was that the appropriate ‘model closure’ for Bayesian inference (and of course all formal models have a closure, as Shelia argues) occurs (or should occur) at the level of model structure and ‘boundary conditions’ (priors) and does not require a probability distribution over the ‘background’ or ‘external’ variables.

Rather, closure occurs via a collection of ‘structural’ conditional probability statements with some variables only appearing on the right-hand side of the conditioning (and hence not possessing/requiring a probability distribution). These provide an assumed separation into ‘inside the system’, a ‘closed boundary’ based on experimentally-controlled variables and the external/irrelevant ‘outside the system’ variables. Once this closure is established, Bayesian inference can be carried out within this boundary, where probability distributions can be normalised, but not outside. This closure is always temporary and falsifiable, however, and requires qualitatively different inference methods for assessing its validity, such as ‘pure significance’ tests, unless again embedded in a further higher-level model. This is why there can be ‘falsificationist Bayesians’ (e.g. Andrew Gelman).

Note the subtle point that we are constructing a sort of ‘meta-model’ of the process of inference itself, into which we embed particular models of interest.

As I stated in the comments on Lars’ blog, in some ways I’m more optimistic about closure – I think the search for model closure is the search for interesting theories and is part of the ‘stupendous beauty of closure’ referenced in my post. I agree, however, that we may not always be able to find it (especially for ‘messy’ subjects like biology, psychology, economics etc). This came up in my exchanges with the philosopher Greg Gandenberger and is something I need to elaborate on at some point.

This latter issue, as I see it, has its clarification through the roles of idealisation and approximation and the separation of the ‘formal/mathematical’ and ‘actual/possible’ worlds. I’ll return to the topic of formalising (to some extent) the processes of idealisation, measurement and ‘seeing’ vs ‘doing’ at some point in the future. For now just keep in mind that every closure establishes a formal model by separating it off from the real world at some point and hence, as the cliche goes, ‘all models are wrong’. They are ‘wrong’ because of the closure; however, the ‘but some are useful‘ part comes in when we find useful closures.

Useful/beautiful closures may or may not exist – that’s the fun and challenge of doing science!

II. A topological metaphor for model closure – manifolds, charts and atlases
A metaphor for the role of limited (closed) theories can be found in topology and geometry: we might imagine a collection of possibilities of the ‘real world’ as a sort of Platonic abstract manifold of some sort to which we have finite access (to do – Plato’s cave…, possible worlds). Our (closed) models form a patchwork of charts, each of which only cover a small part of the possible world manifold. As the Wikipedia page states

It is not generally possible to describe a manifold with just one chart, because the global structure of the manifold is different from the simple structure of the charts. For example, no single flat map can represent the entire Earth without separation of adjacent features across the map’s boundaries or duplication of coverage. When a manifold is constructed from multiple overlapping charts, the regions where they overlap carry information essential to understanding the global structure.

As mathematical modellers using idealisations (closures) we inevitably use limited models (charts) only covering some part of the target world. Furthermore we may/will never be able to fully cover the entire ‘possible world manifold’. Our best hope is a collection of charts – a so-called atlas – covering as much as we can and with certain mutual consistency properties. In the Bayesian account, each chart would be analogous to a probability distribution over possible parameter values/’true’ states of the world. Note that in this interpretation of the level of model closure, the Bayesian account appears to require some sort of ‘possible world‘ interpretation.

Returning to the topological metaphor, it is again helpful for guiding us on understanding consistency properties – to compare two models of our state of knowledge (charts) we require the analogy of a transition map (see the atlas page) between the charts (models). Without these we cannot compare models of our state of knowledge/information (charts).

This corresponds to the intuitive Bayesian and Likelihoodist constraint that many advocate: one should not compare parameter values between different models unless embedded into a larger model or a mapping between models is provided.

An interesting connection to explore further in the epistemology literature is whether Susan Haack’s ‘crossword puzzle’ metaphor relates to the topological metaphor given here. Her crossword puzzle metaphor for epistemology involves the search for a collection of words (think models/knowledge) satisfying both external empirical ‘clues’ (data) and mutual consistency of ‘intersecting words’ (coherence/invariance properties).