🦋Science: On Truth in Models (Quotes)

"A model, like a novel, may resonate with nature, but it is not a ‘real’ thing. Like a novel, a model may be convincing - it may ‘ring true’ if it is consistent with our experience of the natural world. But just as we may wonder how much the characters in a novel are drawn from real life and how much is artifice, we might ask the same of a model: How much is based on observation and measurement of accessible phenomena, how much is convenience? Fundamentally, the reason for modeling is a lack of full access, either in time or space, to the phenomena of interest." (Kenneth Belitz, Science, Vol. 263, 1944)

"Exact truth of a null hypothesis is very unlikely except in a genuine uniformity trial." (David R Cox, "Some problems connected with statistical inference", Annals of Mathematical Statistics 29, 1958)

"[…] no models are [true] = not even the Newtonian laws. When you construct a model you leave out all the details which you, with the knowledge at your disposal, consider inessential. […] Models should not be true, but it is important that they are applicable, and whether they are applicable for any given purpose must of course be investigated. This also means that a model is never accepted finally, only on trial." (Georg Rasch, "Probabilistic Models for Some Intelligence and Attainment Tests", 1960)

"The validation of a model is not that it is 'true' but that it generates good testable hypotheses relevant to important problems." (Richard Levins, "The Strategy of Model Building in Population Biology", 1966)

"A theory has only the alternative of being right or wrong. A model has a third possibility: it may be right, but irrelevant." (Manfred Eigen, 1973)

"Models, of course, are never true, but fortunately it is only necessary that they be useful. For this it is usually needful only that they not be grossly wrong. I think rather simple modifications of our present models will prove adequate to take account of most realities of the outside world. The difficulties of computation which would have been a barrier in the past need not deter us now." (George E P Box, "Some Problems of Statistics and Everyday Life", Journal of the American Statistical Association, Vol. 74 (365), 1979)

"The purpose of an experiment is to answer questions. The truth of this seems so obvious, that it would not be worth emphasizing were it not for the fact that the results of many experiments are interpreted and presented with little or no reference to the questions that were asked in the first place."  (Thomas M Little, "Interpretation and presentation of results", Hortscience 16, 1981)

"The fact that [the model] is an approximation does not necessarily detract from its usefulness because models are approximations. All models are wrong, but some are useful." (George Box, 1987)

"A null hypothesis that yields under two different treatments have identical expectations is scarcely very plausible, and its rejection by a significance test is more dependent upon the size of an experiment than upon its untruth." (David J Finney, "Was this in your statistics textbook?", Experimental Agriculture 24, 1988)

"The motivation for any action on outliers must be to improve interpretation of data without ignoring unwelcome truth. To remove bad and untrustworthy data is a laudable ambition, but naive and untested rules may bring harm rather than benefit." (David Finney, "Calibration Guidelines Challenge Outlier Practices", The American Statistician Vol 60 (4), 2006) 

"You might say that there’s no reason to bother with model checking since all models are false anyway. I do believe that all models are false, but for me the purpose of model checking is not to accept or reject a model, but to reveal aspects of the data that are not captured by the fitted model." (Andrew Gelman, "Some thoughts on the sociology of statistics", 2007)

"If students have students have no experience with hands-on [telescope] observing, they may take all data as ‘truth’ without having an understanding of how the data are obtained and what could potentially go wrong in that process, so I think it becomes crucially important to give a glimpse of what’s happening behind the scenes at telescopes, so they can be appropriately skeptical users of data in the future." (Colette Salyk, Sky & Telescope, 2022)

"On a final note, we would like to stress the importance of design, which often does not receive the attention it deserves. Sometimes, the large number of modeling options for spatial analysis may raise the false impression that design does not matter, and that a sophisticated analysis takes care of everything. Nothing could be further from the truth." (Hans-Peter Piepho et al, "Two-dimensional P-spline smoothing for spatial analysis of plant breeding trials", “Biometrical Journal”, 2022)

🦋Science: On Statistical Models (Quotes)

"The most widely used mathematical tools in the social sciences are statistical, and the prevalence of statistical methods has given rise to theories so abstract and so hugely complicated that they seem a discipline in themselves, divorced from the world outside learned journals. Statistical theories usually assume that the behavior of large numbers of people is a smooth, average 'summing-up' of behavior over a long period of time. It is difficult for them to take into account the sudden, critical points of important qualitative change. The statistical approach leads to models that emphasize the quantitative conditions needed for equilibrium - a balance of wages and prices, say, or of imports and exports. These models are ill suited to describe qualitative change and social discontinuity, and it is here that catastrophe theory may be especially helpful." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"Statistical models for data are never true. The question whether a model is true is irrelevant. A more appropriate question is whether we obtain the correct scientific conclusion if we pretend that the process under study behaves according to a particular statistical model." (Scott Zeger, "Statistical reasoning in epidemiology", American Journal of Epidemiology, 1991)

"[…] it does not seem helpful just to say that all models are wrong. The very word model implies simplification and idealization. The idea that complex physical, biological or sociological systems can be exactly described by a few formulae is patently absurd. The construction of idealized representations that capture important stable aspects of such systems is, however, a vital part of general scientific analysis and statistical models, especially substantive ones, do not seem essentially different from other kinds of model." (Sir David Cox, "Comment on ‘Model uncertainty, data mining and statistical inference’", Journal of the Royal Statistical Society, Series A 158, 1995)

"Building statistical models is just like this. You take a real situation with real data, messy as this is, and build a model that works to explain the behavior of real data." (Martha Stocking, New York Times, 2000)

"The role of graphs in probabilistic and statistical modeling is threefold: (1) to provide convenient means of expressing substantive assumptions; (2) to facilitate economical representation of joint probability functions; and (3) to facilitate efficient inferences from observations." (Judea Pearl, "Causality: Models, Reasoning, and Inference", 2000)

"It is impossible to construct a model that provides an entirely accurate picture of network behavior. Statistical models are almost always based on idealized assumptions, such as independent and identically distributed (i.i.d.) interarrival times, and it is often difficult to capture features such as machine breakdowns, disconnected links, scheduled repairs, or uncertainty in processing rates." (Sean Meyn, "Control Techniques for Complex Networks", 2008)

"Statistical cognition is concerned with obtaining cognitive evidence about various statistical techniques and ways to present data. It’s certainly important to choose an appropriate statistical model, use the correct formulas, and carry out accurate calculations. It’s also important, however, to focus on understanding, and to consider statistics as communication between researchers and readers." (Geoff Cumming, "Understanding the New Statistics", 2012)

"In general, when building statistical models, we must not forget that the aim is to understand something about the real world. Or predict, choose an action, make a decision, summarize evidence, and so on, but always about the real world, not an abstract mathematical world: our models are not the reality - a point well made by George Box in his oft-cited remark that "all models are wrong, but some are useful". (David Hand, "Wonderful examples, but let's not close our eyes", Statistical Science 29, 2014)

"A statistical model is a relatively simple approximation to account for complex phenomena that generate data. A statistical model consists of one or more equations involving both random variables and parameters. The random variables have stated or assumed distributions. The parameters are unknown fixed quantities. The random components of statistical models account for the inherent variability in most observed phenomena." (Richard M Heiberger & Burt Holland, "Statistics Concepts", 2015)

"An oft-repeated rule of thumb in any sort of statistical model fitting is 'you can't fit a model with more parameters than data points'. This idea appears to be as wide-spread as it is incorrect. On the contrary, if you construct your models carefully, you can fit models with more parameters than datapoints [...]. A model with more parameters than datapoints is known as an under-determined system, and it's a common misperception that such a model cannot be solved in any circumstance. [...] this misconception, which I like to call the 'model complexity myth' [...] is not true in general, it is true in the specific case of simple linear models, which perhaps explains why the myth is so pervasive." (Jake Vanderplas, "The Model Complexity Myth", 2015) [source]

"Machine learning takes many different forms and goes by many different names: pattern recognition, statistical modeling, data mining, knowledge discovery, predictive analytics, data science, adaptive systems, self-organizing systems, and more. Each of these is used by different communities and has different associations. Some have a long half-life, some less so." (Pedro Domingos, "The Master Algorithm", 2015)

"In machine learning, knowledge is often in the form of statistical models, because most knowledge is statistical [...] Machine learning is a kind of knowledge pump: we can use it to extract a lot of knowledge from data, but first we have to prime the pump." (Pedro Domingos, "The Master Algorithm", 2015)

"One final warning about the use of statistical models (whether linear or otherwise): The estimated model describes the structure of the data that have been observed. It is unwise to extend this model very far beyond the observed data." (David S Salsburg, "Errors, Blunders, and Lies: How to Tell the Difference", 2017)

"The central limit conjecture states that most errors are the result of many small errors and, as such, have a normal distribution. The assumption of a normal distribution for error has many advantages and has often been made in applications of statistical models." (David S Salsburg, "Errors, Blunders, and Lies: How to Tell the Difference", 2017)

"When we use algebraic notation in statistical models, the problem becomes more complicated because we cannot 'observe' a probability and know its exact number. We can only estimate probabilities on the basis of observations." (David S Salsburg, "Errors, Blunders, and Lies: How to Tell the Difference", 2017)

"Any fool can fit a statistical model, given the data and some software. The real challenge is to decide whether it actually fits the data adequately. It might be the best that can be obtained, but still not good enough to use." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

🪂Mind: On Mechanisms (Quotes)

"The mechanism of thought consists in combinations, separations, and recombinations of representative images or symbols […] the object of thought is adaptation to environment." (Paul Carus,"Le probeme de la conscience du moi", 1893)

"[…] there is something wonderful in the idea that man’s brain is the greatest machine of all, imitating within its tiny network events happening in the most distant stars, […] On our model theory neural or other mechanisms can imitate or parallel the behaviour and interaction of physical objects and so supply us with information on physical processes which are not directly observable to us." (Kenneth Craik, "The Nature of Explanation", 1943)

"Thus there are instances of symbolisation in nature; we use such instances as an aid to thinking; there is evidence of similar mechanisms at work in our own sensory and central nervous systems; and the function of such symbolisation is plain. If the organism carries a ’small-scale model’ of external reality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is the best of them, react to future situations before they arise […]" (Kenneth Craik, "The Nature of Explanation", 1943)

"The words of the language, as they are written or spoken, do not seem to play any role in any mechanism of thought. The physical entities which seem to serve as elements in thought are certain signs and more or less clear images which can be 'voluntarily' reproduced or combined. […] But taken from a psychological viewpoint, this combinatory play seems to be the essential feature in productive thought - before there is any connection with logical construction in words or other kinds of signs which can be communicated to others. The above-mentioned elements are, in my case, of visual and some of muscular type. Conventional words or other signs have to be sought for laboriously only in a secondary stage, when the mentioned associative play is sufficiently established and can be reproduced at will. " (Albert Einstein, [letter to Hadamard, in (Jacques Hadamard, "The Psychology of Invention in the Mathematical Field, 1945)])

"Concepts are inventions of the human mind used to construct a model of the world. They package reality into discrete units for further processing, they support powerful mechanisms for doing logic, and they are indispensable for precise, extended chains of reasoning. […] A mental model is a cognitive construct that describes a person's understanding of a particular content domain in the world." (John Sown, "Conceptual Structures: Information Processing in Mind and Machine", 1984)

"How can we dare to predict the behavior of man? We may predict the movements of a machine, of an automaton; more than this, we many even try to predict the mechanisms or 'dynamisms' of the human psyche as well. But man is more than psyche." (Viktor Frankl, Man's Search for Meaning", 1984)

"Mental models are the mechanisms whereby humans are able to generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions of future system states." (William B Rouse & Nancy M Morris, "On looking into the black box: Prospects and limits in the search for mental models", Psychological Bulletin (3), 1986)

"Model is used as a theory. It becomes theory when the purpose of building a model is to understand the mechanisms involved in the developmental process. Hence as theory, model does not carve up or change the world, but it explains how change takes place and in what way or manner. This leads to build change in the structures." (Laxmi K Patnaik, "Model Building in Political Science", The Indian Journal of Political Science Vol. 50 (2), 1989)

"[…] semantic nets fail to be distinctive in the way they (1) represent propositions, (2) cluster information for access, (3) handle property inheritance, and (4) handle general inference; in other words, they lack distinctive representational properties (i.e., 1) and distinctive computational properties (i.e., 2-4). Certain propagation mechanisms, notably 'spreading activation', 'intersection search', or 'inference propagation' have sometimes been regarded as earmarks of semantic nets, but since most extant semantic nets lack such mechanisms, they cannot be considered criterial in current usage." (Lenhart K Schubert, "Semantic Nets are in the Eye of the Beholder", 1990)

"What it means for a mental model to be a structural analog is that it embodies a representation of the spatial and temporal relations among, and the causal structures connecting the events and entities depicted and whatever other information that is relevant to the problem-solving talks. […] The essential points are that a mental model can be nonlinguistic in form and the mental mechanisms are such that they can satisfy the model-building and simulative constraints necessary for the activity of mental modeling." (Nancy J Nersessian, "Model-based reasoning in conceptual change", 1999)

"Models of bounded rationality describe how a judgement or decision is reached (that is, the heuristic processes or proximal mechanisms) rather than merely the outcome of the decision, and they describe the class of environments in which these heuristics will succeed or fail." (Gerd Gigerenzer & Reinhard Selten [Eds., "Bounded Rationality: The Adaptive Toolbox", 2001)

"A model isolates one or a few causal connections, mechanisms, or processes, to the exclusion of other contributing or interfering factors - while in the actual world, those other factors make their effects felt in what actually happens. Models may seem true in the abstract, and are false in the concrete. The key issue is about whether there is a bridge between the two, the abstract and the concrete, such that a simple model can be relied on as a source of relevantly truthful information about the complex reality." (Uskali Mäki,"Fact and Fiction in Economics: Models, Realism and Social Construction", 2002)

"We must begin by distinguishing between visual mental imagery and visual perception: Visual perception occurs while a stimulus is being viewed, and includes functions such as visual recognition (i. e., registering that a stimulus is familiar) and identification (i. e., recalling the name, context, or other information associated with the object). Two types of mechanisms are used in visual perception: ‘bottom-up’ mechanisms are driven by the input from the eyes; in contrast, ‘top-down’ mechanisms make use of stored information (such as knowledge, belief, expectations, and goals). Visual mental imagery is a set of representations that gives rise to the experience of viewing a stimulus in the absence of appropriate sensory input. In this case, information in memory underlies the internal events that produce the experience. Unlike afterimages, mental images are relatively prolonged." (Stephen M Kosslyn, "Mental images and the brain", Cognitive Neuropsychology 22, 2005)

🦋Science: On Data Models (Quotes)

"Science manipulates things and gives up living in them. It makes its own limited models of things; operating upon these indices or variables to effect whatever transformations are permitted by their definition, it comes face to face with the real world only at rare intervals. Science is and always will be that admirably active, ingenious, and bold way of thinking whose fundamental bias is to treat everything as though it were an object-in-general - as though it meant nothing to us and yet was predestined for our own use." (Maurice Merleau-Ponty, "L'Œil et l'Esprit", 1960)

"For the theory-practice iteration to work, the scientist must be, as it were, mentally ambidextrous; fascinated equally on the one hand by possible meanings, theories, and tentative models to be induced from data and the practical reality of the real world, and on the other with the factual implications deducible from tentative theories, models and hypotheses." (George E P Box, "Science and Statistics", Journal of the American Statistical Association 71, 1976)

“The purpose of models is not to fit the data but to sharpen the questions.” (Samuel Karlin, 1983)

"There are those who try to generalize, synthesize, and build models, and there are those who believe nothing and constantly call for more data. The tension between these two groups is a healthy one; science develops mainly because of the model builders, yet they need the second group to keep them honest." (Andrew Miall, "Principles of Sedimentary Basin Analysis", 1984)

"Models are often used to decide issues in situations marked by uncertainty. However statistical differences from data depend on assumptions about the process which generated these data. If the assumptions do not hold, the inferences may not be reliable either. This limitation is often ignored by applied workers who fail to identify crucial assumptions or subject them to any kind of empirical testing. In such circumstances, using statistical procedures may only compound the uncertainty." (David A Greedman & William C Navidi, "Regression Models for Adjusting the 1980 Census", Statistical Science Vol. 1 (1), 1986)

"Competent scientists do not believe their own models or theories, but rather treat them as convenient fictions. […] The issue to a scientist is not whether a model is true, but rather whether there is another whose predictive power is enough better to justify movement from today's fiction to a new one." (Steve Vardeman," Comment", Journal of the American Statistical Association 82, 1987)

"[…] no good model ever accounted for all the facts, since some data was bound to be misleading if not plain wrong. A theory that did fit all the data would have been ‘carpentered’ to do this and would thus be open to suspicion." (Francis H C Crick, "What Mad Pursuit: A Personal View of Scientific Discovery", 1988)

"Information engineering has been defined with the reference to automated techniques as follows: An interlocking set of automated techniques in which enterprise models, data models and process models are built up in a comprehensive knowledge-base and are used to create and maintain data-processing systems." (James Martin, "Information Engineering, 1989)

"When evaluating a model, at least two broad standards are relevant. One is whether the model is consistent with the data. The other is whether the model is consistent with the ‘real world’." (Kenneth A Bollen, "Structural Equations with Latent Variables", 1989)

"So we pour in data from the past to fuel the decision-making mechanisms created by our models, be they linear or nonlinear. But therein lies the logician's trap: past data from real life constitute a sequence of events rather than a set of independent observations, which is what the laws of probability demand. [...] It is in those outliers and imperfections that the wildness lurks." (Peter L Bernstein, "Against the Gods: The Remarkable Story of Risk", 1996)

"Building statistical models is just like this. You take a real situation with real data, messy as this is, and build a model that works to explain the behavior of real data." (Martha Stocking, New York Times, 2000)

"The point of a model is to get useful information about the relation between the response and predictor variables. Interpretability is a way of getting information. But a model does not have to be simple to provide reliable information about the relation between predictor and response variables; neither does it have to be a data model. The goal is not interpretability, but accurate information." (Leo Breiman, "Statistical Modeling: The Two Cultures", Statistical Science Vol 16 (3), 2001)

"A smaller model with fewer covariates has two advantages: it might give better predictions than a big model and it is more parsimonious (simpler). Generally, as you add more variables to a regression, the bias of the predictions decreases and the variance increases. Too few covariates yields high bias; this called underfitting. Too many covariates yields high variance; this called overfitting. Good predictions result from achieving a good balance between bias and variance. […] fiding a good model involves trading of fit and complexity." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"Bayesian inference is appealing when prior information is available since Bayes’ theorem is a natural way to combine prior information with data. Some people find Bayesian inference psychologically appealing because it allows us to make probability statements about parameters. […] In parametric models, with large samples, Bayesian and frequentist methods give approximately the same inferences. In general, they need not agree." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"[…] studying methods for parametric models is useful for two reasons. First, there are some cases where background knowledge suggests that a parametric model provides a reasonable approximation. […] Second, the inferential concepts for parametric models provide background for understanding certain nonparametric methods." (Larry A Wasserman, "All of Statistics: A concise course in statistical inference", 2004)

"[...] construction of a data model is precisely the selective relevant depiction of the phenomena by the user of the theory required for the possibility of representation of the phenomenon."  (Bas C van Fraassen, "Scientific Representation: Paradoxes of Perspective", 2008)

"There are three possible reasons for [the] absence of predictive power. First, it is possible that the models are misspecified. Second, it is possible that the model’s explanatory factors are measured at too high a level of aggregation [...] Third, [...] the search for statistically significant relationships may not be the strategy best suited for evaluating our model’s ability to explain real world events [...] the lack of predictive power is the result of too much emphasis having been placed on finding statistically significant variables, which may be overdetermined. Statistical significance is generally a flawed way to prune variables in regression models [...] Statistically significant variables may actually degrade the predictive accuracy of a model [...] [By using] models that are constructed on the basis of pruning undertaken with the shears of statistical significance, it is quite possible that we are winnowing our models away from predictive accuracy." (Michael D Ward et al, "The perils of policy by p-value: predicting civil conflicts" Journal of Peace Research 47, 2010)

"In the predictive modeling disciplines an ensemble is a group of algorithms that is used to solve a common problem [...] Each modeling algorithm has specific strengths and weaknesses and each provides a different mathematical perspective on the relationships modeled, just like each instrument in a musical ensemble provides a different voice in the composition. Predictive modeling ensembles use several algorithms to contribute their perspectives on the prediction problem and then combine them together in some way. Usually ensembles will provide more accurate models than individual algorithms which are also more general in their ability to work well on different data sets [...] the approach has proven to yield the best results in many situations." ( Gary Miner et al, "Practical Text Mining and Statistical Analysis for Non-Structured Text Data Applications", 2012)

"Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that’s something only humans can provide. We have to explicitly embed better values into our algorithms, creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit." (Cathy O'Neil, "Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy", 2016)

"The greatest plus of data modeling is that it produces a simple and understandable picture of the relationship between the input variables and responses [...] different models, all of them equally good, may give different pictures of the relation between the predictor and response variables [...] One reason for this multiplicity is that goodness-of-fit tests and other methods for checking fit give a yes–no answer. With the lack of power of these tests with data having more than a small number of dimensions, there will be a large number of models whose fit is acceptable. There is no way, among the yes–no methods for gauging fit, of determining which is the better model." (Leo Breiman, "Statistical modeling: The two cultures" Statistical Science 16(3), 2001)

"There may be no significant difference between the point of view of inferring the true structure and that of making a prediction if an infinitely large quantity of data is available or if the data are noiseless. However, in modeling based on a finite quantity of real data, there is a significant gap between these two points of view, because an optimal model for prediction purposes may be different from one obtained by estimating the 'true model'." (Genshiro Kitagawa & Sadanori Konis, "Information Criteria and Statistical Modeling", 2007)

"Causal inference is different, because a change in the system is contemplated - an intervention. Descriptive statistics tell you about the data that you happen to have. Causal models claim to tell you what will happen to some of the numbers if you intervene to change other numbers." (David A. Freedman, "Statistical Models: Theory and Practice", 2009)

"It is important to remember that predictive data analytics models built using machine learning techniques are tools that we can use to help make better decisions within an organization and are not an end in themselves. It is paramount that, when tasked with creating a predictive model, we fully understand the business problem that this model is being constructed to address and ensure that it does address it." (John D Kelleher et al, "Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, worked examples, and case studies", 2015)

"A popular misconception holds that the era of Big Data means the end of a need for sampling. In fact, the proliferation of data of varying quality and relevance reinforces the need for sampling as a tool to work efficiently with a variety of data, and minimize bias. Even in a Big Data project, predictive models are typically developed and piloted with samples." (Peter C Bruce & Andrew G Bruce, "Statistics for Data Scientists: 50 Essential Concepts", 2016)

"Optimization is more than finding the best simulation results. It is itself a complex and evolving field that, subject to certain information constraints, allows data scientists, statisticians, engineers, and traders alike to perform reality checks on modeling results." (Chris Conlan, "Automated Trading with R: Quantitative Research and Platform Development", 2016)

"Data analysis and data mining are concerned with unsupervised pattern finding and structure determination in data sets. The data sets themselves are explicitly linked as a form of representation to an observational or otherwise empirical domain of interest. 'Structure' has long been understood as symmetry which can take many forms with respect to any transformation, including point, translational, rotational, and many others. Symmetries directly point to invariants, which pinpoint intrinsic properties of the data and of the background empirical domain of interest. As our data models change, so too do our perspectives on analysing data." (Fionn Murtagh, "Data Science Foundations: Geometry and Topology of Complex Hierarchic Systems and Big Data Analytics", 2018)

"Any fool can fit a statistical model, given the data and some software. The real challenge is to decide whether it actually fits the data adequately. It might be the best that can be obtained, but still not good enough to use." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

"Cross-validation is a useful tool for finding optimal predictive models, and it also works well in visualization. The concept is simple: split the data at random into a 'training' and a 'test' set, fit the model to the training data, then see how well it predicts the test data. As the model gets more complex, it will always fit the training data better and better. It will also start off getting better results on the test data, but there comes a point where the test data predictions start going wrong." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

❄️Systems Thinking: On Bounded Rationality (Quotes)

"The principle of bounded rationality [is] the capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problems whose solution is required for objectively rational behavior in the real world - or even for a reasonable approximation to such objective rationality." (Herbert A Simon, "Administrative Behavior", 1947)

"The first consequence of the principle of bounded rationality is that the intended rationality of an actor requires him to construct a simplified model of the real situation in order to deal with it. He behaves rationally with respect to this model, and such behavior is not even approximately optimal with respect to the real world. To predict his behavior we must understand the way in which this simplified model is constructed, and its construction will certainly be related to his psychological properties as a perceiving, thinking, and learning animal." (Herbert A Simon, "Models of Man", 1957)

[*]"A decision theory that rests on the assumptions that human cognitive capabilities are limited and that these limitations are adaptive with respect to the decision environments humans frequently encounter. Decision are thought to be made usually without elaborate calculations, but instead by using fast and frugal heuristics. These heuristics certainly have the advantage of speed and simplicity, but if they are well matched to a decision environment, they can even outperform maximizing calculations with respect to accuracy. The reason for this is that many decision environments are characterized by incomplete information and noise. The information we do have is usually structured in a specific way that clever heuristics can exploit." (E Ebenhoh, "Agent-Based Modelnig with Boundedly Rational Agents", 2007)

"Bounded rationality simultaneously constrains the complexity of our cognitive maps and our ability to use them to anticipate the system dynamics. Mental models in which the world is seen as a sequence of events and in which feedback, nonlinearity, time delays, and multiple consequences are lacking lead to poor performance when these elements of dynamic complexity are present. Dysfunction in complex systems can arise from the misperception of the feedback structure of the environment. But rich mental models that capture these sources of complexity cannot be used reliably to understand the dynamics. Dysfunction in complex systems can arise from faulty mental simulation-the misperception of feedback dynamics. These two different bounds on rationality must both be overcome for effective learning to occur. Perfect mental models without a simulation capability yield little insight; a calculus for reliable inferences about dynamics yields systematically erroneous results when applied to simplistic models." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000) 

"Faced with the overwhelming complexity of the real world, time pressure, and limited cognitive capabilities, we are forced to fall back on rote procedures, habits, rules of thumb, and simple mental models to make decisions. Though we sometimes strive to make the best decisions we can, bounded rationality means we often systematically fall short, limiting our ability to learn from experience." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"The robustness of the misperceptions of feedback and the poor performance they cause are due to two basic and related deficiencies in our mental model. First, our cognitive maps of the causal structure of systems are vastly simplified compared to the complexity of the systems themselves. Second, we are unable to infer correctly the dynamics of all but the simplest causal maps. Both are direct consequences of bounded rationality, that is, the many limitations of attention, memory, recall, information processing capability, and time that constrain human decision making." (John D Sterman, "Business Dynamics: Systems thinking and modeling for a complex world", 2000)

"Bounded rationality [...] is the rationality that takes into account the limitations of the decision maker in terms of information, cognitive capacity, and attention as opposed to substantive rationality, which is not limited to satisficing, but rather aims at fully optimized solutions." (Jean-Charles Pomerol & Frédéric Adam, "Understanding the Legacy of Herbert Simon to Decision Support Systems", 2008)

"You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

[*] "Conceptual model that assumes individuals are intentionally rational, i.e. they try to maximize their decisions. However, this ideal model is almost impossible to apply in practice: actions and decisions are taken and performed by individuals whose knowledge of the alternatives and the consequences is incomplete; in addition, preferences are subject to change and are not always clearly orderable." (Maddalena Sorrentino & Marco De Marco, "Developing an Interdisciplinary Approach to the Evaluation of E-Government Implementation", 2009)

[*] "The assumption that agents have limited ability to acquire and process information and to solve complex economic problems. These limitations imply that expectations can diverge from RE [Rational Expectationa]." (Sebastiano Manzan, Agent Based Modeling in Finance", 2009)

[*] "Refers to the difficulties faced by an individual in obtaining, memorizing, and processing information in an actionable manner. Although he/she may want to act rationally, the individual can only do so in a limited way, without being able to take into account all desirable information or all possible options. This limited way consists in acting on the basis of knowledge that is deemed acceptable and sufficient, rather than complete knowledge, and of simple rules, rather than a comprehensive method; and in taking shortcuts whenever possible." (Humbert Lesca & Nicolas Lesca, "Weak Signals for Strategic Intelligence: Anticipation Tool for Managers", 2011)

[*] "The theory that personal rationality is bounded by our ability to process information, our cognitive limitations, and the finite time we have to make a decision. Although our decisions are still rational, they are rational within these constraints and, therefore, may not always appear to be rational or optimal." (Evan Stubbs, "Delivering Business Analytics: Practical Guidelines for Best Practice", 2013)

[*] "The principle that the rationality of human beings is constrained ('bounded') by the limits of their cognition and capacity to process information." (Robert M Grant, "Contemporary Strategy Analysis" 10th Ed., 2018)

[*] "A situation in which people have a limited capacity to anticipate, solve complex problems, or enumerate all options." (Jeffrey M Perloff & James A Brander, "Managerial Economics and Strategy" 2nd Ed., 2016)

[*] "A concept that explains behavior that diverges from the standard assumption of a fully rational economic agent. It occurs due to limitations of cognitive ability and access to information for decision making." (Ashlesha Khedekar-Swaminathan, "Behavioral Strategies to Achieve Financial Stability in Uncertain Times", 2019)

"Bounded rationality means rationality within limits or bounds set by incomplete information, cognitive limitations of mind and limited time available for taking the decision." (Anubhuti Dwivedi, "Peace in Economic Equilibrium: A Micro-Perspective", 2019)

[*] "Paradigm that explains agents’ strategic decision-making based on the imperfect information available to them and the expectations they have that dictate whether they will view the results as satisfactory. It leads on to the idea of adaptive learning and trial-and-error processes." (César Camisón, "Neurostrategy", 2021)

[*] "The idea that decision making deviates from rationality due to such inherently human factors as limitations in cognitive capacity and willpower, and situational constraints." (Shaun Ruysenaar, "Thinking Critically About the Fourth Industrial Revolution as a Wicked Problem", 2021)

❄️Systems Thinking: On Issues/Problems (Quotes)

"The making of decisions, as everyone knows from personal experience, is a burdensome task. Offsetting the exhilaration that may result from correct and successful decision and the relief that follows the termination of a struggle to determine issues is the depression that comes from failure, or error of decision, and the frustration which ensues from uncertainty." (Chester I Barnard, "The Functions of the Executive", 1938)

"Systems engineering embraces every scientific and technical concept known, including economics, management, operations, maintenance, etc. It is the job of integrating an entire problem or problem to arrive at one overall answer, and the breaking down of this answer into defined units which are selected to function compatibly to achieve the specified objectives. [...] Instrument and control engineering is but one aspect of systems engineering - a vitally important and highly publicized aspect, because the ability to create automatic controls within overall systems has made it possible to achieve objectives never before attainable, While automatic controls are vital to systems which are to be controlled, every aspect of a system is essential. Systems engineering is unbiased, it demands only what is logically required. Control engineers have been the leaders in pulling together a systems approach in the various technologies." (Instrumentation Technology, 1957) 

"Systems engineering is the name given to engineering activity which considers the overall behavior of a system, or more generally which considers all factors bearing on a problem, and the systems approach to control engineering problems is correspondingly that approach which examines the total dynamic behavior of an integrated system. It is concerned more with quality of performance than with sizes, capacities, or efficiencies, although in the most general sense systems engineering is concerned with overall, comprehensive appraisal." (Ernest F Johnson, "Automatic process control", 1958)

"System theory is basically concerned with problems of relationships, of structure, and of interdependence rather than with the constant attributes of objects. In general approach it resembles field theory except that its dynamics deal with temporal as well as spatial patterns. Older formulations of system constructs dealt with the closed systems of the physical sciences, in which relatively self-contained structures could be treated successfully as if they were independent of external forces. But living systems, whether biological organisms or social organizations, are acutely dependent on their external environment and so must be conceived of as open systems." (Daniel Katz, "The Social Psychology of Organizations", 1966)

"It is a commonplace of modern technology that there is a high measure of certainty that problems have solutions before there is knowledge of how they are to be solved." (John K Galbraith, "The New Industrial State", 1967)

"The systems approach to problems focuses on systems taken as a whole, not on their parts taken separately. Such an approach is concerned with total - system performance even when a change in only one or a few of its parts is contemplated because there are some properties of systems that can only be treated adequately from a holistic point of view. These properties derive from the relationship between parts of systems: how the parts interact and fit together." (Russell L Ackoff, "Towards a System of Systems Concepts", 1971)

"Technology can relieve the symptoms of a problem without affecting the underlying causes. Faith in technology as the ultimate solution to all problems can thus divert our attention from the most fundamental problem - the problem of growth in a finite system." (Donella A Meadows, "The Limits to Growth", 1972)

"General systems theory deals with the most fundamental concepts and aspects of systems. Many theories dealing with more specific types of systems (e. g., dynamical systems, automata, control systems, game-theoretic systems, among many others) have been under development for quite some time. General systems theory is concerned with the basic issues common to all these specialized treatments. Also, for truly complex phenomena, such as those found predominantly in the social and biological sciences, the specialized descriptions used in classical theories (which are based on special mathematical structures such as differential or difference equations, numerical or abstract algebras, etc.) do not adequately and properly represent the actual events. Either because of this inadequate match between the events and types of descriptions available or because of the pure lack of knowledge, for many truly complex problems one can give only the most general statements, which are qualitative and too often even only verbal. General systems theory is aimed at providing a description and explanation for such complex phenomena." (Mihajlo D. Mesarovic & Yasuhiko Takahare, "General Systems Theory: Mathematical foundations", 1975)

"The world is a complex, interconnected, finite, ecological–social–psychological–economic system. We treat it as if it were not, as if it were divisible, separable, simple, and infinite. Our persistent, intractable global problems arise directly from this mismatch." (Donella Meadows, "Whole Earth Models and Systems", 1982)

"Models are often used to decide issues in situations marked by uncertainty. However statistical differences from data depend on assumptions about the process which generated these data. If the assumptions do not hold, the inferences may not be reliable either. This limitation is often ignored by applied workers who fail to identify crucial assumptions or subject them to any kind of empirical testing. In such circumstances, using statistical procedures may only compound the uncertainty." (David A Greedman & William C Navidi, "Regression Models for Adjusting the 1980 Census", Statistical Science Vol. 1 (1), 1986)

"If we want to solve problems effectively […] we must keep in mind not only many features but also the influences among them. Complexity is the label we will give to the existence of many interdependent variables in a given system. The more variables and the greater their interdependence, the greater the system's complexity. Great complexity places high demands on a planner's capacity to gather information, integrate findings, and design effective actions. The links between the variables oblige us to attend to a great many features simultaneously, and that, concomitantly, makes it impossible for us to undertake only one action in a complex system." (Dietrich Dorner, "The Logic of Failure: Recognizing and Avoiding Error in Complex Situations", 1989)

"The real leverage in most management situations lies in understanding dynamic complexity, not detail complexity. […] Unfortunately, most 'systems analyses' focus on detail complexity not dynamic complexity. Simulations with thousands of variables and complex arrays of details can actually distract us from seeing patterns and major interrelationships. In fact, sadly, for most people 'systems thinking' means 'fighting complexity with complexity', devising increasingly 'complex' (we should really say 'detailed') solutions to increasingly 'complex' problems. In fact, this is the antithesis of real systems thinking." (Peter M Senge, "The Fifth Discipline: The Art and Practice of the Learning Organization", 1990)

"For strictly scientific or technological purposes all this is irrelevant. On a pragmatic view, as on a religious view, theory and concepts are held in faith. On the pragmatic view the only thing that matters is that the theory is efficacious, that it ‘works’ and that the necessary preliminaries and side issues do not cost too much in time and effort. Beyond that, theory and concepts go to constitute a language in which the scientistic matters at issue can be formulated and discussed." (Bertram N Brockhouse, [lecture] 1994)

"In sharp contrast (with the traditional social planning) the systems design approach seeks to understand a problem situation as a system of interconnected, interdependent, and interacting issues and to create a design as a system of interconnected, interdependent, interacting, and internally consistent solution ideas." (Béla H Bánáthy, "Designing Social Systems in a Changing World", 1996)

"It [system dynamics] focuses on building system dynamics models with teams in order to enhance team learning, to foster consensus and to create commitment with a resulting decision […] System dynamics can be helpful to elicit and integrate mental models into a more holistic view of the problem and to explore the dynamics of this holistic view […] It must be understood that the ultimate goal of the intervention is not to build a system dynamics model. The system dynamics model is a means to achieve other ends […] putting people in a position to learn about a messy problem [...] create a shared social reality […] a shared understanding of the problem and potential solutions [...] to foster consensus within the team [..]" (Jac A M Vennix, "Group Model Building: Facilitating Team Learning Using System Dynamics", 1996)

"The theory of Tipping Points requires, however, that we reframe the way we think about the world. [...] We have trouble estimating dramatic, exponential change. [...] There are abrupt limits to the number of cognitive categories we can make and the number of people we can truly love and the number of acquaintances we can truly know. We throw up our hands at a problem phrased in an abstract way, but have no difficulty at all solving the same problem rephrased as a social dilemma. All of these things are expressions of the peculiarities of the human mind and heart, a refutation of the notion that the way we function and communicate and process information is straightforward and transparent. It is not. It is messy and opaque." (Malcolm T Gladwell, "The Tipping Point: How Little Things Can Make a Big Difference", 2000)

"A model is an imitation of reality and a mathematical model is a particular form of representation. We should never forget this and get so distracted by the model that we forget the real application which is driving the modelling. In the process of model building we are translating our real world problem into an equivalent mathematical problem which we solve and then attempt to interpret. We do this to gain insight into the original real world situation or to use the model for control, optimization or possibly safety studies." (Ian T Cameron & Katalin Hangos, "Process Modelling and Model Analysis", 2001)

"Systems thinking means the ability to see the synergy of the whole rather than just the separate elements of a system and to learn to reinforce or change whole system patterns. Many people have been trained to solve problems by breaking a complex system, such as an organization, into discrete parts and working to make each part perform as well as possible. However, the success of each piece does not add up to the success of the whole. to the success of the whole. In fact, sometimes changing one part to make it better actually makes the whole system function less effectively." (Richard L Daft, "The Leadership Experience", 2002)

"Self-organization can be seen as a spontaneous coordination of the interactions between the components of the system, so as to maximize their synergy. This requires the propagation and processing of information, as different components perceive different aspects of the situation, while their shared goal requires this information to be integrated. The resulting process is characterized by distributed cognition: different components participate in different ways to the overall gathering and processing of information, thus collectively solving the problems posed by any perceived deviation between the present situation and the desired situation." (Carlos Gershenson & Francis Heylighen, "How can we think the complex?", 2004)

"System Thinking is a common concept for understanding how causal relationships and feedbacks work in an everyday problem. Understanding a cause and an effect enables us to analyse, sort out and explain how changes come about both temporarily and spatially in common problems. This is referred to as mental modelling, i.e. to explicitly map the understanding of the problem and making it transparent and visible for others through Causal Loop Diagrams (CLD)." (Hördur V. Haraldsson, "Introduction to System Thinking and Causal Loop Diagrams", 2004)

"We often hear warnings that some social problem is 'epidemic'. This expression suggests that the problem's growth is rapid, widespread, and out of control. If things are getting worse, and particularly if they're getting worse fast, we need to act." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Systemic problems trace back in the end to worldviews. But worldviews themselves are in flux and flow. Our most creative opportunity of all may be to reshape those worldviews themselves. New ideas can change everything." (Anthony Weston, "How to Re-Imagine the World", 2007)

"Internal friction is exacerbated by the fact that in business as in war, we are operating in a nonlinear, semi-chaotic environment in which our endeavors will collide and possibly clash with the actions of other independent wills (customers, suppliers, competitors, regulators, lobbyists, and so on). The internal and external worlds are in constant contact and the effects of our actions are the result of their reciprocal interaction. Friction gives rise to three gaps: the knowledge gap, the alignment gap, and the effects gap. To execute effectively, we must address all three. Our instinctive reaction to the three gaps is to demand more detail. We gather more data in order to craft more detailed plans, issue more detailed instructions, and exercise more detailed control. This not only fails to solve the problem, it usually makes it worse. We need to think about the problem differently and adopt a systemic approach to solving it." (Stephen Bungay, "The Art of Action: How Leaders Close the Gaps between Plans, Actions, and Results", 2010)

[system dynamics:] "The interactions of connected and interdependent components, which may cause change over time and give rise to interconnected risks; emerging, unforeseeable issues; and unclear, disproportional cause-and-effect relationships." (Project Management Institute, "Navigating Complexity: A Practice Guide", 2014)

"System dynamics [...] uses models and computer simulations to understand behavior of an entire system, and has been applied to the behavior of large and complex national issues. It portrays the relationships in systems as feedback loops, lags, and other descriptors to explain dynamics, that is, how a system behaves over time. Its quantitative methodology relies on what are called 'stock-and-flow diagrams' that reflect how levels of specific elements accumulate over time and the rate at which they change. Qualitative systems thinking constructs evolved from this quantitative discipline." (Karen L Higgins, "Economic Growth and Sustainability: Systems Thinking for a Complex World", 2015)

"Man is not born to solve the problems of the universe, but to find out where the problems begin, and then to take his stand within the limits of the intelligible." (Johann Wolfgang von Goethe)

"Some problems are just too complicated for rational logical solutions. They admit of insights, not answers." (Jerome B Wiesner)

❄️Systems Thinking: On Bifurcation Theory (Quotes)

"It is not enough to know the critical stress, that is, the quantitative breaking point of a complex design; one should also know as much as possible of the qualitative geometry of its failure modes, because what will happen beyond the critical stress level can be very different from one case to the next, depending on just which path the buckling takes. And here catastrophe theory, joined with bifurcation theory, can be very helpful by indicating how new failure modes appear." (Alexander Woodcock & Monte Davis, "Catastrophe Theory", 1978)

"The study of changes in the qualitative structure of the flow of a differential equation as parameters are varied is called bifurcation theory. At a given parameter value, a differential equation is said to have stable orbit structure if the qualitative structure of the flow does not change for sufficiently small variations of the parameter. A parameter value for which the flow does not have stable orbit structure is called a bifurcation value, and the equation is said to be at a bifurcation point." (Jack K Hale & Hüseyin Kocak, "Dynamics and Bifurcations", 1991)

"Fundamental to catastrophe theory is the idea of a bifurcation. A bifurcation is an event that occurs in the evolution of a dynamic system in which the characteristic behavior of the system is transformed. This occurs when an attractor in the system changes in response to change in the value of a parameter. A catastrophe is one type of bifurcation. The broader framework within which catastrophes are located is called dynamical bifurcation theory." (Courtney Brown, "Chaos and Catastrophe Theories", 1995)

"The existence of equilibria or steady periodic solutions is not sufficient to determine if a system will actually behave that way. The stability of these solutions must also be checked. As parameters are changed, a stable motion can become unstable and new solutions may appear. The study of the changes in the dynamic behavior of systems as parameters are varied is the subject of bifurcation theory. Values of the parameters at which the qualitative or topological nature of the motion changes are known as critical or bifurcation values." (Francis C Moona, "Nonlinear Dynamics", 2003)

"In parametrized dynamical systems a bifurcation occurs when a qualitative change is invoked by a change of parameters. In models such a qualitative change corresponds to transition between dynamical regimes. In the generic theory a finite list of cases is obtained, containing elements like ‘saddle-node’, ‘period doubling’, ‘Hopf bifurcation’ and many others." (Henk W Broer & Heinz Hanssmann, "Hamiltonian Perturbation Theory (and Transition to Chaos)", 2009)

"The concept of bifurcation, present in the context of non-linear dynamic systems and theory of chaos, refers to the transition between two dynamic modalities qualitatively distinct; both of them are exhibited by the same dynamic system, and the transition (bifurcation) is promoted by the change in value of a relevant numeric parameter of such system. Such parameter is named 'bifurcation parameter', and in highly non-linear dynamic systems, its change can produce a large number of bifurcations between distinct dynamic modalities, with self-similarity and fractal structure. In many of these systems, we have a cascade of numberless bifurcations, culminating with the production of chaotic dynamics." (Emilio Del-Moral-Hernandez, "Chaotic Neural Networks", Encyclopedia of Artificial Intelligence, 2009)

"In dynamical systems, a bifurcation occurs when a small smooth change made to the parameter values (the bifurcation parameters) of a system causes a sudden 'qualitative' or topological change in its behaviour. Generally, at a bifurcation, the local stability properties of equilibria, periodic orbits or other invariant sets changes." (Greegory Faye, "An introduction to bifurcation theory",  2011)

"Catastrophe theory can be thought of as a link between classical analysis, dynamical systems, differential topology (including singularity theory), modern bifurcation theory and the theory of complex systems." (Werner Sanns, "Catastrophe Theory" [Mathematics of Complexity and Dynamical Systems, 2012])

"Roughly spoken, bifurcation theory describes the way in which dynamical system changes due to a small perturbation of the system-parameters. A qualitative change in the phase space of the dynamical system occurs at a bifurcation point, that means that the system is structural unstable against a small perturbation in the parameter space and the dynamic structure of the system has changed due to this slight variation in the parameter space." (Holger I Meinhardt, "Cooperative Decision Making in Common Pool Situations", 2012)

"Bifurcation theory is the mathematical study of changes in the qualitative or topological structure of a given family, such as the integral curves of a family of vector fields, and the solutions of a family of differential equations. Most commonly applied to the mathematical study of dynamical systems, a bifurcation occurs when a small smooth change made to the parameter values (the bifurcation parameters) of a system causes a sudden “qualitative” or topological change in its behavior. Bifurcations can occur in both continuous systems (described by ODEs, DDEs, or PDEs) and discrete systems (described by maps)." (Tianshou Zhou, "Bifurcation", 2013)

"Most commonly applied to the mathematical study of dynamical systems, a bifurcation occurs when a small smooth change made to the parameter values (the bifurcation parameters) of a system causes a sudden 'qualitative' or topological change in its behavior. Bifurcations can occur in both continuous systems (described by ODEs, DDEs, or PDEs) and discrete systems (described by maps)." (Tianshou Zhou, "Bifurcation", 2013)

"The core of bifurcation theory of nonlinear system inevitably falls back to the dynamic analysis of linear ones. Because of that, the fundamental question one may ask is if there exist a linearized DAE system with the same qualitative behavior around fixed points of its nonlinear counterpart." (Ataíde S A.Netoa et al, "Nonlinear dynamic analysis of chemical engineering processes described by differential-algebraic equations systems", 2019)

🏷️Knowledge Representation: On Heuristics (Quotes)

"Heuristic reasoning is reasoning not regarded as final and strict but as provisional and plausible only, whose purpose is to discover the solution of the present problem. We are often obliged to use heuristic reasoning. We shall attain complete certainty when we shall have obtained the complete solution, but before obtaining certainty we must often be satisfied with a more or less plausible guess. We may need the provisional before we attain the final. We need heuristic reasoning when we construct a strict proof as we need scaffolding when we erect a building." (George Pólya, "How to Solve It", 1945)

"The aim of heuristics is to study the methods and rules of discovery and invention. [...] Heuristic, as an adjective, means 'serving to discover'." (George Pólya, "How to Solve It", 1945)

"Heuristic (it is of Greek origin) means discovery. Heuristic methods are based on experience, rational ideas, and rules of thumb. Heuristics are based more on common sense than on mathematics. Heuristics are useful, for example, when the optimal solution needs an exhaustive search that is not realistic in terms of time. In principle, a heuristic does not guarantee the best solution, but a heuristic solution can provide a tremendous shortcut in cost and time." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"Heuristic methods may aim at local optimization rather than at global optimization, that is, the algorithm optimizes the solution stepwise, finding the best solution at each small step of the solution process and 'hoping' that the global solution, which comprises the local ones, would be satisfactory." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"A heuristic is ecologically rational to the degree that it is adapted to the structure of an environment. Thus, simple heuristics and environmental structure can both work hand in hand to provide a realistic alternative to the ideal of optimization, whether unbounded or constrained." (Gerd Gigerenzer & Peter M Todd, "Fast and Frugal Heuristics: The Adaptive Toolbox" [in "Simple Heuristics That Make Us Smart"], 1999)

"Fast and frugal heuristics employ a minimum of time, knowledge, and computation to make adaptive choices in real environments. They can be used to solve problems of sequential search through objects or options, as in satisficing. They can also be used to make choices between simultaneously available objects, where the search for information (in the form of cues, features, consequences, etc.) about the possible options must be limited, rather than the search for the options themselves. Fast and frugal heuristics limit their search of objects or information using easily computable stopping rules, and they make their choices with easily computable decision rules." (Gerd Gigerenzer & Peter M Todd, "Fast and Frugal Heuristics: The Adaptive Toolbox" [in "Simple Heuristics That Make Us Smart"], 1999)

"In the language of mental models, such past experience provided the default assumptions necessary to fill the gaps in the emerging and necessarily incomplete framework of a relativistic theory of gravitation. It was precisely the nature of these default assumptions that allowed them to be discarded again in the light of novel information - provided, for instance, by the further elaboration of the mathematical formalism - without, however, having to abandon the underlying mental models which could thus continue to function as heuristic orientations." (Jürgen Renn, "Before the Riemann Tensor: The Emergence of Einstein’s Double Strategy", [in "The Universe of General Relativity"] 2000)

"Theories of choice are at best approximate and incomplete. One reason for this pessimistic assessment is that choice is a constructive and contingent process. When faced with a complex problem, people employ a variety of heuristic procedures in order to simplify the representation and the evaluation of prospects. These procedures include computational shortcuts and editing operations, such as eliminating common components and discarding nonessential differences. The heuristics of choice do not readily lend themselves to formal analysis because their application depends on the formulation of the problem, the method of elicitation, and the context of choice." (Amos Tversky & Daniel Kahneman, "Advances in Prospect Theory: Cumulative Representation of Uncertainty" [in "Choices, Values, and Frames"], 2000)

"Heuristics are rules of thumb that help constrain the problem in certain ways (in other words they help you to avoid falling back on blind trial and error), but they don't guarantee that you will find a solution. Heuristics are often contrasted with algorithms that will guarantee that you find a solution - it may take forever, but if the problem is algorithmic you will get there. However, heuristics are also algorithms." (S Ian Robertson, "Problem Solving", 2001)

"Models of bounded rationality describe how a judgement or decision is reached (that is, the heuristic processes or proximal mechanisms) rather than merely the outcome of the decision, and they describe the class of environments in which these heuristics will succeed or fail." (Gerd Gigerenzer & Reinhard Selten [Eds., "Bounded Rationality: The Adaptive Toolbox", 2001)

"Mental shortcuts, also called heuristic simplifications, help us analyze situations and make decisions quickly in our daily life. However, this process often leads us astray when analyzing decisions with risk and uncertainty. Because investing decisions involve substantial risk and uncertainty, our decisions are biased in predictable ways. The representativeness bias causes us to extrapolate the past and assume that good companies are good investments. The familiarity bias causes us to believe that firms we are familiar with are better investments than unfamiliar firms. Thus, we own more local firms and our employer’s stock and few international stocks. Thus, these biases lead to low diversification and higher risks." (John R Nofsinger, "The Psychology of Investing", 2002)

"Psychological research has shown that the brain uses shortcuts to reduce the complexity of analyzing information. Psychologists call these heuristic simplifications. These mental shortcuts allow the brain to generate an estimate of an answer before fully digesting all the available information. Two examples of shortcuts are known as representativeness and familiarity. Using these shortcuts allows the brain to organize and quickly process large amounts of information. However, these shortcuts also make it hard for investors to analyze new information correctly and can lead to inaccurate conclusions." (John R Nofsinger, "The Psychology of Investing", 2002)

"Most people give substantial weight to anecdotal evidence, perhaps so much that it will cancel out positive recommendations found in consumer reports. People's tendency to give undue weight to some types of information is called the availability heuristic. A heuristic is a rule of thumb, a mental shortcut." (Barry Schwartz, "The Paradox of Choice: Why More Is Less", 2004)

"A heuristic is defined as a simple rule that exploits both evolved abilities to act fast and struc- tures of the environment to act accurately and frugally. The complexity and uncertainty of an environment cannot be determined independently of the actor. What matters is the degree of complexity and uncertainty encountered by the decision maker." (Christoph Engel & Gerd Gigerenzer, "Law and Heuristics: An interdisciplinary venture" [in "Heuristics and the Law", 2006)

"Heuristics are needed in situations where the world does not permit optimization. For many real-world problems (as opposed to optimization-tuned textbook problems), optimal solutions are unknown because the problems are computationally intractable or poorly defined." (Christoph Engel & Gerd Gigerenzer, "Law and Heuristics: An interdisciplinary venture" [in "Heuristics and the Law", 2006)

"A heuristic is a rule applied to an existing solution represented in a perspective that generates a new (and hopefully better) solution or a new set of possible solutions." (Scott E Page, "The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools and Societies", 2008)

"A second class of metaphors - mathematical algorithms, heuristics, and models - brings us closer to the world of computer science programs, simulations, and approximations of the brain and its cognitive processes." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"A heuristic is a decision rule that utilizes a subset of the information set. Since in virtually all cases people must economize and cannot analyze all contingencies, we use heuristics without even realizing it." (Lucy F Ackert & Richard Deaves, "Behavioral Finance: Psychology, Decision-Making, and Markets", 2010)

"Perception and memory are imprecise filters of information, and the way in which information is presented, that is, the frame, influences how it is received. Because too much information is difficult to deal with, people have developed shortcuts or heuristics in order to come up with reasonable decisions. Unfortunately, sometimes these heuristics lead to bias, especially when used outside their natural domains." (Lucy F Ackert & Richard Deaves, "Behavioral Finance: Psychology, Decision-Making, and Markets", 2010)

"When heuristics don’t yield the results we expect, you’d think we would eventually realize that something’s wrong. Even if we don’t locate the biases, we should be able to see the discrepancy between what we wanted and what we got, right? Well, not necessarily. As it turns out, we have biases that support our biases! If we’re partial to one option - perhaps because it’s more memorable, or framed to minimize loss, or seemingly consistent with a promising pattern - we tend to search for information that will justify choosing that option. On the one hand, it’s sensible to make choices that we can defend with data and a list of reasons. On the other hand, if we’re not careful, we’re likely to conduct an imbalanced analysis, falling prey to a cluster of errors collectively known as 'confirmation biases'." (Sheena Iyengar, "The Art of Choosing", 2010)

"In particular, the accurate intuitions of experts are better explained by the effects of prolonged practice than by heuristics. We can now draw a richer and more balanced picture, in which skill and heuristics are alternative sources of intuitive judgments and choices." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"This is the essence of intuitive heuristics: when faced with a difficult question, we often answer an easier one instead, usually without noticing the substitution." (Daniel Kahneman, "Thinking, Fast and Slow", 2011)

"Heuristics are an evolutionary solution to an ongoing problem: we have limited mental resources. As such, they have a very long and thoroughly time-tested history of helping us - on average - make better decisions." (Peter H Diamandis, "Abundance: The Future is Better Than You Think", 2012)

"Heuristics are simplified rules of thumb that make things simple and easy to implement. But their main advantage is that the user knows that they are not perfect, just expedient, and is therefore less fooled by their powers. They become dangerous when we forget that." (Nassim N Taleb, "Antifragile: Things that gain from disorder", 2012)

"Mental models represent possibilities, and the theory of mental models postulates three systems of mental processes underlying inference: (0) the construction of an intensional representation of a premise’s meaning – a process guided by a parser; (1) the building of an initial mental model from the intension, and the drawing of a conclusion based on heuristics and the model; and (2) on some occasions, the search for alternative models, such as a counterexample in which the conclusion is false. System 0 is linguistic, and it may be autonomous. System 1 is rapid and prone to systematic errors, because it makes no use of a working memory for intermediate results. System 2 has access to working memory, and so it can carry out recursive processes, such as the construction of alternative models." (Sangeet Khemlania & P.N. Johnson-Laird, "The processes of inference", Argument and Computation, 2012)

"The art of reasoned persuasion is an iterative, recursive heuristic, meaning that we must go back and forth between the facts and the rules until we have a good fit. We cannot see the facts properly until we know what framework to place them into, and we cannot determine what framework to place them into until we see the basic contours of the facts." (Joel P Trachtman, "The Tools of Argument", 2013)

"A good heuristic decision is made by 1) knowing what to look for, 2) knowing when enough information is enough (the 'threshold of decision' ), and 3) knowing what decision to make." (Patrick Van Horne, "Left of Bang", 2014)

"A rule of thumb, or heuristic, enables us to make a decision fast, without much searching for information, but nevertheless with high accuracy. [...] A heuristic can be safer and more accurate than a calculation, and the same heuristic can underlie both conscious and unconscious decisions." (Gerd Gigerenzer, "Risk Savvy: How to make good decisions", 2014)

"Even if virtual worlds were tabula rasa, we are encumbered with a great deal of cognitive baggage. Our brains are hardwired with many mental shortcuts to help us make sense of the world. We simply do not have the time to carefully process every piece of information that comes our way. To cope with this inundation of information, our brains have developed automated heuristics that filter and preprocess this information for us. Thus, when we encounter new media and technological devices, we fall back on the existing rules and norms we know." (Nick Yee, "The Proteus Paradox", 2014)

"A heuristic is a strategy that ignores part of the information, with the goal of making decisions more quickly, frugally, and/or accurately than more complex methods." (Gerd Gigerenzer et al, "Simply Rational: Decision Making in the Real World", 2015)

"Because economists go through a similar training and share a common method of analysis, they act very much like a guild. The models themselves may be the product of analysis, reflection, and observation, but practitioners’ views about the real world develop much more heuristically, as a by-product of informal conversations and socialization among themselves. This kind of echo chamber easily produces overconfidence - in the received wisdom or the model of the day. Meanwhile, the guild mentality renders the profession insular and immune to outside criticism. The models may have problems, but only card-carrying members of the profession are allowed to say so. The objections of outsiders are discounted because they do not understand the models. The profession values smarts over judgment, being interesting over being right - so its fads and fashions do not always self-correct." (Dani Rodrik, "Economics Rules: The Rights and Wrongs of the Dismal Science", 2015)

"Heuristic decision making is fast and frugal and is often based on the evaluation of one or two salient bits of information." (Amitav Chakravarti, "Why People (Don’t) Buy: The Go and Stop Signals", 2015)

"Probability theory is not the only tool for rationality. In situations of uncertainty, as opposed to risk, simple heuristics can lead to more accurate judgments, in addition to being faster and more frugal. Under uncertainty, optimal solutions do not exist (except in hindsight) and, by definition, cannot be calculated. Thus, it is illusory to model the mind as a general optimizer, Bayesian or otherwise. Rather, the goal is to achieve satisficing solutions, such as meeting an aspiration level or coming out ahead of a competitor."  (Gerd Gigerenzer et al, "Simply Rational: Decision Making in the Real World", 2015)

"Judgments made in difficult circumstances can be based on a limited number of simple, rapidly-arrived-at rules ('heuristics'), rather than formal, extensive algorithmic calculus and programs. Often, even complex problems can be solved quickly and accurately using such 'quick and dirty' heuristics. However, equally often, such heuristics can be beset by systematic errors or biases." (Jérôme Boutang & Michel De Lara, "The Biased Mind", 2016)

"A heuristic is a strategy we derive from previous experience with a similar problem." (Darius Foroux, "Think Straight", 2017)

"In any analysis of any part of the world, it is mandatory to institute a general reasoning in which the whole - the Absolute - is also included. This is what science scrupulously avoids. Science is all about the parts, and ignoring the whole. Science is non-holistic, which is why it cannot arrive at a grand unified, final theory of everything. From the whole you can get to every part, because the whole defines the parts. If you start with the parts, as science does, you can never get to the whole because the parts are necessarily defined piecemeal, heuristically and with no regard to the whole, since the whole is unknown. A bottom-up approach can never work. Only top-down approaches have any chance of working. Empiricists are always parts people and bottom-up people. Rationalists are holistic and top-down. These are opposite worldviews. The PSR is an explanatory, top-down principle. Randomness is a non-explanatory, bottom-up speculation." (Thomas Stark, "God Is Mathematics: The Proofs of the Eternal Existence of Mathematics", 2018)

"The social world that humans have made for themselves is so complex that the mind simplifies the world by using heuristics, customs, and habits, and by making models or assumptions about how things generally work (the ‘causal structure of the world’). And because people rely upon (and are invested in) these mental models, they usually prefer that they remain uncontested." (Dr James Brennan, "Psychological  Adjustment to Illness and Injury", West of England Medical Journal Vol. 117 (2), 2018)

"We used the word 'heuristics' to describe aspects of software development that tip toward the liberal arts. Its counterpart, 'algorithms', was its alter ego on the technical side. Heuristics and algorithms are like two sides of the same coin. Both are specific procedures for making software do what it does: taking input, applying an operation, and producing output. Yet each had a different purpose." (Ken Kocienda, "Creative Selection: Inside Apple's Design Process During the Golden Age of Steve Jobs", 2018)

🦋Science: On Facts (Quotes)

"[…] to kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact." (Charles R Darwin, "More Letters of Charles Darwin", Vol 2, 1903)

"Entia non sunt multiplicanda praeter necessitatem. That is to say; before you try a complicated hypothesis, you should make quite sure that no simplification of it will explain the facts equally well." (Charles S Peirce," Pragmatism and Pragmaticism", [lecture] 1903)

"But, once again, what the physical states as the result of an experiment is not the recital of observed facts, but the interpretation and the transposing of these facts into the ideal, abstract, symbolic world created by the theories he regards as established." (Pierre-Maurice-Marie Duhem, "The Aim and Structure of Physical Theory", 1908)

"The facts of greatest outcome are those we think simple; may be they really are so, because they are influenced only by a small number of well-defined circumstances, may be they take on an appearance of simplicity because the various circumstances upon which they depend obey the laws of chance and so come to mutually compensate." (Henri Poincaré, "The Foundations of Science", 1913)

"The world is an endless variety of facts, linked together by necessary and immutable bonds." (Émile Boutroux, "Natural law in Science and Philosophy", 1914)

"Statistics may be defined as numerical statements of facts by means of which large aggregates are analyzed, the relations of individual units to their groups are ascertained, comparisons are made between groups, and continuous records are maintained for comparative purposes." (Melvin T Copeland. "Statistical Methods" [in: Harvard Business Studies, Vol. III, Ed. by Melvin T Copeland, 1917])

"Facts are carpet-tacks under the pneumatic tires of theory." (Austin O’Malley, "Keystones of Thought", 1918)

"The aim of science is to seek the simplest explanations of complex facts. We are apt to fall into the error of thinking that the facts are simple because simplicity is the goal of our quest. The guiding motto in the life of every natural philosopher should be, ‘Seek simplicity and distrust it’." (Alfred N Whitehead, "The Concept of Nature", 1919)

"[…] the mere collection of facts, without some basis of theory for guidance and elucidation, is foolish and profitless." (Gamaliel Bradford, "Darwin", 1926)

"Observed facts must be built up, woven together, ordered, arranged, systematized into conclusions and theories by reflection and reason, if they are to have full bearing on life and the universe. Knowledge is the accumulation of facts. Wisdom is the establishment of relations. And just because the latter process is delicate and perilous, it is all the more delightful." (Gamaliel Bradford, "Darwin", 1926)

"[…] facts are too bulky to be lugged about conveniently except on the wheels of theory." (Julian Huxley, "Essays of a Biologist", 1929)

"We can invent as many theories we like, and any one of them can be made to fit the facts. But that theory is always preferred which makes the fewest number of assumptions." (Albert Einstein [interview] 1929)

"A system is said to be coherent if every fact in the system is related every other fact in the system by relations that are not merely conjunctive. A deductive system affords a good example of a coherent system." (Lizzie S Stebbing, "A modern introduction to logic", 1930)

"In experimental science facts of the greatest importance are rarely discovered accidentally: more frequently new ideas point the way towards them." (Erwin Schrödinger, "Science and the Human Temperament", 1935)

"Science is the attempt to discover, by means of observation, and reasoning based upon it, first, particular facts about the world, and then laws connecting facts with one another and (in fortunate cases) making it possible to predict future occurrences." (Bertrand Russell, "Religion and Science, Grounds of Conflict", 1935)

"The fundamental gospel of statistics is to push back the domain of ignorance, prejudice, rule-of-thumb, arbitrary or premature decisions, tradition, and dogmatism and to increase the domain in which decisions are made and principles are formulated on the basis of analyzed quantitative facts." (Robert W Burgess, "The Whole Duty of the Statistical Forecaster", Journal of the American Statistical Association , Vol. 32, No. 200, 1937)

"With the help of physical theories we try to find our way through the maze of observed facts, to order and understand the world of our sense impressions." (Albert Einstein & Leopold Infeld, "The Evolution of Physics", 1938)

"We can put it down as one of the principles learned from the history of science that a theory is only overthrown by a better theory, never merely by contradictory facts." (James B Conant, "On Understanding Science", 1947)

"To some people, statistics is ‘quartered pies, cute little battleships and tapering rows of sturdy soldiers in diversified uniforms’. To others, it is columns and columns of numerical facts. Many regard it as a branch of economics. The beginning student of the subject considers it to be largely mathematics." (The Editors, "Statistics, The Physical Sciences and Engineering", The American Statistician, Vol. 2, No. 4, 1948)

"A conceptual scheme is never discarded merely because of a few stubborn facts with which it cannot be reconciled; a conceptual scheme is either modified or replaced by a better one, never abandoned with nothing left to take its place." (James B Conant, "Science and Common Sense", 1951)

"The act of discovery escapes logical analysis; there are no logical rules in terms of which a 'discovery machine' could be constructed that would take over the creative function of the genius. But it is not the logician’s task to account for scientific discoveries; all he can do is to analyze the relation between given facts and a theory presented to him with the claim that it explains these facts. In other words, logic is concerned with the context of justification." (Hans Reichenbach, "The Rise of Scientific Philosophy", 1951)

"The study of inductive inference belongs to the theory of probability, since observational facts can make a theory only probable but will never make it absolutely certain." (Hans Reichenbach, "The Rise of Scientific Philosophy", 1951)

"[…] the grand aim of all science […] is to cover the greatest possible number of empirical facts by logical deductions from the smallest possible number of hypotheses or axioms." (Albert Einstein, 1954)

"Science does not begin with facts; one of its tasks is to uncover the facts by removing misconceptions." (Lancelot L Whyte, "Accent on Form", 1954)

"Science is the creation of concepts and their exploration in the facts. It has no other test of the concept than its empirical truth to fact." (Jacob Bronowski, "Science and Human Values", 1956)

"When we meet a fact which contradicts a prevailing theory, we must accept the fact and abandon the theory, even when the theory is supported by great names and generally accepted." (Claude Bernard, "An Introduction to the Study of Experimental Medicine", 1957)

"Science aims at the discovery, verification, and organization of fact and information [...] engineering is fundamentally committed to the translation of scientific facts and information to concrete machines, structures, materials, processes, and the like that can be used by men." (Eric A Walker, "Engineers and/or Scientists", Journal of Engineering Education Vol. 51, 1961)

"The important distinction between science and those other systematizations [i.e., art, philosophy, and theology] is that science is self-testing and self-correcting. Here the essential point of science is respect for objective fact. What is correctly observed must be believed [...] the competent scientist does quite the opposite of the popular stereotype of setting out to prove a theory; he seeks to disprove it." (George G Simpson, "Notes on the Nature of Science", 1962)

"It is not impossible that our own Model will die a violent death, ruthlessly smashed by an unprovoked assault of new facts […]. (Clive S Lewis, "The Discarded Image: An Introduction to Medieval and Renaissance Literature", 1964)

"A model is a qualitative or quantitative representation of a process or endeavor that shows the effects of those factors which are significant for the purposes being considered. A model may be pictorial, descriptive, qualitative, or generally approximate in nature; or it may be mathematical and quantitative in nature and reasonably precise. It is important that effective means for modeling be understood such as analog, stochastic, procedural, scheduling, flow chart, schematic, and block diagrams." (Harold Chestnut, "Systems Engineering Tools", 1965)

"A model is a useful (and often indispensable) framework on which to organize our knowledge about a phenomenon. […] It must not be overlooked that the quantitative consequences of any model can be no more reliable than the a priori agreement between the assumptions of the model and the known facts about the real phenomenon. When the model is known to diverge significantly from the facts, it is self-deceiving to claim quantitative usefulness for it by appeal to agreement between a prediction of the model and observation." (John R Philip, 1966)

"To do science is to search for repeated patterns, not simply to accumulate facts, and to do the science of geographical ecology is to search for patterns of plants and animal life that can be put on a map." (Robert H. MacArthur, "Geographical Ecology", 1972)

"Models are not assigned per se uniquely to their originals. They perform their replacement function: a) for definite – cognitive and/or handling, model-using – subjects, b) within definite time intervals, c) under restrictions of definite operations of thought or fact. […] Models are not only models of something. They are also models for somebody, a human or an artificial model user. They perform thereby their functions in time, within a time interval. And finally, they are models for a definite purpose." (Herbert Stachowiak, "Allgemeine Modelltheorie", 1973)

"No theory ever agrees with all the facts in its domain, yet it is not always the theory that is to blame. Facts are constituted by older ideologies, and a clash between facts and theories may be proof of progress. It is also a first step in our attempt to find the principles implicit in familiar observational notions." (Paul K Feyerabend, "Against Method: Outline of an Anarchistic Theory of Knowledge", 1975)

"Facts do not ‘speak for themselves’; they are read in the light of theory. Creative thought, in science as much as in the arts, is the motor of changing opinion. Science is a quintessentially human activity, not a mechanized, robot-like accumulation of objective information, leading by laws of logic to inescapable interpretation." (Stephen J Gould, "Ever Since Darwin", 1977)

"Science has so accustomed us to devising and accepting theories to account for the facts we observe, however fantastic, that our minds must begin their manufacture before we are aware of it." (Gene Wolfe, "Seven American Nights", 1978)

"Science, since people must do it, is a socially embedded activity. It progresses by hunch, vision, and intuition. Much of its change through time does not record a closer approach to absolute truth, but the alteration of cultural contexts that influence it so strongly. Facts are not pure and unsullied bits of information; culture also influences what we see and how we see it. Theories, moreover, are not inexorable inductions from facts. The most creative theories are often imaginative visions imposed upon facts; the source of imagination is also strongly cultural." (Stephen J Gould, "The Mismeasure of Man", 1980)

"Facts and theories are different things, not rungs in a hierarchy of increasing certainty. Facts are the world's data. Theories are structures of ideas that explain and interpret facts. Facts do not go away while scientists debate rival theories for explaining them." (Stephen J Gould "Evolution as Fact and Theory", 1981)

"All great theories are expansive, and all notions so rich in scope and implication are underpinned by visions about the nature of things. You may call these visions ‘philosophy’, or ‘metaphor’, or ‘organizing principle’, but one thing they are surely not - they are not simple inductions from observed facts of the natural world." (Stephen J Gould, "Time’s Arrow, Time’s Cycle", 1987)

"Although science literally means ‘knowledge’, the scientific attitude is concerned much more with rational perception through the mind and with testing such perceptions against actual fact, in the form of experiments and observations." (David Bohm & F David Peat, "Science, Order, and Creativity", 1987)

"Facts do not 'speak for themselves'. They speak for or against competing theories. Facts divorced from theory or visions are mere isolated curiosities." (Thomas Sowell, "A Conflict of Visions: Ideological Origins of Political Struggles", 1987)

"[…] no good model ever accounted for all the facts, since some data was bound to be misleading if not plain wrong. A theory that did fit all the data would have been ‘carpentered’ to do this and would thus be open to suspicion." (Francis H C Crick, "What Mad Pursuit: A Personal View of Scientific Discovery", 1988)

"The common perception of science as a rational activity, in which one confronts the evidence of fact with an open mind, could not be more false. Facts assume significance only within a pre-existing intellectual structure, which may be based as much on intuition and prejudice as on reason." (Walter Gratzer, The Guardian, 1989)

"On this view, we recognize science to be the search for algorithmic compressions. We list sequences of observed data. We try to formulate algorithms that compactly represent the information content of those sequences. Then we test the correctness of our hypothetical abbreviations by using them to predict the next terms in the string. These predictions can then be compared with the future direction of the data sequence. Without the development of algorithmic compressions of data all science would be replaced by mindless stamp collecting - the indiscriminate accumulation of every available fact. Science is predicated upon the belief that the Universe is algorithmically compressible and the modern search for a Theory of Everything is the ultimate expression of that belief, a belief that there is an abbreviated representation of the logic behind the Universe's properties that can be written down in finite form by human beings." (John D Barrow, New Theories of Everything", 1991)

"The word theory, as used in the natural sciences, doesn’t mean an idea tentatively held for purposes of argument - that we call a hypothesis. Rather, a theory is a set of logically consistent abstract principles that explain a body of concrete facts. It is the logical connections among the principles and the facts that characterize a theory as truth. No one element of a theory [...] can be changed without creating a logical contradiction that invalidates the entire system. Thus, although it may not be possible to substantiate directly a particular principle in the theory, the principle is validated by the consistency of the entire logical structure." (Alan Cromer, "Uncommon Sense: The Heretical Nature of Science", 1993)

"Worldviews are social constructions, and they channel the search for facts. But facts are found and knowledge progresses, however fitfully. Fact and theory are intertwined, and all great scientists understand the interaction." (Stephen J Gould, "Shields of Expectation - and Actuality", 1993)

"As a result, surprisingly enough, scientific advance rarely comes solely through the accumulation of new facts. It comes most often through the construction of new theoretical frameworks. [..]  To understand scientific development, it is not enough merely to chronicle new discoveries and inventions. We must also trace the succession of worldviews" (Nancy R Pearcey & Charles B Thaxton, "The Soul of Science: Christian Faith and Natural Philosophy", 1994)

"Theories rarely arise as patient inferences forced by accumulated facts. Theories are mental constructs potentiated by complex external prods (including, in idealized cases, a commanding push from empirical reality)." (Stephen J Gould, "Leonardo's Mountain of Clams and the Diet of Worms" , 1998)

"When we entrust the domain of values to those whose intellectual concerns are essentially centred on empirical facts, and whose conceptual frameworks are inevitably constructed around sets of empirical facts, we need not be surprised if the result is moral confusion." (Ronald W K Paterson, "The New Patricians", 1998)

"Modeling involves a style of scientific thinking in which the argument is structured by the model, but in which the application is achieved via a narrative prompted by an external fact, an imagined event or question to be answered." (Uskali Mäki, "Fact and Fiction in Economics: Models, Realism and Social Construction", 2002)

"We tackle a multifaceted universe one face at a time, tailoring our models and equations to fit the facts at hand. Whatever mechanical conception proves appropriate, that is the one to use. Discovering worlds within worlds, a practical observer will deal with each realm on its own terms. It is the only sensible approach to take." (Michael Munowitz, "Knowing: The Nature of Physical Law", 2005)

"Although fiction is not fact, paradoxically we need some fictions, particularly mathematical ideas and highly idealized models, to describe, explain, and predict facts.  This is not because the universe is mathematical, but because our brains invent or use refined and law-abiding fictions, not only for intellectual pleasure but also to construct conceptual models of reality." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"There are no surprising facts, only models that are surprised by facts; and if a model is surprised by the facts, it is no credit to that model." (Eliezer S Yudkowsky, "Quantum Explanations", 2008)

"Obviously, the final goal of scientists and mathematicians is not simply the accumulation of facts and lists of formulas, but rather they seek to understand the patterns, organizing principles, and relationships between these facts to form theorems and entirely new branches of human thought." (Clifford A Pickover, "The Math Book", 2009)

"Each person has a different mental model and, therefore, potentially a different interpretation of the Facts. The danger comes when we start to assume that our interpretation of the Facts is the only interpretation and we believe that what we see and think is the Truth, and that there is only one Truth." (Robina Chatham & Brian Sutton, "Changing the IT Leader’s Mindset", 2010)

"Relevance is not something you can predict. It is something you discover after the fact." (Thomas Sowell, "The Thomas Sowell Reader", 2011)

"Science does not live with facts alone. In addition to facts, it needs models. Scientific models fulfill two main functions with respect to empirical facts." (Andreas Bartels [in "Models, Simulations, and the Reduction of Complexity", Ed. by Ulrich Gähde et al, 2013)

"A mental representation is a mental structure that corresponds to an object, an idea, a collection of information, or anything else, concrete or abstract, that the brain is thinking about. […] Because the details of mental representations can differ dramatically from field to field, it’s hard to offer an overarching definition that is not too vague, but in essence these representations are preexisting patterns of information - facts, images, rules, relationships, and so on - that are held in long-term memory and that can be used to respond quickly and effectively in certain types of situations." (Anders Ericsson & Robert Pool," Peak: Secrets from  the  New  Science  of  Expertise", 2016)

"Statistics is the science of collecting, organizing, and interpreting numerical facts, which we call data. […] Statistics is the science of learning from data." (Moore McCabe & Alwan Craig, "The Practice of Statistics for Business and Economics" 4th Ed., 2016)

"That is the trouble with facts: they sometimes force you to conclusions that differ with your intuition." (Steven G Krantz, "A Primer of Mathematical Writing" 2nd Ed., 2016)