Systems Thinking: On Connectivity (Quotes)

“Concepts can only acquire content when they are connected, however indirectly, with sensible experience. But no logical investigation can reveal this connection; it can only be experienced. […] this connection […] determines the cognitive value of systems of concepts.” (Albert Einstein, "The Problem of Space, Ether, and the Field in Physics", Mein Weltbild, 1934)

"The first attempts to consider the behavior of so-called 'random neural nets' in a systematic way have led to a series of problems concerned with relations between the 'structure' and the 'function' of such nets. The 'structure' of a random net is not a clearly defined topological manifold such as could be used to describe a circuit with explicitly given connections. In a random neural net, one does not speak of 'this' neuron synapsing on 'that' one, but rather in terms of tendencies and probabilities associated with points or regions in the net." (Anatol Rapoport, "Cycle distributions in random nets", The Bulletin of Mathematical Biophysics 10(3), 1948)

"For understanding the general principles of dynamic systems, therefore, the concept of feedback is inadequate in itself. What is important is that complex systems, richly cross-connected internally, have complex behaviours, and that these behaviours can be goal-seeking in complex patterns." (W Ross Ashby, "An Introduction to Cybernetics", 1956)

"I am using the term 'network' in a general sense, to imply any set of interconnected and measurably active physical entities. Naturally occurring networks, of interest because they have a, self-organizing character, are, for example, a marsh, a colony of microorganisms, a research team, and a man." (Gordon Pask, "The Natural History of Networks", 1960)

"To say a system is 'self-organizing' leaves open two quite different meanings. There is a first meaning that is simple and unobjectionable. This refers to the system that starts with its parts separate (so that the behavior of each is independent of the others' states) and whose parts then act so that they change towards forming connections of some type. Such a system is 'self-organizing' in the sense that it changes from 'parts separated' to 'parts joined'. […] In general such systems can be more simply characterized as 'self-connecting', for the change from independence between the parts to conditionality can always be seen as some form of 'connection', even if it is as purely functional […]" (W Ross Ashby, "Principles of the self-organizing system", 1962)

"In general, one might define a complex of semantic components connected by logical constants as a concept. The dictionary of a language is then a system of concepts in which a phonological form and certain syntactic and morphological characteristics are assigned to each concept. This system of concepts is structured by several types of relations. It is supplemented, furthermore, by redundancy or implicational rules […] representing general properties of the whole system of concepts. […] At least a relevant part of these general rules is not bound to particular languages, but represents presumably universal structures of natural languages. They are not learned, but are rather a part of the human ability to acquire an arbitrary natural language." (Manfred Bierwisch, "Semantics", 1970)

"Whatever the system, adaptive change depends upon feedback loops, be it those provided by natural selection or those of individual reinforcement. In all cases, then, there must be a process of trial and error and a mechanism of comparison. […] By superposing and interconnecting many feedback loops, we (and all other biological systems) not only solve particular problems but also form habits which we apply to the solution of classes of problems." (Gregory Bateson, "Steps to an Ecology of Mind", 1972)

"There is a strong current in contemporary culture advocating ‘holistic’ views as some sort of cure-all […] Reductionism implies attention to a lower level while holistic implies attention to higher level. These are intertwined in any satisfactory description: and each entails some loss relative to our cognitive preferences, as well as some gain [...] there is no whole system without an interconnection of its parts and there is no whole system without an environment." (Francisco Varela, "On being autonomous: The lessons of natural history for systems theory", 1977)

"Information is recorded in vast interconnecting networks. Each idea or image has hundreds, perhaps thousands, of associations and is connected to numerous other points in the mental network." (Peter Russell, "The Brain Book: Know Your Own Mind and How to Use it", 1979)

"When loops are present, the network is no longer singly connected and local propagation schemes will invariably run into trouble. [...] If we ignore the existence of loops and permit the nodes to continue communicating with each other as if the network were singly connected, messages may circulate indefinitely around the loops and process may not converges to a stable equilibrium. […] Such oscillations do not normally occur in probabilistic networks […] which tend to bring all messages to some stable equilibrium as time goes on. However, this asymptotic equilibrium is not coherent, in the sense that it does not represent the posterior probabilities of all nodes of the network." (Judea Pearl, "Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference", 1988)

"Systems thinking is a discipline for seeing wholes. It is a framework for seeing interrelationships rather than things, for seeing patterns of change rather than static 'snapshots'. It is a set of general principles- distilled over the course of the twentieth century, spanning fields as diverse as the physical and social sciences, engineering, and management. [...] During the last thirty years, these tools have been applied to understand a wide range of corporate, urban, regional, economic, political, ecological, and even psychological systems. And systems thinking is a sensibility for the subtle interconnectedness that gives living systems their unique character." (Peter Senge, "The Fifth Discipline", 1990)

"An artificial neural network is an information-processing system that has certain performance characteristics in common with biological neural networks. Artificial neural networks have been developed as generalizations of mathematical models of human cognition or neural biology, based on the assumptions that: (1) Information processing occurs at many simple elements called neurons. (2) Signals are passed between neurons over connection links. (3) Each connection link has an associated weight, which, in a typical neural net, multiplies the signal transmitted. (4) Each neuron applies an activation function (usually nonlinear) to its net input (sum of weighted input signals) to determine its output signal." (Laurene Fausett, "Fundamentals of Neural Networks", 1994)

"It has long been appreciated by science that large numbers behave differently than small numbers. Mobs breed a requisite measure of complexity for emergent entities. The total number of possible interactions between two or more members accumulates exponentially as the number of members increases. At a high level of connectivity, and a high number of members, the dynamics of mobs takes hold. " (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"In sharp contrast (with the traditional social planning) the systems design approach seeks to understand a problem situation as a system of interconnected, interdependent, and interacting issues and to create a design as a system of interconnected, interdependent, interacting, and internally consistent solution ideas." (Béla H Bánáthy, "Designing Social Systems in a Changing World", 1996)

"In the new systems thinking, the metaphor of knowledge as a building is being replaced by that of the network. As we perceive reality as a network of relationships, our descriptions, too, form an interconnected network of concepts and models in which there are no foundations. For most scientists such a view of knowledge as a network with no firm foundations is extremely unsettling, and today it is by no means generally accepted. But as the network approach expands throughout the scientific community, the idea of knowledge as a network will undoubtedly find increasing acceptance." (Fritjof Capra," The Web of Life: a new scientific understanding of living systems", 1996)

"The more complex the network is, the more complex its pattern of interconnections, the more resilient it will be." (Fritjof Capra, "The Web of Life: A New Scientific Understanding of Living Systems", 1996)

"A standalone object, no matter how well designed, has limited potential for new weirdness. A connected object, one that is a node in a network that interacts in some way with other nodes, can give birth to a hundred unique relationships that it never could do while unconnected. Out of this tangle of possible links come myriad new niches for innovations and interactions." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"We are connecting everything to everything. […] When we permit any object to transmit a small amount of data and to receive input from its neighborhood, we change an inert object into an animated node." (Kevin Kelly, "New Rules for the New Economy: 10 radical strategies for a connected world", 1998)

"The three basic mechanisms of averaging, feedback and division of labor give us a first idea of a how a CMM [Collective Mental Map] can be developed in the most efficient way, that is, how a given number of individuals can achieve a maximum of collective problem-solving competence. A collective mental map is developed basically by superposing a number of individual mental maps. There must be sufficient diversity among these individual maps to cover an as large as possible domain, yet sufficient redundancy so that the overlap between maps is large enough to make the resulting graph fully connected, and so that each preference in the map is the superposition of a number of individual preferences that is large enough to cancel out individual fluctuations. The best way to quickly expand and improve the map and fill in gaps is to use a positive feedback that encourages individuals to use high preference paths discovered by others, yet is not so strong that it discourages the exploration of new paths." (Francis Heylighen, "Collective Intelligence and its Implementation on the Web", 1999)

"Most systems displaying a high degree of tolerance against failures are a common feature: Their functionality is guaranteed by a highly interconnected complex network. A cell's robustness is hidden in its intricate regulatory and metabolic network; society's resilience is rooted in the interwoven social web; the economy's stability is maintained by a delicate network of financial and regulator organizations; an ecosystem's survivability is encoded in a carefully crafted web of species interactions. It seems that nature strives to achieve robustness through interconnectivity. Such universal choice of a network architecture is perhaps more than mere coincidences." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"A self-organizing system not only regulates or adapts its behavior, it creates its own organization. In that respect it differs fundamentally from our present systems, which are created by their designer. We define organization as structure with function. Structure means that the components of a system are arranged in a particular order. It requires both connections, that integrate the parts into a whole, and separations that differentiate subsystems, so as to avoid interference. Function means that this structure fulfils a purpose." (Francis Heylighen & Carlos Gershenson, "The Meaning of Self-organization in Computing", IEEE Intelligent Systems, 2003)

"At an anatomical level - the level of pure, abstract connectivity - we seem to have stumbled upon a universal pattern of complexity. Disparate networks show the same three tendencies: short chains, high clustering, and scale-free link distributions. The coincidences are eerie, and baffling to interpret." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"Average path length reflects the global structure; it depends on the way the entire network is connected, and cannot be inferred from any local measurement. Clustering reflects the local structure; it depends only on the interconnectedness of a typical neighborhood, the inbreeding among nodes tied to a common center. Roughly speaking, path length measures how big the network is. Clustering measures how incestuous it is." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"[…] topology, the study of continuous shape, a kind of generalized geometry where rigidity is replaced by elasticity. It's as if everything is made of rubber. Shapes can be continuously deformed, bent, or twisted, but not cut - that's never allowed. A square is topologically equivalent to a circle, because you can round off the corners. On the other hand, a circle is different from a figure eight, because there's no way to get rid of the crossing point without resorting to scissors. In that sense, topology is ideal for sorting shapes into broad classes, based on their pure connectivity." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"We’re accustomed to  in terms of centralized control, clear chains of command, the straightforward logic of cause and effect. But in huge, interconnected systems, where every player ultimately affects every other, our standard ways of thinking fall apart. Simple pictures and verbal arguments are too feeble, too myopic." (Steven Strogatz, "Sync: The Emerging Science of Spontaneous Order", 2003)

"Learning is the process of creating networks. Nodes are external entities which we can use to form a network. Or nodes may be people, organizations, libraries, web sites, books, journals, database, or any other source of information. The act of learning (things become a bit tricky here) is one of creating an external network of nodes - where we connect and form information and knowledge sources. The learning that happens in our heads is an internal network (neural). Learning networks can then be perceived as structures that we create in order to stay current and continually acquire, experience, create, and connect new knowledge (external). And learning networks can be perceived as structures that exist within our minds (internal) in connecting and creating patterns of understanding. (George Siemens, "Knowing Knowledge", 2006)

"The addition of new elements or agents to a particular system multiplies exponentially the number of connections or potential interactions among those elements or agents, and hence the number of possible outcomes. This is an important attribute of complexity theory." (Mark Marson, "What Are Its Implications for Educational Change?", 2008)

"You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays." (Donella H Meadow, "Thinking in Systems: A Primer", 2008)

"An artificial neural network, often just called a 'neural network' (NN), is an interconnected group of artificial neurons that uses a mathematical model or computational model for information processing based on a connectionist approach to computation. Knowledge is acquired by the network from its environment through a learning process, and interneuron connection strengths (synaptic weighs) are used to store the acquired knowledge." (Larbi Esmahi et al, "Adaptive Neuro-Fuzzy Systems", 2009)

"Complexity theory embraces things that are complicated, involve many elements and many interactions, are not deterministic, and are given to unexpected outcomes. […] A fundamental aspect of complexity theory is the overall or aggregate behavior of a large number of items, parts, or units that are entangled, connected, or networked together. […] In contrast to classical scientific methods that directly link theory and outcome, complexity theory does not typically provide simple cause-and-effect explanations." (Robert E Gunther et al, "The Network Challenge: Strategy, Profit, and Risk in an Interlinked World", 2009)

"We are beginning to see the entire universe as a holographically interlinked network of energy and information, organically whole and self-referential at all scales of its existence. We, and all things in the universe, are non-locally connected with each other and with all other things in ways that are unfettered by the hitherto known limitations of space and time." (Ervin László,"Cosmos: A Co-creator's Guide to the Whole-World", 2010)

"A self–organizing system acts autonomously, as if the interconnecting components had a single mind. And as these components spontaneously march to the beat of their own drummer, they organize, adapt, and evolve toward a greater complexity than one would ever expect by just looking at the parts by themselves." (Lawrence K Samuels, "Defense of Chaos: The Chaology of Politics, Economics and Human Action", 2013)

"Deep ecology does not separate humans - or anything else-from the natural environment. It sees the world not as a collection of isolated objects, but as a network of phenomena that are fundamentally interconnected and interdependent. Deep ecology recognizes the intrinsic value of all living beings and views humans as just one particular strand in the web of life." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

"Shallow ecology is anthropocentric, or human-centered. It views humans as above or outside of nature, as the source of all value, and ascribes only instrumental, or ‘use’, value to nature. Deep ecology does not separate humans - or anything else-from the natural environment. It sees the world not as a collection of isolated objects, but as a network of phenomena that are fundamentally interconnected and interdependent. Deep ecology recognizes the intrinsic value of all living beings and views humans as just one particular strand in the web of life." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

"System Dynamics is a dynamic modelling approach at system level which is primarily used to understand interconnected systems and their evolution over time. Basic elements to represent the systems are internal feedback loops as well as stocks and flows." (Catalina Spataru et al, "Multi-Scale, Multi-Dimensional Modelling of Future Energy Systems", 2015)

Systems Thinking: On Hierarchies (Quotes)

"A second possible approach to general systems theory is through the arrangement of theoretical systems and constructs in a hierarchy of complexity, roughly corresponding to the complexity of the ‘individuals’ of the various empirical fields […] leading towards a ‘system of systems’." (Kenneth E Boulding, "General Systems Theory - The Skeleton of Science", 1956)

"One advantage of exhibiting a hierarchy of systems in this way is that it gives us some idea of the present gaps in both theoretical and empirical knowledge. Adequate theoretical models extend up to about the fourth level, and not much beyond. Empirical knowledge is deficient at practically all levels." (Kenneth E Boulding, "General Systems Theory: The Skeleton of Science", 1956)

"Two possible approaches to the organization of general systems theory suggest themselves, which are to be thought of as complementary rather than competitive, or at least as two roads each of which is worth exploring. The first approach is to look over the empirical universe and to pick out certain general phenomena which are found in many different disciplines, and to seek to build up general theoretical models relevant to these phenomena. The second approach is to arrange the empirical fields in a hierarchy of complexity of organization of their basic 'individual' or unit of behavior, and to try to develop a level of abstraction appropriate to each." (Kenneth E. Boulding, General Systems Theory - The Skeleton of Science, Management Science Vol. 2 (3), 1956)

"Thus, the central theme that runs through my remarks is that complexity frequently takes the form of hierarchy, and that hierarchic systems have some common properties that are independent of their specific content. Hierarchy, I shall argue, is one of the central structural schemes that the architect of complexity uses." (Herbert A Simon, "The Architecture of Complexity", Proceedings of the American Philosophical Society Vol. 106 (6), 1962)

"To model the dynamic behavior of a system, four hierarchies of structure should be recognized: closed boundary around the system; feedback loops as the basic structural elements within the boundary; level variables representing accumulations within the feedback loops; rate variables representing activity within the feedback loops." (Jay W Forrester, "Urban Dynamics", 1969)

"The parallelism of general conceptions or even special laws in different fields therefore is a consequence of the fact that these are concerned with 'systems' and that certain general principles apply to systems irrespective of their nature. Hence principles such as those of wholeness and sum, mechanization, hierarchic order, approached to steady states, equifinality, etc., may appear in quite different disciplines. The isomorphism found in different realms is based of the existence of general system principles, of a more or less well-developed ‘general system theory’." (Ludwig von Bertalanffy, "General System Theory", 1968)

"General systems theory is the scientific exploration of 'wholes' and 'wholeness' which, not so long ago, were considered metaphysical notions transcending the boundaries of science. Hierarchic structure, stability, teleology, differentiation, approach to and maintenance of steady states, goal-directedness - these are a few of such general system properties." (Ervin László, "Introduction to Systems Philosophy", 1972)

"[The] system may evolve through a whole succession of transitions leading to a hierarchy of more and more complex and organized states. Such transitions can arise in nonlinear systems that are maintained far from equilibrium: that is, beyond a certain critical threshold the steady-state regime become unstable and the system evolves into a new configuration." (Ilya Prigogine, Gregoire Micolis & Agnes Babloyantz, "Thermodynamics of Evolution", Physics Today 25 (11), 1972)

"The main object of cybernetics is to supply adaptive, hierarchical models, involving feedback and the like, to all aspects of our environment. Often such modelling implies simulation of a system where the simulation should achieve the object of copying both the method of achievement and the end result. Synthesis, as opposed to simulation, is concerned with achieving only the end result and is less concerned (or completely unconcerned) with the method by which the end result is achieved. In the case of behaviour, psychology is concerned with simulation, while cybernetics, although also interested in simulation, is primarily concerned with synthesis." (Frank H George, "Soviet Cybernetics, the militairy and Professor Lerner", New Scientist, 1973)

"[Hierarchy is] the principle according to which entities meaningfully treated as wholes are built up of smaller entities which are themselves wholes […] and so on. In hierarchy, emergent properties denote the levels." (Peter Checkland, "Systems Thinking, Systems Practice", 1981)

"Systems thinking is a special form of holistic thinking - dealing with wholes rather than parts. One way of thinking about this is in terms of a hierarchy of levels of biological organization and of the different 'emergent' properties that are evident in say, the whole plant (e.g. wilting) that are not evident at the level of the cell (loss of turgor). It is also possible to bring different perspectives to bear on these different levels of organization. Holistic thinking starts by looking at the nature and behaviour of the whole system that those participating have agreed to be worthy of study. This involves: (i) taking multiple partial views of 'reality' […] (ii) placing conceptual boundaries around the whole, or system of interest and (iii) devising ways of representing systems of interest." (C J Pearson and R L Ison, "Agronomy of Grassland Systems", 1987)

"This is why a 'web' of notes with links (like references) between them is far more useful than a fixed hierarchical system. When describing a complex system, many people resort to diagrams with circles and arrows. Circles and arrows leave one free to describe the interrelationships between things in a way that tables, for example, do not. The system we need is like a diagram of circles and arrows, where circles and arrows can stand for anything." (Tim Berners-Lee, "Information Management: A Proposal", 1989)

"Light a fire, build up the steam, turn on a switch, and a linear system awakens. It’s ready to serve you. If it stalls, restart it. Simple collective systems can be awakened simply. But complex swarm systems with rich hierarchies take time to boot up. The more complex, the longer it takes to warm up. Each hierarchical layer has to settle down; lateral causes have to slosh around and come to rest; a million autonomous agents have to acquaint themselves. I think this will be the hardest lesson for humans to learn: that organic complexity will entail organic time." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"In a random network the peak of the distribution implies that the vast majority of nodes have the same number of links and that nodes deviating from the average are extremely rare. Therefore, a random network has a characteristic scale in its node connectivity, embodied by the average node and fixed by the peak of the degree distribution. In contrast, the absence of a peak in a power-law degree distribution implies that in a real network there is no such thing as a characteristic node. We see a continuous hierarchy of nodes, spanning from rare hubs to the numerous tiny nodes. The largest hub is closely followed by two or three somewhat smaller hubs, followed by dozens that are even smaller, and so on, eventually arriving at the numerous small nodes." (Albert-László Barabási, "Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life", 2002)

"Another typical feature of theories of emergence is the layered view of nature. On this view, all things in nature belong to a certain level of existence, each according to its characteristic properties. These levels of existence constitute a hierarchy of increasing complexity that also corresponds to their order of appearance in the course of evolution." (Markus Eronen, "Emergence in the Philosophy of Mind", 2004)

"An ecology provides the special formations needed by organizations. Ecologies are: loose, free, dynamic, adaptable, messy, and chaotic. Innovation does not arise through hierarchies. As a function of creativity, innovation requires trust, openness, and a spirit of experimentation - where random ideas and thoughts can collide for re-creation." (George Siemens, "Knowing Knowledge", 2006)

"Change pressures arise from different sectors of a system. At times it is mandated from the top of a hierarchy, other times it forms from participants at a grass-roots level. Some changes are absorbed by the organization without significant impact on, or alterations of, existing methods. In other cases, change takes root. It causes the formation of new methods (how things are done and what is possible) within the organization." (George Siemens, "Knowing Knowledge", 2006)

"Hierarchy adapts knowledge to the organization; a network adapts the organization to the knowledge." (George Siemens, "Knowing Knowledge", 2006)

"[…] our mental models fail to take into account the complications of the real world - at least those ways that one can see from a systems perspective. It is a warning list. Here is where hidden snags lie. You can’t navigate well in an interconnected, feedback-dominated world unless you take your eyes off short-term events and look for long-term behavior and structure; unless you are aware of false boundaries and bounded rationality; unless you take into account limiting factors, nonlinearities and delays. You are likely to mistreat, misdesign, or misread systems if you don’t respect their properties of resilience, self-organization, and hierarchy." (Donella H Meadows, "Thinking in Systems: A Primer", 2008)

"A hierarchy is a diagram that shows how various components of a system are related, often with a downward direction (or alternatively a left-to-right direction) that moves from more general to more specific. One way to envision a hierarchy is as an inverted tree: We start with a single component (referred to as the root node or topmost node), typically denoted by a square, and then we draw one or more paths leading from it to other nodes. Each of these nodes, in turn, may subdivide into additional subpaths to other nodes. This process may be repeated any number of times to arrive at a multitiered, tree-like structure." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Laws of complexity hold universally across hierarchical scales (scalar, self-similarity) and are not influenced by the detailed behavior of constituent parts." (Jamshid Gharajedaghi, "Systems Thinking: Managing Chaos and Complexity A Platform for Designing Business Architecture" 3rd Ed., 2011)

"The notion of emergence is used in a variety of disciplines such as evolutionary biology, the philosophy of mind and sociology, as well as in computational and complexity theory. It is associated with non-reductive naturalism, which claims that a hierarchy of levels of reality exist. While the emergent level is constituted by the underlying level, it is nevertheless autonomous from the constituting level. As a naturalistic theory, it excludes non-natural explanations such as vitalistic forces or entelechy. As non-reductive naturalism, emergence theory claims that higher-level entities cannot be explained by lower-level entities." (Martin Neumann, "An Epistemological Gap in Simulation Technologies and the Science of Society", 2011)

"In other words, the web of life consists of networks within networks. At each scale, under closer scrutiny, the nodes of the network reveal themselves as smaller networks. We tend to arrange these systems, all nesting within larger systems, in a hierarchical scheme by placing the larger systems above the smaller ones in pyramid fashion. But this is a human projection. In nature there is no 'above' or 'below', and there are no hierarchies. There are only networks nesting within other networks." (Fritjof Capra, "The Systems View of Life: A Unifying Vision", 2014)

[the principle of hierarchy:] "Generally, a control system has a hierarchical structure. It must agree with the functional structure of a controlled system and not contradict the hierarchy of (horizontally or vertically) adjacent systems. Tasks and resources supporting the activity of a controlled system must be decomposed according to its structure." (Dmitry A Novikov, "Cybernetics: From Past to Future", 2016)

Knowledge Representation: On Domains (Quotes)

"Great discoveries which give a new direction to currents of thoughts and research are not, as a rule, gained by the accumulation of vast quantities of figures and statistics. These are apt to stifle and asphyxiate and they usually follow rather than precede discovery. The great discoveries are due to the eruption of genius into a closely related field, and the transfer of the precious knowledge there found to his own domain." (Theobald Smith, Boston Medical and Surgical Journal Volume 172, 1915)

"For a certain domain of facts let no theory be known. If we replace our study of this domain by the study of another set of facts for which a theory is well known, and that has certain important characteristics in common with the field under investigation, then we use a model to develop our knowledge from a zero (or near zero) starting point." (Leo Apostel, "Towards the formal study of models in the non-formal sciences", Synthese Vol. 12 (2-3), 1960)

"Learning is any change in a system that produces a more or less permanent change in its capacity for adapting to its environment. Understanding systems, especially systems capable of understanding problems in new task domains, are learning systems." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"A cognitive system is a system whose organization defines a domain of interactions in which it can act with relevance to the maintenance of itself, and the process of cognition is the actual (inductive) acting or behaving in this domain. Living systems are cognitive systems, and living as a process is a process of cognition. This statement is valid for all organisms, with and without a nervous system." (Humberto R Maturana, "Biology of Cognition", 1970)

"No theory ever agrees with all the facts in its domain, yet it is not always the theory that is to blame. Facts are constituted by older ideologies, and a clash between facts and theories may be proof of progress. It is also a first step in our attempt to find the principles implicit in familiar observational notions." (Paul K Feyerabend, "Against Method: Outline of an Anarchistic Theory of Knowledge", 1975)

"A cognitive map is a specific way of representing a person's assertions about some limited domain, such as a policy problem. It is designed to capture the structure of the person's causal assertions and to generate the consequences that follow front this structure. […]  a person might use his cognitive map to derive explanations of the past, make predictions for the future, and choose policies in the present." (Robert M Axelrod, "Structure of Decision: The cognitive maps of political elites", 1976)

"The thinking person goes over the same ground many times. He looks at it from varying points of view - his own, his arch-enemy’s, others’. He diagrams it, verbalizes it, formulates equations, constructs visual images of the whole problem, or of troublesome parts, or of what is clearly known. But he does not keep a detailed record of all this mental work, indeed could not. […] Deep understanding of a domain of knowledge requires knowing it in various ways. This multiplicity of perspectives grows slowly through hard work and sets the state for the re-cognition we experience as a new insight." (Howard E Gruber, "Darwin on Man", 1981)

"Metaphor [is] a pervasive mode of understanding by which we project patterns from one domain of experience in order to structure another domain of a different kind. So conceived metaphor is not merely a linguistic mode of expression; rather, it is one of the chief cognitive structures by which we are able to have coherent, ordered experiences that we can reason about and make sense of. Through metaphor, we make use of patterns that obtain in our physical experience to organise our more abstract understanding." (Mark Johnson, "The Body in the Mind", 1987)

"There is no coherent knowledge, i.e. no uniform comprehensive account of the world and the events in it. There is no comprehensive truth that goes beyond an enumeration of details, but there are many pieces of information, obtained in different ways from different sources and collected for the benefit of the curious. The best way of presenting such knowledge is the list - and the oldest scientific works were indeed lists of facts, parts, coincidences, problems in several specialized domains." (Paul K Feyerabend, "Farewell to Reason", 1987)

"[…] a mental model is a mapping from a domain into a mental representation which contains the main characteristics of the domain; a model can be ‘run’ to generate explanations and expectations with respect to potential states. Mental models have been proposed in particular as the kind of knowledge structures that people use to understand a specific domain […]" (Helmut Jungermann, Holger Schütz & Manfred Thuering, "Mental models in risk assessment: Informing people about drugs", Risk Analysis 8 (1), 1988)

"Algorithmic complexity theory and nonlinear dynamics together establish the fact that determinism reigns only over a quite finite domain; outside this small haven of order lies a largely uncharted, vast wasteland of chaos." (Joseph Ford, "Progress in Chaotic Dynamics: Essays in Honor of Joseph Ford's 60th Birthday", 1988)

"When partitioning a domain, we divide the information model so that the clusters remain intact. [...] Each section of the information model then becomes a separate subsystem. Note that when the information model is partitioned into subsystems, each object is assigned to exactly one subsystem."  (Stephen J Mellor, "Object-Oriented Systems Analysis: Modeling the World In Data", 1988) 

"While a small domain (consisting of fifty or fewer objects) can generally be analyzed as a unit, large domains must be partitioned to make the analysis a manageable task. To make such a partitioning, we take advantage of the fact that objects on an information model tend to fall into clusters: groups of objects that are interconnected with one another by many relationships. By contrast, relatively few relationships connect objects in different clusters." (Stephen J Mellor, "Object-Oriented Systems Analysis: Modeling the World In Data", 1988) 

"A law explains a set of observations; a theory explains a set of laws. […] a law applies to observed phenomena in one domain (e.g., planetary bodies and their movements), while a theory is intended to unify phenomena in many domains. […] Unlike laws, theories often postulate unobservable objects as part of their explanatory mechanism." (John L Casti, "Searching for Certainty: How Scientists Predict the Future", 1990)

"Generally speaking, problem knowledge for solving a given problem may consist of heuristic rules or formulas that comprise the explicit knowledge, and past-experience data that comprise the implicit, hidden knowledge. Knowledge represents links between the domain space and the solution space, the space of the independent variables and the space of the dependent variables." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"Inference is the process of matching current facts from the domain space to the existing knowledge and inferring new facts. An inference process is a chain of matchings. The intermediate results obtained during the inference process are matched against the existing knowledge. The length of the chain is different. It depends on the knowledge base and on the inference method applied." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)

"An individual understands a concept, skill, theory, or domain of knowledge to the extent that he or she can apply it appropriately in a new situation." (Howard Gardner, "The Disciplined Mind", 1999)

"Knowledge maps are node-link representations in which ideas are located in nodes and connected to other related ideas through a series of labeled links. They differ from other similar representations such as mind maps, concept maps, and graphic organizers in the deliberate use of a common set of labeled links that connect ideas. Some links are domain specific (e.g., function is very useful for some topic domains...) whereas other links (e.g., part) are more broadly used. Links have arrowheads to indicate the direction of the relationship between ideas." (Angela M. O’Donnell et al, "Knowledge Maps as Scaffolds for Cognitive Processing", Educational Psychology Review Vol. 14 (1), 2002) 

"We build models to increase productivity, under the justified assumption that it's cheaper to manipulate the model than the real thing. Models then enable cheaper exploration and reasoning about some universe of discourse. One important application of models is to understand a real, abstract, or hypothetical problem domain that a computer system will reflect. This is done by abstraction, classification, and generalization of subject-matter entities into an appropriate set of classes and their behavior." (Stephen J Mellor, "Executable UML: A Foundation for Model-Driven Architecture", 2002)

"The domain of systems science consists thus of all kinds of relational properties which are valid for particular classes of systems, or, in some rare instances, are valid for all systems. The chosen relational classification of systems determines the way in which the domain of systems is divided into subdomains, in a similar fashion as the domain of traditional science has been divided into subdomains of the various disciplines and specializations." (George J Klir & Doug Elias, "Architecture of Systems Problem Solving" 2nd Ed, 2003) 

"Perception and memory are imprecise filters of information, and the way in which information is presented, that is, the frame, influences how it is received. Because too much information is difficult to deal with, people have developed shortcuts or heuristics in order to come up with reasonable decisions. Unfortunately, sometimes these heuristics lead to bias, especially when used outside their natural domains." (Lucy F Ackert & Richard Deaves, "Behavioral Finance: Psychology, Decision-Making, and Markets", 2010)

"This is always the case in analogical reasoning: Relations between two dissimilar domains never map completely to one another. In fact, it is often the salient similarities between the base and target domains that provoke thought and increase the usefulness of an analogy as a problem-solving tool." (Robbie T Nakatsu, "Diagrammatic Reasoning in AI", 2010)

"Conceptual models are best thought of as design-tools - a way for designers to straighten out and simplify the design and match it to the users’ task-domain, thereby making it clearer to users how they should think about the application. The designers’ responsibility is to devise a conceptual model that seems natural to users based on the users’ familiarity with the task domain. If designers do their job well, the conceptual model will be the basis for users’ mental models of the application." (Jeff Johnson & Austin Henderson, "Conceptual Models", 2011)

"Stated loosely, models are simplified, idealized and approximate representations of the structure, mechanism and behavior of real-world systems. From the standpoint of set-theoretic model theory, a mathematical model of a target system is specified by a nonempty set - called the model’s domain, endowed with some operations and relations, delineated by suitable axioms and intended empirical interpretation." (Zoltan Domotor, "Mathematical Models in Philosophy of Science" [Mathematics of Complexity and Dynamical Systems, 2012])

"A model or conceptual model is a schematic or representation that describes how something works. We create and adapt models all the time without realizing it. Over time, as you gain more information about a problem domain, your model will improve to better match reality." (James Padolsey, "Clean Code in JavaScript", 2020)

"Knowledge graphs use an organizing principle so that a user (or a computer system) can reason about the underlying data. The organizing principle gives us an additional layer of organizing data (metadata) that adds connected context to support reasoning and knowledge discovery. […] Importantly, some processing can be done without knowledge of the domain, just by leveraging the features of the property graph model (the organizing principle)." (Jesús Barrasa et al, "Knowledge Graphs: Data in Context for Responsive Businesses", 2021)

Knowledge Representation: On Understanding (Quotes)

"We invoke the imagination and the intervals that it furnishes, since the form itself is without motion or genesis, indivisible and free of all underlying matter, though the elements latent in the form are produced distinctly and individually on the screen of imagination. What projects the images is the understanding; the source of what is projected is the form in the understanding; and what they are projected in is this 'passive nous' that unfolds in revolution about the partlessness of genuine Nous." (Proclus Lycaeus, "A Commentary on the First Book of Euclid’s Elements", cca 5th century)

"Since it seems that there is nothing outside and separate in existence from sensible spatial magnitudes, the objects of thought are in the sensible forms, viz. both the abstract objects and all the states and affections of sensible things. Hence no one can learn or understand anything in the absence of sense, and when the mind is actively aware of anything it is necessarily aware of it along with an image; for images are like sensuous contents except in that they contain no matter. Imagination is different from assertion and denial; for what is true or false involves a synthesis of thoughts. In what will the primary thoughts differ from images? Must we not say that neither these nor even our other thoughts are images, though they necessarily involve them?" (Aristotle, "De Anima", cca. 350 BC)

"[Intuitive] Understanding is consequent upon deliberation, and firmly embraces the better part. For [intuitive] understanding concerns itself with divine truths, and the relish, love, and observance of the latter constitutes true wisdom. Rather than being the [mere] product of nature, these successive steps are the result of grace. The latter, according to its own free determination, derives the various rivulets of the sciences and wisdom from the fountainhead of sense perception. Grace reveals hidden divine truths by means of those things which have been made, and by that unity which belongs to love, communicates what it has made manifest, thus uniting man to God." (John of Salisbury, "Metalogicon", 1159)

"There are three kinds of intelligence: one kind understands things for itself, the other appreciates what others can understand, the third understands neither for itself nor through others. This first kind is excellent, the second good, and the third kind useless." (Niccolò Machiavelli, "The Prince", 1532)

"The human understanding resembles not a dry light, but admits a tincture of the will and passions, which generate their own system accordingly; for man always believes more readily that which he prefers." (Sir Francis Bacon, "Novum Organum", 1620)

"Nay farther, even with relation to that succession, we cou'd only admit of those perceptions, which are immediately present to our consciousness, nor cou'd those lively images, with which the memory presents us, be ever receiv'd as true pictures of past perceptions. The memory, senses, and understanding are, therefore, all of them founded on the imagination, or the vivacity of our ideas."(David Hume, "A Treatise of Human Nature A Treatise of Human Nature", 1739)

"If an inquiry thus carefully conducted should fail at last of discovering the truth, it may answer an end perhaps as useful, in discovering to us the weakness of our own understanding. If it does not make us knowing, it may make us modest. If it does not preserve us from error, it may at least from the spirit of error; and may make us cautious of pronouncing with positiveness or with haste, when so much labour may end in so much uncertainty." (Edmund Burke, "Essay on the Sublime and Beautiful", 1756)

"The analysis of concepts is for the understanding nothing more than what the magnifying glass is for sight." (Moses Mendelssohn, 1763)

"This formal and pure condition of sensibility to which the employment of the concept of understanding is restricted, we shall entitle the schema of the concept. The procedure of understanding in these schemata we shall entitle the schematism of pure understanding.
 The schema is in itself always a product of imagination. Since, however, the synthesis of imagination aims at no special intuition, but only at unity in the determination of sensibility, the schema has to be distinguished from the image." (Immanuel Kant," Critique of Pure Reason", 1781)

“This schematism of our understanding, in its application to appearances and their mere form, is an art concealed in the depths of the human soul, whose real modes of activity nature is hardly likely ever to allow us to discover, and to have open to our gaze.” (Immanuel Kant, “Critique of Pure Reason”, 1781)

"Thoughts without content are empty, intuitions without concepts are blind. The understanding can intuit nothing, the senses can think nothing. Only through their unison can knowledge arise." (Immanuel Kant, "Critique of Pure Reason", 1781)

"As facts and knowledge accumulate, the claim of the scientist to an understanding of the world in a certain sense diminishes." (Werner K Heisenberg, "Zur Geschichte der physikalischen Naturerklärung", 1933)

"We have discovered that it is actually an aid in the search for knowledge to understand the nature of the knowledge we seek." (Arthur S Eddington, "The Philosophy of Physical Science", 1938)

"[…] as a model of a complex system becomes more complete, it becomes less understandable. Alternatively, as a model grows more realistic, it also becomes just as difficult to understand as the real world processes it represents." (Jay M Dutton & William H Starbuck," Computer simulation models of human behavior: A history of an intellectual technology", IEEE Transactions on Systems, 1971)

"The aim of the model is of course not to reproduce reality in all its complexity. It is rather to capture in a vivid, often formal, way what is essential to understanding some aspect of its structure or behavior." (Joseph Weizenbaum, "Computer power and human reason: From judgment to calculation" , 1976)

"New metaphors are capable of creating new understandings and, therefore, new realities. This should be obvious in the case of poetic metaphor, where language is the medium through which new conceptual metaphors are created." (George Lakoff and Mark Johnson, "Metaphors We Live By", 1980)

"The essence of metaphor is understanding and experiencing one kind of thing in terms of another. […] Metaphor is pervasive in everyday life, not just in language but in thought and action. Our ordinary conceptual system, in terms of which we both think and act, is fundamentally metaphorical in nature." (George Lakoff & Mark Johnson, "Metaphors We Live By", 1980)

"After all of this it is a miracle that our models describe anything at all successfully. In fact, they describe many things well: we observe what they have predicted, and we understand what we observe. However, this last act of observation and understanding always eludes physical description." (Yuri I Manin, "Mathematics and Physics", 1981)

"The thinking person goes over the same ground many times. He looks at it from varying points of view - his own, his arch-enemy’s, others’. He diagrams it, verbalizes it, formulates equations, constructs visual images of the whole problem, or of troublesome parts, or of what is clearly known. But he does not keep a detailed record of all this mental work, indeed could not. […] Deep understanding of a domain of knowledge requires knowing it in various ways. This multiplicity of perspectives grows slowly through hard work and sets the state for the re-cognition we experience as a new insight." (Howard E Gruber, "Darwin on Man", 1981)

"Metaphor [is] a pervasive mode of understanding by which we project patterns from one domain of experience in order to structure another domain of a different kind. So conceived metaphor is not merely a linguistic mode of expression; rather, it is one of the chief cognitive structures by which we are able to have coherent, ordered experiences that we can reason about and make sense of. Through metaphor, we make use of patterns that obtain in our physical experience to organise our more abstract understanding. " (Mark Johnson, "The Body in the Mind", 1987)

"The model and the theory it represents must be accepted, at least temporarily, or rejected, depending on the agreement or disagreement between observed data and the hypotheses and implications of the model. When neither the hypotheses nor the implications of a theory can be confronted with the real world, that theory is devoid of any scientific interest. Mere logical, even mathematical, deduction remains worthless in terms of the understanding of reality if it is not closely linked to that reality." (Maurice Allais, "An Outline of My Main Contributions to Economic Science", [Noble lecture] 1988)

"Wisdom is your perspective on life, your sense of balance, your understanding of how the various parts and principles apply and relate to each other." (Stephen R Covey, "The 7 Habits of Highly Effective People", 1989)

"An important symptom of an emerging understanding is the capacity to represent a problem in a number of different ways and to approach its solution from varied vantage points; a single, rigid representation is unlikely to suffice." (Howard Gardner, "The Unschooled Mind", 1991)

"[...] cognitive maps can be seen as a picture or visual aid in comprehending the mappers' understanding of particular, and selective, elements of the thoughts (rather than thinking) of an individual, group or organization. They may also be seen as a representation that is amenable to analysis by both the mapper and others." (Colin Eden, "One the nature of cognitive maps", Journal of Management Studies 29 (3), 1992)

"From a very early age, we form concepts. Each concept is a particular idea or understanding we have about our world. These concepts allow us to make sense of and reason about the things in our world. These things to which our concepts apply are called objects." (James Martin, 1993)

"[For] us to be able to speak and understand novel sentences, we have to store in our heads not just the words of our language but also the patterns of sentences possible in our language. These patterns, in turn, describe not just patterns of words but also patterns of patterns. Linguists refer to these patterns as the rules of language stored in memory; they refer to the complete collection of rules as the mental grammar of the language, or grammar for short." (Ray Jackendoff, "Patterns in the Mind", 1994)

"A mental model is not normally based on formal definitions but rather on concrete properties that have been drawn from life experience. Mental models are typically analogs, and they comprise specific contents, but this does not necessarily restrict their power to deal with abstract concepts, as we will see. The important thing about mental models, especially in the context of mathematics, is the relations they represent. […]  The essence of understanding a concept is to have a mental representation or mental model that faithfully reflects the structure of that concept. (Lyn D. English & Graeme S. Halford, "Mathematics Education: Models and Processes", 1995)

"We all depend on models to interpret our everyday experiences. We interpret what we see in terms of mental models constructed on past experience and education. They are constructs that we use to understand the pattern of our experiences." (David Bartholomew, "What is Statistics?", 1995)

"If we are to have meaningful, connected experiences; ones that we can comprehend and reason about; we must be able to discern patterns to our actions, perceptions, and conceptions. Underlying our vast network of interrelated literal meanings (all of those words about objects and actions) are those imaginative structures of understanding such as schema and metaphor, such as the mental imagery that allows us to extrapolate a path, or zoom in on one part of the whole, or zoom out until the trees merge into a forest." (William H Calvin, "The Cerebral Code", 1996)

"Models of the real world are not always easy to formulate because of the richness, variety, and ambiguity that exists in the real world or because of our ambiguous understanding of it." (George Dantzig & Mukund N Thapa, Linear Programming, Vol I, 1997)

"An individual understands a concept, skill, theory, or domain of knowledge to the extent that he or she can apply it appropriately in a new situation." (Howard Gardner, "The Disciplined Mind", 1999)

"Simple observation generally gets us nowhere. It is the creative imagination that increases our understanding by finding connections between apparently unrelated phenomena, and forming logical, consistent theories to explain them. And if a theory turns out to be wrong, as many do, all is not lost. The struggle to create an imaginative, correct picture of reality frequently tells us where to go next, even when science has temporarily followed the wrong path." (Richard Morris, "The Universe, the Eleventh Dimension, and Everything: What We Know and How We Know It", 1999)

"A mental model is a representation of some domain or situation that supports understanding, reasoning, and prediction. Mental models permit reasoning about situations not directly experienced. They allow people to mentally simulate the behavior of a system. Many mental models are based on generalizations and analogies from experience." (D Gentner, "Psychology of Mental Models" [in "International Encyclopedia of the Social & Behavioral Sciences"], 2001)

"An internal model corresponds to a specific concrete situation in the external world and allows us to reason about the external situation. To do so you used information about the problem presented in the problem statement. The process of understanding, then, refers to constructing an initial mental representation of what the problem is, based on the information in the problem statement about the goal, the initial state, what you are not allowed to do, and what operator to apply, as well as your own personal past experience." (S Ian Robertson, "Problem Solving", 2001)

"The understanding of a thing begins and ends with some conceptual model of it. The model is the better, the more accurate, and inclusive. But even rough models can be used to guide - or misguide - research." (Bunge A Mario, "Philosophy in Crisis: The Need for Reconstruction", 2001)

"We reach wisdom when we achieve a deep understanding of acquired knowledge, when we not only 'get it', but when new information blends with prior experience so completely that it makes us better at knowing what to do in other situations, even if they are only loosely related to the information from which our original knowledge came. Just as not all the information we absorb leads to knowledge, not all of the knowledge we acquire leads to wisdom." (Alberto Cairo, "The Functional Art", 2011)

"Understanding reduces the complexity of data by collapsing the dimensionality of information to a lower set of known variables. s revolutions, be they tiny or vast, technological or social." (Beau Lotto, "Deviate: The Science of Seeing Differently", 2017)

"Understanding transcends context, since the different contexts collapse according to their previously unknown similarity, which the principle contains. That is what understanding does. And you actually feel it in your brain when it happens. Your 'cognitive load' decreases, your level of stress and anxiety decrease, and your emotional state improves." (Beau Lotto, "Deviate: The Science of Seeing Differently", 2017)

"[…] many-model thinking produces wisdom through a diverse ensemble of logical frames. The various models accentuate different causal forces. Their insights and implications overlap and interweave. By engaging many models as frames, we develop nuanced, deep understandings." (Scott E Page, "The Model Thinker", 2018)