Systems Thinking: On Synergy (Quotes)

"The constructive process inheres in all forms of synergy, and the cooperation of antithetical forces in nature always results in making, that is, in creating something that did not exist before. But in the organic world this character of structure becomes the leading feature, and we have synthetic products consisting of tissues and organs serving definite purposes, which we call functions." (Lester F Ward, "Pure Sociology", 1903)

"[...] there is a universal principle, operating in every department of nature and at every stage of evolution, which is conservative, creative and constructive. [...] I have at last fixed upon the word synergy, as the term best adapted to express its twofold character of ‘energy’ and ‘mutuality’ or the systematic and organic ‘working together’ of the antithetical forces of nature. [...] Synergy is a synthesis of work, or synthetic work, and this is what is everywhere taking place. It may be said to begin with the primary atomic collision in which mass, motion, time, and space are involved, and to find its simplest expression in the formula for force, which implies a plurality of elements, and signifies an interaction of these elements." (Lester F Ward, "Pure Sociology", 1903)

"This compromise among the contending forces of nature was effected through organization and the formation of chemical systems, which are so many reservoirs of power, this power being represented by what we call the properties of matter. These systems store up energy and expend it in work, but the work is always a collaboration or cooperation of all the competing forces involved. It is synergy." (Lester F Ward, "Pure Sociology", 1903)

"Social structures are the products of social synergy, i.e., of the interaction of different social forces, all of which, in and of themselves, are destructive, but whose combined effect, mutually checking, constraining, and equilibrating one another, is to produce structures. The entire drift is toward economy, conservatism, and the prevention of waste. Social structures are mechanisms for the production of results, and the results cannot be secured without them. They are reservoirs of power." (James Q Dealey & Lester F Ward, "A Text-book of Sociology", 1905)

"The true nature of the universal principle of synergy pervading all nature and creating all the different kinds of structure that we observe to exist, must now be made clearer. Primarily and essentially it is a process of equilibration, i.e., the several forces are first brought into a state of partial equilibrium. It begins in collision, conflict, antagonism, and opposition, and then we have the milder phases of antithesis, competition, and interaction, passing next into a modus vivendi, or compromise, and ending in collaboration and cooperation. […] The entire drift is toward economy, conservatism, and the prevention of waste." (James Q Dealey & Lester F Ward, "A Text-book of Sociology", 1905)

"Synergy is the only word in our language that means behavior of whole systems unpredicted by the separately observed behaviors of any of the system's separate parts or any subassembly of the system's parts." (R Buckminster Fuller, "Operating Manual for Spaceship Earth", 1963)

"Synergy means behavior of whole systems unpredicted by the behavior of their parts taken separately." (R Buckminster Fuller, "Synergetics: Explorations in the Geometry of Thinking", 1975)

"[...] synergy is the consequence of the energy expended in creating order. It is locked up in the viable system created, be it an organism or a social system. It is at the level of the system. It is not discernible at the level of the system. It is not discernible at the level of the system’s components. Whenever the system is dismembered to examine its components, this binding energy dissipates." (J-C Spender, "Organizational Knowledge, Collective Practice and Penrose Rents", 1999)

"There is a multilayering of global networks in the key strategic activities that structure and destructure the planet. When these multilayered networks overlap in some node, when there is a node that belongs to different networks, two major consequences follow. First, economies of synergy between these different networks take place in that node: between financial markets and media businesses; or between academic research and technology development and innovation; between politics and media." (Manuel Castells, "The Rise of the Network Society", 1996)

"With the growing interest in complex adaptive systems, artificial life, swarms and simulated societies, the concept of “collective intelligence” is coming more and more to the fore. The basic idea is that a group of individuals (e. g. people, insects, robots, or software agents) can be smart in a way that none of its members is. Complex, apparently intelligent behavior may emerge from the synergy created by simple interactions between individuals that follow simple rules." (Francis Heylighen, "Collective Intelligence and its Implementation on the Web", 1999)

"Systems thinking means the ability to see the synergy of the whole rather than just the separate elements of a system and to learn to reinforce or change whole system patterns. Many people have been trained to solve problems by breaking a complex system, such as an organization, into discrete parts and working to make each part perform as well as possible. However, the success of each piece does not add up to the success of the whole. to the success of the whole. In fact, sometimes changing one part to make it better actually makes the whole system function less effectively." (Richard L Daft, "The Leadership Experience", 2002)

"Self-organization can be seen as a spontaneous coordination of the interactions between the components of the system, so as to maximize their synergy. This requires the propagation and processing of information, as different components perceive different aspects of the situation, while their shared goal requires this information to be integrated. The resulting process is characterized by distributed cognition: different components participate in different ways to the overall gathering and processing of information, thus collectively solving the problems posed by any perceived deviation between the present situation and the desired situation." (Carlos Gershenson & Francis Heylighen, "How can we think the complex?", 2004)

[synergy:] "Measure describing how one agent or system increases the satisfaction of other agents or systems." (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"To develop a Control, the designer should find aspect systems, subsystems, or constraints that will prevent the negative interferences between elements (friction) and promote positive interferences (synergy). In other words, the designer should search for ways of minimizing frictions that will result in maximization of the global satisfaction" (Carlos Gershenson, "Design and Control of Self-organizing Systems", 2007)

"Synergy is the combined action that occurs when people work together to create new alternatives and solutions. In addition, the greatest opportunity for synergy occurs when people have different viewpoints, because the differences present new opportunities. The essence of synergy is to value and respect differences and take advantage of them to build on strengths and compensate for weaknesses." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"Synergy occurs when organizational parts interact to produce a joint effect that is greater than the sum of the parts acting alone. As a result the organization may attain a special advantage with respect to cost, market power, technology, or employee." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

"In short, synergy is the consequence of the energy expended in creating order. It is locked up in the viable system created, be it an organism or a social system. It is at the level of the system. It is not discernible at the level of the system. It is not discernible at the level of the system's components. Whenever the system is dismembered to examine its components, this binding energy dissipates." (J-C Spender, "Organizational Knowledge, Collective Practice and Penrose Rents", 2009)

"Synergy is defined as the surplus gained by working together. A task which couldn’t be fulfilled by one individual, can be completed by the work of different individuals together. To maximize synergy, first, the initial task is divided into different sub-tasks. Different agents perform different tasks, which is called division of labor. An end product of one work is used for another work, which is called workflow. Finally, everything needs to be put together. We call this aggregation. This isn’t as linear as it looks. At every step in the process it can happen that a task is divided into sub tasks or aggregated with other tasks." (Evo Busseniers, "Self-organization versus hierarchical organization", [thesis] 2018)

Systems Thinking: On Paradigm (Quotes)

"All crises begin with the blurring of a paradigm and the consequent loosening of the rules for normal research […] Or finally, the case that will most concern us here, a crisis may end with the emergence of a new candidate for paradigm and with the ensuing battle over its acceptance." (Thomas S Kuhn, "The Structure of Scientific Revolutions", 1962)

"Discovery commences with the awareness of anomaly, i.e., with the recognition that nature has somehow violated the paradigm-induced expectations that govern normal science. It then continues with a more or less extended exploration of the area of anomaly. And it closes only when the paradigm theory has been adjusted so that the anomalous has become the expected. […] Until he has learned to see nature in a different way - the new fact is not quite a scientific fact at all." (Thomas S Kuhn, "The Structure of Scientific Revolutions", 1962)

"Probably, the single most prevalent claim advanced by the proponents of a new paradigm is that they can solve the problems that led the old one to a crisis." (Thomas S Kuhn, "The Structure of Scientific Revolutions", 1962)

"The transition from a paradigm to a new one from which a new tradition of normal science can emerge is far from a cumulative process, one achieved by an articulation or extension of the old paradigm. Rather it is a reconstruction of the field from new fundamentals, a reconstruction that changes some of the field’s most elementary theoretical generalizations as well as many of its paradigm methods and applications." (Thomas S Kuhn, "The Structure of Scientific Revolutions", 1962)

"[…] paradigms, the core of the culture of science, are transmitted and sustained just as is culture generally: scientists accept them and become committed to them as a result of training and socialization, and the commitment is maintained by a developed system of social control." (Barry Barnes, "Thomas Kuhn", 1985)

"For our purposes, a simple way to understand paradigms is to see them as maps. We all know that ‘the map is not the territory’. A map is simply an explanation of certain aspects of the territory. That’s exactly what a paradigm is. It is a theory, an explanation, or model of something else." (Stephen R Covey, "The 7 Habits of Highly Effective People", 1989)

"The word paradigm comes from the Greek. It was originally a scientific term, and is more commonly used today to mean a model, theory, perception, assumption, or frame of reference. In the more general sense, it's the way we 'see' the world - not in terms of our visual sense of sight, but in terms of perceiving, understanding, and interpreting." (Stephen Covey, "The 7 Habits of Highly Effective People", 1989)

“Each of us carries within us a worldview, a set of assumptions about how the world works - what some call a paradigm - that forms the very questions we allow ourselves to ask, and determines our view of future possibilities.” (Frances M Lappé, “Rediscovering America's Values”, 1991)

"Paradigms are powerful because they create the lens through which we see the world." (Stephen Covey, "Daily Reflections for Highly Effective People", 1994)

"The realm of the particularity of each experienced item differs from the formal realm of concepts. [...] The power of paradigmatic thought is to bring order to experience by seeing individual things as belonging to a category." (Donald E Polkinghorne, “Narrative configuration in qualitative analysis", International Journal of Qualitative Studies in Education Vol. 8 (1), 1995)

"The shift of paradigms requires an expansion not only of our perceptions and ways of thinking, but also of our values. […] scientific facts emerge out of an entire constellation of human perceptions, values, and actions-in one word, out of a paradigm-from which they cannot be separated. […] Today the paradigm shift in science, at its deepest level, implies a shift from physics to the life sciences." (Fritjof Capra, "The Web of Life", 1996)

"All scientific theories, even those in the physical sciences, are developed in a particular cultural context. Although the context may help to explain the persistence of a theory in the face of apparently falsifying evidence, the fact that a theory arises from a particular context is not sufficient to condemn it. Theories and paradigms must be accepted, modified or rejected on the basis of evidence." (Richard P Bentall,  "Madness Explained: Psychosis and Human Nature", 2003)

"Paradigms are the most general-rather like a philosophical or ideological framework. Theories are more specific, based on the paradigm and designed to describe what happens in one of the many realms of events encompassed by the paradigm. Models are even more specific providing the mechanisms by which events occur in a particular part of the theory's realm. Of all three, models are most affected by empirical data - models come and go, theories only give way when evidence is overwhelmingly against them and paradigms stay put until a radically better idea comes along." (Lee R Beach, "The Psychology of Decision Making: People in Organizations", 2005)

"A paradigm is a shared mindset that represents a fundamental way of thinking about, perceiving, and understanding the world." (Richard L Daft, "The Leadership Experience" 4th Ed., 2008)

“The crucial concept that brings all of this together is one that is perhaps as rich and suggestive as that of a paradigm: the concept of a model." (Otávio Bueno, [in" Springer Handbook of Model-Based Science", Ed. by Lorenzo Magnani & Tommaso Bertolotti, 2017])

Knowledge Representation: On Problem Solving (Quotes)

"Reflexion is careful and laborious thought, and watchful attention directed to the agreeable effect of one's plan. Invention, on the other hand, is the solving of intricate problems and the discovery of new principles by means of brilliancy and versatility." (Marcus Vitruvius Pollio, "De architectura" ["On Architecture], cca. 15BC)

"This diagrammatic method has, however, serious inconveniences as a method for solving logical problems. It does not show how the data are exhibited by cancelling certain constituents, nor does it show how to combine the remaining constituents so as to obtain the consequences sought. In short, it serves only to exhibit one single step in the argument, namely the equation of the problem; it dispenses neither with the previous steps, i.e., 'throwing of the problem into an equation' and the transformation of the premises, nor with the subsequent steps, i.e., the combinations that lead to the various consequences. Hence it is of very little use, inasmuch as the constituents can be represented by algebraic symbols quite as well as by plane regions, and are much easier to deal with in this form." (Louis Couturat, "The Algebra of Logic", 1914)

"A great discovery solves a great problem but there is a grain of discovery in the solution of any problem. Your problem may be modest; but if it challenges your curiosity and brings into play your inventive faculties, and if you solve it by your own means, you may experience the tension and enjoy the triumph of discovery." (George Polya, "How to solve it", 1944)

"[The] function of thinking is not just solving an actual problem but discovering, envisaging, going into deeper questions. Often, in great discovery the most important thing is that a certain question is found." (Max Wertheimer, "Productive Thinking", 1945)

“We can scarcely imagine a problem absolutely new, unlike and unrelated to any formerly solved problem; but if such a problem could exist, it would be insoluble. In fact, when solving a problem, we should always profit from previously solved problems, using their result or their method, or the experience acquired in solving them.” (George Polya, 1945)

"I believe, that the decisive idea which brings the solution of a problem is rather often connected with a well-turned word or sentence. The word or the sentence enlightens the situation, gives things, as you say, a physiognomy. It can precede by little the decisive idea or follow on it immediately; perhaps, it arises at the same time as the decisive idea. […]  The right word, the subtly appropriate word, helps us to recall the mathematical idea, perhaps less completely and less objectively than a diagram or a mathematical notation, but in an analogous way. […] It may contribute to fix it in the mind." (George Pólya [in a letter to Jaque Hadamard, "The Psychology of Invention in the Mathematical Field", 1949])

"Solving problems is the specific achievement of intelligence." (George Pólya, 1957)

"Systems engineering embraces every scientific and technical concept known, including economics, management, operations, maintenance, etc. It is the job of integrating an entire problem or problem to arrive at one overall answer, and the breaking down of this answer into defined units which are selected to function compatibly to achieve the specified objectives." (Instrumentation Technology, 1957)

"Solving problems can be regarded as the most characteristically human activity." (George Pólya, "Mathematical Discovery", 1962)

"The final test of a theory is its capacity to solve the problems which originated it." (George Dantzig, "Linear Programming and Extensions", 1963)

"It is a commonplace of modern technology that there is a high measure of certainty that problems have solutions before there is knowledge of how they are to be solved." (John K Galbraith, "The New Industrial State", 1967)

"Every problem-solving effort must begin with creating a representation for the problem - a problem space in which the search for the solution can take place. Of course, for most of the problems we encounter in our daily personal or professional lives, we simply retrieve from memory a representation that we have already stored and used on previous occasions. Sometimes, we have to adapt the representation a bit to the new situation, but that is usually a rather simple matter." (Herbert A Simon, "The Sciences of the Artificial", 1968)

"An expert problem solver must be endowed with two incompatible qualities, a restless imagination and a patient pertinacity.” (Howard W Eves, “In Mathematical Circles”, 1969)

"In general, complexity and precision bear an inverse relation to one another in the sense that, as the complexity of a problem increases, the possibility of analysing it in precise terms diminishes. Thus 'fuzzy thinking' may not be deplorable, after all, if it makes possible the solution of problems which are much too complex for precise analysis." (Lotfi A Zadeh, "Fuzzy languages and their relation to human intelligence", 1972)

"If we deal with our problem not knowing, or pretending not to know the general theory encompassing the concrete case before us, if we tackle the problem 'with bare hands', we have a better chance to understand the scientist's attitude in general, and especially the task of the applied mathematician." (George Pólya, "Mathematical Methods in Science", 1977)

“Solving problems can be regarded as the most characteristically human activity.” (George Polya, 1981)

"The problem solver needs to stand back and examine problem contexts in the light of different 'Ws' (Weltanschauungen). Perhaps he can then decide which 'W' seems to capture the essence of the particular problem context he is faced with. This whole process needs formalizing if it is to be carried out successfully. The problem solver needs to be aware of different paradigms in the social sciences, and he must be prepared to view the problem context through each of these paradigms." (Michael C Jackson, "Towards a System of Systems Methodologies", 1984)

“A great many problems are easier to solve rigorously if you know in advance what the answer is.” (Ian Stewart, “From Here to Infinity”, 1987)

"There are many things you can do with problems besides solving them. First you must define them, pose them. But then of course you can also refine them, depose them, or expose them or even dissolve them! A given problem may send you looking for analogies, and some of these may lead you astray, suggesting new and different problems, related or not to the original. Ends and means can get reversed. You had a goal, but the means you found didn’t lead to it, so you found a new goal they did lead to. It’s called play. Creative mathematicians play a lot; around any problem really interesting they develop a whole cluster of analogies, of playthings." (David Hawkins, "The Spirit of Play", Los Alamos Science, 1987)

"A scientific problem can be illuminated by the discovery of a profound analogy, and a mundane problem can be solved in a similar way." (Philip Johnson-Laird, "The Computer and the Mind", 1988)

“A mental model is a knowledge structure that incorporates both declarative knowledge (e.g., device models) and procedural knowledge (e.g., procedures for determining distributions of voltages within a circuit), and a control structure that determines how the procedural and declarative knowledge are used in solving problems (e.g., mentally simulating the behavior of a circuit).” (Barbara Y White & John R Frederiksen, “Causal Model Progressions as a Foundation for Intelligent Learning Environments”, Artificial Intelligence 42, 1990)

"An important symptom of an emerging understanding is the capacity to represent a problem in a number of different ways and to approach its solution from varied vantage points; a single, rigid representation is unlikely to suffice." (Howard Gardner, "The Unschooled Mind", 1991)

“[By understanding] I mean simply a sufficient grasp of concepts, principles, or skills so that one can bring them to bear on new problems and situations, deciding in which ways one’s present competencies can suffice and in which ways one may require new skills or knowledge.” (Howard Gardner, “The Unschooled Mind”, 1991)

"We consider the notion of ‘system’ as an organising concept, before going on to look in detail at various systemic metaphors that may be used as a basis for structuring thinking about organisations and problem situations." (Michael C Jackson, "Creative Problem Solving: Total Systems Intervention", 1991)

“But our ways of learning about the world are strongly influenced by the social preconceptions and biased modes of thinking that each scientist must apply to any problem. The stereotype of a fully rational and objective ‘scientific method’, with individual scientists as logical (and interchangeable) robots, is self-serving mythology.” (Stephen Jay Gould, “This View of Life: In the Mind of the Beholder”, “Natural History”, Vol. 103, No. 2, 1994)

“The term mental model refers to knowledge structures utilized in the solving of problems. Mental models are causal and thus may be functionally defined in the sense that they allow a problem solver to engage in description, explanation, and prediction. Mental models may also be defined in a structural sense as consisting of objects, states that those objects exist in, and processes that are responsible for those objects’ changing states.” (Robert Hafner & Jim Stewart, “Revising Explanatory Models to Accommodate Anomalous Genetic Phenomena: Problem Solving in the ‘Context of Discovery’”, Science Education 79 (2), 1995)

"The purpose of a conceptual model is to provide a vocabulary of terms and concepts that can be used to describe problems and/or solutions of design. It is not the purpose of a model to address specific problems, and even less to propose solutions for them. Drawing an analogy with linguistics, a conceptual model is analogous to a language, while design patterns are analogous to rhetorical figures, which are predefined templates of language usages, suited particularly to specific problems." (Peter P Chen [Ed.], "Advances in Conceptual Modeling", 1999)

"The three basic mechanisms of averaging, feedback and division of labor give us a first idea of a how a CMM [Collective Mental Map] can be developed in the most efficient way, that is, how a given number of individuals can achieve a maximum of collective problem-solving competence. A collective mental map is developed basically by superposing a number of individual mental maps. There must be sufficient diversity among these individual maps to cover an as large as possible domain, yet sufficient redundancy so that the overlap between maps is large enough to make the resulting graph fully connected, and so that each preference in the map is the superposition of a number of individual preferences that is large enough to cancel out individual fluctuations. The best way to quickly expand and improve the map and fill in gaps is to use a positive feedback that encourages individuals to use high preference paths discovered by others, yet is not so strong that it discourages the exploration of new paths." (Francis Heylighen, "Collective Intelligence and its Implementation on the Web", 1999)

"What it means for a mental model to be a structural analog is that it embodies a representation of the spatial and temporal relations among, and the causal structures connecting the events and entities depicted and whatever other information that is relevant to the problem-solving talks. […] The essential points are that a mental model can be nonlinguistic in form and the mental mechanisms are such that they can satisfy the model-building and simulative constraints necessary for the activity of mental modeling." (Nancy J Nersessian, "Model-based reasoning in conceptual change", 1999)

"A model is an imitation of reality and a mathematical model is a particular form of representation. We should never forget this and get so distracted by the model that we forget the real application which is driving the modelling. In the process of model building we are translating our real world problem into an equivalent mathematical problem which we solve and then attempt to interpret. We do this to gain insight into the original real world situation or to use the model for control, optimization or possibly safety studies." (Ian T Cameron & Katalin Hangos, "Process Modelling and Model Analysis", 2001)

"Good numeric representation is a key to effective thinking that is not limited to understanding risks. Natural languages show the traces of various attempts at finding a proper representation of numbers. [...] The key role of representation in thinking is often downplayed because of an ideal of rationality that dictates that whenever two statements are mathematically or logically the same, representing them in different forms should not matter. Evidence that it does matter is regarded as a sign of human irrationality. This view ignores the fact that finding a good representation is an indispensable part of problem solving and that playing with different representations is a tool of creative thinking." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"The key role of representation in thinking is often downplayed because of an ideal of rationality that dictates that whenever two statements are mathematically or logically the same, representing them in different forms should not matter. Evidence that it does matter is regarded as a sign of human irrationality. This view ignores the fact that finding a good representation is an indispensable part of problem solving and that playing with different representations is a tool of creative thinking." (Gerd Gigerenzer, "Calculated Risks: How to know when numbers deceive you", 2002)

"Mathematical modeling is as much ‘art’ as ‘science’: it requires the practitioner to (i) identify a so-called ‘real world’ problem (whatever the context may be); (ii) formulate it in mathematical terms (the ‘word problem’ so beloved of undergraduates); (iii) solve the problem thus formulated (if possible; perhaps approximate solutions will suffice, especially if the complete problem is intractable); and (iv) interpret the solution in the context of the original problem." (John A Adam, "Mathematics in Nature", 2003)

"What is a mathematical model? One basic answer is that it is the formulation in mathematical terms of the assumptions and their consequences believed to underlie a particular ‘real world’ problem. The aim of mathematical modeling is the practical application of mathematics to help unravel the underlying mechanisms involved in, for example, economic, physical, biological, or other systems and processes." (John A Adam, "Mathematics in Nature", 2003)

"Alternative models are neither right nor wrong, just more or less useful in allowing us to operate in the world and discover more and better options for solving problems." (Andrew Weil," The Natural Mind: A Revolutionary Approach to the Drug Problem", 2004)

“A conceptual model is a mental image of a system, its components, its interactions. It lays the foundation for more elaborate models, such as physical or numerical models. A conceptual model provides a framework in which to think about the workings of a system or about problem solving in general. An ensuing operational model can be no better than its underlying conceptualization.” (Henry N Pollack, “Uncertain Science … Uncertain World”, 2005)

"In specific cases, we think by applying mental rules, which are similar to rules in computer programs. In most of the cases, however, we reason by constructing, inspecting, and manipulating mental models. These models and the processes that manipulate them are the basis of our competence to reason. In general, it is believed that humans have the competence to perform such inferences error-free. Errors do occur, however, because reasoning performance is limited by capacities of the cognitive system, misunderstanding of the premises, ambiguity of problems, and motivational factors. Moreover, background knowledge can significantly influence our reasoning performance. This influence can either be facilitation or an impedance of the reasoning process." (Carsten Held et al, "Mental Models and the Mind", 2006)

"Every problem has a solution; it may sometimes just need another perspective." (Rebecca Mallery et al, "NLP for Rookies", 2009)

"Mental acuity of any kind comes from solving problems yourself, not from being told how to solve them.” (Paul Lockhart, “A Mathematician's Lament”, 2009)

"Mental models are formed over time through a deep enculturation process, so it follows that any attempt to align mental models must focus heavily on collective sense making. Alignment only happens through a process of socialisation; people working together, solving problems together, making sense of the world together." (Robina Chatham & Brian Sutton, "Changing the IT Leader’s Mindset", 2010)

"Mathematical modeling is the application of mathematics to describe real-world problems and investigating important questions that arise from it." (Sandip Banerjee, "Mathematical Modeling: Models, Analysis and Applications", 2014)

"Mental imagery is often useful in problem solving. Verbal descriptions of problems can become confusing, and a mental image can clear away excessive detail to bring out important aspects of the problem. Imagery is most useful with problems that hinge on some spatial relationship. However, if the problem requires an unusual solution, mental imagery alone can be misleading, since it is difficult to change one’s understanding of a mental image. In many cases, it helps to draw a concrete picture since a picture can be turned around, played with, and reinterpreted, yielding new solutions in a way that a mental image cannot." (James Schindler, "Followership", 2014)

“Framing the right problem is equally or even more important than solving it.” (Pearl Zhu, “Change, Creativity and Problem-Solving”, 2017)

Cybernetics: On Neural Networks (Quotes)

"The first attempts to consider the behavior of so-called 'random neural nets' in a systematic way have led to a series of problems concerned with relations between the 'structure' and the 'function' of such nets. The 'structure' of a random net is not a clearly defined topological manifold such as could be used to describe a circuit with explicitly given connections. In a random neural net, one does not speak of 'this' neuron synapsing on 'that' one, but rather in terms of tendencies and probabilities associated with points or regions in the net." (Anatol Rapoport, "Cycle distributions in random nets", The Bulletin of Mathematical Biophysics 10(3), 1948)

"The terms 'black box' and 'white box' are convenient and figurative expressions of not very well determined usage. I shall understand by a black box a piece of apparatus, such as four-terminal networks with two input and two output terminals, which performs a definite operation on the present and past of the input potential, but for which we do not necessarily have any information of the structure by which this operation is performed. On the other hand, a white box will be similar network in which we have built in the relation between input and output potentials in accordance with a definite structural plan for securing a previously determined input-output relation." (Norbert Wiener, "Cybernetics: Or Control and Communication in the Animal and the Machine", 1948)

"Neural computing is the study of cellular networks that have a natural property for storing experimental knowledge. Such systems bear a resemblance to the brain in the sense that knowledge is acquired through training rather than programming and is retained due to changes in node functions. The knowledge takes the form of stable states or cycles of states in the operation of the net. A central property of such nets is to recall these states or cycles in response to the presentation of cues." (Igor Aleksander & Helen Morton, "Neural computing architectures: the design of brain-like machines", 1989)

"A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects: 1. Knowledge is acquired by the network through a learning process. 2. Interneuron connection strengths known as synaptic weights are used to store the knowledge." (Igor Aleksander & Helen Morton, "An introduction to neural computing", 1990) 

"Neural Computing is the study of networks of adaptable nodes which through a process of learning from task examples, store experiential knowledge and make it available for use." (Igor Aleksander & Helen Morton, "An introduction to neural computing", 1990)

"A neural network is characterized by (1) its pattern of connections between the neurons (called its architecture), (2) its method of determining the weights on the connections (called its training, or learning, algorithm), and (3) its activation function." (Laurene Fausett, "Fundamentals of Neural Networks", 1994)

"An artificial neural network is an information-processing system that has certain performance characteristics in common with biological neural networks. Artificial neural networks have been developed as generalizations of mathematical models of human cognition or neural biology, based on the assumptions that: (1) Information processing occurs at many simple elements called neurons. (2) Signals are passed between neurons over connection links. (3) Each connection link has an associated weight, which, in a typical neural net, multiplies the signal transmitted. (4) Each neuron applies an activation function (usually nonlinear) to its net input (sum of weighted input signals) to determine its output signal." (Laurene Fausett, "Fundamentals of Neural Networks", 1994)

"At the other far extreme, we find many systems ordered as a patchwork of parallel operations, very much as in the neural network of a brain or in a colony of ants. Action in these systems proceeds in a messy cascade of interdependent events. Instead of the discrete ticks of cause and effect that run a clock, a thousand clock springs try to simultaneously run a parallel system. Since there is no chain of command, the particular action of any single spring diffuses into the whole, making it easier for the sum of the whole to overwhelm the parts of the whole. What emerges from the collective is not a series of critical individual actions but a multitude of simultaneous actions whose collective pattern is far more important. This is the swarm model." (Kevin Kelly, "Out of Control: The New Biology of Machines, Social Systems and the Economic World", 1995)

"The most familiar example of swarm intelligence is the human brain. Memory, perception and thought all arise out of the nett actions of billions of individual neurons. As we saw earlier, artificial neural networks (ANNs) try to mimic this idea. Signals from the outside world enter via an input layer of neurons. These pass the signal through a series of hidden layers, until the result emerges from an output layer. Each neuron modifies the signal in some simple way. It might, for instance, convert the inputs by plugging them into a polynomial, or some other simple function. Also, the network can learn by modifying the strength of the connections between neurons in different layers." (David G Green, "The Serendipity Machine: A voyage of discovery through the unexpected world of computers", 2004)

"It is not only a metaphor to transform the Internet to a superbrain with self-organizing features of learning and adapting. Information retrieval is already realized by neural networks adapting to the information preferences of a human user with synaptic plasticity. In sociobiology, we can 1 earn from populations of ants and termites how to organize traffic and information processing by swarm intelligence. From a technical point of view, we need intelligent programs distributed in the nets. There are already more or less intelligent virtual organisms {'agents'), learning, self-organizing and adapting to our individual preferences of information, to select our e-mails, to prepare economic transactions or to defend the attacks of hostile computer viruses, like the immune system of our body." (Klaus Mainzer, "Complexity Management in the Age of Globalization", 2006)

"A neural network is a particular kind of computer program, originally developed to try to mimic the way the human brain works. It is essentially a computer simulation of a complex circuit through which electric current flows." (Keith J Devlin & Gary Lorden, "The Numbers behind NUMB3RS: Solving crime with mathematics", 2007)

"A network of many simple processors ('units' or 'neurons') that imitates a biological neural network. The units are connected by unidirectional communication channels, which carry numeric data. Neural networks can be trained to find nonlinear relationships in data, and are used in various applications such as robotics, speech recognition, signal processing, medical diagnosis, or power systems." (Adnan Khashman et al, "Voltage Instability Detection Using Neural Networks", 2009)

"An artificial neural network, often just called a 'neural network' (NN), is an interconnected group of artificial neurons that uses a mathematical model or computational model for information processing based on a connectionist approach to computation. Knowledge is acquired by the network from its environment through a learning process, and interneuron connection strengths (synaptic weighs) are used to store the acquired knowledge." (Larbi Esmahi et al, "Adaptive Neuro-Fuzzy Systems", 2009)

"An artificial neural network is a massive parallel distributed processor made up of simple processing units. It has the ability to learn from experiential knowledge expressed through interunit connections strengths, and can make such knowledge available for use." (Yorgos Goletsis et al, "Bankruptcy Prediction through Artificial Intelligence", 2009)

"The simplest basic architecture of an artificial neural network is composed of three layers of neurons - input, output, and intermediary (historically called perceptron). When the input layer is stimulated, each node responds in a particular way by sending information to the intermediary level nodes, which in turn distribute it to the output layer nodes and thereby generate a response. The key to artificial neural networks is in the ways that the nodes are connected and how each node reacts to the stimuli coming from the nodes it is connected to. Just as with the architecture of the brain, the nodes allow information to pass only if a specific stimulus threshold is passed. This threshold is governed by a mathematical equation that can take different forms. The response depends on the sum of the stimuli coming from the input node connections and is 'all or nothing'." (Diego Rasskin-Gutman, "Chess Metaphors: Artificial Intelligence and the Human Mind", 2009)

"Intelligent systems that mimic processing of information by human brain neurons. They are capable of learning attributes, generalizing, parallel processing of information and error minimization. As a result, they are capable to model and solve complex systems." (Salim Lahmiri , "Modeling Stock Market Industrial Sectors as Dynamic Systems and Forecasting", 2015)

"Just as they did thirty years ago, machine learning programs (including those with deep neural networks) operate almost entirely in an associational mode. They are driven by a stream of observations to which they attempt to fit a function, in much the same way that a statistician tries to fit a line to a collection of points. Deep neural networks have added many more layers to the complexity of the fitted function, but raw data still drives the fitting process. They continue to improve in accuracy as more data are fitted, but they do not benefit from the 'super-evolutionary speedup'."  (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

"A neural-network algorithm is simply a statistical procedure for classifying inputs (such as numbers, words, pixels, or sound waves) so that these data can mapped into outputs. The process of training a neural-network model is advertised as machine learning, suggesting that neural networks function like the human mind, but neural networks estimate coefficients like other data-mining algorithms, by finding the values for which the model’s predictions are closest to the observed values, with no consideration of what is being modeled or whether the coefficients are sensible." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"It is a computational technique inspired by the human brain. It consists of nodes (neurons) and connections (also known as synapses) between them, to exchange and transfer data. The network learns automatically according to the flow of the data." (Shradha Verma, "Deep Learning-Based Mobile Application for Plant Disease Diagnosis", 2019)

"The label neural networks suggests that these algorithms replicate the neural networks in human brains that connect electrically excitable cells called neurons. They don’t. We have barely scratched the surface in trying to figure out how neurons receive, store, and process information, so we cannot conceivably mimic them with computers." (Gary Smith & Jay Cordes, "The 9 Pitfalls of Data Science", 2019)

"[a neural network is] a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs." (Robert Hecht-Nielsen)

Systems Thinking: On Automata (Quotes)

"Ninety-nine [students] out of a hundred are automata, careful to walk in prescribed paths, careful to follow the prescribed custom. This is not an accident but the result of substantial education, which, scientifically defined, is the subsumption of the individual." (William T Harris, "The Philosophy of Education", 1889)

"We are automata entirely controlled by the forces of the medium being tossed about like corks on the surface of the water, but mistaking the resultant of the impulses from the outside for free will. The movements and other actions we perform are always life preservative and tho seemingly quite independent from one another, we are connected by invisible links." (Nikola Tesla, "My Inventions", 1919)

"Besides electrical engineering theory of the transmission of messages, there is a larger field [cybernetics] which includes not only the study of language but the study of messages as a means of controlling machinery and society, the development of computing machines and other such automata, certain reflections upon psychology and the nervous system, and a tentative new theory of scientific method." (Norbert Wiener, "Cybernetics", 1948)

"Automata have begun to invade certain parts of mathematics too, particularly but not exclusively mathematical physics or applied mathematics. The natural systems (e.g., central nervous system) are of enormous complexity and it is clearly necessary first to subdivide what they represent into several parts that to a certain extent are independent, elementary units. The problem then consists of understanding how these elements are organized as a whole. It is the latter problem which is likely to attract those who have the background and tastes of the mathematician or a logician. With this attitude, he will be inclined to forget the origins and then, after the process of axiomatization is complete, concentrate on the mathematical aspects." (John Von Neumann, "The General and Logical Theory of Automata", 1951)

"A world of automata - of creatures that worked like machines - would hardly be worth creating." (Clive S Lewis, Mere Christianity, 1952)

"It bothers me that, according to the laws as we understand them today, it takes a computing machine an infinite number of logical operations to figure out what goes on in no matter how tiny a region of space, and no matter how tiny a region of time. How can all that be going on in that tiny space? Why should it take an infinite amount of logic to figure out what a tiny piece of space-time is going to do? So I have often made the hypothesis that ultimately physics will not require a mathematical statement, that in the end the machinery will be revealed and the laws will turn out to be simple, like the checker board with all its apparent complexities." (Richard P Feynman, "The Character of Physical Law", 1965)

"Cellular automata are discrete dynamical systems with simple construction but complex self-organizing behaviour. Evidence is presented that all one-dimensional cellular automata fall into four distinct universality classes. Characterizations of the structures generated in these classes are discussed. Three classes exhibit behaviour analogous to limit points, limit cycles and chaotic attractors. The fourth class is probably capable of universal computation, so that properties of its infinite time behaviour are undecidable." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Cellular automata are mathematical models for complex natural systems containing large numbers of simple identical components with local interactions. They consist of a lattice of sites, each with a finite set of possible values. The value of the sites evolve synchronously in discrete time steps according to identical rules. The value of a particular site is determined by the previous values of a neighbourhood of sites around it." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica 10D, 1984)

"Cellular automata may be considered as discrete dynamical systems. In almost all cases, cellular automaton evolution is irreversible. Trajectories in the configuration space for cellular automata therefore merge with time, and after many time steps, trajectories starting from almost all initial states become concentrated onto 'attractors'. These attractors typically contain only a very small fraction of possible states. Evolution to attractors from arbitrary initial states allows for 'self-organizing' behaviour, in which structure may evolve at large times from structureless initial states. The nature of the attractors determines the form and extent of such structures." (Stephen Wolfram, "Nonlinear Phenomena, Universality and complexity in cellular automata", Physica D (10), 1984)

"Perhaps the most exciting implication [of CA representation of biological phenomena] is the possibility that life had its origin in the vicinity of a phase transition and that evolution reflects the process by which life has gained local control over a successively greater number of environmental parameters affecting its ability to maintain itself at a critical balance point between order and chaos." ( Chris G Langton, "Computation at the Edge of Chaos: Phase Transitions and Emergent Computation", Physica D (42), 1990)

"Cellular automata are now being used to model varied physical phenomena normally modelled by wave equations, fluid dynamics, Ising models, etc. We hypothesize that there will be found a single cellular automaton rule that models all of microscopic physics; and models it exactly." (Edward Fredkin, "Nonlinear Phenomena", Physica D (45), 1990)

"Finite Nature is a hypothesis that ultimately every quantity of physics, including space and time, will turn out to be discrete and finite; that the amount of information in any small volume of space-time will be finite and equal to one of a small number of possibilities. [...] We take the position that Finite Nature implies that the basic substrate of physics operates in a manner similar to the workings of certain specialized computers called cellular automata." (Edward Fredkin, "A New Cosmogony", PhysComp ’92: Proceedings of the Workshop on Physics and Computation, 1993)

"Over and over again we will see the same kind of thing: that even though the underlying rules for a system are simple, and even though the system is started from simple initial conditions, the behavior that the system shows can nevertheless be highly complex." (Stephen Wolfram, "A New Kind of Science", 2002)

"Cellular automata (henceforth: CA) are discrete, abstract computational systems that have proved useful both as general models of complexity and as more specific representations of non-linear dynamics in a variety of scientific fields. Firstly, CA are (typically) spatially and temporally discrete: they are composed of a finite or denumerable set of homogenous, simple units, the atoms or cells. [...] Secondly, CA are abstract: they can be specified in purely mathematical terms and physical structures can implement them. Thirdly, CA are computational systems: they can compute functions and solve algorithmic problems." (Francesco Berto & Jacopo Tagliabue, "Cellular Automata", Stanford Encyclopedia of Philosophy, 2012) [source]