🦋Science: On Reinforcement Learning (Quotes)

"[reinforcement learning is a]  training paradigm where the neural network is presented with a sequence of input data, followed by a reinforcement signal." (Joseph P Bigus, "Data Mining with Neural Networks: Solving Business Problems from Application Development to Decision Support", 1996)

"[reinforcement learning is a] learning mode in which adaptive changes of the parameters due to reward or punishment depend on the final outcome of a whole sequence of behavior. The results of learning are evaluated by some performance index." (Teuvo Kohonen, "Self-Organizing Maps" 3rd Ed., 2001)

"[reinforcement learning is a] learning method which interprets feedback from an environment to learn optimal sets of condition/response relationships for problem solving within that environment" (Pi-Sheng Deng, "Genetic Algorithm Applications to Optimization Modeling", Encyclopedia of Artificial Intelligence, 2009)

"[reinforcement learning is a] sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to maximize some notion of long-term reward. Reinforcement learning algorithms attempt to find a policy that maps states of the world to the actions the agent ought to take in those states. Differently from supervised learning, in this case there is no target value for each input pattern, only a reward based of how good or bad was the action taken by the agent in the existent environment." (Marley Vellasco et al, "Hierarchical Neuro-Fuzzy Systems" Part II, Encyclopedia of Artificial Intelligence, 2009)

"[reinforcement learning is a] a type of machine learning in which an agent learns, through its own experience, to navigate through an environment, choosing actions in order to maximize the sum of rewards." (Lisa Torrey & Jude Shavlik, "Transfer Learning",  2010)

"[reinforcement learning is a] a machine learning technique whereby actions are associated with credits or penalties, sometimes with delay, and whereby, after a series of learning episodes, the learning agent has developed a model of which action to choose in a particular environment, based on the expectation of accumulated rewards." (Apostolos Georgas, "Scientific Workflows for Game Analytics", Encyclopedia of Business Analytics and Optimization", 2014)

"[reinforcement learning is a]  type of machine learning in which the machine learns what to do by discovering through trial and error the way to maximize a reward." (Gloria Phillips-Wren, "Intelligent Systems to Support Human Decision Making", 2014)

"[reinforcement learning] stands, in the context of computational learning, for a family of algorithms aimed at approximating the best policy to play in a certain environment (without building an explicit model of it) by increasing the probability of playing actions that improve the rewards received by the agent." (Fernando S Oliveira, "Reinforcement Learning for Business Modeling", 2014)

"The knowledge is obtained using rewards and punishments which there is an agent (learner) that acts autonomously and receives a scalar reward signal that is used to evaluate the consequences of its actions." (Nuno Pombo et al, "Machine Learning Approaches to Automated Medical Decision Support Systems", 2015)

"It is also known as learning with a critic. The agent takes a sequence of actions and receives a reward/penalty only at the very end, with no feedback during the intermediate actions. Using this limited information, the agent should learn to generate the actions to maximize the reward in later trials. For example, in chess, we do a set of moves, and at the very end, we win or lose the game; so we need to figure out what the actions that led us to this result were and correspondingly credit them." (Ethem Alpaydın, "Machine learning : the new AI", 2016)

"[reinforcement learning is a] learning algorithm for a robot or a software agent to take actions in an environment so as to maximize the sum of rewards through trial and error." (Tomohiro Yamaguchi et al, "Analyzing the Goal-Finding Process of Human Learning With the Reflection Subtask", 2018)

"Training/learning method aiming to automatically determine the ideal behavior within a specific context based on rewarding desired behaviors and/or punishing undesired one." (Ioan-Sorin Comşa et al, "Guaranteeing User Rates With Reinforcement Learning in 5G Radio Access Networks", 2019)

"Brach of the Artificial Intelligence field devoted to obtaining optimal control sequences for agents only by interacting with a concrete dynamical system." (Juan Parras & Santiago Zazo, "The Threat of Intelligent Attackers Using Deep Learning: The Backoff Attack Case", 2020)

"Machine learning approaches often used in robotics. A reward is used to teach a system a desired behavior." (Jörg Frochte et al, "Concerning the Integration of Machine Learning Content in Mechatronics Curricula", 2020)

"This area of deep learning includes methods which iterates over various steps in a process to get the desired results. Steps that yield desirable outcomes are content and steps that yield undesired outcomes are reprimanded until the algorithm is able to learn the given optimal process. In unassuming terms, learning is finished on its own or effort on feedback or content-based learning." (Amit K Tyagi & Poonam Chahal, "Artificial Intelligence and Machine Learning Algorithms", 2020)

"Reinforcement learning is also a subset of AI algorithms which creates independent, self-learning systems through trial and error. Any positive action is assigned a reward and any negative action would result in a punishment. Reinforcement learning can be used in training autonomous vehicles where the goal would be obtaining the maximum rewards." (Vijayaraghavan Varadharajan & Akanksha Rajendra Singh, "Building Intelligent Cities: Concepts, Principles, and Technologies", 2021)

❄️Systems Thinking: On Postulates (Quotes)

"As we continue the great adventure of scientific exploration our models must often be recast. New laws and postulates will be required, while those that we already have must be broadened, extended and generalized in ways that we are now hardly able to surmise." (Gilbert Newton Lewis, "The Anatomy of Science", 1926)

"Postulate 1. All chance systems of causes are not alike in the sense that they enable us to predict the future in terms of the past. Postulate 2. Constant systems of chance causes do exist in nature. Postulate 3. Assignable causes of variation may be found and eliminated."(Walter A Shewhart, "Economic Control of Quality of Manufactured Product", 1931)

"The functional validity of a working hypothesis is not a priori certain, because often it is initially based on intuition. However, logical deductions from such a hypothesis provide expectations (so called prognoses) as to the circumstances under which certain phenomena will appear in nature. Such a postulate or working hypothesis can then be substantiated by additional observations or by experiments especially arranged to test details. The value of the hypothesis is strengthened if the observed facts fit the expectation within the limits of permissible error." (R Willem van Bemmelen, "The Scientific Character of Geology", The Journal of Geology Vol 69 (4), 1961)

"Statistics provides a quantitative example of the scientific process usually described qualitatively by saying that scientists observe nature, study the measurements, postulate models to predict new measurements, and validate the model by the success of prediction." (Marshall J Walker, "The Nature of Scientific Thought", 1963)

"A model […] is a story with a specified structure: to explain this catch phrase is to explain what a model is. The structure is given by the logical and mathematical form of a set of postulates, the assumptions of the model. The structure forms an uninterpreted system, in much the way the postulates of a pure geometry are now commonly regarded as doing. The theorems that follow from the postulates tell us things about the structure that may not be apparent from an examination of the postulates alone." (Allan Gibbard & Hal R. Varian, "Economic Models", The Journal of Philosophy, Vol. 75, No. 11, 1978)

"A law explains a set of observations; a theory explains a set of laws. […] Unlike laws, theories often postulate unobservable objects as part of their explanatory mechanism." (John L Casti, "Searching for Certainty", 1990)

"In order to understand how mathematics is applied to understanding of the real world it is convenient to subdivide it into the following three modes of functioning: model, theory, metaphor. A mathematical model describes a certain range of phenomena qualitatively or quantitatively. […] A (mathematical) metaphor, when it aspires to be a cognitive tool, postulates that some complex range of phenomena might be compared to a mathematical construction." (Yuri I Manin," Mathematics as Metaphor: Selected Essays of Yuri I. Manin" , 2007)

"Mental models represent possibilities, and the theory of mental models postulates three systems of mental processes underlying inference: (0) the construction of an intensional representation of a premise’s meaning – a process guided by a parser; (1) the building of an initial mental model from the intension, and the drawing of a conclusion based on heuristics and the model; and (2) on some occasions, the search for alternative models, such as a counterexample in which the conclusion is false. System 0 is linguistic, and it may be autonomous. System 1 is rapid and prone to systematic errors, because it makes no use of a working memory for intermediate results. System 2 has access to working memory, and so it can carry out recursive processes, such as the construction of alternative models." (Sangeet Khemlania & P.N. Johnson-Laird, "The processes of inference", Argument and Computation, 2012)

❄️Systems Thinking: On Criteria (Quotes)

"For Science in its totality, the ultimate goal is the creation of a monistic system in which - on the symbolic level and in terms of the inferred components of invisibility and intangibly fine structure - the world’s enormous multiplicity is reduced to something like unity, and the endless successions of unique events of a great many different kinds get tidied and simplified into a single rational order. Whether this goal will ever be reached remains to be seen. Meanwhile we have the various sciences, each with its own system coordinating concepts, its own criterion of explanation." (Aldous Huxley, "Literature and Science", 1963)

"Adaptive system - whether on the biological, psychological, or sociocultural level - must manifest (1) some degree of 'plasticity' and 'irritability' vis-a-vis its environment such that it carries on a constant interchange with acting on and reacting to it; (2) some source or mechanism for variety, to act as a potential pool of adaptive variability to meet the problem of mapping new or more detailed variety and constraints in a changeable environment; (3) a set of selective criteria or mechanisms against which the 'variety pool' may be sifted into those variations in the organization or system that more closely map the environment and those that do not; and (4) an arrangement for preserving and/or propagating these 'successful' mappings." (Walter F Buckley," Sociology and modern systems theory", 1967)

"Most of our beliefs about complex organizations follow from one or the other of two distinct strategies. The closed-system strategy seeks certainty by incorporating only those variables positively associated with goal achievement and subjecting them to a monolithic control network. The open-system strategy shifts attention from goal achievement to survival and incorporates uncertainty by recognizing organizational interdependence with environment. A newer tradition enables us to conceive of the organization as an open system, indeterminate and faced with uncertainty, but subject to criteria of rationality and hence needing certainty." (James D Thompson, "Organizations in Action", 1967)

"Heavy dependence on direct observation is essential to biology not only because of the complexity of biological phenomena, but because of the intervention of natural selection with its criterion of adequacy rather than perfection. In a system shaped by natural selection it is inevitable that logic will lose its way." (George A Bartholomew, "Scientific innovation and creativity: a zoologist’s point of view", American Zoologist Vol. 22, 1982)

"[…] semantic nets fail to be distinctive in the way they (1) represent propositions, (2) cluster information for access, (3) handle property inheritance, and (4) handle general inference; in other words, they lack distinctive representational properties (i.e., 1) and distinctive computational properties (i.e., 2-4). Certain propagation mechanisms, notably 'spreading activation', 'intersection search', or 'inference propagation' have sometimes been regarded as earmarks of semantic nets, but since most extant semantic nets lack such mechanisms, they cannot be considered criterial in current usage." (Lenhart K Schubert, "Semantic Nets are in the Eye of the Beholder", 1990)

"A model for simulating dynamic system behavior requires formal policy descriptions to specify how individual decisions are to be made. Flows of information are continuously converted into decisions and actions. No plea about the inadequacy of our understanding of the decision-making processes can excuse us from estimating decision-making criteria. To omit a decision point is to deny its presence - a mistake of far greater magnitude than any errors in our best estimate of the process." (Jay W Forrester, "Policies, decisions and information sources for modeling", 1994)

“The amount of understanding produced by a theory is determined by how well it meets the criteria of adequacy - testability, fruitfulness, scope, simplicity, conservatism - because these criteria indicate the extent to which a theory systematizes and unifies our knowledge.” (Theodore Schick Jr.,  “How to Think about Weird Things: Critical Thinking for a New Age”, 1995)

"Sensitive dependence on initial conditions is one of the criteria necessary for showing a solution to a difference equation exhibits chaotic behavior." (Linda J S Allen, "An Introduction to Mathematical Biology", 2007)

🦋Science: On Criteria (Quotes)

"The modern age has a false sense of superiority because of the great mass of data at its disposal. But the valid criterion of distinction is rather the extent to which man knows how to form and master the material at his command." (Johann Wolfgang von Goethe, "On Theory of Color", 1810)

“[Precision] is the very soul of science; and its attainment afford the only criterion, or at least the best, of the truth of theories, and the correctness of experiments.” (John F W Herschel, “A Preliminary Discourse on the Study of Natural Philosophy”, 1830)

"When the hypothesis, of itself and without adjustment for the purpose, gives us the rule and reason of a class of facts not contemplated in its construction, we have a criterion of its reality, which has never yet been produced in favour of falsehood." (William Whewell, "The Philosophy of the Inductive Sciences", 1840) 

"In scientific thought we adopt the simplest theory which will explain all the facts under consideration and enable us to predict facts of the same kind. The  catch in this criterion lies in the world 'simplest'." (John B S Haldane, "Possible Worlds and Other Essays", 1928)

"When the hypothesis, of itself and without adjustment for the purpose, gives us the rule and reason of a class of facts not contemplated in its construction, we have a criterion of its reality, which has never yet been produced in favour of falsehood." (William Whewell, "The Philosophy of the Inductive Sciences", 1840)

"A primary goal of any learning model is to predict correctly the learning curve - proportions of correct responses versus trials. Almost any sensible model with two or three free parameters, however, can closely fit the curve, and so other criteria must be invoked when one is comparing several models." (Robert R Bush & Frederick Mosteller, "A Comparison of Eight Models?", Studies in Mathematical Learning Theory, 1959)

"A satisfactory prediction of the sequential properties of learning data from a single experiment is by no means a final test of a model. Numerous other criteria - and some more demanding - can be specified. For example, a model with specific numerical parameter values should be invariant to changes in independent variables that explicitly enter in the model." (Robert R Bush & Frederick Mosteller,"A Comparison of Eight Models?", Studies in Mathematical Learning Theory, 1959)

"[...] sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work - that is, correctly to describe phenomena from a reasonably wide area. Furthermore, it must satisfy certain aesthetic criteria - that is, in relation to how much it describes, it must be rather simple." (John von Neumann, "Method in the physical sciences", 1961)

"For Science in its totality, the ultimate goal is the creation of a monistic system in which - on the symbolic level and in terms of the inferred components of invisibility and intangibly fine structure - the world’s enormous multiplicity is reduced to something like unity, and the endless successions of unique events of a great many different kinds get tidied and simplified into a single rational order. Whether this goal will ever be reached remains to be seen. Meanwhile we have the various sciences, each with its own system coordinating concepts, its own criterion of explanation." (Aldous Huxley, "Literature and Science", 1963)

"The mediation of theory and praxis can only be clarified if to begin with we distinguish three functions, which are measured in terms of different criteria: the formation and extension of critical theorems, which can stand up to scientific discourse; the organization of processes of enlightenment, in which such theorems are applied and can be tested in a unique manner by the initiation of processes of reflection carried on within certain groups toward which these processes have been directed; and the selection of appropriate strategies, the solution of tactical questions, and the conduct of the political struggle. On the first level, the aim is true statements, on the second, authentic insights, and on the third, prudent decisions." (Jürgen Habermas, "Introduction to Theory and Practice", 1963)

"In practice, let us note, the determination of sets by means of characterizing criteria runs into difficulty because of the ambiguity of our language. The task of separating the objects belonging to a set from those that do not is often made difficult by the large number of objects of intermediate type." (Naum Ya. Vilenkin, "Stories about Sets", 1968)

"Any theory starts off with an observer or experimenter. He has in mind a collection of abstract models with predictive capabilities. Using various criteria of relevance, he selects one of them. In order to actually make predictions, this model must be interpreted and identified with a real assembly to form a theory. The interpretation may be prescriptive or predictive, as when the model is used like a blueprint for designing a machine and predicting its states. On the other hand, it may be descriptive and predictive as it is when the model is used to explain and predict the behaviour of a given organism." (Gordon Pask, "The meaning of cybernetics in the behavioural sciences", 1969)

"The principal aim of physical theories is understanding. A theory's ability to find a number is merely a useful criterion for a correct understanding." (Yuri I Manin, "Mathematics and Physics", 1981)

"It is often the scientist’s experience that he senses the nearness of truth when such connections are envisioned. A connection is a step toward simplification, unification. Simplicity is indeed often the sign of truth and a criterion of beauty.” (Mahlon B Hoagland, “Toward the Habit of Truth”, 1990)

"The ability of a scientific theory to be refuted is the key criterion that distinguishes science from metaphysics. If a theory cannot be refuted, if there is no observation that will disprove it, then nothing can prove it - it cannot predict anything, it is a worthless myth." (Eric Lerner, "The Big Bang Never Happened", 1991)

"[...] there is no criterion for appreciation which does not vary from one epoch to another and from one mathematician to another. [...] These divergences in taste recall the quarrels aroused by works of art, and it is a fact that mathematicians often discuss among themselves whether a theorem is more or less ‚beautiful‘. This never fails to surprise practitioners of other sciences: for them the sole criterion is the 'truth' of a theory or formula." (Jean Dieudonné, "Mathematics - The Music of Reason" , 1992)

"Indeed, knowledge that one will be judged on some criterion of ‘creativeness’ or ‘originality’ tends to narrow the scope of what one can produce (leading to products that are then judged as relatively conventional); in contrast, the absence of an evaluations seems to liberate creativity." (Howard Gardner,  "Creating Minds", 1993)

"No one has yet succeeded in deriving the second law from any other law of nature. It stands on its own feet. It is the only law in our everyday world that gives a direction to time, which tells us that the universe is moving toward equilibrium and which gives us a criteria for that state, namely, the point of maximum entropy, of maximum probability. The second law involves no new forces. On the contrary, it says nothing about forces whatsoever." (Brian L Silver, "The Ascent of Science", 1998)

"No plea about inadequacy of our understanding of the decision-making processes can excuse us from estimating decision making criteria. To omit a decision point is to deny its presence - a mistake of far greater magnitude than any errors in our best estimate of the process." (Jay W Forrester, "Perspectives on the modelling process", 2000)

"A full definition of an object must include the whole of human experience, both as a criterion of truth and a practical indicator of its connection with human wants." (Vladimir Lenin)