🦋Science: On Trial and Error (Quotes)

"The one lesson that comes out of all our theorizing and experimenting is that there is only one really scientific progressive method; and that is the method of trial and error." (George B Shaw, "The Doctor's Dilemma: Preface on Doctors", 1909)

"There are many men now living who were in the habit of using the age-old expression: 'It is as impossible as flying.' The discoveries in physical science, the triumphs in invention, attest the value of the process of trial and error. In large measure, these advances have been due to experimentation." (Louis Brandeis, "Dissent, New State Ice Co. v. Liebmann, 285 U.S. 262", 1932)

"The discoveries in physical science, the triumphs in invention, attest the value of the process of trial and error. In large measure, these advances have been due to experimentation." (Louis Brandeis, "Judicial opinions", 1932)

"But I believe that there is no philosophical high-road in science, with epistemological signposts. No, we are in a jungle and find our way by trial and error, building our road behind us as we proceed. We do not find signposts at cross-roads, but our own scouts erect them, to help the rest." (Max Born, "Experiment and Theory in Physics", 1943)

"The method of learning by trial and error - of learning from our mistakes - seems to be fundamentally the same whether it is practised by lower or by higher animals, by chimpanzees or by men of science." (Karl Popper, "Conjectures and Refutations: The Growth of Scientific Knowledge", 1963)

"The difference between the amoeba and Einstein is that, although both make use of the method of trial and error elimination, the amoeba dislikes erring while Einstein is intrigued by it [...]" (Karl R Popper, "Objective Knowledge: An Evolutionary Approach", 1972) 

"Science progresses by trial and error, by conjectures and refutations. Only the fittest theories survive." (Alan Chalmers, "What Is This Thing Called Science?", 1976)

"The great revolutions in science are almost always the result of unexpected intuitive leaps. After all, what is science if not the posing of difficult puzzles by the universe? Mother Nature does something interesting, and challenges the scientist to figure out how she does it. In many cases the solution is not found by exhaustive trial and error […] or even by a deduction based on the relevant knowledge." (Martin Gardner, "Aha! Insight", 1978)

"I believe people can solve complex problems eventually. By repeated trial and error they will get there; but they need a long time. At this point I agree with Herbert Simon. People do not learn immediately, as those rational expectations models seem to imply. I don't believe that. The statement that assumptions do not matter is nonsense. It is funny. Yes, I assume people are consistent in their behavior. I assume that not because I believe everybody actually is, but because I believe, on the average, you do not get too far from it." (Franco Modigliani, "Conversations with Economists", 1983)

"Science usually amounts to a lot more than blind trial and error. Good statistics consists of much more than just significance tests; there are more sophisticated tools available for the analysis of results, such as confidence statements, multiple comparisons, and Bayesian analysis, to drop a few names. However, not all scientists are good statisticians, or want to be, and not all people who are called scientists by the media deserve to be so described." (Robert Hooke, "How to Tell the Liars from the Statisticians", 1983)

"Whatever humans have learned had to be learned as a consequence only of trial and error experience. Humans have learned only through mistakes." (R Buckminster Fuller, "Intuition", 1983)

"Growth is a process of experimentation, a series of trials, errors, and occasional victories. The failed experiments are as much as part of the process as the experiments that work." (Chérie Carter-Scott, "If Life Is a Game, These Are the Rules", 1998)

"The natural as well as the social sciences always start from problems, from the fact that something inspires amazement in us, as the Greek philosophers used to say. To solve these problems, the sciences use fundamentally the same method that common sense employs, the method of trial and error. To be more precise, it is the method of trying out solutions to our problem and then discarding the false ones as erroneous. This method assumes that we work with a large number of experimental solutions. One solution after another is put to the test and eliminated." (Karl R Popper, "All Life is Problem Solving", 1999)

"We can simplify the relationships between fragility, errors, and antifragility as follows. When you are fragile, you depend on things following the exact planned course, with as little deviation as possible - for deviations are more harmful than helpful. This is why the fragile needs to be very predictive in its approach, and, conversely, predictive systems cause fragility. When you want deviations, and you don’t care about the possible dispersion of outcomes that the future can bring, since most will be helpful, you are antifragile. Further, the random element in trial and error is not quite random, if it is carried out rationally, using error as a source of information. If every trial provides you with information about what does not work, you start zooming in on a solution - so every attempt becomes more valuable, more like an expense than an error. And of course you make discoveries along the way." (Nassim N Taleb, "Antifragile: Things that gain from disorder", 2012)

🦋Science: On Sampling (Quotes)

"By a small sample we may judge of the whole piece." (Miguel de Cervantes, "Don Quixote de la Mancha", 1605–1615)

"If the number of experiments be very large, we may have precise information as to the value of the mean, but if our sample be small, we have two sources of uncertainty: (I) owing to the 'error of random sampling' the mean of our series of experiments deviates more or less widely from the mean of the population, and (2) the sample is not sufficiently large to determine what is the law of distribution of individuals." (William S Gosset, "The Probable Error of a Mean", Biometrika, 1908)

"The postulate of randomness thus resolves itself into the question, 'of what population is this a random sample?' which must frequently be asked by every practical statistician." (Ronald Fisher, "On the Mathematical Foundation of Theoretical Statistics", Philosophical Transactions of the Royal Society of London Vol. A222, 1922)

"If the chance of error alone were the sole basis for evaluating methods of inference, we would never reach a decision, but would merely keep increasing the sample size indefinitely." (C West Churchman, "Theory of Experimental Inference", 1948)

"If significance tests are required for still larger samples, graphical accuracy is insufficient, and arithmetical methods are advised. A word to the wise is in order here, however. Almost never does it make sense to use exact binomial significance tests on such data - for the inevitable small deviations from the mathematical model of independence and constant split have piled up to such an extent that the binomial variability is deeply buried and unnoticeable. Graphical treatment of such large samples may still be worthwhile because it brings the results more vividly to the eye." (Frederick Mosteller & John W Tukey, "The Uses and Usefulness of Binomial Probability Paper?", Journal of the American Statistical Association 44, 1949)

"A good sample-design is lost if it is not carried out according to plans." (W Edwards Deming, "Some Theory of Sampling", 1950)

"Sampling is the science and art of controlling and measuring the reliability of useful statistical information through the theory of probability." (William E Deming, "Some Theory of Sampling", 1950)

"Almost any sort of inquiry that is general and not particular involves both sampling and measurement […]. Further, both the measurement and the sampling will be imperfect in almost every case. We can define away either imperfection in certain cases. But the resulting appearance of perfection is usually only an illusion." (Frederick Mosteller et al, "Principles of Sampling", Journal of the American Statistical Association Vol. 49 (265), 1954)

"By sampling we can learn only about collective properties of populations, not about properties of individuals. We can study the average height, the percentage who wear hats, or the variability in weight of college juniors [...]. The population we study may be small or large, but there must be a population - and what we are studying must be a population characteristic. By sampling, we cannot study individuals as particular entities with unique idiosyncrasies; we can study regularities (including typical variabilities as well as typical levels) in a population as exemplified by the individuals in the sample." (Frederick Mosteller et al, "Principles of Sampling", Journal of the American Statistical Association Vol. 49 (265), 1954)

"In many cases general probability samples can be thought of in terms of (1) a subdivision of the population into strata, (2) a self-weighting probability sample in each stratum, and (3) combination of the stratum sample means weighted by the size of the stratum." (Frederick Mosteller et al, "Principles of Sampling", Journal of the American Statistical Association Vol. 49 (265), 1954)

"Precision is expressed by an international standard, viz., the standard error. It measures the average of the difference between a complete coverage and a long series of estimates formed from samples drawn from this complete coverage by a particular procedure or drawing, and processed by a particular estimating formula." (W Edwards Deming, "On the Presentation of the Results of Sample Surveys as Legal Evidence", Journal of the American Statistical Association Vol 49 (268), 1954)

"The purely random sample is the only kind that can be examined with entire confidence by means of statistical theory, but there is one thing wrong with it. It is so difficult and expensive to obtain for many uses that sheer cost eliminates it." (Darell Huff, "How to Lie with Statistics", 1954)

"To be worth much, a report based on sampling must use a representative sample, which is one from which every source of bias has been removed." (Darell Huff, "How to Lie with Statistics", 1954)

"Null hypotheses of no difference are usually known to be false before the data are collected [...] when they are, their rejection or acceptance simply reflects the size of the sample and the power of the test, and is not a contribution to science." (I Richard Savage, "Nonparametric statistics", Journal of the American Statistical Association 52, 1957)

"[A] sequence is random if it has every property that is shared by all infinite sequences of independent samples of random variables from the uniform distribution." (Joel N Franklin, 1962)

"Weighing a sample appropriately is no more fudging the data than is correcting a gas volume for barometric pressure." (Frederick Mosteller, "Principles of Sampling", Journal of the American Statistical Association Vol. 49 (265), 1964)

"Entropy theory is indeed a first attempt to deal with global form; but it has not been dealing with structure. All it says is that a large sum of elements may have properties not found in a smaller sample of them." (Rudolf Arnheim, "Entropy and Art: An Essay on Disorder and Order", 1974) 

"The fact must be expressed as data, but there is a problem in that the correct data is difficult to catch. So that I always say 'When you see the data, doubt it!' 'When you see the measurement instrument, doubt it!' [...] For example, if the methods such as sampling, measurement, testing and chemical analysis methods were incorrect, data. […] to measure true characteristics and in an unavoidable case, using statistical sensory test and express them as data." (Kaoru Ishikawa, Annual Quality Congress Transactions, 1981)

"The law of truly large numbers states: With a large enough sample, any outrageous thing is likely to happen." (Frederick Mosteller, "Methods for Studying Coincidences", Journal of the American Statistical Association Vol. 84, 1989)

"A little thought reveals a fact widely understood among statisticians: The null hypothesis, taken literally (and that’s the only way you can take it in formal hypothesis testing), is always false in the real world. [...] If it is false, even to a tiny degree, it must be the case that a large enough sample will produce a significant result and lead to its rejection. So if the null hypothesis is always false, what’s the big deal about rejecting it?" (Jacob Cohen, "Things I Have Learned (So Far)", American Psychologist, 1990)

"When looking at the end result of any statistical analysis, one must be very cautious not to over interpret the data. Care must be taken to know the size of the sample, and to be certain the method forg athering information is consistent with other samples gathered. […] No one should ever base conclusions without knowing the size of the sample and how random a sample it was. But all too often such data is not mentioned when the statistics are given - perhaps it is overlooked or even intentionally omitted." (Theoni Pappas, "More Joy of Mathematics: Exploring mathematical insights & concepts", 1991)

"When the sample size is small or the study is of one organization, descriptive use of the thematic coding is desirable." (Richard Boyatzis, "Transforming qualitative information", 1998)

"Statisticians can calculate the probability that such random samples represent the population; this is usually expressed in terms of sampling error [...]. The real problem is that few samples are random. Even when researchers know the nature of the population, it can be time-consuming and expensive to draw a random sample; all too often, it is impossible to draw a true random sample because the population cannot be defined. This is particularly true for studies of social problems.[...] The best samples are those that come as close as possible to being random." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"There are two problems with sampling - one obvious, and  the other more subtle. The obvious problem is sample size. Samples tend to be much smaller than their populations. [...] Obviously, it is possible to question results based on small samples. The smaller the sample, the less confidence we have that the sample accurately reflects the population. However, large samples aren't necessarily good samples. This leads to the second issue: the representativeness of a sample is actually far more important than sample size. A good sample accurately reflects (or 'represents') the population." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Traditional statistics is strong in devising ways of describing data and inferring distributional parameters from sample. Causal inference requires two additional ingredients: a science-friendly language for articulating causal knowledge, and a mathematical machinery for processing that knowledge, combining it with data and drawing new causal conclusions about a phenomenon." (Judea Pearl, "Causal inference in statistics: An overview", Statistics Surveys 3, 2009)

"The closer that sample-selection procedures approach the gold standard of random selection - for which the definition is that every individual in the population has an equal chance of appearing in the sample - the more we should trust them. If we don’t know whether a sample is random, any statistical measure we conduct may be biased in some unknown way." (Richard E Nisbett, "Mindware: Tools for Smart Thinking", 2015)

"A popular misconception holds that the era of Big Data means the end of a need for sampling. In fact, the proliferation of data of varying quality and relevance reinforces the need for sampling as a tool to work efficiently with a variety of data, and minimize bias. Even in a Big Data project, predictive models are typically developed and piloted with samples." (Peter C Bruce & Andrew G Bruce, "Statistics for Data Scientists: 50 Essential Concepts", 2016)

"Repeated observations of the same phenomenon do not always produce the same results, due to random noise or error. Sampling errors result when our observations capture unrepresentative circumstances, like measuring rush hour traffic on weekends as well as during the work week. Measurement errors reflect the limits of precision inherent in any sensing device. The notion of signal to noise ratio captures the degree to which a series of observations reflects a quantity of interest as opposed to data variance. As data scientists, we care about changes in the signal instead of the noise, and such variance often makes this problem surprisingly difficult." (Steven S Skiena, "The Data Science Design Manual", 2017)

"Samples give us estimates of something, and they will almost always deviate from the true number by some amount, large or small, and that is the margin of error. […] The margin of error does not address underlying flaws in the research, only the degree of error in the sampling procedure. But ignoring those deeper possible flaws for the moment, there is another measurement or statistic that accompanies any rigorously defined sample: the confidence interval." (Daniel J Levitin, "Weaponized Lies", 2017)

"To be any good, a sample has to be representative. A sample is representative if every person or thing in the group you’re studying has an equally likely chance of being chosen. If not, your sample is biased. […] The job of the statistician is to formulate an inventory of all those things that matter in order to obtain a representative sample. Researchers have to avoid the tendency to capture variables that are easy to identify or collect data on - sometimes the things that matter are not obvious or are difficult to measure." (Daniel J Levitin, "Weaponized Lies", 2017)

🦋Science: On Errors in Statistics (Quotes)

"[It] may be laid down as a general rule that, if the result of a long series of precise observations approximates a simple relation so closely that the remaining difference is undetectable by observation and may be attributed to the errors to which they are liable, then this relation is probably that of nature." (Pierre-Simon Laplace, "Mémoire sur les Inégalites Séculaires des Planètes et des Satellites", 1787)

"It is surprising to learn the number of causes of error which enter into the simplest experiment, when we strive to attain rigid accuracy." (William S Jevons, "The Principles of Science: A Treatise on Logic and Scientific Method", 1874)

"Some of the common ways of producing a false statistical argument are to quote figures without their context, omitting the cautions as to their incompleteness, or to apply them to a group of phenomena quite different to that to which they in reality relate; to take these estimates referring to only part of a group as complete; to enumerate the events favorable to an argument, omitting the other side; and to argue hastily from effect to cause, this last error being the one most often fathered on to statistics. For all these elementary mistakes in logic, statistics is held responsible." (Sir Arthur L Bowley, "Elements of Statistics", 1901)

"If the number of experiments be very large, we may have precise information as to the value of the mean, but if our sample be small, we have two sources of uncertainty: (I) owing to the 'error of random sampling' the mean of our series of experiments deviates more or less widely from the mean of the population, and (2) the sample is not sufficiently large to determine what is the law of distribution of individuals." (William S Gosset, "The Probable Error of a Mean", Biometrika, 1908)

"We know not to what are due the accidental errors, and precisely because we do not know, we are aware they obey the law of Gauss. Such is the paradox." (Henri Poincaré, "The Foundations of Science", 1913)

"No observations are absolutely trustworthy. In no field of observation can we entirely rule out the possibility that an observation is vitiated by a large measurement or execution error. If a reading is found to lie a very long way from its fellows in a series of replicate observations, there must be a suspicion that the deviation is caused by a blunder or gross error of some kind. [...] One sufficiently erroneous reading can wreck the whole of a statistical analysis, however many observations there are." (Francis J Anscombe, "Rejection of Outliers", Technometrics Vol. 2 (2), 1960)

"It might be reasonable to expect that the more we know about any set of statistics, the greater the confidence we would have in using them, since we would know in which directions they were defective; and that the less we know about a set of figures, the more timid and hesitant we would be in using them. But, in fact, it is the exact opposite which is normally the case; in this field, as in many others, knowledge leads to caution and hesitation, it is ignorance that gives confidence and boldness. For knowledge about any set of statistics reveals the possibility of error at every stage of the statistical process; the difficulty of getting complete coverage in the returns, the difficulty of framing answers precisely and unequivocally, doubts about the reliability of the answers, arbitrary decisions about classification, the roughness of some of the estimates that are made before publishing the final results. Knowledge of all this, and much else, in detail, about any set of figures makes one hesitant and cautious, perhaps even timid, in using them." (Ely Devons, "Essays in Economics", 1961)

"The art of using the language of figures correctly is not to be over-impressed by the apparent ai

"Measurement, we have seen, always has an element of error in it. The most exact description or prediction that a scientist can make is still only approximate." (Abraham Kaplan, "The Conduct of Inquiry: Methodology for Behavioral Science", 1964)

"A mature science, with respect to the matter of errors in variables, is not one that measures its variables without error, for this is impossible. It is, rather, a science which properly manages its errors, controlling their magnitudes and correctly calculating their implications for substantive conclusions." (Otis D Duncan, "Introduction to Structural Equation Models", 1975)

"Pencil and paper for construction of distributions, scatter diagrams, and run-charts to compare small groups and to detect trends are more efficient methods of estimation than statistical inference that depends on variances and standard errors, as the simple techniques preserve the information in the original data." (William E Deming, "On Probability as Basis for Action" American Statistician Vol. 29 (4), 1975)

"Linear regression assumes that in the population a normal distribution of error values around the predicted Y is associated with each X value, and that the dispersion of the error values for each X value is the same. The assumptions imply normal and similarly dispersed error distributions." (Fred C Pampel, "Linear Regression: A primer", 2000)

"Compound errors can begin with any of the standard sorts of bad statistics - a guess, a poor sample, an inadvertent transformation, perhaps confusion over the meaning of a complex statistic. People inevitably want to put statistics to use, to explore a number's implications. [...] The strengths and weaknesses of those original numbers should affect our confidence in the second-generation statistics." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Trimming potentially theoretically meaningful variables is not advisable unless one is quite certain that the coefficient for the variable is near zero, that the variable is inconsequential, and that trimming will not introduce misspecification error." (James Jaccard, "Interaction Effects in Logistic Regression", 2001)

"The central limit theorem says that, under conditions almost always satisfied in the real world of experimentation, the distribution of such a linear function of errors will tend to normality as the number of its components becomes large. The tendency to normality occurs almost regardless of the individual distributions of the component errors. An important proviso is that several sources of error must make important contributions to the overall error and that no particular source of error dominate the rest." (George E P Box et al, "Statistics for Experimenters: Design, discovery, and innovation" 2^nd Ed., 2005)

"Two things explain the importance of the normal distribution: (1) The central limit effect that produces a tendency for real error distributions to be 'normal like'. (2) The robustness to nonnormality of some common statistical procedures, where 'robustness' means insensitivity to deviations from theoretical normality." (George E P Box et al, "Statistics for Experimenters: Design, discovery, and innovation" 2^nd Ed., 2005)

"Always expect to find at least one error when you proofread your own statistics. If you don’t, you are probably making the same mistake twice." (Cheryl Russell)

[Murphy’s Laws of Analysis:] "(1) In any collection of data, the figures that are obviously correct contain errors. (2) It is customary for a decimal to be misplaced. (3) An error that can creep into a calculation, will. Also, it will always be in the direction that will cause the most damage to the calculation." (G C Deakly)

🦋Science: On Failure (Quotes)

"Every detection of what is false directs us towards what is true: every trial exhausts some tempting form of error. Not only so; but scarcely any attempt is entirely a failure; scarcely any theory, the result of steady thought, is altogether false; no tempting form of error is without some latent charm derived from truth." (William Whewell, "Lectures on the History of Moral Philosophy in England", 1852)

"Scarcely any attempt is entirely a failure; scarcely any theory, the result of steady thought, is altogether false; no tempting form of Error is without some latent charm derived from Truth." (William Whewell, "Lectures on the History of Moral Philosophy in England", 1852)

"We learn wisdom from failure much more than from success. We often discover what will do, by finding out what will not do; and probably he who never made a mistake never made a discovery." (Samuel Smiles, "Facilities and Difficulties", 1859)

"One word characterizes the most strenuous of the efforts for the advancement of science that I have made perseveringly during fifty-five years, and that word is failure." (William Thomson [Kelvin], Nature Vol. 54, 1896)

"This history constitutes a mirror of past and present conditions in mathematics which can be made to bear on the notational problems now confronting mathematics. The successes and failures of the past will contribute to a more speedy solution of notational problems of the present time." (Florian Cajori, "A History of Mathematical Notations", 1928)

"[…] the statistical prediction of the future from the past cannot be generally valid, because whatever is future to any given past, is in tum past for some future. That is, whoever continually revises his judgment of the probability of a statistical generalization by its successively observed verifications and failures, cannot fail to make more successful predictions than if he should disregard the past in his anticipation of the future. This might be called the ‘Principle of statistical accumulation’." (Clarence I Lewis, "Mind and the World-Order: Outline of a Theory of Knowledge", 1929)

"The failure of the social sciences to think through and to integrate their several responsibilities for the common problem of relating the analysis of parts to the analysis of the whole constitutes one of the major lags crippling their utility as human tools of knowledge." (Robert S Lynd, "Knowledge of What?", 1939)

"Science condemns itself to failure when, yielding to the infatuation of the serious, it aspires to attain being, to contain it, and to possess it; but it finds its truth if it considers itself as a free engagement of thought in the given, aiming, at each discovery, not at fusion with the thing, but at the possibility of new discoveries; what the mind then projects is the concrete accomplishment of its freedom." (Simone de Beauvoir, "The Ethics of Ambiguity", 1947)

"Common sense […] may be thought of as a series of concepts and conceptual schemes which have proved highly satisfactory for the practical uses of mankind. Some of those concepts and conceptual schemes were carried over into science with only a little pruning and whittling and for a long time proved useful. As the recent revolutions in physics indicate, however, many errors can be made by failure to examine carefully just how common sense ideas should be defined in terms of what the experimenter plans to do." (James B Conant, "Science and Common Sense", 1951)

"Scientific method is the way to truth, but it affords, even in principle, no unique definition of truth. Any so-called pragmatic definition of truth is doomed to failure equally." (Willard v O Quine, "Word and Object", 1960)

"Catastrophes are often stimulated by the failure to feel the emergence of a domain, and so what cannot be felt in the imagination is experienced as embodied sensation in the catastrophe. (William I Thompson, "Gaia, a Way of Knowing: Political Implications of the New Biology", 1987)

"What about confusing clutter? Information overload? Doesn't data have to be ‘boiled down’ and  ‘simplified’? These common questions miss the point, for the quantity of detail is an issue completely separate from the difficulty of reading. Clutter and confusion are failures of design, not attributes of information." (Edward R Tufte, "Envisioning Information", 1990)

"When a system is predictable, it is already performing as consistently as possible. Looking for assignable causes is a waste of time and effort. Instead, you can meaningfully work on making improvements and modifications to the process. When a system is unpredictable, it will be futile to try and improve or modify the process. Instead you must seek to identify the assignable causes which affect the system. The failure to distinguish between these two different courses of action is a major source of confusion and wasted effort in business today." (Donald J Wheeler, "Understanding Variation: The Key to Managing Chaos" 2nd Ed., 2000)

"Good science is more than the mechanics of research and experimentation. Good science requires that scientists look inward-to contemplate the origin of their thoughts. The failures of science do not begin with flawed evidence or fumbled statistics; they begin with personal self-deception and an unjustified sense of knowing." (Robert A Burton, "On Being Certain: Believing You Are Right Even When You're Not", 2008)

"[…] in cybernetics, control is seen not as a function of one agent over something else, but as residing within circular causal networks, maintaining stabilities in a system. Circularities have no beginning, no end and no asymmetries. The control metaphor of communication, by contrast, punctuates this circularity unevenly. It privileges the conceptions and actions of a designated controller by distinguishing between messages sent in order to cause desired effects and feedback that informs the controller of successes or failures." (Klaus Krippendorff, "On Communicating: Otherness, Meaning, and Information", 2009)

"To get a true understanding of the work of mathematicians, and the need for proof, it is important for you to experiment with your own intuitions, to see where they lead, and then to experience the same failures and sense of accomplishment that mathematicians experienced when they obtained the correct results. Through this, it should become clear that, when doing any level of mathematics, the roads to correct solutions are rarely straight, can be quite different, and take patience and persistence to explore." (Alan Sultan & Alice F Artzt, "The Mathematics that every Secondary School Math Teacher Needs to Know", 2011)

"Failure is so much more interesting because you learn from it. That's what we should be teaching children at school, that being successful the first time, there's nothing in it. There's no interest, you learn nothing actually." (Sir James Dyson, The Observer, [interview] 2014)

"Although cascading failures may appear random and unpredictable, they follow reproducible laws that can be quantified and even predicted using the tools of network science. First, to avoid damaging cascades, we must understand the structure of the network on which the cascade propagates. Second, we must be able to model the dynamical processes taking place on these networks, like the flow of electricity. Finally, we need to uncover how the interplay between the network structure and dynamics affects the robustness of the whole system." (Albert-László Barabási, "Network Science", 2016)

🦋Science: On Errors (Quotes)

"The probable is something which lies midway between truth and error" (Christian Thomasius, "Institutes of Divine Jurisprudence", 1688)

"Knowledge being to be had only of visible and certain truth, error is not a fault of our knowledge, but a mistake of our judgment, giving assent to that which is not true." (John Locke, "An Essay Concerning Human Understanding", 1689)

"The errors of definitions multiply themselves according as the reckoning proceeds; and lead men into absurdities, which at last they see but cannot avoid, without reckoning anew from the beginning." (Thomas Hobbes, "The Moral and Political Works of Thomas Hobbes of Malmesbury", 1750)

"Men are often led into errors by the love of simplicity, which disposes us to reduce things to few principles, and to conceive a greater simplicity in nature than there really is." (Thomas Reid, "Essays on the Intellectual Powers of Man", 1785)

"The orbits of certainties touch one another; but in the interstices there is room enough for error to go forth and prevail." (Johann Wolfgang von Goethe, "Maxims and Reflections", 1833)

"Nothing hurts a new truth more than an old error." (Johann Wolfgang von Goethe, "Sprüche in Prosa", 1840)

"Every detection of what is false directs us towards what is true: every trial exhausts some tempting form of error. Not only so; but scarcely any attempt is entirely a failure; scarcely any theory, the result of steady thought, is altogether false; no tempting form of error is without some latent charm derived from truth." (William Whewell, "Lectures on the History of Moral Philosophy in England", 1852)

"[…] ideas may be both novel and important, and yet, if they are incorrect - if they lack the very essential support of incontrovertible fact, they are unworthy of credence. Without this, a theory may be both beautiful and grand, but must be as evanescent as it is beautiful, and as unsubstantial as it is grand." (George Brewster, "A New Philosophy of Matter", 1858)

"When a power of nature, invisible and impalpable, is the subject of scientific inquiry, it is necessary, if we would comprehend its essence and properties, to study its manifestations and effects. For this purpose simple observation is insufficient, since error always lies on the surface, whilst truth must be sought in deeper regions." (Justus von Liebig," Familiar Letters on Chemistry", 1859)

"As in the experimental sciences, truth cannot be distinguished from error as long as firm principles have not been established through the rigorous observation of facts." (Louis Pasteur, "Étude sur la maladie des vers à soie", 1870)

"It would be an error to suppose that the great discoverer seizes at once upon the truth, or has any unerring method of divining it. In all probability the errors of the great mind exceed in number those of the less vigorous one. Fertility of imagination and abundance of guesses at truth are among the first requisites of discovery; but the erroneous guesses must be many times as numerous as those that prove well founded. The weakest analogies, the most whimsical notions, the most apparently absurd theories, may pass through the teeming brain, and no record remain of more than the hundredth part. […] The truest theories involve suppositions which are inconceivable, and no limit can really be placed to the freedom of hypotheses." (W Stanley Jevons, "The Principles of Science: A Treatise on Logic and Scientific Method", 1877)

"Perfect readiness to reject a theory inconsistent with fact is a primary requisite of the philosophic mind. But it, would be a mistake to suppose that this candour has anything akin to fickleness; on the contrary, readiness to reject a false theory may be combined with a peculiar pertinacity and courage in maintaining an hypothesis as long as its falsity is not actually apparent." (William S Jevons, "The Principles of Science", 1887)

"One is almost tempted to assert that quite apart from its intellectual mission, theory is the most practical thing conceivable, the quintessence of practice as it were, since the precision of its conclusions cannot be reached by any routine of estimating or trial and error; although given the hidden ways of theory, this will hold only for those who walk them with complete confidence." (Ludwig E Boltzmann, "On the Significance of Theories", 1890)

"[…] to kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact." (Charles R Darwin, "More Letters of Charles Darwin", Vol 2, 1903)

"Man's determination not to be deceived is precisely the origin of the problem of knowledge. The question is always and only this: to learn to know and to grasp reality in the midst of a thousand causes of error which tend to vitiate our observation." (Federigo Enriques, "Problems of Science", 1906)

"The aim of science is to seek the simplest explanations of complex facts. We are apt to fall into the error of thinking that the facts are simple because simplicity is the goal of our quest. The guiding motto in the life of every natural philosopher should be, ‘Seek simplicity and distrust it’." (Alfred N Whitehead, "The Concept of Nature", 1919)

"One grievous error in interpreting approximations is to allow only good approximations." (Preston C Hammer, "Mind Pollution", Cybernetics, Vol. 14, 1971)

"A mature science, with respect to the matter of errors in variables, is not one that measures its variables without error, for this is impossible. It is, rather, a science which properly manages its errors, controlling their magnitudes and correctly calculating their implications for substantive conclusions." (Otis D Duncan, "Introduction to Structural Equation Models", 1975)

"Humans may crave absolute certainty; they may aspire to it; they may pretend, as partisans of certain religions do, to have attained it. But the history of science - by far the most successful claim to knowledge accessible to humans - teaches that the most we can hope for is successive improvement in our understanding, learning from our mistakes, an asymptotic approach to the Universe, but with the proviso that absolute certainty will always elude us. We will always be mired in error. The most each generation can hope for is to reduce the error bars a little, and to add to the body of data to which error bars apply." (Carl Sagan, "The Demon-Haunted World: Science as a Candle in the Dark", 1995)

"Repeated observations of the same phenomenon do not always produce the same results, due to random noise or error. Sampling errors result when our observations capture unrepresentative circumstances, like measuring rush hour traffic on weekends as well as during the work week. Measurement errors reflect the limits of precision inherent in any sensing device. The notion of signal to noise ratio captures the degree to which a series of observations reflects a quantity of interest as opposed to data variance. As data scientists, we care about changes in the signal instead of the noise, and such variance often makes this problem surprisingly difficult." (Steven S Skiena, "The Data Science Design Manual", 2017)

🦋Science: On Definitions (Quotes)

"Rules for Demonstrations. I. Not to undertake to demonstrate any thing that is so evident of itself that nothing can be given that is clearer to prove it. II. To prove all propositions at all obscure, and to employ in their proof only very evident maxims or propositions already admitted or demonstrated. III. To always mentally substitute definitions in the place of things defined, in order not to be misled by the ambiguity of terms which have been restricted by definitions." (Blaise Pascal, "Pensées", 1670)

"Rules necessary for definitions. Not to leave any terms at all obscure or ambiguous without definition; Not to employ in definitions any but terms perfectly known or already explained. […] A few rules include all that is necessary for the perfection of the definitions, the axioms, and the demonstrations, and consequently of the entire method of the geometrical proofs of the art of persuading." (Blaise Pascal, "Pensées", 1670)

"The errors of definitions multiply themselves according as the reckoning proceeds; and lead men into absurdities, which at last they see but cannot avoid, without reckoning anew from the beginning." (Thomas Hobbes, "The Moral and Political Works of Thomas Hobbes of Malmesbury", 1750)

"Definitions might be good if we did not employ words in making them." (Jean-Jacques Rousseau, "Emile, or, Treatise on Education", 1762)

"A definition is nothing else but an explication of the meaning of a word, by words whose meaning is already known. Hence it is evident that every word cannot be defined; for the definition must consist of words; and there could be no definition, if there were not words previously understood without definition." (Thomas Reid, "Essays on the Intellectual Powers of Man", 1785)

"There is nothing more difficult than a good definition, for it is scarcely possible to express, in a few words, the abstracted view of an infinite variety of facts." (Humphry Davy, "Consolations in Travel, or the Last Days of a Philosopher", 1830)

"It is the essence of a scientific definition to be causative, not by introduction of imaginary somewhats, natural or supernatural, under the name of causes, but by announcing the law of action in the particular case, in subordination to the common law of which all the phenomena are modifications or results." (Samuel T Coleridge, "Hints Towards the Formation of a More Comprehensive Theory of Life, The Nature of Life", 1847)

"The dimmed outlines of phenomenal things all merge into one another unless we put on the focusing-glass of theory, and screw it up sometimes to one pitch of definition and sometimes to another, so as to see down into different depths through the great millstone of the world." (James C Maxwell, "Are There Real Analogies in Nature?", 1856)

"Questions of Definition are of the very highest importance in Philosophy, and they need to be watched accordingly." (George Campbell, "A Fourth State of Matter, Nature", 1880)

"We say it is ‘explanation’ but it is only in ‘description’ that we are in advance of the older stages of knowledge and science. We describe better we explain just as little as our predecessors." (Friedrich W Nietzsche, "The Joyful Wisdom", 1887)

"The more elevated a culture, the richer its language. The number of words and their combinations depends directly on a sum of conceptions and ideas; without the latter there can be no understandings, no definitions, and, as a result, no reason to enrich a language. (Anton Chekhov, [letter to A.S. Suvorin] 1892)

"A definition is the enclosing a wilderness of idea within a wall of words." (Samuel Butler, "The Note-Books of Samuel Butler", 1912)

"In every new and growing science there are many working hypotheses that never attain to any sort of reality. On the other hand, in the old and abstract sciences of mathematics, where it is hard to tell how much is mere definition or convention, the problem of reality is not so much doubtful as it is meaningless." (Gilbert N Lewis, "The Anatomy of Science", 1926)

"If we wish to express our ideas in terms of the concepts synthetic and analytic, we would have to point out that these concepts are applicable only to sentences that can be either true of false, and not to definitions. The mathematical axioms are therefore neither synthetic nor analytic, but definitions. [...] Hence the question of whether axioms are a priori becomes pointless since they are arbitrary." (Hans Reichenbach, "The Philosophy of Space and Time", 1928)

"A Weltanschauung [worldview] is an intellectual construction which solves all the problems of our existence uniformly on the basis of one overriding hypothesis, which, accordingly, leaves no question unanswered and in which everything that interests us finds its fixed place [...] the worldview of science already departs noticeably from our definition. It is true that it too assumes the uniformity of the explanation of the universe; but it does so only as a programme, the fulfillment of which is relegated to the future." Sigmund Freud, "New introductory lectures on psycho-analysis", 1932)

"Scientific Ideas can often be adequately exhibited for all the purposes of reasoning, by means of Definitions and Axioms; all attempts to reason by means of Definitions from common Notions, lead to empty forms or entire confusion." (William Whewell, "History of the Inductive Sciences from the Earliest to the Present Time", 1937)

"The view is often defended that sciences should be built up on clear and sharply defined basal concepts. In actual fact no science, not even the most exact, begins with such definitions. The true beginning of scientific activity consists rather in describing phenomena and then in proceeding to group, classify and correlate them." (Sigmund Freud, "Collected Papers", 1950)

"Being built on concepts, hypotheses, and experiments, laws are no more accurate or trustworthy than the wording of the definitions and the accuracy and extent of the supporting experiments." (Gerald Holton, "Introduction to Concepts and Theories in Physical Science", 1952)

"The word 'definition' has come to have a dangerously reassuring sound, owing no doubt to its frequent occurrence in logical and mathematical writings." (Willard van Orman Quine, "From a Logical Point of View", 1953)

"We cannot define truth in science until we move from fact to law. And within the body of laws in turn, what impresses us as truth is the orderly coherence of the pieces. They fit together like the characters of a great novel, or like the words of a poem. Indeed, we should keep that last analogy by us always, for science is a language, and like a language it defines its parts by the way they make up a meaning. Every word in a sentence has some uncertainty of definition, and yet the sentence defines its own meaning and that of its words conclusively. It is the internal unity and coherence of science which gives it truth, and which makes it a better system of prediction than any less orderly language." (Jacob Bronowski, "The Common Sense of Science", 1953)

"The exact verbal definition of qualitative concepts is more often the province of philosophy than of physical science." (Ronnie Bell, "The Proton in Chemistry", 1959)

"In order that a mathematical science of any importance may be founded upon conventional definitions, the entities created by them must have properties which bear some affinity to the properties of existing things." (John H C Whitehead, A Treatise on Universal Algebra, with Applications, 1960)

"The view is often defended that sciences should be built up on clear and sharply defined basal concepts. In actual fact no science, not even the most exact, begins with such definitions. The true beginning of scientific activity consists rather in describing phenomena and then in proceeding to group, classify and correlate them." (Sigmund Freud, "General Psychological Theory", 1963)

"This other world is the so-called physical world image; it is merely an intellectual structure. To a certain extent it is arbitrary. It is a kind of model or idealization created in order to avoid the inaccuracy inherent in every measurement and to facilitate exact definition." (Max Planck, "The Philosophy of Physics", 1963)

"If all the theories pertinent to systems engineering could be discussed within a common framework by means of a standard set of nomenclature and definitions, many separate courses might not be required." (A Wayne Wymore, "A Mathematical Theory of Systems Engineering", 1967)

"Definitions are the guardians of rationality, the first line of defense against the chaos of mental disintegration. (Ayn Rand, The Romantic Manifesto, 1969)

"Concepts form the basis for any science. These are ideas, usually somewhat vague (especially when first encountered), which often defy really adequate definition. The meaning of a new concept can seldom be grasped from reading a one-paragraph discussion. There must be time to become accustomed to the concept, to investigate it with prior knowledge, and to associate it with personal experience. Inability to work with details of a new subject can often be traced to inadequate understanding of its basic concepts." (William C Reynolds & Harry C Perkins, "Engineering Thermodynamics", 1977)

"Definitions, like questions and metaphors, are instruments for thinking. Their authority rests entirely on their usefulness, not their correctness. We use definitions in order to delineate problems we wish to investigate, or to further interests we wish to promote. In other words, we invent definitions and discard them as suits our purposes." (Neil Postman, "Language Education in a Knowledge Context", 1980)

"When terms [...] evolve and change definition with time; and when the social reality which terms are intended to organize and render intelligible is also seen to be in flux, capturing the truth in a net of words becomes a matter of intuition and style more than of any scientific method that can be replicated by others and made to achieve the same result every time someone asks the same question, or undertakes the same operations." (William H McNeill, "Discrepancies among the Social Sciences", 1981)

"Definitions are temporary verbalizations of concepts, and concepts - particularly difficult concepts - are usually revised repeatedly as our knowledge and understanding grows." (Ernst Mayr, "The Growth of Biological Thought", 1982)

"The assumptions and definitions of mathematics and science come from our intuition, which is based ultimately on experience. They then get shaped by further experience in using them and are occasionally revised. They are not fixed for all eternity." (Richard Hamming, "Methods of Mathematics Applied to Calculus, Probability, and Statistics", 1985)

"A full definition of an object must include the whole of human experience, both as a criterion of truth and a practical indicator of its connection with human wants." (Vladimir Lenin)

"Every physical concept must be given a definition such that one can in principle describe, in virtue of this definition, whether or not it applies in each particular case." (Albert Einstein)

"Fundamental definitions do not arise at the start but at the end of the exploration, because in order to define a thing you must know what it is and what it is good for." (Hans Freudenthal)

"We begin to reason from sensible objects, and definition is the end and epilogue of science. It is not the beginning of our knowing, but only of our teaching." (Tommaso Campanella)

🦋Science: On Facts (Quotes)

"[…] to kill an error is as good a service as, and sometimes even better than, the establishing of a new truth or fact." (Charles R Darwin, "More Letters of Charles Darwin", Vol 2, 1903)

"Entia non sunt multiplicanda praeter necessitatem. That is to say; before you try a complicated hypothesis, you should make quite sure that no simplification of it will explain the facts equally well." (Charles S Peirce," Pragmatism and Pragmaticism", [lecture] 1903)

"But, once again, what the physical states as the result of an experiment is not the recital of observed facts, but the interpretation and the transposing of these facts into the ideal, abstract, symbolic world created by the theories he regards as established." (Pierre-Maurice-Marie Duhem, "The Aim and Structure of Physical Theory", 1908)

"The facts of greatest outcome are those we think simple; may be they really are so, because they are influenced only by a small number of well-defined circumstances, may be they take on an appearance of simplicity because the various circumstances upon which they depend obey the laws of chance and so come to mutually compensate." (Henri Poincaré, "The Foundations of Science", 1913)

"The world is an endless variety of facts, linked together by necessary and immutable bonds." (Émile Boutroux, "Natural law in Science and Philosophy", 1914)

"Statistics may be defined as numerical statements of facts by means of which large aggregates are analyzed, the relations of individual units to their groups are ascertained, comparisons are made between groups, and continuous records are maintained for comparative purposes." (Melvin T Copeland. "Statistical Methods" [in: Harvard Business Studies, Vol. III, Ed. by Melvin T Copeland, 1917])

"Facts are carpet-tacks under the pneumatic tires of theory." (Austin O’Malley, "Keystones of Thought", 1918)

"The aim of science is to seek the simplest explanations of complex facts. We are apt to fall into the error of thinking that the facts are simple because simplicity is the goal of our quest. The guiding motto in the life of every natural philosopher should be, ‘Seek simplicity and distrust it’." (Alfred N Whitehead, "The Concept of Nature", 1919)

"[…] the mere collection of facts, without some basis of theory for guidance and elucidation, is foolish and profitless." (Gamaliel Bradford, "Darwin", 1926)

"Observed facts must be built up, woven together, ordered, arranged, systematized into conclusions and theories by reflection and reason, if they are to have full bearing on life and the universe. Knowledge is the accumulation of facts. Wisdom is the establishment of relations. And just because the latter process is delicate and perilous, it is all the more delightful." (Gamaliel Bradford, "Darwin", 1926)

"[…] facts are too bulky to be lugged about conveniently except on the wheels of theory." (Julian Huxley, "Essays of a Biologist", 1929)

"We can invent as many theories we like, and any one of them can be made to fit the facts. But that theory is always preferred which makes the fewest number of assumptions." (Albert Einstein [interview] 1929)

"A system is said to be coherent if every fact in the system is related every other fact in the system by relations that are not merely conjunctive. A deductive system affords a good example of a coherent system." (Lizzie S Stebbing, "A modern introduction to logic", 1930)

"In experimental science facts of the greatest importance are rarely discovered accidentally: more frequently new ideas point the way towards them." (Erwin Schrödinger, "Science and the Human Temperament", 1935)

"Science is the attempt to discover, by means of observation, and reasoning based upon it, first, particular facts about the world, and then laws connecting facts with one another and (in fortunate cases) making it possible to predict future occurrences." (Bertrand Russell, "Religion and Science, Grounds of Conflict", 1935)

"The fundamental gospel of statistics is to push back the domain of ignorance, prejudice, rule-of-thumb, arbitrary or premature decisions, tradition, and dogmatism and to increase the domain in which decisions are made and principles are formulated on the basis of analyzed quantitative facts." (Robert W Burgess, "The Whole Duty of the Statistical Forecaster", Journal of the American Statistical Association , Vol. 32, No. 200, 1937)

"With the help of physical theories we try to find our way through the maze of observed facts, to order and understand the world of our sense impressions." (Albert Einstein & Leopold Infeld, "The Evolution of Physics", 1938)

"We can put it down as one of the principles learned from the history of science that a theory is only overthrown by a better theory, never merely by contradictory facts." (James B Conant, "On Understanding Science", 1947)

"To some people, statistics is ‘quartered pies, cute little battleships and tapering rows of sturdy soldiers in diversified uniforms’. To others, it is columns and columns of numerical facts. Many regard it as a branch of economics. The beginning student of the subject considers it to be largely mathematics." (The Editors, "Statistics, The Physical Sciences and Engineering", The American Statistician, Vol. 2, No. 4, 1948)

"A conceptual scheme is never discarded merely because of a few stubborn facts with which it cannot be reconciled; a conceptual scheme is either modified or replaced by a better one, never abandoned with nothing left to take its place." (James B Conant, "Science and Common Sense", 1951)

"The act of discovery escapes logical analysis; there are no logical rules in terms of which a 'discovery machine' could be constructed that would take over the creative function of the genius. But it is not the logician’s task to account for scientific discoveries; all he can do is to analyze the relation between given facts and a theory presented to him with the claim that it explains these facts. In other words, logic is concerned with the context of justification." (Hans Reichenbach, "The Rise of Scientific Philosophy", 1951)

"The study of inductive inference belongs to the theory of probability, since observational facts can make a theory only probable but will never make it absolutely certain." (Hans Reichenbach, "The Rise of Scientific Philosophy", 1951)

"[…] the grand aim of all science […] is to cover the greatest possible number of empirical facts by logical deductions from the smallest possible number of hypotheses or axioms." (Albert Einstein, 1954)

"Science does not begin with facts; one of its tasks is to uncover the facts by removing misconceptions." (Lancelot L Whyte, "Accent on Form", 1954)

"Science is the creation of concepts and their exploration in the facts. It has no other test of the concept than its empirical truth to fact." (Jacob Bronowski, "Science and Human Values", 1956)

"When we meet a fact which contradicts a prevailing theory, we must accept the fact and abandon the theory, even when the theory is supported by great names and generally accepted." (Claude Bernard, "An Introduction to the Study of Experimental Medicine", 1957)

"Science aims at the discovery, verification, and organization of fact and information [...] engineering is fundamentally committed to the translation of scientific facts and information to concrete machines, structures, materials, processes, and the like that can be used by men." (Eric A Walker, "Engineers and/or Scientists", Journal of Engineering Education Vol. 51, 1961)

"The important distinction between science and those other systematizations [i.e., art, philosophy, and theology] is that science is self-testing and self-correcting. Here the essential point of science is respect for objective fact. What is correctly observed must be believed [...] the competent scientist does quite the opposite of the popular stereotype of setting out to prove a theory; he seeks to disprove it." (George G Simpson, "Notes on the Nature of Science", 1962)

"It is not impossible that our own Model will die a violent death, ruthlessly smashed by an unprovoked assault of new facts […]. (Clive S Lewis, "The Discarded Image: An Introduction to Medieval and Renaissance Literature", 1964)

"A model is a qualitative or quantitative representation of a process or endeavor that shows the effects of those factors which are significant for the purposes being considered. A model may be pictorial, descriptive, qualitative, or generally approximate in nature; or it may be mathematical and quantitative in nature and reasonably precise. It is important that effective means for modeling be understood such as analog, stochastic, procedural, scheduling, flow chart, schematic, and block diagrams." (Harold Chestnut, "Systems Engineering Tools", 1965)

"A model is a useful (and often indispensable) framework on which to organize our knowledge about a phenomenon. […] It must not be overlooked that the quantitative consequences of any model can be no more reliable than the a priori agreement between the assumptions of the model and the known facts about the real phenomenon. When the model is known to diverge significantly from the facts, it is self-deceiving to claim quantitative usefulness for it by appeal to agreement between a prediction of the model and observation." (John R Philip, 1966)

"To do science is to search for repeated patterns, not simply to accumulate facts, and to do the science of geographical ecology is to search for patterns of plants and animal life that can be put on a map." (Robert H. MacArthur, "Geographical Ecology", 1972)

"Models are not assigned per se uniquely to their originals. They perform their replacement function: a) for definite – cognitive and/or handling, model-using – subjects, b) within definite time intervals, c) under restrictions of definite operations of thought or fact. […] Models are not only models of something. They are also models for somebody, a human or an artificial model user. They perform thereby their functions in time, within a time interval. And finally, they are models for a definite purpose." (Herbert Stachowiak, "Allgemeine Modelltheorie", 1973)

"No theory ever agrees with all the facts in its domain, yet it is not always the theory that is to blame. Facts are constituted by older ideologies, and a clash between facts and theories may be proof of progress. It is also a first step in our attempt to find the principles implicit in familiar observational notions." (Paul K Feyerabend, "Against Method: Outline of an Anarchistic Theory of Knowledge", 1975)

"Facts do not ‘speak for themselves’; they are read in the light of theory. Creative thought, in science as much as in the arts, is the motor of changing opinion. Science is a quintessentially human activity, not a mechanized, robot-like accumulation of objective information, leading by laws of logic to inescapable interpretation." (Stephen J Gould, "Ever Since Darwin", 1977)

"Science has so accustomed us to devising and accepting theories to account for the facts we observe, however fantastic, that our minds must begin their manufacture before we are aware of it." (Gene Wolfe, "Seven American Nights", 1978)

"Science, since people must do it, is a socially embedded activity. It progresses by hunch, vision, and intuition. Much of its change through time does not record a closer approach to absolute truth, but the alteration of cultural contexts that influence it so strongly. Facts are not pure and unsullied bits of information; culture also influences what we see and how we see it. Theories, moreover, are not inexorable inductions from facts. The most creative theories are often imaginative visions imposed upon facts; the source of imagination is also strongly cultural." (Stephen J Gould, "The Mismeasure of Man", 1980)

"Facts and theories are different things, not rungs in a hierarchy of increasing certainty. Facts are the world's data. Theories are structures of ideas that explain and interpret facts. Facts do not go away while scientists debate rival theories for explaining them." (Stephen J Gould "Evolution as Fact and Theory", 1981)

"All great theories are expansive, and all notions so rich in scope and implication are underpinned by visions about the nature of things. You may call these visions ‘philosophy’, or ‘metaphor’, or ‘organizing principle’, but one thing they are surely not - they are not simple inductions from observed facts of the natural world." (Stephen J Gould, "Time’s Arrow, Time’s Cycle", 1987)

"Although science literally means ‘knowledge’, the scientific attitude is concerned much more with rational perception through the mind and with testing such perceptions against actual fact, in the form of experiments and observations." (David Bohm & F David Peat, "Science, Order, and Creativity", 1987)

"Facts do not 'speak for themselves'. They speak for or against competing theories. Facts divorced from theory or visions are mere isolated curiosities." (Thomas Sowell, "A Conflict of Visions: Ideological Origins of Political Struggles", 1987)

"[…] no good model ever accounted for all the facts, since some data was bound to be misleading if not plain wrong. A theory that did fit all the data would have been ‘carpentered’ to do this and would thus be open to suspicion." (Francis H C Crick, "What Mad Pursuit: A Personal View of Scientific Discovery", 1988)

"The common perception of science as a rational activity, in which one confronts the evidence of fact with an open mind, could not be more false. Facts assume significance only within a pre-existing intellectual structure, which may be based as much on intuition and prejudice as on reason." (Walter Gratzer, The Guardian, 1989)

"On this view, we recognize science to be the search for algorithmic compressions. We list sequences of observed data. We try to formulate algorithms that compactly represent the information content of those sequences. Then we test the correctness of our hypothetical abbreviations by using them to predict the next terms in the string. These predictions can then be compared with the future direction of the data sequence. Without the development of algorithmic compressions of data all science would be replaced by mindless stamp collecting - the indiscriminate accumulation of every available fact. Science is predicated upon the belief that the Universe is algorithmically compressible and the modern search for a Theory of Everything is the ultimate expression of that belief, a belief that there is an abbreviated representation of the logic behind the Universe's properties that can be written down in finite form by human beings." (John D Barrow, New Theories of Everything", 1991)

"The word theory, as used in the natural sciences, doesn’t mean an idea tentatively held for purposes of argument - that we call a hypothesis. Rather, a theory is a set of logically consistent abstract principles that explain a body of concrete facts. It is the logical connections among the principles and the facts that characterize a theory as truth. No one element of a theory [...] can be changed without creating a logical contradiction that invalidates the entire system. Thus, although it may not be possible to substantiate directly a particular principle in the theory, the principle is validated by the consistency of the entire logical structure." (Alan Cromer, "Uncommon Sense: The Heretical Nature of Science", 1993)

"Worldviews are social constructions, and they channel the search for facts. But facts are found and knowledge progresses, however fitfully. Fact and theory are intertwined, and all great scientists understand the interaction." (Stephen J Gould, "Shields of Expectation - and Actuality", 1993)

"As a result, surprisingly enough, scientific advance rarely comes solely through the accumulation of new facts. It comes most often through the construction of new theoretical frameworks. [..]  To understand scientific development, it is not enough merely to chronicle new discoveries and inventions. We must also trace the succession of worldviews" (Nancy R Pearcey & Charles B Thaxton, "The Soul of Science: Christian Faith and Natural Philosophy", 1994)

"Theories rarely arise as patient inferences forced by accumulated facts. Theories are mental constructs potentiated by complex external prods (including, in idealized cases, a commanding push from empirical reality)." (Stephen J Gould, "Leonardo's Mountain of Clams and the Diet of Worms" , 1998)

"When we entrust the domain of values to those whose intellectual concerns are essentially centred on empirical facts, and whose conceptual frameworks are inevitably constructed around sets of empirical facts, we need not be surprised if the result is moral confusion." (Ronald W K Paterson, "The New Patricians", 1998)

"Modeling involves a style of scientific thinking in which the argument is structured by the model, but in which the application is achieved via a narrative prompted by an external fact, an imagined event or question to be answered." (Uskali Mäki, "Fact and Fiction in Economics: Models, Realism and Social Construction", 2002)

"We tackle a multifaceted universe one face at a time, tailoring our models and equations to fit the facts at hand. Whatever mechanical conception proves appropriate, that is the one to use. Discovering worlds within worlds, a practical observer will deal with each realm on its own terms. It is the only sensible approach to take." (Michael Munowitz, "Knowing: The Nature of Physical Law", 2005)

"Although fiction is not fact, paradoxically we need some fictions, particularly mathematical ideas and highly idealized models, to describe, explain, and predict facts.  This is not because the universe is mathematical, but because our brains invent or use refined and law-abiding fictions, not only for intellectual pleasure but also to construct conceptual models of reality." (Mario Bunge, "Chasing Reality: Strife over Realism", 2006)

"There are no surprising facts, only models that are surprised by facts; and if a model is surprised by the facts, it is no credit to that model." (Eliezer S Yudkowsky, "Quantum Explanations", 2008)

"Obviously, the final goal of scientists and mathematicians is not simply the accumulation of facts and lists of formulas, but rather they seek to understand the patterns, organizing principles, and relationships between these facts to form theorems and entirely new branches of human thought." (Clifford A Pickover, "The Math Book", 2009)

"Each person has a different mental model and, therefore, potentially a different interpretation of the Facts. The danger comes when we start to assume that our interpretation of the Facts is the only interpretation and we believe that what we see and think is the Truth, and that there is only one Truth." (Robina Chatham & Brian Sutton, "Changing the IT Leader’s Mindset", 2010)

"Relevance is not something you can predict. It is something you discover after the fact." (Thomas Sowell, "The Thomas Sowell Reader", 2011)

"Science does not live with facts alone. In addition to facts, it needs models. Scientific models fulfill two main functions with respect to empirical facts." (Andreas Bartels [in "Models, Simulations, and the Reduction of Complexity", Ed. by Ulrich Gähde et al, 2013)

"A mental representation is a mental structure that corresponds to an object, an idea, a collection of information, or anything else, concrete or abstract, that the brain is thinking about. […] Because the details of mental representations can differ dramatically from field to field, it’s hard to offer an overarching definition that is not too vague, but in essence these representations are preexisting patterns of information - facts, images, rules, relationships, and so on - that are held in long-term memory and that can be used to respond quickly and effectively in certain types of situations." (Anders Ericsson & Robert Pool," Peak: Secrets from  the  New  Science  of  Expertise", 2016)

"Statistics is the science of collecting, organizing, and interpreting numerical facts, which we call data. […] Statistics is the science of learning from data." (Moore McCabe & Alwan Craig, "The Practice of Statistics for Business and Economics" 4th Ed., 2016)

"That is the trouble with facts: they sometimes force you to conclusions that differ with your intuition." (Steven G Krantz, "A Primer of Mathematical Writing" 2nd Ed., 2016)

🦋Science: On Observation (Quotes)

"[…] it is not necessary that these hypotheses should be true, or even probably; but it is enough if they provide a calculus which fits the observations […]" (Andrew Osiander, "On the Revolutions of the Heavenly Spheres", 1543)

"[…] it is from long experience chiefly that we are to expect the most certain rules of practice, yet it is withal to be remembered, that observations, and to put us upon the most probable means of improving any art, is to get the best insight we can into the nature and properties of those things which we are desirous to cultivate and improve." (Stephen Hales, "Vegetable Staticks", 1727) 

"Those who have not imbibed the prejudices of philosophers, are easily convinced that natural knowledge is to be founded on experiment and observation." (Colin Maclaurin, "An Account of Sir Isaac Newton’s Philosophical Discoveries", 1748)

"We have three principal means: observation of nature, reflection, and experiment. Observation gathers the facts reflection combines them, experiment verifies the result of the combination. It is essential that the observation of nature be assiduous, that reflection be profound, and that experimentation be exact. Rarely does one see these abilities in combination. And so, creative geniuses are not common." (Denis Diderot, "On the Interpretation of Nature", 1753)

"Facts, observations, experiments - these are the materials of a great edifice, but in assembling them we must combine them into classes, distinguish which belongs to which order and to which part of the whole each pertains." (Antoine L Lavoisier, "Mémoires de l’Académie Royale des Sciences", 1777)

"On the other hand, if we add observation to observation, without attempting to draw no only certain conclusions, but also conjectural views from them, we offend against the very end for which only observations ought to be made." (Friedrich W Herschel, "On the Construction of the Heavens", Philosophical Transactions of the Royal Society of London Vol. LXXV, 1785)

"[It] may be laid down as a general rule that, if the result of a long series of precise observations approximates a simple relation so closely that the remaining difference is undetectable by observation and may be attributed to the errors to which they are liable, then this relation is probably that of nature." (Pierre-Simon Laplace, "Mémoire sur les Inégalites Séculaires des Planètes et des Satellites", 1787)

"The art of drawing conclusions from experiments and observations consists in evaluating probabilities and in estimating whether they are sufficiently great or numerous enough to constitute proofs. This kind of calculation is more complicated and more difficult than it is commonly thought to be […]" (Antoine-Laurent Lavoisier, cca. 1790)

"We must trust to nothing but facts: These are presented to us by Nature, and cannot deceive. We ought, in every instance, to submit our reasoning to the test of experiment, and never to search for truth but by the natural road of experiment and observation." (Antoin-Laurent de Lavoisiere, "Elements of Chemistry", 1790)

"Conjecture may lead you to form opinions, but it cannot produce knowledge. Natural philosophy must be built upon the phenomena of nature discovered by observation and experiment." (George Adams, "Lectures on Natural and Experimental Philosophy" Vol. 1, 1794)

"In order to supply the defects of experience, we will have recourse to the probable conjectures of analogy, conclusions which we will bequeath to our posterity to be ascertained by new observations, which, if we augur rightly, will serve to establish our theory and to carry it gradually nearer to absolute certainty." (Johann H Lambert, "The System of the World", 1800)

"[…] we must not measure the simplicity of the laws of nature by our facility of conception; but when those which appear to us the most simple, accord perfectly with observations of the phenomena, we are justified in supposing them rigorously exact." (Pierre-Simon Laplace, "The System of the World", 1809)

"Primary causes are unknown to us; but are subject to simple and constant laws, which may be discovered by observation, the study of them being the object of natural philosophy." (Jean-Baptiste-Joseph Fourier, "The Analytical Theory of Heat", 1822)

"The aim of every science is foresight. For the laws of established observation of phenomena are generally employed to foresee their succession. All men, however little advanced make true predictions, which are always based on the same principle, the knowledge of the future from the past." (Auguste Compte, "Plan des travaux scientifiques nécessaires pour réorganiser la société", 1822)

"The framing of hypotheses is, for the enquirer after truth, not the end, but the beginning of his work. Each of his systems is invented, not that he may admire it and follow it into all its consistent consequences, but that he may make it the occasion of a course of active experiment and observation. And if the results of this process contradict his fundamental assumptions, however ingenious, however symmetrical, however elegant his system may be, he rejects it without hesitation. He allows no natural yearning for the offspring of his own mind to draw him aside from the higher duty of loyalty to his sovereign, Truth, to her he not only gives his affections and his wishes, but strenuous labour and scrupulous minuteness of attention." (William Whewell, "Philosophy of the Inductive Sciences" Vol. 2, 1847)

"In the fields of observation chance favors only the prepared mind." (Louis Pasteur, [lecture] 1854)

"When a power of nature, invisible and impalpable, is the subject of scientific inquiry, it is necessary, if we would comprehend its essence and properties, to study its manifestations and effects. For this purpose simple observation is insufficient, since error always lies on the surface, whilst truth must be sought in deeper regions." (Justus von Liebig," Familiar Letters on Chemistry", 1859)

"Observation is so wide awake, and facts are being so rapidly added to the sum of human experience, that it appears as if the theorizer would always be in arrears, and were doomed forever to arrive at imperfect conclusion; but the power to perceive a law is equally rare in all ages of the world, and depends but little on the number of facts observed." (Henry D Thoreau, "A Week on the Concord and Merrimack Rivers", 1862)

"The process of discovery is very simple. An unwearied and systematic application of known laws to nature, causes the unknown to reveal themselves. Almost any mode of observation will be successful at last, for what is most wanted is method." (Henry D Thoreau, "A Week on the Concord and Merrimack Rivers", 1862)

"An anticipative idea or an hypothesis is, then, the necessary starting point for all experimental reasoning. Without it, we could not make any investigation at all nor learn anything; we could only pile up sterile observations. If we experiment without a preconceived idea, we should move at random […]" (Claude Bernard, "An Introduction to the Study of Experimental Medicine", 1865)

"Men who have excessive faith in their theories or ideas are not only ill prepared for making discoveries; they also make very poor observations." (Claude Bernard, "An Introduction to the Study of Experimental Medicine", 1865)

"Only within very narrow boundaries can man observe the phenomena which surround him; most of them naturally escape his senses, and mere observation is not enough." (Claude Bernard, "An Introduction to the Study of Experimental Medicine", 1865)

"[…] wrong hypotheses, rightly worked from, have produced more useful results than unguided observation." (Augustus de Morgan, "A Budget of Paradoxes", 1872)

"Every science begins by accumulating observations, and presently generalizes these empirically; but only when it reaches the stage at which its empirical generalizations are included in a rational generalization does it become developed science." (Herbert Spencer, "The Data of Ethics", 1879)

"Science is the observation of things possible, whether present or past; prescience is the knowledge of things which may come to pass, though but slowly." (Leonardo da Vinci, "The Notebooks of Leonardo da Vinci", 1883)

"Even one well-made observation will be enough in many cases, just as one well-constructed experiment often suffices for the establishment of a law." (Émile Durkheim, "The Rules of Sociological Method", "The Rules of Sociological Method", 1895)

"Every experiment, every observation has, besides its immediate result, effects which, in proportion to its value, spread always on all sides into ever distant parts of knowledge." (Sir Michael Foster, "Annual Report of the Board of Regents of the Smithsonian Institution", 1898)

"The primary basis of all scientific thinking is observation." (Douglas Marsland, "Principles of Modern Biology", 1899)

"To observe is not enough. We must use our observations, and to do that we must generalize." (Henri Poincaré, "Science and Hypothesis", 1902)

"An isolated sensation teaches us nothing, for it does not amount to an observation. Observation is a putting together of several results of sensation which are or are supposed to be connected with each other according to the law of causality, so that some represent causes and others their effects." (Thorvald N Thiele, "Theory of Observations", 1903)

"Man's determination not to be deceived is precisely the origin of the problem of knowledge. The question is always and only this: to learn to know and to grasp reality in the midst of a thousand causes of error which tend to vitiate our observation." (Federigo Enriques, "Problems of Science", 1906)

"An experiment is an observation that can be repeated, isolated and varied. The more frequently you can repeat an observation, the more likely are you to see clearly what is there and to describe accurately what you have seen. The more strictly you can isolate an observation, the easier does your task of observation become, and the less danger is there of your being led astray by irrelevant circumstances, or of placing emphasis on the wrong point. The more widely you can vary an observation, the more clearly will be the uniformity of experience stand out, and the better is your chance of discovering laws." (Edward B Titchener, "A Text-Book of Psychology", 1909)

"Neither logic without observation, nor observation without logic, can move one step in the formation of science." (Alfred N Whitehead, "The Organization of Thought", 1916)

"A discovery is rarely, if ever, a sudden achievement, nor is it the work of one man; a long series of observations, each in turn received in doubt and discussed in hostility, are familiarized by time, and lead at last to the gradual disclosure of truth." (Sir Berkeley Moynihan, "Surgery, Gynecology & Obstetrics" Vol. 31, 1920)

"In the world of natural knowledge, no authority is great enough to support a theory when a crucial observation has shown it to be untenable." (Sir Richard A Gregory, "Discovery; or, The Spirit and Service of Science", 1928)

"Science is but a method. Whatever its material, an observation accurately made and free of compromise to bias and desire, and undeterred by consequence, is science." (Hans Zinsser, "Untheological Reflections", The Atlantic Monthly, 1929)

"Abstraction is the detection of a common quality in the characteristics of a number of diverse observations […] A hypothesis serves the same purpose, but in a different way. It relates apparently diverse experiences, not by directly detecting a common quality in the experiences themselves, but by inventing a fictitious substance or process or idea, in terms of which the experience can be expressed. A hypothesis, in brief, correlates observations by adding something to them, while abstraction achieves the same end by subtracting something." (Herbert Dingle, Science and Human Experience, 1931)

"A scientist, whether theorist or experimenter, puts forward statements, or systems of statements, and tests them step by step. In the field of the empirical sciences, more particularly, he constructs hypotheses, or systems of theories, and tests them against experience by observation and experiment." (Karl Popper, "The Logic of Scientific Discovery", 1934)

"Science is the attempt to discover, by means of observation, and reasoning based upon it, first, particular facts about the world, and then laws connecting facts with one another and (in fortunate cases) making it possible to predict future occurrences." (Bertrand Russell, "Religion and Science, Grounds of Conflict", 1935)

"Starting from statistical observations, it is possible to arrive at conclusions which not less reliable or useful than those obtained in any other exact science. It is only necessary to apply a clear and precise concept of probability to such observations. " (Richard von Mises, "Probability, Statistics, and Truth", 1939)

"Experiment as compared with mere observation has some of the characteristics of cross-examining nature rather than merely overhearing her." (Alan Gregg, "The Furtherance of Medical Research", 1941)

"Science, in the broadest sense, is the entire body of the most accurately tested, critically established, systematized knowledge available about that part of the universe which has come under human observation. For the most part this knowledge concerns the forces impinging upon human beings in the serious business of living and thus affecting man’s adjustment to and of the physical and the social world. […] Pure science is more interested in understanding, and applied science is more interested in control […]" (Austin L Porterfield, "Creative Factors in Scientific Research", 1941)

"We see what we want to see, and observation conforms to hypothesis." (Bergen Evans, "The Natural History of Nonsense", 1947)

"[...] the conception of chance enters in the very first steps of scientific activity in virtue of the fact that no observation is absolutely correct. I think chance is a more fundamental conception that causality; for whether in a concrete case, a cause-effect relation holds or not can only be judged by applying the laws of chance to the observation." (Max Born, 1949)

"Every bit of knowledge we gain and every conclusion we draw about the universe or about any part or feature of it depends finally upon some observation or measurement. Mankind has had again and again the humiliating experience of trusting to intuitive, apparently logical conclusions without observations, and has seen Nature sail by in her radiant chariot of gold in an entirely different direction." (Oliver J Lee, "Measuring Our Universe: From the Inner Atom to Outer Space", 1950)

"Science is an interconnected series of concepts and schemes that have developed as a result of experimentation and observation and are fruitful of further experimentation and observation."(James B Conant, "Science and Common Sense", 1951)

"The stumbling way in which even the ablest of the scientists in every generation have had to fight through thickets of erroneous observations, misleading generalizations, inadequate formulations, and unconscious prejudice is rarely appreciated by those who obtain their scientific knowledge from textbooks." (James B Conant, "Science and Common Sense", 1951)

"The methods of science may be described as the discovery of laws, the explanation of laws by theories, and the testing of theories by new observations. A good analogy is that of the jigsaw puzzle, for which the laws are the individual pieces, the theories local patterns suggested by a few pieces, and the tests the completion of these patterns with pieces previously unconsidered." (Edwin P Hubble, "The Nature of Science and Other Lectures", 1954)

"The discrete change has only to become small enough in its jump to approximate as closely as is desired to the continuous change. It must further be remembered that in natural phenomena the observations are almost invariably made at discrete intervals; the 'continuity' ascribed to natural events has often been put there by the observer's imagina- tion, not by actual observation at each of an infinite number of points. Thus the real truth is that the natural system is observed at discrete points, and our transformation represents it at discrete points. There can, therefore, be no real incompatibility." (W Ross Ashby, "An Introduction to Cybernetics", 1956)

"Scientists whose work has no clear, practical implications would want to make their decisions considering such things as: the relative worth of (1) more observations, (2) greater scope of his conceptual model, (3) simplicity, (4) precision of language, (5) accuracy of the probability assignment." (C West Churchman, "Costs, Utilities, and Values", 1956)

"No observations are absolutely trustworthy. In no field of observation can we entirely rule out the possibility that an observation is vitiated by a large measurement or execution error. If a reading is found to lie a very long way from its fellows in a series of replicate observations, there must be a suspicion that the deviation is caused by a blunder or gross error of some kind. [...] One sufficiently erroneous reading can wreck the whole of a statistical analysis, however many observations there are." (Francis J Anscombe, "Rejection of Outliers", Technometrics Vol. 2 (2), 1960)

"Observation, reason, and experiment make up what we call the scientific method. (Richard Feynman, "Mainly mechanics, radiation, and heat", 1963)

"As soon as we inquire into the reasons for the phenomena, we enter the domain of theory, which connects the observed phenomena and traces them back to a single ‘pure’ phenomena, thus bringing about a logical arrangement of an enormous amount of observational material." (Georg Joos, "Theoretical Physics", 1968)

"[…] the link between observation and formulation is one of the most difficult and crucial in the scientific enterprise. It is the process of interpreting our theory or, as some say, of ‘operationalizing our concepts’. Our creations in the world of possibility must be fitted in the world of probability; in Kant’s epigram, ‘Concepts without precepts are empty’. It is also the process of relating our observations to theory; to finish the epigram, ‘Precepts without concepts are blind’." (Scott Greer, "The Logic of Social Inquiry", 1969)

"Innocent, unbiased observation is a myth." (Sir Peter B Medawar, Induction and Intuition in Scientific Thought, 1969)

"The advantages of models are, on one hand, that they force us to present a 'complete' theory by which I mean a theory taking into account all relevant phenomena and relations and, on the other hand, the confrontation with observation, that is, reality." (Jan Tinbergen, "The Use of Models: Experience," 1969)

"Science consists simply of the formulation and testing of hypotheses based on observational evidence; experiments are important where applicable, but their function is merely to simplify observation by imposing controlled conditions." (Henry L Batten, "Evolution of the Earth", 1971)

"All perceiving is also thinking, all reasoning is also intuition, all observation is also invention." (Rudolf Arnheim, "Entropy and Art: An Essay on Disorder and Order", 1974)

"No theory ever agrees with all the facts in its domain, yet it is not always the theory that is to blame. Facts are constituted by older ideologies, and a clash between facts and theories may be proof of progress. It is also a first step in our attempt to find the principles implicit in familiar observational notions." (Paul K Feyerabend, "Against Method: Outline of an Anarchistic Theory of Knowledge", 1975)

"The essential function of a hypothesis consists in the guidance it affords to new observations and experiments, by which our conjecture is either confirmed or refuted." (Ernst Mach, "Knowledge and Error: Sketches on the Psychology of Enquiry", 1976)

"After all of this it is a miracle that our models describe anything at all successfully. In fact, they describe many things well: we observe what they have predicted, and we understand what we observe. However, this last act of observation and understanding always eludes physical description." (Yuri I Manin, "Mathematics and Physics", 1981)

"Science is a process. It is a way of thinking, a manner of approaching and of possibly resolving problems, a route by which one can produce order and sense out of disorganized and chaotic observations. Through it we achieve useful conclusions and results that are compelling and upon which there is a tendency to agree." (Isaac Asimov, "‘X’ Stands for Unknown", 1984)

"Science is defined as a set of observations and theories about observations." (F Albert Matsen, "The Role of Theory in Chemistry", Journal of Chemical Education Vol. 62 (5), 1985)

"The only touchstone for empirical truth is experiment and observation." (Heinz Pagels, "Perfect Symmetry: The Search for the Beginning of Time", 1985)

"The model is only a suggestive metaphor, a fiction about the messy and unwieldy observations of the real world. In order for it to be persuasive, to convey a sense of credibility, it is important that it not be too complicated and that the assumptions that are made be clearly in evidence. In short, the model must be simple, transparent, and verifiable." (Edward Beltrami, "Mathematics for Dynamic Modeling", 1987)

"A theory is a good theory if it satisfies two requirements: it must accurately describe a large class of observations on the basis of a model that contains only a few arbitrary elements, and it must make definite predictions about the results of future observations." (Stephen Hawking, "A Brief History of Time: From Big Bang To Black Holes", 1988)

"A law explains a set of observations; a theory explains a set of laws. […] a law applies to observed phenomena in one domain (e.g., planetary bodies and their movements), while a theory is intended to unify phenomena in many domains. […] Unlike laws, theories often postulate unobservable objects as part of their explanatory mechanism." (John L Casti, "Searching for Certainty: How Scientists Predict the Future", 1990)

"A model is often judged by how well it 'explains' some observations. There need not be a unique model for a particular situation, nor need a model cover every possible special case. A model is not reality, it merely helps to explain some of our impressions of reality. [...] Different models may thus seem to contradict each other, yet we may use both in their appropriate places." (Richard W Hamming, "The Art of Probability for Scientists and Engineers", 1991)

"The ability of a scientific theory to be refuted is the key criterion that distinguishes science from metaphysics. If a theory cannot be refuted, if there is no observation that will disprove it, then nothing can prove it - it cannot predict anything, it is a worthless myth." (Eric Lerner, "The Big Bang Never Happened", 1991)

"It is in the nature of theoretical science that there can be no such thing as certainty. A theory is only ‘true’ for as long as the majority of the scientific community maintain the view that the theory is the one best able to explain the observations." (Jim Baggott, "The Meaning of Quantum Theory", 1992)

"The art of science is knowing which observations to ignore and which are the key to the puzzle." (Edward W Kolb, "Blind Watchers of the Sky", 1996)

"The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test." (Richard Feynman, "The Meaning of It All", 1998)

"[…] because observations are all we have, we take them seriously. We choose hard data and the framework of mathematics as our guides, not unrestrained imagination or unrelenting skepticism, and seek the simplest yet most wide-reaching theories capable of explaining and predicting the outcome of today’s and future experiments." (Brian Greene, "The Fabric of the Cosmos", 2004)

"A model is a good model if it:1. Is elegant 2. Contains few arbitrary or adjustable elements 3. Agrees with and explains all existing observations 4. Makes detailed predictions about future observations that can disprove or falsify the model if they are not borne out." (Stephen Hawking & Leonard Mlodinow, "The Grand Design", 2010)