14 June 2010

DIKW – An Introduction

    Collocations like “Web of data”, “Web of Information” “Web of Knowledge” “Data Management”, “Information Management”, “Knowledge Management”, “Data Technologies”, “Information Technologies”, “Knowledge Technologies” and other similar such compounds involving Data, Information and Knowledge (DIK) are increasingly and interchangeably used in various contexts by vendors, customers, researchers and bloggers, sometimes being misused from confusion or exaggeration. On the other side it is true that the border between data, information and knowledge can be shallow, and from here rise up divergent opinions and theories, and as always people like to fight about their ideas and believes. To the three adds wisdom forming what is known as DIKW, the Knowledge Pyramid, however wisdom is less met in the technical landscape presumptively because it’s harder to express and quantify.

    DIKW reveals the layered evolution of information from data to knowledge and further to wisdom; it’s just a wide accepted theory, a pyramidal representation to which there were attempts of including proto-facts and facts [2], understanding [1] [6], transaction and intuition [7], vision and “One Mindness” [6], and probably other similar attempts might be there in the deep web. Sometimes such theories are pure philosophy apparently without applicability and quite far away from the work of scientists who are used to juggle with data, information and knowledge, relatively easy to quantify and grasp compared to wisdom, intuition, meaning, creativity, vision, consciousness, awareness or any other similar terms that made, since ancient times, philosophers’ delight. On the other side the scientific work is built upon the philosophic work, and seemingly as new advancements are made the “New Age” philosophies tend to be integrated in scientific work. From the concepts from the second group, only meaning has started to stir up researcher’s attention, and this mainly in AI domain, which attempts to bridge the gap between humans and machines. I suppose there are attempts also in the area of creativity, vision, intuition or wisdom, though it will be difficult to do concludent steps until the barriers of meaning are not broken.

    Data are considered to be “simple isolated facts” [4], a combination of "symbols or isolated and non-interpreted facts" [5], symbols, raw that “simply exists and has no significance beyond its existence” [10], quantified or qualified symbol sets, signals or sensory stimuli perceived [3], etc. Be it facts, symbols, events, signals or sensory stimuli, it is highlighted their isolated and not-interpretable characteristics. When we think of data we refer mainly to the written symbols, independent on whether they have or not meaning for us – a digit or set of digits, a letter and the words they form, an image or any combination of them. In a apparently collection of signs, each sign could be regarded as a data chunk, though it happens that we decipher patterns of signs that we know, for example words, phone numbers, addresses, coordinates, time, etc. In database terminology data refers to a matrix structure (columns vs. records) in which each cells of the matrix contains a data chunk that nowadays could be any aggregation of signs and data types a database system supports. 

    As we can see interpretations of data could have different granularity dependent on the form of aggregation and purpose, thus a natural question: when could we discuss about information? For sure a collection of data is not information, then what ingredient(s) is/are essential in having a chunk of information? I believe that there are two magic words – relation and meaning, actually meaningful relation. Using various algorithms, and even from a simple review of the data, researchers could establish various relations between two data chunks, for example that they always occur together, that one causes the other, that XYZ is the address of John Smith, and so on. To make it even more complicated, for an IT professional, a structure, and I’m referring here mainly to databases, supposes already the existence of relations between data, thus statements like XYZ is the address of John Smith who was born on 1st of July and works for ABC are already available. Of course, only few relations are useful, just because two chunks of data occur together, in one or several locations, this doesn’t mean that such a relation is meaningful, while in other situations that might be quite important. When we discuss about a meaningful relation we actually imply a context, a multidimensional (mental) frame in which meaning is formed and derived from. Thus a relation could be meaningful in one context and irrelevant in another, and that’s why many say that data don’t equate with information. Discovering that the evolution from data to information is context dependent could maybe solve some of the philosophical disagreements, though this aspect makes it difficult to find a common accepted definition for information. Basically information is just chunks of data found in meaningful relation(s).

    In what concerns knowledge things are not less complex, the just formulated definition for information could be adapted from my point of view also to knowledge, advancing the idea that knowledge is information found in meaningful relation. Establishing meaningful relations between information implies a certain level of understanding, conscious or unconscious processing and systematization of information, their integration in the existing mental structures - contexts. Mental structures apply here not only to individuals but also to communities, in the past years the later gaining considerable importance. Communities equate with networks, and networked is also the structure of the knowledge, this involving a dynamic (evolutional) nature, with multiple values of truth, levels of organization and granularity. The systematization and integration of knowledge relies on existing or new formed patterns based on characteristics of knowledge. There were written many books about knowledge, trying to depict its characteristics, and there are so many facets that it’s difficult to find a definition that accommodates all the types of knowledge – tacit vs. explicit knowledge, personal vs. networked knowledge, declarative vs. procedural, practical vs. theoretical knowledge, personal vs. collective knowledge, contextual, situational or definitional knowledge, and so on. I find it quite important to understand the various types of characteristics because each of them comes with their own characteristics and uses.

    Wisdom is skillful use of knowledge, knowledge put into use, transcended knowledge, understanding the patterns and principles of knowledge, knowing when, where, how, why and by what means to apply the knowledge. Wisdom involves also keeping the balance between the various aspects of situations, strengths and weaknesses altogether, minimizing or totally overcoming biases. In spiritual literature and tradition, wisdom plays a fundamental role, the many books on this topic attempting to reveal the qualities (dimensions) and implications of wisdom. I will limit myself only to give two quotes, namely one from A. Einstein who was remarking that "wisdom is not a product of schooling of the lifelong attempt to acquire it", and B. Russel was remarking that “although our age far surpasses all previous ages in knowledge, there has been no correlative increase in wisdom” [8].

     The new added scales to the DIKW pyramid are questionable, it’s kind of a gray area, based maybe on deeper considerations not so easy to grasp, while the lack of definitions and argumentation makes the understanding process even more difficult.  The best example regarding the lack of definitions is the proto-fact, in such situations we could eventually call into help the possible etymological formation. The prefix proto-, from Greek protos, when used in compounds has the meaning of “first” or “before”, thus that proto-facts could be considered as basic/primary facts, maybe axioms, idea that matches also [5]’s view, who’s considering them as assumptions that are used to extract new facts from the world. Following this definition or assumption, proto-facts and facts are rather information, data describing them at most so I really don’t see why the facts and proto-facts are considered as the lower layer on which data layer is based, this because I see data as describing facts, mainly as quantitative or qualitative associations.

    From a creationist point of view, everything is vibration, what we perceive is nothing but grossier vibrations taking various forms, some of them recepted and processed by our senses. The layer that comes into contact with the medium (environment) is called by [7] the transaction layer, where the “packets” of sensations are put onto or taken off of the medium, and before any data can exist, the packets have to be received, unpacked and assembled into data. Even if the denomination of transaction layer is not maybe the best, and the description is quite simplistic, this should be at high level the process in which the stimuli from the outside and inside world are transformed by our senses and brain in the so called “data”, forming our continuous reality. [7] believes that somewhere at this level the intuition occurs, though it could be also at higher level as long everything happens within the unconscious mind. Without going to deep into philosophical debate I would limit myself to say that I find well-founded the adding of intuition into DIKW scale, and I could put it on the same level with creativity, innovation, and vision, all three dimensions transcend knowledge and have their important role in understanding and further in wisdom’s attainment. The role of intuition and creative is reinforced also within Edward de Bono’s Six Thinking Hats, red hat representing the intuition, while the green hat, the creativity.

    Vision and “one mindness” seem to come from the New Age vocabulary though they are not so far away from modern science, vision in technical terms is reflected in forecasting, approximating the future based on past trends, while “one mindness” is the principle or the state of art behind Noosphere. It’s true, in what concerns the human being and the actual stage of development, such terms sound to be inapplicable, on the other side when we consider that everything is in continuous evolution/involution, everything is possible. From the point of view of machines (computers), given their interconnectedness and processing power everything is possible.

    When we look from a relational point of view, data are symbols with no explicit relation between them, information are data with an explicit relation, knowledge is relational patterns of information, wisdom is patterns of knowledge, intuition is the comprehension of patterns without sensing the patterns consciously, creativity is the discovery or transmutation of new patterns, and maybe it worth to mention also innovation, seen by [9] as the rearranging of relations. More difficult to define from the perspective of relations are the “one mindness” and symbols. The “one mindness” could be eventually regarded as a reflection of the interconnectedness and comprehensions of all existing patterns, including the divine patterns in case of humans. Symbols are at their turn a pattern but of a lower refinement, at form-energetic level. The complexity or relations or patterns denotes the overall complexity, though our senses already hide a higher level of complexity, and most probably here are the barriers of our spiritual and scientific worlds.

    In what concerns the meaning, symbols can have or not meaning, or different meanings in different contexts, same could be said about data, information and eventually knowledge, but I would expect that at wisdom and upper levels meaning is fundamental, even conscious or unconscious. With the evolution from data to wisdom and further, the meaning and the relations become richer and more complex. And we can’t discuss about meaning without discussing about context. I tend to believe data have an extrinsic context given by the medium in which they occur, though that’s not necessarily relevant. As per my above given definition, information supposes the existence of a meaningful relation and thus the placement in one or more contexts, while with the integration within a context we could talk already about knowledge.  Probably wisdom acts like a bridge point between contexts, visions extrapolates contexts while in “one mindness” everything becomes one context.

    Most of the resources I’ve seen focus mainly on the evolution from data to wisdom, though there is also a feedback mechanism, for example the more high quality data we collect the more are the chances to validate current theories or discover more generalized theories, enriching thus our luggage of information, possibly further evolved to upper layers. It’s interesting also to study the interaction between the various layers, for that having to take into account also the existing theories of learning, of dynamic systems, etc. 

Note:
    I left out understanding from this post because is a complex (philosophical) concept even if everybody has an idea what is about. Please note that I haven’t tried to review the existing DIKW models, but just to write down my thoughts, to create a basis for further posts.

References:
[1] Wikipedia. (2009). DIKW. [Online] Available from: http://en.wikipedia.org/wiki/DIKW (Accessed: 7 May 2009)
[2]  Visual Concept. (1998). Making Meaning. [Online] Available from: http://www.visual-concept.co.uk/makingmeaning.htm (Accessed: 10 February 2009)
[3] Zins, C. (2007). Journal of American Society for Information Science and Technology, 58(4): 279-293, 2007. Wiley InterScience. [Online] Available from: http://www.success.co.il/is/zins_definitions_dik.pdf (Accessed: 16 May 2009)
[4] (Ken) McLennan, K. J.. "Chapter 20 - Knowledge". The Virtual World of Work: How to Gain Competitive
[5] M. J. Coombs ,  H. D. Pfeiffer ,  R. T. Hartley (). e-MGR: an architecture for symbolic plasticity. Int. J. Man-Machine Studies http://www.cs.nmsu.edu/~hdp/PDF/plastic.pdf
[6] OneMind. (2009). Seven Levels of Interaction [Online] Available from: http://onemind.com/2009/12/19/ (Accessed: 7 June 2010)
[7] G. Czerepak. (2007) TIDIKW. [Online] Available from: http://relationary.wordpress.com/2007/07/14/dikw (Accessed: 8 June 2010)
[8] B. Russel. Knowledge and Wisdom
[9] [Online] Available from:  http://thingamy.typepad.com/sigs_blog/2010/01/information-knowledge-wisdom-and-innovation.html (Accessed: 14 June 2010)
[10] G. Bellinger, D. Castro, A. Mills (2004) Data, Information, Knowledge, and Wisdom. [Online] Available from: http://www.systems-thinking.org/dikw/dikw.htm (Accessed: 14 June 2010)