Scribbles on dimensions of technology

• For Tech in MTSs review dimension intro make-public

• Communication richness

  • Graham Bell's telephone and David Alter's telegraph bridged distances, sending our voices and words across the miles and the continents. These new mediums for communication allowed us to talk to more people, more distant people, than ever before. It was not long before we dreamed of talking with video connections, and of holographic images projected into reality. The first of these dreams is already reality, and the second is sure to follow soon. This use of technology lets us send our words to each other, and the continuum of how realistic these communications are, how closely they approximate the experience of talking to someone face-to-face, is a measure of communication richness [@Kirkman.Mathieu2005].
    • Communication richness reflects the dimension of how technology facilitates communication between human actors, and contains two subdimensions. The first element is the virtuality of the communication, and has been the focus of research to date. We suggest a second subdimension to allow for additional mediums of communication.
      • The virtuality of communication reflects how faithfully technology reproduces the experience of talking with other people. On the low end of this dimension, communication is purely textual. Where all communication already requires that we spend effort encoding our thoughts into words (CITE), purely textual communication strips any non-verbal communication abilities and requires that we write out our words.
      • Higher levels of communication virtuality incorporate non-verbal communication elements, mimicking the realism of in-person communication. Whether through the use of audio calls, or video calls, or a realistic video call that places the other in a three-dimensional form in our very presence through the use of augmented reality or holographic image, high levels of virtuality allow for non-verbal communication elements such as vocal tone, facial expression, and body movements.
        • Assuredly, there is some clarity that often comes with writing out our thoughts in textual form for others to read. However, that is a result of a process similar to the one at play when handwritten notes result in a clearer understanding and retention of material than when notes are typed: the extra encoding steps require additional steps that clarify our understanding. The medium of text itself is still a more bare medium than verbalized communication.
    • Communication richness cannot be wholly captured by the virtuality of the medium. There is a second spect of communication richness that has not received attention in the literature: medium for communication, or the incorporation of additional methods or mediums for us to transfer our thoughts to others.
      • Mediums for communication explore how communication goes beyond words and gestures, which are what we use when we have no tools but our voices and bodies. Communication is the process of encoding our thoughts into a medium that others work to decode, ascertaining meaning in their own minds (CITE). This process can utilize other tools.
        • trying different things:
        • Communication is not only about words and gestures...
        • In an analogue format, these can include napkin diagrams during dinner meetings, whiteboard use in a classroom, or visual diagramming during a conference talk...
        • For example, in a classroom or over dinner, we might grab a whiteboard or napkin and sketch out a diagram; not as something to be given over separately from our words, but to be used as a complement to our words, punctuating points on a diagram with explanations. These...
        • For example, in a classroom or over dinner, we might grab a whiteboard or napkin and sketch out a diagram. In a design meeting, we might utilize props or moldable mediums to construct physical aids to accompany our words. These tools are part of the communication process, used as complements to our words, where points of explanation are punctuated with visual or kinesthetic or other experiential elements to aid in encoding our thoughts more clearly.
        • For example, in a classroom or over dinner, we might grab a whiteboard or napkin and sketch out a diagram. In a design meeting, we might utilize props or moldable mediums to construct physical aids to accompany our words. These tools are part of the communication process, used as complements to our words, where points of explanation are punctuated with visual or kinesthetic or other experiential elements to aid in encoding our thoughts more clearly.
        • Mediums for communication can be analogue, or they can be part of the digital technologies we use to communicate. Including a diagram in an email is adding a visual element to our words; even more communicative is using mediums in a compilational manner, using verbal and non-verbal communication together with additional tools, whether by sending a video accompanying the building of a diagram with an explanation, interfacing with colleagues over a large interactive display [e.g., @Mateescu.Pimmer.ea2019], or participating in a live holographic conference call where digital three-dimensional diagrams or physical props can be conjured and manipulated at will.
          • think this is too long an explanation - especially if also want to include, as part of mediums of communication, the inclusion of the context behind our thoughts, and the possibility of _removing_ rather than adding to the encoding process, like sending the direct artifacts of our thinking. Can work to shorten from after "including a diagram in an email..."
      • Mediums for communication are not limited to two and three dimensional diagrams. The ability to include the context that gave birth to our ideas, technological affordances that allow us to send simple messages that are accompanied by access to a knowledge graph of the sources that underly ones thinking, would allow others to explore more deeply any elements in our words that were unclear, or where subject matter familiarity was falsely assumed.
        • Included, too, are methods where the encoding process is skipped entirely, where one does not consider how to put thoughts into words meant purely for communication, but shares the artifacts of their thinking directly. Live synchronous document editing is such a medium, where collaborators working together might not explain their thinking directly, but allow others to see the artifacts of their thinking directly - the writing, deleting, and restructuring that are all part of the process. Of course, skipping the encoding process does not mean that our thoughts are effectively visible, nor that the receiver does not still have to go through the decoding process. But this method of communication is one afforded by digital technologies, and not captured in examining the virtuality of communication alone.
          • Left this out of the version in Tech in Teams March 1
    • Of course, the challenges of face-to-face communication apply to digital communication as well: while we encode our thoughts into words, and try to include any context necessary for understanding, the receiver must go through a process of decoding whereby they hear those words through the lenses of their own experiences, knowledge, and biases.
    • place for this is not here - but had the thought and wanted to get it downWhen we use live synchronous documents to work together, we skip the encoding process where we must consider how to put our thoughts into words meant purely for communication. Others may look directly at the artifact of our thinking. Of course, skipping the encoding process does not always mean that our thoughts are effectively visible, nor that the receiver does not still have to go through the decoding process.
    • Diagramming, drawing pictures or representations for others, are often more effective ways of encoding meaning, particularly when paired with written or spoken words.
    • The common element in all of communication richness is the existence of the thought in the mind of one individual, where the richness of communication affords mediums for the process of conveying that thought to others. There is place for technology to help formulate the thoughts in the first place, but such leverages will be discussed later.

• Artificial Intelligence

  • A second type, or dimension, of technology has captured our imaginations from the earliest days of science fiction: machines that can reason independently of humans, and make novel contributions. We are in the early days of Artificial Intelligence (AI), and machines cannot quite think on their own, but they can and do make physical and intellectual contributions that are not the direct products of queries from the humans they are working with. The intelligence of machines can range on how independent they are from their human counterparts.
    • In considering how machines can range in their intellectual ability, it is useful to consider varying levels of human intelligence. While all humans possess the ability to think and act independently, their ability to do so in a useful way varies considerably. At low levels of intelligence, humans can carry out repetitive tasks that they are taught how to do, and with rising intelligence comes the ability to learn more quickly and to reason independently. While machines and humans differ qualitatively in the way they find solutions to problems, the varying intelligences of humans does provide a useful analogy for understanding the variance in artificial intelligence.
    • Just like humans of simple intelligence, machines with low levels of AI can act in specified ways, carrying out repetitive tasks, usually following a set of if-then rules. Machines of this intelligence have been part of our world for decades: they may call the fire department if there is smoke, triage our emails, or keep a car at a set speed and slow it down when it approaches a car in front.
    • (Need to find that paper on the continuum of AI I came across sometime over the past week or two).
    • At low levels, machines act in specified ways, usually following a set of if-then rules. Machines of this intelligence have been part of our world for decades: they call the fire department if there's smoke, triage our emails, keep the car at a set speed and slow it down if it comes close to a car in front. *
    • At higher levels, machines can learn, taking information from the environment, making predictions about the future, and adjusting future predictions based on the accuracy of past predictions. At higher levels yet, machines can create new ideas. Emerging technologies allow machines to write essays, to to create art and design and unique illustrations. *
    • A current limitation of AI is that it needs to be fed a lot of information to learn. Machines need to see millions of pictures, paintings, and illustrations of an elephant to "understand" what it is. Humans, by contrast, can see just a single elephant, and then recognize a drawing or abstraction of it as an elephant (provided they already have developed nuanced enough schema to recognize it as a species unique from other, similar animals). Should AI develop the ability to extrapolate from limited sets of information, that would be a higher level of AI. And should AI ever reach the ability to evolve on its own, to adjust its own programming (software) and physical makeup (hardware), that might be the highest level of AI. *

• Augmenting Intelligence

  • Original writing here - since supplanted:
    • But there is another use for technology, wherein it functions with the sense of the word as it predates the digital age *. Technologies are tools, affording humans a way to deal with information /complex data, or with physical tasks, or with ideas that are still imprecise and nascent. Technologies as tools give us better ways of doing things, or even enable new capabilities not previously possible. Technologies in this sense have been with us every time we invented a new medium that gave birth to new capabilities. Paper and writing was a technology, giving us an external memory whereby we could store more information than we could memorize, and share it with future others that did not live nearby. Arabic numbers replaced Roman numerals, easing the use of math and paving the way for future mathematicians and their discoveries. Need to find a good third example.
      • update - Seely's pieces of facilitation, expansion, etc are probably not limited to just augmented intelligence. This should fit with AI and communication richness as well. So maybe it's part of how we overlay these 3 dimensions onto the taxonomy of team/mts processes. Of course, these examples are tools that were not merely useful, they were transformative. Most technologies that are useful tools help us improve existing processes, in process facilitation (Seely, 2015). Others are more useful, expanding the boundaries of what is possible and opening up new ways we can work, in process expansion (Seely, 2015). Simultaneous live document editing is one such technology, changing the way a team can draft documents, allowing them to view thinking of the other and providing context to every comment. And a few technologies have the power to be truly transformative. It is difficult to predict what these technologies will look like, and sometimes it is difficult to perceive how an emerging technology might be the next truly transformative tool. Perhaps augmented reality, where we can introduce spatial and visual elements into our world and pin them in places we can return to, is this next tool. But the difficulty of prediction is part of the point: At the highest levels, technology as a tool is not merely useful. At the highest levels, a technology gives rise to an emergent phenomenon, and truly transforms our thought. If we could predict this, if we could articulate the paradigm change we desired, the tool would not truly be transformative, for we would already think that way (Matuschak & Nielsen, 2019).
      • Note - had thought to include a definition here, but it talks about using to accomplish a task - I don't know that I want to do that.
        • (...with the sense of the word as it predates the digital age): technology (noun): a manner of accomplishing a task especially using technical processes, methods, or knowledge (Webster online dictionary).
    • Footnote:
      • The idea that tools and technologies can both be used to refer, in a broad sense, to environmental objects that can increase our sensorimotor or cognitive capacities may come from the following source (cited in @Osiurak.Navarro.ea2018):
        • Osiurak, F., Jarry, C., and Le Gall, D. (2010). Grasping the affordances, understanding the reasoning: toward a dialectical theory of human tool use. Psychol. Rev. 117, 517–540. doi: 10.1037/a0019004
  • But there is another dimension of technology, wherein it functions with the sense of the word as it predates the digital age. Technologies are tools, affording humans a way to deal with complex data, or with physical tasks, or with ideas that are still imprecise and nascent. This third dimension sees computers not as independently intelligent creatures of artificial origin, but as tools crafted by humans, for humans, as leverages of our own rich intelligence [@Engelbart1962].
  • Technologies in this sense have been with us every time we invented a medium that gave birth to new capabilities. Paper and writing was a technology, giving us an external memory whereby we could store more information than we could memorize, and share it with future others. Arabic numbers replaced Roman numerals, easing the use of math and paving the way for future mathematicians and their discoveries. Language itself was a technology, laying the groundwork for the rich social structure of humanity. And computers are both a technology and a medium for new technologies, being a tool in their simplest forms that computed firing tables in World War II, and allowing for the creation of many new tools to augment our intelligence.
  • rewrite_this In examining the pathways through which technology augments our intelligence, there are three key components to consider. First, technology serves as a tool for thought, providing leverage for our intelligences. Second, technology can tap into one or more modalities? types of human intelligence, potentially engaging humans with ideas through more channels than has traditionally been possible. might combine these 1st two into one And third, the ease and richness of the interface between human and computer affects our ability to use it as a tool.
    • Computers as a Tool for thought:
      • Early computers worked with existing paradigms, mimicking existing modes of thinking. Computers wrote linear documents, just as had been written by hand and then by typewriter, albeit computers added richer editing capabilities.
        • Some trace many modern computing schemas to thinking that is still rooted in the printing press, and fault this way of thinking for the limited ways in which computers currently augment our intelligence. (Personal conversations * - need to see if this was an idea they had found/written in the lit somewhere.) - edit, this is more relevant to types of human intelligence
      • But the power of computers as levers for our use is richer than being fancy printing machines. Computers are a tool that allows us to interface with symbols, with representations of ideas, in new ways.
      • Picture a computer taking a complex set of paragraphs, each paragraph representing an atomic thought or idea, that might be explicitly linked to some other related paragraphs. Some of those paragraphs are related hierarchically to other paragraphs, with those higher up more abstract and those farther down, more nuanced. For a computer to allow those paragraphs to be shown in multiple places, where an edit in one place propagates across all instances; to further place those paragraphs onto a two or three dimensional spatial plane that allows for rearranging and playing around; to allow the saving of multiple instances of those spatial planes; to offer visual libraries of drawings or icons to give representational anchors to ideas such that the human sees the icon and recalls the idea without reading; to seamlessly switch back and forth between spatial and more purely textual representations of ideas; to, in short, allow a human to play with and rearrange ideas in looking to compare, contrast, and find patterns across a large repository of thoughts - for a computer to do this is nothing. Nothing, in that it requires little effort on the part of the computer, but nothing too, in that it means nothing to the computer. No additional insight can be obtained by a computer by changing the abstraction, the representation of an idea. But a human that is given the power of such a computer sees a tool that allows for the generation of new ideas, insights that come from multiple representations of thought and the ready recall of information at unprecedented scale. This is the computer functioning as a tool for thought, as a tool that allows for more innovative and insightful human thought. Computers and humans each have powerful and unique sets of capabilities, and the computer as a tool for thought allows for a complementary synergy.
      • A computer can easily take a complex set of information or ideas and display it from multiple perspectives, including or excluding specific sources or types of data, and shift back and forth between any of these perspectives on demand. This is something that is difficult to replicate with analogue tools- and computers can do this at a scale that the pre-computer world could not dream of. Insights drawn from these representations are generated by the human rather than a computer; the computer here serves as a tool rather than as an Artificial Intelligence (to the extent that the computer generates insight, it displays a higher level of the Artificial Intelligence dimension).
      • Computers allow us to manipulate information in ways we could not otherwise do easily, or at all. Computers can present the same information in multiple perspectives, such that the human can examine ideas and data from multiple views, and ask the computer to include or exclude sources of date or ideas. Computers in this way allow for a dynamic manipulation of data and ideas, crafting a medium that could not be represented using analogue methods. New representations of thought allow us to tap into latent capabilities that we already have, where humans rather than computers are the ones generating insight (as opposed to when technology functions as Artificial Intelligence), but it is the representation of ideas or data, through the capabilities of the computer as a tool, that makes this insight possible.
        • Many great ideas in history have been realizations that data could be represented in different ways: maps of the world, algebraic notation, data visualization.
    • Related to the computer as a tool for thought, is the dimension of the types of human intelligence that the computer taps into. just laying out thoughts for now - will make the writing pretty later
      • Picture knowledge work, often picture working over books and paper, or a computer. The medium shapes how we envision it. @Victor2014HumaneRepresentThought
        • Reading, writing, talking about ideas with others - taps into visual, aural modes of understanding.
          • According to Jerome Bruner, taps into symbolic, but not enactive or iconic.
        • If can add ability to diagram in 2D add spatial; for Jerome Bruner, add iconic
        • If can move into 3D space and allow for dynamic manipulation, tap into kinesthetic, maybe even tactile; for Jerome Bruner, maybe add enactive
      • Technology today largely taps into the same types of human intelligence that knowledge work tapped into when we worked with pen and paper. Read and think, put your ideas into words and 2D diagrams, dialogue with others. But human intelligence is broader than this.
        • When we ride a bike, we understand how it works kinesthetically (understanding lives in performing of it), iconically (graphic understanding of how gear ratios work, wordless way of understanding), symbolically (language). need to show an example that is more clearly the realm of knowledge workers but still has non-language representations. Maybe understanding of something in astronomy? Or engineering. Bike could still work, but have to talk from engineering perspective.
        • Bret Victor goes so far as to call this style of knowledge work "inhumane," in that it forces someone to live in a way that severely constrains them - our method of knowledge work limits how we think, and takes away from our full range of intellectual capabilities @Victor2014HumaneRepresentThought.
      • Technology can enable multiple types of intelligence. Spatial representation of thought, possible today. Picture a simple technology that draws upon a visual library to offer icons to anchor to ideas - allows easy adding of visual elements - possible today. Allow humans to explore complex dynamic systems, in 3D, that are accurate and explorable models that draw on real data - maybe not quite possible today, maybe beginnings of this with AR/VR.
    • While the capabilities of the computer as a tool for thought form the foundation for its augmentation of intelligence, equally important is our ability to use that tool - to give instruction to the computer.
      • In early days of computing, all instructions had to be input via punch cards, and visual displays were largely text-based [@Rheingold1985]. Even the addition of some basic graphics required a supplementary screen with a dedicated technology. Communication between human and machine was very rudimentary. With the introduction of successively more sophisticated programming languages, programmers were able to give richer instructions to computers, and with time, create interfaces such that non-programmers could talk to computers in specific ways [@Rheingold1985]. This last step was an important one, as it finally presented some of the power of computing to those who did not program - in other words, to those who did not speak computer. Computers were finally learning to speak our language, rather than we, theirs. Recent advances in natural language recognition have improved further the ability of computers to understand our language, and the field of Brain-Computer Interfacing [BCI; @Wolpaw.Birbaumer.ea2002] is exploring how in the future we might interface more symbiotically with computers and realize early visions of the computer as a symbiotic tool [@Licklider1960a].