The Semiotic Engineering of Human-Computer Interaction

From Acting with Technology

The Semiotic Engineering of Human-Computer Interaction

By Clarisse Sieckenius de Souza

A theory of HCI that uses concepts from semiotics and computer science to focus on the communication between designers and users during interaction.





A theory of HCI that uses concepts from semiotics and computer science to focus on the communication between designers and users during interaction.

In The Semiotic Engineering of Human-Computer Interaction, Clarisse Sieckenius de Souza proposes an account of HCI that draws on concepts from semiotics and computer science to investigate the relationship between user and designer. Semiotics is the study of signs, and the essence of semiotic engineering is the communication between designers and users at interaction time; designers must somehow be present in the interface to tell users how to use the signs that make up a system or program. This approach, which builds on—but goes further than—the currently dominant user-centered approach, allows designers to communicate their overall vision and therefore helps users understand designs—rather than simply which icon to click.

According to de Souza's account, both designers and users are interlocutors in an overall communication process that takes place through an interface of words, graphics, and behavior. Designers must tell users what they mean by the artifact they have created, and users must understand and respond to what they are being told. By coupling semiotic theory and engineering, de Souza's approach to HCI design encompasses the principles, the materials, the processes, and the possibilities for producing meaningful interactive computer system discourse and achieves a broader perspective than cognitive, ethnographic, or ergonomic approaches.

De Souza begins with a theoretical overview and detailed exposition of the semiotic engineering account of HCI. She then shows how this approach can be applied specifically to HCI evaluation and design of online help systems, customization and end-user programming, and multiuser applications. Finally, she reflects on the potential and opportunities for research in semiotic engineering.


Out of Print ISBN: 9780262042208 312 pp. | 7 in x 9 in 61 illus.


$45.00 X ISBN: 9780262527095 312 pp. | 7 in x 9 in 61 illus.


  • By using new terminology to recast received HCI wisdom, de Souza suddenly expands our horizon with innovative and promising conceptual tools. She is mercifully sparing with semiotic jargon and her choice of examples is compelling. Enjoyable, stimulating, and unexpected.

    Thomas Green

    Department of Computer Science, University of Leeds

  • Semiotics has a reputation for being difficult, abstract, and obtuse. This is most unfortunate, for as Clarisse de Souza illustrates in this powerful book, semiotics offers a compelling and novel viewpoint. It is common to think of interaction between a person and technology as communication with the technology. De Souza shows that the real communication is between designer and person, where the technology is the medium. Once designs are thought of as shared communication and technologies as media, the entire design philosophy changes radically, but in a positive and constructive way. This book has changed my thinking: read it and change yours.

    Don Norman

    Nielsen Norman Group, author of Emotional Design

  • De Souza has brought an important new dimension to our understanding of the design of human-computer interaction. Through clear examples and discussion of the principles, she shows how semiotics offers new ways to think about the relationship between designer and user, ideas we can use to create computer systems that communicate better and therefore work better. Although much of the literature on semiotics has been abstract and abstruse, de Souza is able to explain the theory clearly and highlight its many links to HCI. Her clear focus on software as a medium for communication can have a profound effect on the way we understand and design the software and systems that permeate our lives.

    Terry Winograd

    TerryProfessor of Computer Science, Stanford University