Designing a Pictogram-Based Visual Language Tool for Self-Determination

  • Herbert Spencer Auckland University of Technology / PUC-V of Technology
  • Marcos Mortensen Steagall  (Translator) Auckland University of Technology
Keywords: Communication Design, Pictogram-Based, Practice-Based Research, Self-Determination, Visual Language

Abstract

This presentation focuses on a doctoral research project aimed at developing a pictogram-based visual language system to support individuals with complex communication needs (CCN) in Chile and Aotearoa New Zealand. The project seeks to enhance the self-determination and independence of adults with CCN by creating a universally accessible tool that promotes self-expression. The research employs a practice-based methodology, combining theoretical frameworks in augmentative communication with practical design and testing processes. It builds on existing research, such as the PICTOS project in Chile, which has advanced accessibility in public services through the use of pictograms. This project aims to extend such applications into a cohesive visual language system that enables improved decision-making and autonomy for users with CCN. The research design includes co-creation workshops with stakeholders, iterative prototyping, and extensive field testing to ensure that the tool is responsive to the real-world needs and challenges faced by individuals with CCN. Accessible communication methods, such as easy-to-read documents and visual aids, are integrated into the research process to ensure participant comprehension. The project follows a mixed-methods approach, collecting both quantitative and qualitative data to evaluate the tool’s effectiveness, with self-determination measured using standardised instruments alongside interviews, testimonials, and focus groups. Funded by a FONDEF IDEA project grant in Chile and supplemented by scholarships, the research contributes to the fields of augmentative communication and visual language systems, with the ultimate goal of improving communication accessibility for underserved populations in both Chile and Aotearoa New Zealand.

Author Biographies

Herbert Spencer, Auckland University of Technology / PUC-V of Technology

Herbert Spencer is an Interaction Designer and professor at the Pontifical Catholic University of Valparaíso. In 2024, he will begin a practice-based PhD at Auckland University of Technology (AUT). A Fulbright Scholar with an MDes from Carnegie Mellon University, Herbert has over 20 years of teaching experience and has supervised more than 70 theses. His research focuses on using technology to support civic engagement and empower adults with intellectual disabilities, with significant contributions in projects like PiX and PICTOS. He also leads the Masters Program in Architecture and Design (MAD) at his home institution.

Marcos Mortensen Steagall, Auckland University of Technology

Marcos Mortensen Steagall is an Associate Professor in the Department of Communication Design at Auckland University of Technology (AUT). In his research and professional pursuits, Dr. Mortensen Steagall explores the intersection of visual semiotics and practice-oriented methodologies in Art, Design, Communication, and Technology. His artistic practice, primarily centred on lens-based and digital image-making, serves as a method for knowledge production.  Dr. Mortensen Steagall's work is characterised by an interdisciplinary approach that merges academic research with artistic practice, highlighting the significance of embracing diverse cultural narratives and knowledge systems in Design. Additionally, he is the editor of the academic journal LINK Praxis and chairs the LINK International Conference, focusing on Practice-led Research and the Global South.

References

Archer, L. A. (1977). Blissymbolics—A Nonverbal Communication System. https://pubs.asha.org/doi/10.1044/jshd.4204.568

Bliss, C. K. (1949). Semantography, a non-alphabetical symbol writing readable in all languages. Institute for Semantography.

Bondy, A., & Frost, L. (2002). The Picture Exchange Communication System Training Manual (Second). Pyramid Educational Consultants. https://archive.org/details/pecstrainingmanu0000lori

Brydon, S. (2023). Empowering parents to use a core board with children who have complex communication needs: A multiple case study : a thesis presented in fulfilment of the requirements for Doctor of Philosophy in Speech and Language Therapy at Massey University, Albany Campus, Aotearoa New Zealand [Massey University]. https://mro.massey.ac.nz/handle/10179/69251

Bühler, D. (2021). Universal, Intuitive, and Permanent Pictograms: A Human-Centered Design Process Grounded in Embodied Cognition, Semiotics, and Visual Perception. Springer Fachmedien. https://doi.org/10.1007/978-3-658-32310-3

Bühler, D., Hemmert, F., Hurtienne, J., & Petersen, C. (2022). Designing Universal and Intuitive Pictograms (UIPP) – A Detailed Process for More Suitable Visual Representations. International Journal of Human-Computer Studies, 163, 102816. https://doi.org/10.1016/j.ijhcs.2022.102816

Carrasco, E. (2011). Conversaciones con Matta.

Cat, J., & Tuboly, A. T. (Eds.). (2019). Neurath Reconsidered: New Sources and Perspectives (Vol. 336). Springer International Publishing. https://doi.org/10.1007/978-3-030-02128-3

Ćwiek, A., Fuchs, S., Draxler, C., Asu, E. L., Dediu, D., Hiovain, K., Kawahara, S., Koutalidis, S., Krifka, M., Lippus, P., Lupyan, G., Oh, G. E., Paul, J., Petrone, C., Ridouane, R., Reiter, S., Schümchen, N., Szalontai, Á., Ünal-Logacev, Ö., … Winter, B. (2021). The bouba/kiki effect is robust across cultures and writing systems. Philosophical Transactions of the Royal Society B: Biological Sciences, 377(1841), 20200390. https://doi.org/10.1098/rstb.2020.0390

Davis, C. P., Libben, G., & Segalowitz, S. J. (2019). Compounding matters: Event-related potential evidence for early semantic access to compound words. Cognition, 184, 44–52. https://doi.org/10.1016/j.cognition.2018.12.006

Dunne, A. (with Raby, F.). (2013). Speculative Everything: Design, Fiction, and Social Dreaming. MIT Press.

Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023). How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions. Proceedings of the 2023 ACM Designing Interactive Systems Conference, 2263–2280. https://doi.org/10.1145/3563657.3596101

Goddard, C. (2018). Ten Lectures on Natural Semantic MetaLanguage: Exploring language, thought and culture using simple, translatable words. BRILL.

Goddard, C., & Wierzbicka, A. (2014). Words and Meanings: Lexical Semantics Across Domains, Languages, and Cultures. OUP Oxford.

Haseman, B. (2006). A Manifesto for Performative Research. Media International Australia, 118(1), 98–106. https://doi.org/10.1177/1329878X0611800113

Kaptelinin, V., & Nardi, B. (2012). Activity and experience. In V. Kaptelinin & B. Nardi (Eds.), Activity Theory in HCI: Fundamentals and Reflections (pp. 61–72). Springer International Publishing. https://doi.org/10.1007/978-3-031-02196-1_4

Luftig, R. L., & Bersani, H. A. (1985). An Initial Investigation of Translucency, Transparency, and Component Complexity of Blissymbolics. Journal of Childhood Communication Disorders, 8(2), 191–209. https://doi.org/10.1177/152574018500800209

Mäkelä, M., & Nimkulrat, N. (2018). Documentation as a practice-led research tool for reflection on experiential knowledge. FormAkademisk, 11(2), Article 2. https://doi.org/10.7577/formakademisk.1818

Murturi, I., Donta, P. K., & Dustdar, S. (2023). Community AI: Towards Community-based Federated Learning. 2023 IEEE 5th International Conference on Cognitive Machine Intelligence (CogMI), 1–9. https://doi.org/10.1109/CogMI58952.2023.00029

Neurath, O. (1936). International Picture Language. K. Paul, Trench, Trubner & co., ltd.

Olesen Toft, K. (2024). Translating Khora. Derrida Today, 17(1), 82–96. https://doi.org/10.3366/drt.2024.0327

Rimbaud, A. (2010). Una temporada en el infierno, Iluminaciones, Carta del Vidente. CreateSpace Independent Publishing Platform.

Shogren, K. A., Wehmeyer, M. L., & Palmer, S. B. (2017). Causal Agency Theory. In M. L. Wehmeyer, K. A. Shogren, T. D. Little, & S. J. Lopez (Eds.), Development of Self-Determination Through the Life-Course (pp. 55–67). Springer Netherlands. https://doi.org/10.1007/978-94-024-1042-6_5

Spencer González, H., Rodo Iunissi, R., & Pasten Bernales, A. (2022). Pictogramas PICTOS. Visual library, Github repository. https://pictogramas.pictos.cl/

Spencer González, H., Vega Córdova, V., Exss Cid, K., & Álvarez-Aguado, I. (2023). PICTOS es una aplicación para hacer al mundo más inclusivo y accesible [PWA]. PICTOS. https://www.pictos.cl/

Suchman, L. (2006). Human-Machine Reconfigurations: Plans and Situated Actions (2nd ed.). Cambridge University Press; Cambridge Core. https://doi.org/10.1017/CBO9780511808418

Uddin, M. I., & Mashwani, W. K. (2024). Federated Learning: Unlocking the Power of Collaborative Intelligence.

Vaughan, L. (2017). Practice-Based Design Research. Bloomsbury Academic. https://books.google.com/books?id=JKbeDQAAQBAJ

W3C. (2018). Scalable Vector Graphics (SVG) 2 [W3C Candidate Recommendation]. https://www.w3.org/TR/SVG2/

Wakkary, R., & Oogjes, D. (2024). Continuous Present. In R. Wakkary & D. Oogjes (Eds.), The Importance of Speculation in Design Research (pp. 37–54). Springer International Publishing. https://doi.org/10.1007/978-3-031-67095-4_3

Wehmeyer, M. L., & Shogren, K. A. (2017). Applications of the Self-Determination Construct to Disability. In M. L. Wehmeyer, K. A. Shogren, T. D. Little, & S. J. Lopez (Eds.), Development of Self-Determination Through the Life-Course (pp. 111–123). Springer Netherlands. https://doi.org/10.1007/978-94-024-1042-6_9

Wehmeyer, M. L., Shogren, K. A., Little, T. D., & Lopez, S. J. (2017). Introduction to the Self-Determination Construct. In M. L. Wehmeyer, K. A. Shogren, T. D. Little, & S. J. Lopez (Eds.), Development of Self-Determination Through the Life-Course (pp. 3–16). Springer Netherlands. https://doi.org/10.1007/978-94-024-1042-6_1

Wierzbicka, A. (1999). Emotions across Languages and Cultures: Diversity and Universals. Cambridge University Press. https://doi.org/10.1017/CBO9780511521256

Wierzbicka, A. (2003). Cross-cultural pragmatics: The semantics of human interaction (2nd ed). Mouton de Gruyter. http://www.degruyter.com/doi/book/10.1515/9783110220964

Wolfram, S. (2023). Twenty Years of a New Kind of Science. Wolfram Media, Incorporated.

Published
2025-03-25