Intelligent Soft Wearables

UIST 2024 Workshop - CMU (Pittsburgh), Oct 13

Cedric Honnet (MIT), Catherine Yu (Cornell), Irmandy Wicaksono (MIT), Tingyu Cheng (Georgia Tech),
Andreea Danielescu (Accenture Labs), Cheng Zhang (Cornell), Stefanie Mueller (MIT), Joe Paradiso (MIT), Yiyue Luo (MIT-UW).

banner Illustrating the breadth of wearable research, from materials science (1), to energy harvesting (2), haptics for spacesuits (3), sustainable devices (4), to knitted neural networks (5).

Workshop location: CMU Gates Hillman Center GHC 4303

Workshop participation:

Contact: SoftWearables(AT)mit.edu

News 📢

1) The schedule is up!

(more details below)

Time Duration Event
8:00 AM 1 h 00 Registrations start
9:00 AM 0 h 15 Welcome and Introductions
9:15 AM 1 h 00 Keynotes by Scott Hudson and Carmel Madjidi
10:15 AM 0 h 30 Coffee Break
10:45 AM 1 h 15 Panel Discussion
12:00 PM 1 h 30 Lunch Break
1:30 PM 1 h 30 Spotlight and Demos
3:00 PM 0 h 30 Coffee Break
3:30 PM 1 h 00 Group Discussion on Challenges & Opportunities
4:30 PM 0 h 30 Closing Session

2) Keynote Abstracts

“Soft is Hard: Some thoughts on organizing the challenges, and starting on the work”

Abstract: Fabricating things which are soft is a very important goal for the HCI component of the computational fabrication community. Soft objects have critically important attributes when considered for human use. Soft material – textiles in particular – are the only material which we keep in contact with our bodies 99% of the day. And there are good reasons why this is true. But while there is a very long history of humans creating with these materials, they can be challenging to use in a computational fabrication context. For example, they can be unruly to handle in a fabrication machine, and hard to model or simulate, compared to rigid materials. In this talk, I will first try to give some structure to the several different types of challenges we need to address in this area, and then turn to just a few specific exemplar projects I and my collaborators have undertaken within that structure.

“Energy Harvesting for Battery-Free Soft Wearables”

Abstract: Over the past decade, there’s been tremendous advancements in soft and highly stretchable circuitry for use in epidermal electronics for health monitoring, wearable computing, and soft robotics. As these technologies continue to improve, there is increasing interest in new material architectures for energy harvesting and battery-free operation. Several promising approaches leverage various sources of energy that are created by the body through repeated surface contact, kinetic motion, or heat. As with the sensors and circuits that they power, these energy harvesting transducers should be soft and stretchable so that they conform to the human body without causing discomfort or impairing natural motion. In this talk, I will present several methods for achieving fully battery-free functionality of soft wearable devices that rely on various energy harvesting modalities. Specifically, I will show how liquid metal alloys like eutectic gallium indium (EGaIn) can be used to produce soft material architectures that can convert body heat or limb motion into electricity using thermoelectric or triboelectric coupling, respectively. To demonstrate the practicality of these battery-free wearables, I will show how they can be used to power a variety of body-mounted electronic devices for temperature sensing, humidity sensing, pulse oximetry, signal processing, and wireless radio communication. As time permits, I will also show how these EGaIn-based architectures can be extended to other applications related to soft robotics and intelligent soft materials.

Details about the workshop

Keywords

Wearables, Applied Machine Learning, e-Textiles, Personal Fabrication, Sensing, Actuation, Materials Sciences, Energy Harvesting.

Abstract

Wearables have long been integral to human culture and daily life. Recent advances in intelligent soft wearables have dramatically transformed how we interact with the world, enhancing our health, productivity, and overall well-being. These innovations, combining advanced sensor design, fabrication, and computational power, offer unprecedented opportunities for monitoring, assistance, and augmentation. However, the benefits of these advancements are not yet universally accessible. Economic and technical barriers often limit the reach of these technologies to domain-specific experts. There is a growing need for democratizing intelligent wearables that are scalable, seamlessly integrated, customized, and adaptive. By bringing researchers from relevant disciplines together, this workshop aims to identify the challenges and investigate opportunities for democratizing intelligent soft wearables within the HCI community via interactive demos, invited keynotes, and focused panel discussions.

Introduction

Recent developments in materials science, digital fabrication, and applied machine learning have enabled significant advances in intelligent soft wearables. At the intersection of wearable technology and applied artificial intelligence, these wearables seamlessly integrate functionalities with the human body, serving as conformal sensing and actuation interfaces, and unobtrusively extract information and generate optimal feedback combined with advanced computational techniques. Some research efforts have been prioritizing functionality over conformability, either for sensing [2, 7, 10, 24] or actuation [4, 22, 25], but we propose to further investigate soft interfaces that conform to the body. They have already served as interactive interfaces [1, 8, 19, 23, 28], behavior monitoring and learning systems [13, 27, 29, 30], assistive wearables [5, 11, 14], human augmentation [9, 12, 18, 21], and entertainment [3, 6, 16, 20, 26], demonstrating great potential in HCI, healthcare, AR/VR, robotics, and space, among other applications.

However, despite their significant potential, most current intelligent wearables are still localized to specific parts of the body, function for only short periods, and are tailored to specific users and scenarios (as in ElectroDermis [15] or SmartSleeve[17]). Additionally, the design, fabrication, and application of intelligent soft wearables generally require domain-specific knowledge and interdisciplinary integration.

In this workshop, we aim to analyze the challenges and explore the opportunities in the democratization of such intelligent soft wearables, enabling them to be scalable, robust, customizable, and adaptive. The structure of the workshop is designed to maximize engagement through a combination of keynotes, interactive demos, group discussions, and expert panels.

Scope: Goals & Topics

The democratization of intelligent soft wearables requires interdisciplinary efforts. In this workshop, we focus on the following research questions:

The goal of our workshop includes:

Topics of interest for the workshop include, for example:

Schedule Details

Opening Session (15 min)

The workshop will commence with a brief welcome session to introduce the agenda and objectives for the day.

Keynotes (1 hour)

In the morning, we plan to invite 2 keynote speakers in the field to present for 30 minutes. They will talk about insights into the latest advancements and future trends in intelligent soft wearables. The keynotes will provide some background information for interactive discussion and panels in the afternoon.

Coffee break (30 min)

Panel Discussion (1 hour 15 min)

To prepare for a group discussion on challenges and opportunities of intelligent wearables, organizers will invite a panel of 3-5 experts in the fields. During the panel session, participants will be encouraged to engage actively with the panelists. They will have the opportunity to explore in-depth the challenges and opportunities previously identified during group discussions. Organizers will moderate this dialogue by collecting questions from participants in advance through Slido, ensuring that the session addresses the most pressing issues and interests of the audience. By leveraging the expertise of the panelists and the input from attendees, the session aims to generate valuable discussions and actionable strategies for advancing the field of intelligent wearables.

Lunch break (1 hour 30 min)

Spotlight and Demo Session (1 hour 30 min)

To prepare for this session, we will invite submissions of interactive demos from a diverse HCI community. During the session, participants will briefly introduce themselves, present their interests, and showcase their demo through a spotlight/lightning talk. After that, participants will be divided into subgroups for interactive demos. During these subgroup demo sessions, participants will present and demo their hardware or software prototypes, discuss the design process, and share any insights that arose during development. General attendees are encouraged to join any subgroup session to engage in discussion and explore the showcased demos. This format aims to promote collaboration, inspire new ideas, and facilitate networking among researchers, designers, and practitioners in the HCI field.

Coffee break (30 min)

Group Discussion on Challenges and Opportunities (1 hour)

Participants will be divided into subgroups and engage in a collaborative discussion to identify and analyze the major challenges and opportunities in democratizing intelligent wearables. This session will facilitate the exchange of ideas and experiences from various perspectives. Organizers will provide example discussion topics and research questions that touch upon critical aspects of democratization. Building on the previous keynotes, this session will then focus on exploring opportunities and potential solutions in the design, fabrication, and computational aspects of intelligent wearables.

Closing Session (30 min)

The workshop will conclude with a summary of key takeaways and a discussion of the next steps for fostering ongoing collaboration and innovation in the field of intelligent soft wearables.

Call for Participation

We invite participants and interactive demos from the UIST community who are experienced in or interested in the democratization of intelligent soft wearables, from design and fabrication of various applications. We particularly welcome researchers, designers, and practitioners in digital fabrication, wearables, toolkit development, embedded AI, computational sensing, and personalized modeling. We will advertise for participation not only within the UIST community but also within human-robot interaction and other AI venues.

References

[1] Laura Devendorf, Joanne Lo, Noura Howell, Jung Lin Lee, Nan-Wei Gong, M Emre Karagozler, Shiho Fukuhara, Ivan Poupyrev, Eric Paulos, and Kimiko Ryokai. 2016. “ I don’t Want to Wear a Screen” Probing Perceptions of and Possibilities for Dynamic Displays on Clothing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 6028–6039. 

[2] Nathan Devrio and Chris Harrison. 2022. DiscoBand: Multiview Depth-Sensing Smartwatch Strap for Hand, Body and Environment Tracking. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–13. 

[3] Maurin Donneaud, Cedric Honnet, and Paul Strohmeier. 2017. Designing a Multi-Touch eTextile for Music Performances. In Proceedings of the International Conference on New Interfaces for Musical Expression. 

[4] Zhu et al.2024. PortaChrome: A Portable Contact Light Source for Integrated Re-Programmable Multi-Color Textures. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 

[5] Jack Forman, Ozgun Kilic-Afsar, Sarah Nicita, Rosalie Hsin-Ju Lin, Liu Yang, Megan Hofmann, Akshay Kothakonda, Zachary Gordon, Cedric Honnet, Kristen Dorsey, Neil Gershenfeld, and Hiroshi Ishii. 2023. FibeRobo: Fabricating 4D Fiber Interfaces by Continuous Drawing of Temperature Tunable Liquid Crystal Elastomers. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(UIST ’23). Association for Computing Machinery.

[6] Rachel Freire, Cedric Honnet, and Paul Strohmeier. 2017. Second Skin: An Exploration of eTextile Stretch Circuits on the Body. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction. 653–658. 

[7] Cédric Honnet and Gonçalo Lopes. 2019. HiveTracker: 3D positioning for ubiquitous embedded systems. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 288–291. 

[8] Cedric Honnet, Hannah Perner-Wilson, Marc Teyssier, Bruno Fruchard, Jürgen Steimle, Ana C. Baptista, and Paul Strohmeier. 2020. PolySense: Augmenting Textiles with Electrical Functionality Using In-Situ Polymerization. Association for Computing Machinery, New York, NY, USA, 1–13.

[9] Ozgun Kilic Afsar, Ali Shtarbanov, Hila Mor, Ken Nakagaki, Jack Forman, Karen Modrei, Seung Hee Jeong, Klas Hjort, Kristina Höök, and Hiroshi Ishii. 2021. OmniFiber: Integrated fluidic fiber actuators for weaving movement based interactions into the ‘fabric of everyday life’. In The 34th Annual ACM Symposium on User Interface Software and Technology. 1010–1026. 

[10] Daehwa Kim and Chris Harrison. 2022. EtherPose: Continuous hand pose tracking with wrist-worn antenna impedance characteristic sensing. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–12. 

[11] Jin Hee Kim, Joan Stilling, Michael O’Dell, and Cindy Hsin-Liu Kao. 2023. Knitdema: robotic textile as personalized edema mobilization device. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19. 

[12] Pedro Lopes, Alexandra Ion, and Patrick Baudisch. 2015. Impacto: Simulating physical impact by combining tactile stimulation with electrical muscle stimulation. In Proceedings of the 28th annual ACM symposium on user interface software & technology. 11–19. 

[13] Yiyue Luo, Yunzhu Li, Pratyusha Sharma, Wan Shou, Kui Wu, Michael Foshey, Beichen Li, Tomás Palacios, Antonio Torralba, and Wojciech Matusik. 2021. Learning human–environment interactions using conformal tactile textiles. Nature Electronics 4, 3 (2021), 193–201. 

[14] Yiyue Luo, Kui Wu, Andrew Spielberg, Michael Foshey, Daniela Rus, Tomás Palacios, and Wojciech Matusik. 2022. Digital fabrication of pneumatic actuators with integrated sensing by machine knitting. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–13. 

[15] Eric Markvicka, Guanyun Wang, Yi-Chin Lee, Gierad Laput, Carmel Majidi, and Lining Yao. 2019. Electrodermis: Fully untethered, stretchable, and highly-customizable electronic bandages. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–10. 

[16] Alex Olwal, Jon Moeller, Greg Priest-Dorman, Thad Starner, and Ben Carroll. 2018. I/O Braid: Scalable touch-sensitive lighted cords using spiraling, repeating sensing textiles and fiber optics. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 485–497. 

[17] Patrick Parzer, Adwait Sharma, Anita Vogl, Jürgen Steimle, Alex Olwal, and Michael Haller. 2017. SmartSleeve: real-time sensing of surface and deformation gestures on flexible, interactive textiles, using a hybrid gesture detection pipeline. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 565–577. 

[18] Syamantak Payra, Irmandy Wicaksono, Juliana Cherston, Cedric Honnet, Valentina Sumini, and Joseph A Paradiso. 2021. Feeling through spacesuits: Application of space-resilient E-Textiles to enable haptic feedback on pressurized extravehicular suits. In 2021 IEEE Aerospace Conference (50100). IEEE, 1–12. 

[19] Ivan Poupyrev, Nan-Wei Gong, Shiho Fukuhara, Mustafa Emre Karagozler, Carsten Schwesig, and Karen E Robinson. 2016. Project Jacquard: interactive digital textiles at scale. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 4216–4227. 

[20] Thomas Preindl, Cedric Honnet, Andreas Pointner, Roland Aigner, Joseph A Paradiso, and Michael Haller. 2020. Sonoflex: Embroidered speakers without permanent magnets. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 675–685. 

[21] Jun Rekimoto. 2001. Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In Proceedings Fifth International Symposium on Wearable Computers. IEEE, 21–27. 

[22] Vivian Shen, Tucker Rae-Grant, Joe Mullenbach, Chris Harrison, and Craig Shultz. 2023. Fluid reality: High-resolution, untethered haptic gloves using electroosmotic pump arrays. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–20. 

[23] Zilin Si, Tianhong Catherine Yu, Katrene Morozov, James McCann, and Wenzhen Yuan. 2023. RobotSweater: scalable, generalizable, and customizable machine-knitted tactile skins for robots. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 10352–10358. 

[24] Paul Strohmeier, Cedric Honnet, and Samppa Von Cyborg. 2016. Developing an ecosystem for interactive electronic implants. In Biomimetic and Biohybrid Systems: 5th International Conference, Living Machines 2016, Edinburgh, UK, July 19-22, 2016. Proceedings 5. Springer, 518–525. 

[25] Gabriela Vega, Valentin Martinez-Missir, Dennis Wittchen, Nihar Sabnis, Audrey Girouard, Karen Anne Cochrane, and Paul Strohmeier. 2024. vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–20. 

[26] Nicolas Villar, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, William Field, and Haiyan Zhang. 2018. Project zanzibar: A portable and flexible tangible interaction platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13. 

[27] Irmandy Wicaksono, Peter G Hwang, Samir Droubi, Franny Xi Wu, Allison N Serio, Wei Yan, and Joseph A Paradiso. 2022. 3dknits: Three-dimensional digital knitting of intelligent textile sensor for activity recognition and biomechanical monitoring. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2403–2409. 

[28] Irmandy Wicaksono and Joseph A Paradiso. 2017. Fabrickeyboard: multimodal textile sensate media as an expressive and deformable musical interface.. In NIME, Vol. 17. 348–353. 

[29] Tianhong Catherine Yu, Riku Arakawa, James McCann, and Mayank Goel. 2023. uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. 

[30] Bo Zhou, Daniel Geissler, Marc Faulhaber, Clara Elisabeth Gleiss, Esther Friederike Zahn, Lala Shakti Swarup Ray, David Gamarra, Vitor Fortes Rey, Sungho Suh, Sizhen Bian, et al. 2023. Mocapose: Motion capturing with textile-integrated capacitive sensors in loose-fitting smart garments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 1 (2023), 1–40.

Cite the paper (PDF):

Cedric Honnet, Catherine Yu, Irmandy Wicaksono, Tingyu Cheng, Andreea Danielescu, Cheng Zhang, Stefanie Mueller, Joe Paradiso, Yiyue Luo.
"Democratizing Intelligent Soft Wearables."
In Proceedings of UIST 2024, https://doi.org/10.1145/3672539.3686707

LaTex:

@inproceedings{UIST24_SoftWearables,
  title={Democratizing Intelligent Soft Wearables},
  author={Honnet, Cedric and Yu, Catherine and Wicaksono, Irmandy and Cheng, Tingyu and Danielescu, Andreea and Zhang, Cheng and Mueller, Stefanie and Paradiso, Joe and Luo, Yiyue},
  booktitle={In Proceedings of UIST 2024},
  year={2024},
  url={https://doi.org/10.1145/3672539.3686707},
}
This site is open source. Improve this page.