Archinect's Lexicon focuses on newly invented or adopted vocabulary within the architectural community. For this installment, we're featuring a term highly relevant to the recent wave of generative AI tools released for public use in the AEC sector and beyond.

"Multimodal AI" refers to artificial intelligence systems that are capable of understanding, processing, and integrating data from multiple modes or types of inputs. These systems can process and interpret data from various sources such as text, images, video, voice, or other sensory inputs. Multimodal AI allows for a more nuanced understanding of information as it can leverage the rich context provided by combining multiple data modes.

In the context of architecture and design, Multimodal AI can manifest in many ways, such as: 

• Enhanced Design Processes: Multimodal AI can analyze multiple types of data simultaneously to generate insights. For example, it can process data from images, blueprints, 3D models, and other visual material while also understanding related textual information. It can help architects and designers make more informed decisions by providing data-backed insights.
• Virtual and Augmented Reality: With the ability to understand visual, auditory, and text data, multimodal AI can significantly improve virtual and augmented reality experiences. This can be invaluable for architecture and design, allowing designers to create interactive, virtual models of their designs and enabling clients to explore these models in a highly immersive way.
• User Interaction: Multimodal AI can help develop systems where architects and designers interact with AI in multiple ways — they can speak, sketch, or even use gestures. This could make design software more intuitive and user-friendly.
• Smart Buildings: In the context of built environments, multimodal AI could be used to design intelligent systems within buildings. These systems could process data from various sources — like temperature sensors, security cameras, and user feedback — to optimize everything from energy use to space management.
• Historic Preservation and Urban Planning: Multimodal AI can analyze historical data, satellite images, population statistics, and city plans to aid in historic preservation and informed urban planning.

This article is part of the Archinect In-Depth: Artificial Intelligence series.