An AI Foundation Model for Contextual Interpretation of Radar Data

In this talk, we present a physical AI foundation model that can fuse real-time radar data with other sensor modalities and natural language for contextual understanding of human activity. Context is an essential component for interpreting human behavior because the meaning of physical activity is highly dependent on the circumstances, environment, and background in which the activity takes place. Our foundation model combines radar data with other contextual inputs in a unified embedding space, allowing deep fusion and reasoning about real-time activity and intent. We discuss the main components and architecture of the model that enable encoding radar data along with other modalities, then decoding from latent representations into semantic insights about the sensor data. By fusing with natural language, the model can produce responses to open-ended queries, generating rich contextual descriptions of human behavior across a broad variety of environments and scenarios. We show several applications and demos of the model for real-world use cases. We close with an outlook on directions for developing and applying AI for RF-based sensing technologies.