Can LLM be a Construction Engineer?: Interpretable, Knowledge-Infused AI for Construction Automation

Details

Title: Can LLM be a Construction Engineer?: Interpretable, Knowledge-Infused AI for Construction Automation
Speaker: Yoonhwa Jung, PH.D., Assistant Professor, Louisiana State University
Zoom: https://ufl.zoom.us/j/4369404526?omn=92606232997

Digital twin models offer powerful end-to-end project lifecycle management and project control capabilities. However, significant
challenges remain between fragmented project data and actionable knowledge, which limits reliable automated
construction systems. At the core of this transformation is the project schedule—the “brain” of construction engineering that
orchestrates spatial logistics, time, and resources. In this talk, I will present my research on Knowledge-Infused AI
frameworks that embed this core domain intelligence into autonomous workflows. To move beyond recognizing what exists
on a jobsite toward understanding how work is executed and why decisions are made, I introduce novel architectures, including an
implicit schedule checker that decipher the underlying engineering logic for schedule revision; UniformatBridge, a language model that
semantically synchronizes project schedules with 3D design models for automated 4D and digital twin backbone creation ;
ExpertPlanner, a Mixture-of-Experts model that automates granular look-ahead task decomposition for jobsite execution ; and
VisualSiteDiary, a vision-language model for real-time progress tracking and multimodal information retrieval.
I will also discuss multimodal agentic AI systems for seamless human-AI collaboration and the MineCEraft benchmark for
evaluating coordinate-based spatial and engineering reasoning in large language models (LLMs).