PhD-CD Dissertation Defense Presentation: Jingyang Liu

Title: Spatial Interfaces: A Computational Framework for Spatial Computing in Robotically-Assisted Construction 

Name: Jingyang Liu, PhD Candidate in Computational Design

Date: Tuesday, May 21, 2024
Time: 10:00am-12:15pm ET
Location: Intelligent Workplace (IW) Conference Room, MM 415 

Dissertation Committee:

Dr. Daniel Cardoso Llach
Associate Professor
School of Architecture
Carnegie Mellon University

Prof. Josh Bard
Associate Professor
School of Architecture
Carnegie Mellon University

Dr. Mayank Goel
Associate Professor
Software and Societal Systems Department (S3D) and Human-Computer Interaction Institute
Carnegie Mellon University

Abstract:
Since the early 1970s, many studies have explored integrating robotic systems into architecture, engineering, and construction (AEC) to address challenges like low productivity, worker shortages, and safety concerns. Efforts have aimed to automate construction. However, complexities such as job site environments, worker safety, and economic factors limit fully unmanned systems in AEC. The increasing trend of robotically-assisted practices underscores the need for an effective human-machine interface for robotic manipulation.

Recent advancements in spatial interfaces (spatial interface refers to a set of immersive technologies, such as augmented reality (AR), where users can interact with virtual information in three-dimensional space), offer a promising direction for Human-Robot Interaction (HRI) in construction. This dissertation introduces an AR for HRI framework tailored for robotic-assisted construction, empowering operators to leverage physical space as a shared canvas for information visualization, robot supervision, and manipulation. The framework optimizes components such as indoor localization and point cloud processing for AEC’s unique demands. For instance, a multimodal localization approach combining visual simultaneous localization and mapping (vSLAM) with Ultra-Wideband (UWB) signals enhances robustness under complex lighting conditions.

Building on this framework, bespoke spatial interfaces are devised and tested in two use cases: proxemic-aware AR for HRI and robotic-assisted quality assessment and management. Case studies illustrate how AR interfaces facilitate robotic manipulation in construction contexts.

This dissertation contributes a novel AR-enabled HRI framework for robotic-assisted construction, initiating broader discussions on spatial interfaces’ role in the built environment’s future.

More information.