e-SPAN Newsletter v025: Materiality & Artificial Intelligence
Artistic Style Robotic Painting is a project that integrates human agents into machine learning and robotic workflows. It was developed by Ardavan Bidgoli and Manuel Rodríguez Ladrón de Guevara, founding members of a research group called Creative AI + Design Launchpad (CRAIDL).
Dear School of Architecture Community,
There has been much press lately on how artificial intelligence (AI) tools will impact and disrupt higher education and industry. The popular launch of OpenAI’s ChatGPT has made the power of these tools accessible to the public, and the response has ranged from awestruck to alarmist.
At the School of Architecture, artificial intelligence is one of our three central pedagogical challenges, along with climate change and social justice.
We recognize the profound effect AI will have on the future of architecture, design, and the built environment. The fusing of the information environment with the built environment is more or less complete: the smart city of driverless cars, responsive buildings, and real-time data-controlled services is here and quickly being adopted by municipalities around the world. Our educational task is to help our students navigate the adoption of these powerful new tools, critique them when they undermine human agency, and oppose them when they become a means to manipulate and control people.
CMU is recognized as one of the birthplaces of AI. Hence, the SoA has a responsibility to be a leader in the adoption of these technologies into architecture. With this in mind, a group of Computational Design doctoral students, Ardavan Bidgoli, Manuel Rodríguez Ladrón de Guevara, Jinmo Rhee and Pedro Veloso created a research group called Creative AI + Design Launchpad (CRAIDL) that uses a variety of AI tools—from natural language processing to reinforcement learning, deep learning, and machine learning—to undertake art, architecture, and design projects. The group also developed and taught a course, Inquiry into Machine Learning and Design, that helped students create their own machine learning-based projects. (Learn more in the profile of Ardavan, below.)
In our graduate programs, the adoption of AI tools for creative and diagnostic purposes is well established; Ji-Hyun Lee, profiled below, has been exploring AI since her graduate student days in the late ‘90s. We are now introducing AI tools into our undergraduate design program’s pedagogies, as well: studios are working with Midjourney- and DALL·E-generated imagery for creative inquiry. It is vital that undergraduate students use the design studios as a means to critically engage these tools and dispel both the alarmist and utopian visions associated with them.
Recently, I had a wonderful virtual meeting with Chinese parents of current undergraduate students regarding the school and its missions. Their concerns about AI are familiar: how does it shape students’ learning? How will it affect employment opportunities for future architects?
In our meeting I shared my educated optimism about the future of AI and architecture, and I’ll do so here, as well. The relational and design thinking that we train architects to do is not under threat by AI. To the contrary, architectural design thinking will become more valued as automation is widely adopted across various industries. The complexity that buildings must engage with—energy, occupancy, life cycles, adaptation—will require both human ingenuity and sophisticated computational tools.
It is with this in mind that we prepare our students to augment their natural creativity with thoughtful use of these tools. Our graduate students in computational design, building performance, and sustainable design will continue creating the tools themselves. Artificial intelligence may be largely unknown territory, but it’s territory into which we as a school are well prepared to lead.
As ever, please stay in touch, and let us know whom we should profile next.
Omar Khan
Head of School
A Leader in AI Education: Ardavan Bidgoli, PhD-CD ‘22
For anyone overwhelmed by the rapid growth of artificial intelligence, fascinated by its design applications, and wondering what comes next, Ardavan Bidgoli (PhD-CD ‘22) has a message: welcome, and buckle up.
Like many architects, Bidgoli began designing with pencil and paper. It was the late 1990s, and he interned at his father’s architecture firm in Gorgan, Iran. As an undergraduate architecture student, he made CAD his primary design interface; in the intervening years, he’s added a number of programming languages and parametric modeling to his design toolkit. Now, he’s working with artificial intelligence—more specifically, machine learning, the fast-changing AI subfield associated with phenomena like ChatGPT.
Even within architecture’s constant flux, he says, the last few years have felt especially turbulent. “I’ve been in machine learning for six years,” he laments, “and I already feel old!” For example: so many new tools became available while he made final dissertation edits last spring, he finally had to content himself with a footnote indicating that his literature review reflected the state of the art as of that date.
Machine learning: a quick introduction
Bidgoli understands that not everyone sees the application of machine learning to architectural design—or even has a confident understanding of what machine learning is.
By way of definition, he distinguishes between machine learning and more traditional computational design. “In classic programming, you design an algorithm to get from inputs to outputs. For example, you might plug a circle’s radius into a formula, and the output is its area. You find the algorithm; then you simply need to translate it into a language the computer understands.
“In machine learning, by contrast, you start with a fairly complex general algorithm that’s not designed to solve any one specific problem. You feed it input data, see what it returns, compare that output with a prepared answer, and use the difference to tweak the algorithm for a slightly better output.
“The machine repeats this process hundreds, thousands, or millions of times, and with each iteration, the algorithm improves. Instead of explicitly telling the computer what to do, you provide it with the experience of looking at enormous data sets, finding patterns. It finds its own, extremely powerful algorithm.”
Bidgoli notes that machine learning is a particularly interesting process for complex, broadly-defined problems like architecture, in which an algorithm needs to translate from one data modality—language, for example—to another, like image. The data sets are huge: millions of descriptions of buildings (input) plus the buildings themselves (output). Full 3D renderings are still in the future, as machine learning models cannot yet associate extremely complex dependencies among design elements. At present, some SoA students are training machine learning models to, for example, predict a building’s thermal behavior or find reusable materials in construction waste.
The result of these models, Bidgoli is careful to point out, is in most cases essentially a rough draft, produced with a fraction of the time and energy necessary for more traditional simulations. The goal of a machine learning algorithm is not to replace human creativity but to speed up the calculation necessary to arrive at an almost-perfect mathematical simulation.
Machine learning for architectural design
Within and outside of design, Bidgoli sees an “avalanche of machine learning achievements completely outpacing anything else in the realm of AI.” Because the field is changing so fast, he urges architects and designers to be nimble and responsive.
Part of that responsiveness is remembering what an algorithm can and can’t do. Just as the existence of Rhino doesn’t preempt the need to express complex ideas through hand-drawing, the power of machine learning algorithms doesn’t obviate the need for human creativity. The algorithms are trained on existing data sets; they haven’t “seen” every idea humans can generate. Therefore, they cannot produce something drastically different than what has been seen before. It’s imperative, then, that a designer consider whether a given task can be cast as a machine-learning problem, and whether it’s useful, practical, and worthwhile to do so.
It is this—the exhilaration of finding out what else AI can do, combined with the centrality of human creativity and expertise—that Bidgoli finds so compelling about machine learning.
He arrived at the SoA in 2016, when the field was just approaching its current peak. By 2020, he and three of his friends in the computational design program, Manuel Rodríguez Ladrón de Guevara (MAAD ‘18, PhD-CD expected ‘23), Jinmo Rhee (MSCD ‘19, PhD-CD expected ‘23), and Pedro Veloso (PhD-CD ‘22), joined forces to form a collaborative platform for better understanding machine learning processes and their application in architectural design.
In the early months of the pandemic, the four founded the Creative AI + Design Launchpad (CRAIDL) research group to collect their research findings—and, crucially, to create a curriculum for other SoA students to explore machine learning in design.
With support from Head Omar Khan and Computational Design’s Ramesh Krishnamurti, Daniel Cardoso Llach and Daragh Byrne, the four PhD students crafted one of the world’s first university courses specifically dedicated to machine learning in architecture and design.
In it, they taught senior undergraduate and graduate students to care for the entire life cycle of a machine learning algorithm, to ensure that young designers understand every phase of the process. “Our students had to be extremely proficient with Python upon arrival, because we didn’t simply give them off-the-shelf tools to play with,” says Bidgoli. “We told them: ‘you’re going to collect your own data set, clean it up, curate it, audit it, mix and match with other students’ data sets, and care for every aspect of it before training an algorithm on it.’ Then they fine-tuned the model and presented the results.”
All of this ensured that their students weren’t mere users of prepackaged algorithmic tools; instead, they understood every aspect of a machine learning project, from the ground up. The pilot class ran for three semesters; its students took those skills into their thesis projects and on into their professional work.
When asked what architects can expect in the future of AI and machine learning, Bidgoli laughs. Given the field’s hair-raising pace of change, he declines to make predictions. To current SoA students, he urges flexibility and a willingness to embrace change. After all, over the course of his career he’s mastered everything from hand-drawing to machine-learning. He knows he hasn’t finished engaging with new interfaces.
When SoA students graduate with technical skill “a step ahead of everyone else,” he says, they’re well placed to be thoughtful decision-makers about which tools to embrace and how to use them well. Human expertise, says Bidgoli, is “still our best source of knowledge and skill.”
Sequencing Memes into Cultural DNA: Ji-Hyun Lee, PhD-CD ‘02
Twenty years before Ardavan Bidgoli arrived at the SoA’s PhD program in Computational Design (CD), Ji-Hyun Lee (PhD-CD ‘02) found herself in Hamburg Hall, sharing learning space with other CD students, as well as students in electrical and computer engineering, mechanical engineering, and computer science.
She credits that interdisciplinary experience with the flexibility that defines her scholarship and teaching at the Korea Advanced Institute of Science and Technology (KAIST), where she is a professor in the Graduate School of Culture Technology.
There, she studies the role of AI in architectural design from a wide range of perspectives.
As a member of Korea’s National Research Foundation, she spent years assessing the country’s humanities and social science education system, considering how liberal arts graduates might be empowered to collaborate with science graduates in the creation of ethical guidelines for AI.
As a teacher of graduate students, she helps young designers accept their role in “making the computer smart” by fully understanding the technology they employ. She teaches them to see sophisticated data analysis as the beginning, not the end, of a thoughtful design process. “What does your analysis mean,” she asks them, “for design? For space? For the city? For architecture?”
As director of KAIST’s Information-Based Design Lab, she combines cognitive science, design theory, and mathematics to extend beyond computational design and into “computational culture.”
And as a scholar, she takes inspiration from her school’s name. “‘Culture’ is bigger than architecture, and ‘technology’ is bigger than computation,” she says. “I’m always considering how I can merge the two.” Recently, that endeavor has taken the form of a project she calls “Cultural DNA.” At conferences hosted by Lee, she and computational design colleagues from all over the world use shape grammars and other computational tools to map individual elements of national cultures, seeking patterns in cultural creation and dissemination. The project provocatively situates a meme as a gene. The next Cultural DNA conference will investigate the Metaverse.
Space Information-Based Virtual Environment testbed for Metaverse GIS Design, as part of KAIST-IBD’s Cultural DNA conference (courtesy of Ji-Hyun Lee)
Space Information-Based Virtual Environment testbed for Metaverse GIS Design, as part of KAIST-IBD’s Cultural DNA conference (courtesy of Ji-Hyun Lee)
Student & Faculty Research at the SoA: CRAIDL founding members share their work
DeepCloud (above), developed by Ardavan Bidgoli and Pedro Veloso, is a data-driven modeling system that enables the user to quickly generate innovative and unexpected objects of any class in its database – such as cars, chairs, tables, and hats. It learns the common characteristics of these classes and enables the user to manipulate them in a meaningful way. It depicts a symbiotic design future where designers can curate their own design culture on the cloud, while an artificial intelligence system defines the appropriate design space and operations accordingly.
Two distinct building configurations generated by 96 spatial agents in real-time. Project developed by Pedro Veloso as part of his PhD research at CMU under supervision of Prof. Ramesh Krishnamurti.
Axonometric models of a row house designed by students with support of interactive deep learning workflow as part of the course “48-770: Inquiry into Machine Learning and Design” (Credit: CRAIDL)
Design intents prediction, developed by Manuel Rodríguez Ladrón de Guevara, is a deep learning approach to predict design intents from images.
Design intents visual grounding, developed by Manuel Rodríguez Ladrón de Guevara, is a multimodal approach to ground vague design adjectives in images.
Attention-based painting is an attention-based optimization method to paint in different styles (Manuel Rodriguez Ladron de Guevara).
Artistic Style Robotic Painting, developed by Ardavan Bidgoli and Manuel Rodríguez Ladrón de Guevara, is a project that integrates human agents into machine learning and robotic workflows. The project proposed a method to integrate an artistic style into brushstrokes and the painting process through collaboration with a human artist. The approach consisted of three pillars: 1) collecting brushstrokes and hand-brush motion samples from an artist, 2) training a generative model to generate brushstrokes that pertain to the artist's style, and 3) integrating the learned model on a robot arm to paint on a canvas.
SecondHand is a case study on “A Collaborative Framework for Machine Learning-Based Toolmaking for Creative Practices” developed by Ardavan Bidgoli. In this study, a group of participants utilized various mediums to collect their own handwriting samples and trained a machine learning model to generate a bespoke handwriting typeface. Instead of the standard practice of using off-the-shelf databases and pure quantitative metrics to train and evaluate their models, participants relied on their own data, subjective measures, and personal preferences to make their tools.
ThirdHand is a case study on “A Collaborative Framework for Machine Learning-Based Toolmaking for Creative Practices” developed by Ardavan Bidgoli. This study proposes a framework for making robotic musical instruments to augment an artist’s capability to play santur, a hammered dulcimer of Iranian origins. It is not destined to replace the artist, but it is an effort to explore the affordances of machine learning for musical toolmaking. This project utilizes a dataset of six-degree-of-freedom motions provided by the musician as a vehicle to convey the specific idiom of the artist.
Alumni News & Updates
We invite all Carnegie Mellon Architecture alumni to keep us up to date on their awards, professional milestones and more. Send us your updates with a brief description and link to more information.