In this work, we present progress on the application of contemporary ML techniques to the design process in the architecture, engineering, and construction (AEC) industry. We introduce a technique to partially circumvent the data hungriness of neural networks, which is a significant impediment to their application outside of the ML research community. We also show results on the applicability of this technique to real-world drawings and present research that addresses how some fundamental attributes of drawings as images affect the way they are interpreted in deep neural networks. Our primary contribution is a technique to train a neural network to segment real-world architectural drawings after using only generated pseudodrawings.

The paper contextualizes the approach with respect to the current state of the art of the usage of artificial neural networks both in architecture and beyond. It illustrates the cyber physical system that is at the core of this research, with a focus on the employed neural network–based computational method. Finally, the paper discusses the repercussions of these design tools on the contemporary design paradigm.

CityMatrix was introduced to address these challenges. Machine learning techniques were applied to achieve real-time prediction of multiple urban simulations, and thousands of city configurations were simulated. The simulation results were used to train a convolutional neural network (CNN) to predict the traffic and solar performance of unseen city configurations. The prediction with the CNN is thousands of times faster than the original simulations and maintains a high-quality representation of the results. This machine learning approach was applied as a versatile, quick, accurate, and computationally efficient method not only for real-time feedback, but also for optimized design recommendations. Users involved in the evaluation of this project had a better understanding of the embodied trade-offs of the city and achieved their goals in an efficient manner.