History Repeats Itself

If you were a designer of physical things like buildings or cars or bridges in December 1982, you would have been forgiven for thinking that computer-aided design was a novelty. It probably seemed like an abstract technical exercise that would never be able to supplant the maylines, rotring pens, and tilt-y tables that were stalwart in design offices. There was simply too much material complexity in this endeavor, it requires too much human creativity and too much mental plasticity to be all nicely packaged up into an IBM 5150.

More than three decades later, CAD has not supplanted human creativity. What it did was solve a problem that wasn’t evident at the time. Cabinets full of spec sheets, flat files of past design solutions — these things disappeared into magnetic memory where they lost their mass and became searchable. CAD reduced the amount of energy that needed to be expended to resolve design’s nebulous conception into the ordered structures and specifications used to describe changes to the physical world. There was some conceptual space that existed between the inception of a project and opening day; CAD let us explore that space faster and develop more complex solutions.

Reading tech news on a pocket computer in 2019, with 200,000 times the storage of an IBM 5150, there’s an awful lot of noise around what is known variously as deep learning, machine learning, neural networks, or (worst of all) artificial intelligence. Largely, it seems like a novelty. The kind of thing that is an abstract technical exercise with potential in robotics or playing board games or making autocorrect actually work, for once. AI could never design a car or a bridge or a building...

A Space of Spaces

The work of work in architecture, engineering, and construction, the value that it adds, is the ability to see the future. As architects, engineers, and designers, we are responsible for planning the confluence of a huge number of flows of material, people, and information. Currently, our ability to see the future is based on modeling. We make models of the material products of these confluences — buildings, infrastructure, renovations — in order to map out the topography of the space and time of decisions that lie between the state of the world right now and our intended destination. A successful operator in the work of A/E/C is one who is capable of charting a course through the space between now and that point in the future. The tools with which we will soon navigate this space are known, collectively, as machine learning.

We work with our clients and collaborators to structure, to the best of our abilities, the forking paths of various decisions between the Prow of Now and the Island of Intent, the latent space of all possible timelines that could take us from where we are to where we would like to be. Any given step along any of these paths describes the same project as the product of different choices. The idea of enumerating these possibilities seems ridiculous, ostensibly. But there is a building out there, somewhere in the space of all possible buildings, that has all the qualities that suits our client’s vision. This has to be true or there would be no point in continuing our work. But the space of all possible buildings is very different from the space of all achievable buildings. The straightest, most achievable line is probably something like buying one of those prefab steel sheds that will cover the necessary area and just zip-tying all of the electrical and telecom equipment to the rafters. And, in some cases, that might be the solution. On the other hand, one of these possible timelines, way off to one side, describes a building that is made of solid gold. Another, very far away from that, has a structure completely composed of carbon composites which weighs so little it can be moved by half a dozen ordinary people but can support a tractor trailer.

A successful operator in the work of A/E/C is one who is capable of charting a course through the space between now and that point in the future. The tools with which we will soon navigate this space are known, collectively, as machine learning.

However, most clients are looking for a solution somewhere between 23rd century carbon nanotube construction and a prefabricated steel shed with those yellow work lights dangling from the frame. Thus, in the space of all possible solutions, we are looking for a kind of geodesic that leads us from Now to Then, not in the Valley of Banality, and not too high on the Slopes of the Ridiculous.

There is some space of known time steps that lead directly to where we would like to go, where we know that we will eventually arrive. There is also some space of steps that leads us away from the immediate present. Graphing the number of unknowns at stages of a project does not increase linearly with time, but rather has its maxima somewhere between now and then.

Even more valuable is the ability to map out how the choices you make between these points affects the landscape of possible events that can exist after a project is complete. After all, the lifetime of a project is typically ten to 50 times longer than its development. Because time is short, you don’t want to spend your time budget exploring the massive combinatorial space of possible solutions, otherwise known as Schematic Development. A good consultant today will bring experience and expertise to the beginning stages of a project when the ability to affect the lifetime of the project is the highest.

In contrast to the typical rules-of-thumb, the tradeoffs in machine learning will not be simply “Good, Fast, or Cheap. Pick any Two.” Rather, engineers and designers will be responsible for the creation of a long vector of design parameters and a weighting scheme for the ones that are most important.

The real benefit of machine learning will be the ability to reach forward into our conceptual space and pull back decisions and consequences to the present stage of a project, back to when there is still opportunity to greatly affect its lifetime and before its monetary and material budget has been expended.

An expert consultant will be able to enumerate the possible decisions given some budget, timeline, scope, etc. and pull them out of the future chain of events into the present to be evaluated for their efficacy.

The real benefit of machine learning will be the ability to reach forward into our conceptual space and pull back decisions and consequences to the present stage of a project, back to when there is still opportunity to greatly affect its lifetime and before its monetary and material budget has been expended.

Why Now?

All of this is to say that machine learning solves a problem that we haven’t known that we’ve had. In the same way that CAD (and its offspring, BIM) made complexity cheap by reducing vellum to PDFs and tracking schedules and areas automatically, machine learning will automate exploring your solution space. If you can reimagine your work not as the day-to-day but as steps along a manifold defined by the many dimensions of the problems you are confronted with, then you have the beginnings of a machine learning application.

Why is it important to be working on this now? When, with respect to neural networks, even the head of Facebook’s Artificial Intelligence Research admits, “...we make those networks bigger and bigger, and they just work better. Even without additional data, which is a big mystery.”  Surely there is no reason for architects or civil and electrical engineers to try to build their own applications. There are two reasons that this is wrong. If you accept that you think through your tools, then it is the responsibility of experts to be experts in designing their tools as much as the product of their craft. The second is that complexity is just the summation of a lot of simplicity. Although it is unlikely that most architects or engineers could look at the formulation of the backpropagation or stochastic gradient descent and know exactly what is happening, the basic principles of what goes on in a neural network are simply multiplication, division, addition, and subtraction. They just happen in complicated shapes, and a lot, all at once. To take a lesson from machine learning itself, learning progresses asymptotically. At the beginning, at least, even a little bit of knowledge will make you exponentially better at this task, though this implies that the rate at which you learn decreases as you improve. There are always trade-offs.

We are at a moment when it is possible for people to experiment with these tools without knowing how to translate pages of mathematical notation into C code that is compiled to run on a GPU. This daunting task was the case as recently as four years ago. Now, libraries like Google’s TensorFlow and Facebook’s PyTorch have lowered the software barrier to entry to the point that those without advanced degrees in math or computer science can make interesting things happen with them. This progression is not new. In the late 1970s, rendering an image of Cookie Monster onto a screen took the concerted effort and expenditure of Xerox PARC, but by 1986, advanced CAD applications had become ubiquitous in design offices industry-wide.

Within just the last few months, the Google Colaboratory platform has lowered the hardware barrier to entry. As recently as two years ago, configuring hardware to do meaningful experiments in deep learning was an excruciating task. Now it’s as simple as opening a Google doc and checking a box.

This is what led us at TEECOM to begin working with these tools in-house. At the end of February, the R&D team hosted a Machine Learning Coding Club as an effort to show how we see machine learning being applied in our field, and to expose our engineers and designers to thinking about how to solve problems in this way.

Our experience with machine learning scholarship has been that most resources focus on the mathematical formulations of key principles like linear regression, convolution, and so on. Our approach is twofold: to focus on coding implementations, and to demonstrate principles visually. When you are already dealing with highly trained biological neural networks, we believe that it is more beneficial to introduce machine learning as a way of reformulating problems, rather than to explain the mathematical principles of learning systems. To that end, our coding club was based around introducing an autoencoder that is trained to redraw architectural plans as a conceptual demonstration, the pieces of which can then be rearranged or modified for image classification, segmentation, generative adversarial networks, etc.

What can we add?

For all of its abilities, and for all of the hype, machine learning is just another tool. It is not magic, and it requires deep engagement to leverage its abilities to suit an organization’s goals. After all, CAD didn’t change how far a beam could span. In the same way, machine learning’s limiting factor is still domain-specific knowledge about the real world. It is the responsibility of engaged experts to draw out the connections between their problem space and the types of problems that machine learning can be applied to. Even Stephen Wolfram, who has created some of the most sophisticated knowledge exploration systems conceded in a recent lecture, “You need human experts, or you are going to get it wrong.”

The first and most important task for human experts is the development of datasets that describe learnable tasks in your area of expertise. This, however, is a kind of bidirectional opportunity. The process of creating a dataset can be a productive intellectual exercise that answers the questions:

• What does my data look like?

• Is it labeled? If so, how?

• How do I learn about problems that are described by my data?

• How do I enumerate solutions to these problems and choose a path among them?

• How would I describe how to learn to map from raw data to problem identification to solution?

What changes?

The most significant change to A/E/C in the machine learning-powered future is the differentiability of the problem space. Designers will become responsible not so much for the actual design of the solution, but for posing the design questions in a way that the space can be easily searched. In that same lecture, Stephen Wolfram stated that, “AI will let us automate anything that we can describe.” This is what the future of A/E/C work looks like. Designers will need to cultivate this ability to describe problems in ways that lend themselves to convex optimization or other kinds of differentiable search.

The most significant change to A/E/C in the machine learning-powered future is the differentiability of the problem space. Designers will become responsible not so much for the actual design of the solution, but for posing the design questions in a way that the space can be easily searched.

There are some sci-fi possibilities which sound more exciting than their real impact is likely to be. Recently, researchers in Japan were able to read minds of subjects in real time by using a deep neural network to map patterns of brain activity to reconstructions of images. If we foresee a confluence of a commercialized version of this technique and current trends in VR design applications, we can imagine a sketching environment for designing buildings where a vague internal rendering of some spatial/structural idea could be resolved instantly into workable digital models/drawings.

There is a lot of hype around deep learning and its apparently limitless applications. What it really allows are just high-dimensional correlations and a smooth and systematic way to improve the models doing that correlation. Ultimately, it will change how we think about problems rather than creating some post-human world of auto-design. CAD gave us new ways to organize complexity. The internet gave us new ways to search for answers. Machine learning’s real impact will be to give us new ways to search the complex space that we deal with every day.