It all started with a drawing

Following the success of our first collaboration, Mining Structures, Peter (@lax_raven) and I often talked about doing a sequel. I was particularly fascinated by one of his pieces, a very intricate ink drawing of a large city. Peter had graciously sent me a print of the drawing, and not a day went by where I wasn't studying it, wondering how to convert this into a generative piece.

There is something deceptive about this drawing. The first impression is that it's crazily intricate, and that only a madman would attempt to replicate this in code !

Second impression, well, after all that aren't that many different building types, maybe half a doze in total, so maybe it's not that crazy after all.

Third impression, is when you look at the details in each building, and you realize your first impression was the right one after all. This is an insane undertaking.

Or is it ? Let's find out...

Ink lines

For generating natural-looking lines in p5.js, I used the code that was fist developed for Mining Structure, then improved with my other piece Cloverfield. I ended up adding a lot more enhancements to it for The City.

I've already documented the process elsewhere, the general principle is that instead of drawing a line between 2 points, the line is split into a large number of smaller lines, each sub-point is randomly displaced, and each sub-line has its own line weight and opacity. This lets me write code like this:

    for (var i=0; i<10; ++i)
    {
        lineDrawer.draw(drawingLayer, 10, 10+20*i, 1000, 10+20*i);
    }

and get this:

For this project I added many little features like random splatters at the end of lines, smoothing of the variation in line weight, distance-based amount of variation (so smaller lines get less displacement than longer ones). It would be an entire article by itself to go into all the details.

First attempt - chronology of a failure

I initially approached this piece by picking up a specific building and trying to reproduce it, with as many details and texture as possible.

I figured out that I could them vary the parameters slightly to generate a lot of slightly different buildings, then slap them together to get the finished piece.

Let me tell you right now, I was so wrong....

My first mistake was to chose a building that was not even part of the original piece. I don't know what got into me really. Here is the drawing I chose from Peter's collection, and to the right, my first approximation of it.

Clearly this wasn't looking too nice, but I thought that the final piece would have hundreds of these, and they wouldn't be so large, so the difference in details would not be visible. So I tried putting a lot of these buildings together, together with a second building type, and here's what I got.

It was awful ! I should have realized by then it was the wrong approach, but yet I persisted. Maybe it needed more details, more textures ?

I went down the rabbit hole of writing a texture generator, which, by the way, produced some pretty nice outputs like these:

which I used to generate nice looking columns:

However it still didn't quite work out when putting all of these together in a complete piece:

Maybe it was a matter of varying the shapes ? I wrote more code for rounded top buildings, like in Peter's original piece. It still didn't work out.

The higher the resolution, the worse it was !

At this point I was ready to give up. I had spent countless hours on detailed buildings, a texture generator, fine tuning ink lines, generating images at 16000px resolution, and it wasn't working.

I was burned out. The project was put on hold while I took a break.

Project restart

I took a long needed break from social media, but Peter's drawing was still hanging in my living room, taunting me. I would look at it and marvel at all the details, and one day it hit me - focusing on the details early on was the reason for the initial failure !

Suddenly the path forward was very clear: generate the piece using rough shapes first, make sure the composition makes sense, then progressively add detail so that it looks good booth zoomed out, and up close.

I couldn't wait to get back to code !

The picture below shows 4 successive iterations of this approach. The one on the right highlights the logic for layering the buildings, the composition is split in 6 layers, each further split in 5 sub-layers. In the final piece we ended up using different combinations, such as 6x4, 5x5 and 4x6. They all generate slightly different layouts that are all pleasing.

This was very promising, as the generated silhouettes were very pleasing and were very much in the spirit of the original drawing.

We were both very excited. Peter printed one of the outputs, and started shading it manually.

It was beautiful ! At this point we knew that it could work !

I started working on the shading logic. We decided that, like in the original, the left side of the buildings would be using darker shadows. Subtle variations in angle and length would be used to make the hatching look more natural. Some buildings would randomly be given 4-directions hatching to create more depth.

Peter would keep printing iterations and manually correct the shading. Here are some of these early blueprints, which in my opinion, are stunning for works in progress.

This last image is where we decided to let scaffolds have a more prominent role than in the original. Peter is an architect and it is very important to him, that the final pieces are structurally sound.

I started adding supporting structures, working from the top down so the scaffolds would either be hidden by the layers below, or visible on the side. Scaffolds were randomly given an angled area, to avoid looking too rectangular.

Image editor

Progress was steady but very slow. I kept adding settings to the line drawing system, scaffolds logic, buildings parameters, and it was taking a lot of time to iterate, especially as Peter and I are on different time zones.

I decided to add a hidden editor where Peter could directly tweak all the parameters, so we could work in parallel.

After that, every new feature I added had settings in the editor. Soon enough we needed more pages to display the settings. We ended up with dozens of pages and hundreds of tweakable parameters !

For whoever read this far, we kept the editor available in the published mint. You can access it with the 'e' key.

Improved hatching

Things were progressing nicely but I still wasn't satisfied with the hatching, that didn't look as nice as in the manual version. Peter came up with a breakthrough suggestion: add randomly broken vertical lines on some of the larger buildings. This was a huge improvement.

Layer split

I thought the layout was a little too rectangular, and that we needed a little more variation for a decent size collection. So I added a setting to enable layers to randomly be split it two. This led to some nice structures, like parallel stacks of buildings.

Clearly the scaffolds needed more work. I tried a lot of different algorithms, and got this really nice output from a buggy version where the scaffolds got drawn upwards by accident:

Progress loader

Things were starting to look interesting, but it was also taking a long time to render a piece. We felt the waiting time was too long and we needed a loading screen. This turned out to be harder than I thought, and I had to move around a lot of code to get this to work.

The javascript yield() keyword turned out to be a lifesaver. I first converted the code so it would show the rendering of the progress, but somehow I felt it was taking away from the wow factor when the final piece was revealed - a little bit like explaining a magic trick before performing it. So, even though step-by-step rendering is available in the editor for anyone curious to see the progress, I made a special screen that grabs random areas of the drawing being rendered, and displays them in a circular progress loader. We both really liked the effect and this is the version in the final mint.

Final phase: details

Eventually, we got to the point where we were satisfied with the overall look, we had nice city shapes, with interesting shading. It was time for the tedious, but also fun, part: adding details.

We had decided very early on that, contrary to Mining Structure, this project would not use hand-drawn element. Instead, I would convert Peter's drawings to code so we could easily generate variations of the details. I first added cranes like in the original.

These were looking nice, but someone rightfully pointed out they looked too modern for the style of the city. So Peter sketched medieval style cranes and I converted these to code.

I made some very lose, black trees that ended up looking very much like Peter's hand drawn ones when zoomed all the way out. These trees are randomly drawn at the base of the layers, and also on top of the flat larger buildings.

We also started focusing on rare items, creating special buildings such as a pyramid, monolith, or obelisk. There was even a Stargate portal, that we didn't keep in the final version.

This is also when we started adding different paper colors for extra variety. Then Peter came up with a wonderful suggestion, have some cities painted with a red gradient.

We loved the effect, so soon enough we added a white and black wash too, and even a rainbow wash that we eventually disabled in the final version (but is still accessible via the editor)

Peter kept sketching details for the buildings and towers, and sending me the hand drawn sketches.

I would convert these drawings into parameterized models that could be used to generate variations.

While working on a fractal tree to add as a special item, I accidentally ended up generating what looked like pine trees. We decided to keep these at the bottom of the piece in some of the iterations.

More layouts !

We were getting close to the final version now, with some really nice outputs. However, after generating several batches of hundreds of cities, we both felt the outputs were a little too uniform and could use more variations. I added some extra logic to the layers engine so that instead of piling up constant width layers with a random probability of splitting, the width of each layer would also be controlled by its position. This allowed me to guide the composition to rectangular (the original), triangular, inverted triangle or hourglass shapes.

I also added an "elevated" variation where all layers except the top one, would be hidden.

This change opened up the collection drastically, and we got some beautiful compositions while playing with the settings. I have included some of my favorites below. Although the project was initially inspired by a very busy composition, I find the minimalist outputs just as interesting.

At this point we felt like the piece was ready for release.

We generated several batches to fine tune the various probabilities, and called it done (which was not an easy decision, as you always want to add "just one more thing").

Closing words

I hope you found this long article useful. We learned a lot while doing this piece. The main takeaways:

• use a top-down approach, focusing on shapes and composition and gradually adding details until the piece feels complete. Don't get caught up in details early !
• if you know the piece will require a progress loader, integrate it from the start.
• expose all internal parameters via an editor.