Arttu began his career in visual effects, creating FX simulations for multiple award-winning adverts before pivoting towards real-time graphics. Drawing on his professional background and a lifelong passion for music and sound technology, his artistic medium heavily revolves around creating interactive and generative real-time art. His works often incorporate physics simulations and generative musical instruments, while exploring the themes of playfulness and self-reflection.

→ shaderism

TGAM: Hello ser! Thank you very much for joining us today. How are you?

S: Thanks for having me, I’m doing great! I recently moved back to Finland, so there’s been a lot happening. The days are getting longer, and it finally feels like summer is starting to arrive here also!

TGAM: Can you share with us the story of how you discovered generative art and what drew you to this particular medium?

S: I’ve been experimenting with generative art in various forms for quite some time before knowing of its existence. Before pursuing a career in 3D graphics, my focus was primarily on music, and I would spend countless evenings immersed in synthesizers and software like Max/MSP. When I shifted my focus to 3D graphics, fairly quickly SideFX Houdini became my software of choice. While Houdini is especially known for VFX simulations, it’s also tremendously powerful for creating procedural systems for generating diverse elements, such as trees or virtually anything else. It was through this exploration of building systems that mimic nature in an aesthetically pleasing manner, be it through audio or visual formats, that my interest in generative art started. I believe I first encountered the term "generative art" around 2021 when I began delving into the creative possibilities of NFTs and stumbled upon ArtBlocks.

TGAM: What inspired you to explore and work with shaders specifically in your generative art practice? Can you share some of the primary tools or software you use to create your art?

S: Exploring and working with shaders came very naturally, as even in Houdini, creating shaders was a regular part of my daily workflow, although mostly in a node-based format. When it comes to the way I approach generative art with WebGL, not utilizing shaders would feel quite limiting. Firstly in terms of visual aesthetics as that’s where a lot of the juicy details happen, but also from a performance perspective as well. I always strive to make my work run smoothly in real-time on a wide range of devices.

As for the primary tools I use, they are mostly the same as with my other (non-generative art) WebGL projects. Three.js combined with React, with the help of react-three-fiber ecosystem. Additionally, there’s Tone.js which I currently use for synthesizing sounds into my artwork.

TGAM: Can you briefly walk us through your creative process, from idea to final product?

S: The creative process for me can often involve a significant time gap between having an idea and actually beginning to work on it. I’ve got a long list of ideas that come out of the ether, often inspired by my time spent in nature, for instance. One of these ideas then gets paired with a spark of inspiration started by some recent technical development that catch my attention, forming a fitting combination. For instance, recently the improvements in CSG (Constructive Solid Geometry) within WebGL have inspired me, and the projects have been centered around that. So by the time I start a project, I usually have a strong vision in mind. I then delve into exploring the possibilities and limitations of real-time graphics (as limited by my skills) and that sort of shapes the ambiance of the project. Finally, in generative art, there's the crucial step of curating and refining the outcomes, to something that I find appealing and cohesive as a project.

TGAM: How do you balance the creative freedom of generative art with the need to create something that resonates with an audience?

S: I often find myself struggling to make progress with my creativity if I have too much unrestricted freedom in the tools and processes. Especially when I aim to combine aspects like real-time graphics & audio with interactivity, it's easy to get caught up endlessly tinkering with the tools. To overcome this, I prefer to set boundaries, and often these boundaries come in the form of elements that the audience resonates with. For example, I frequently use isometric projection, even when everything is created with 3D elements.

TGAM: You’ve worked with some big brands, Mclaren & Ikea to name a couple. What has been your favorite project or collaboration to work on as an audio visual artist, and why?

S: Over the years, I've had the opportunity to work on some exciting things as a creative developer, projects such as tools related to VR storyboarding and point cloud visualizations. However, from an artist's perspective, Heineken | The Trailblazers advert comes to mind. Mostly the sheer diversity of visual elements (eg. water simulations and explosions) that were to be created made it very interesting and enjoyable. Audio-visual projects have been a side endeavor for me until now, but I am looking forward to future collaborations in this field!

TGAM: How do you see the field of generative art evolving in the coming years, and where do you see yourself fitting into that landscape?

S: That's an excellent question, one that I haven’t thought about much before! I do feel there is a strong connection and numerous interesting possibilities at the intersection of generative art and mixed realities. However, as the industry has felt somewhat stagnant recently, I’m excited to see how Apple's new headset might shake things up.

On the physical side of generative art, but outside of printing and pen plotters (which I find fascinating, but haven’t had the time for unfortunately), I see significant potential in interactive installations and audio-reactive concert visuals. I believe there are a lot of possibilities in creating immersive and interactive experiences through generative art, and I'm excited to explore and contribute to that field in the coming years!

TGAM: Shaders often involve complex coding and programming. How did you approach learning and mastering this aspect of generative art?

S: When I was initially learning about 3D graphics, I relied heavily on watching tutorials. One person that I used to follow, who helped with understanding shaders, was Rohan Dalvi. Most of my learning during that time involved using nodes rather than coding, which made it feel more approachable while also teaching in a way that transferred easily to code later. When I transitioned to working with WebGL, I found classic sources such as Inigo Quilez and The Book of Shaders to be immensely helpful. Additionally, I often find myself exploring codebases and reading through them to learn about new concepts and techniques related to shaders.

TGAM: How do you strike a balance between control and randomness in your generative art? Is there a specific approach you follow?

S: Honestly, I don’t have any specific approach to it! When I release a project, I view it as singular artwork, and I appreciate it when the outputs that the algorithm generates are somewhat cohesive. Initially, I start with broad ranges for randomness, allowing for a wide variety of outcomes. However, as I look through thousands of batch-generated outputs, I narrow down the parameters by filtering out results that I feel have "gone too far" and don't align with my desired aesthetic or vision for the artwork.

TGAM: What are some of the challenges you face when working with shaders, and how do you overcome them?

S: One of the biggest challenges for me is to avoid the temptation to add more and more details. It's easy to get carried away and end up with visuals that appear cluttered or even chaotic. To overcome this challenge, I use the same approach as with picking colors, sounds and so on. That is by observing the artwork of other artists, I sort of calibrate my senses which helps me to gain a sense of direction the project is heading towards.

The other set of challenges are more technical in nature. Performance is a constant battle, especially when pushing the boundaries of what is possible while still ensuring smooth playback on lower-end devices. The solution here obviously is to test on multiple devices, optimizing code when encountering performance bottlenecks, and potentially providing slightly different variations of the graphics depending on the device. Additionally, one specific issue that occasionally frustrates me with shaders is achieving a correct rendering order, which is not always as simple as it may seem!

TGAM: In your generative art practice, you often incorporate sound and audio elements. How did you first begin integrating audio into your visual works, and what does sound add to the overall experience for you?

S: Integrating audio into my visual works in generative art came naturally to me, as generative art for me is in part about letting myself explore my passions. I wanted to reconnect with music, which I felt I had lost touch with, and incorporating it into my creative practice was a perfect fit. Audio-reactive graphics is something that I’ve explored already for several years, for example by developing a music visualizer that utilized Spotify's API data and live playback to guide the visual elements.

In nature, we perceive and connect with things through multiple senses, like the sight of leaves rustling or the sound of crashing waves, which I love observing. As art is often something that is attempting to portray one viewpoint into our world, by adding audio I aim to create a more immersive and lifelike experience. I think this is why I gravitate towards real-time graphics that are combined with audio and interactivity, as it allows me to create artworks that come closer to replicating the multisensory experience we encounter in the natural world.

TGAM: Could you describe the relationship between the visual and audio components in your artworks? How do you approach creating harmony or synergy between these two elements? Can you provide some insights into your creative decision-making in this aspect?

S: In my artworks on fx(hash), I’ve attempted to go with a “graphics-first” approach, where the visual components take the lead and influence (often through something related to physics) the audio components. To make the relationship between the visual and audio components more clear, I tend to create animations within the shader of the visual element that is responsible for triggering the sound. This animation would then try to follow what the audio sounds like!

TGAM: Have there been any challenges or unique considerations you've encountered when integrating sound into your generative art? How have you overcome or navigated these challenges?

S: One challenge I've encountered is related to the fact that for performance reasons there needs to be a limit on the number of sounds that the synthesizer can play simultaneously. While some other synthesizers drop the oldest sound when the limit is reached, Tone.js (the synthesizer I use), is designed to ignore new sounds in such cases. To overcome this, I've had to stick to using sounds that are as short as possible while still trying to make them sound pleasing.

TGAM: Let’s jump into your Responsive Dreams project, “Chordal Reveries”, can you explain a bit more about the project and the concept behind it?

S: The concept of "Chordal Reveries" revolves around two main aspects that I’m trying to reach toward. Firstly, there is a focus on creating self-playing audiovisual instruments that are visually captivating and enjoyable to listen to in their own right. This time the inspiration for the instrument came from various different marble games, which I then incorporated with the signature touch of physics simulations.

The second aspect delves into something more complex. I place great value on playfulness without a specific end goal. For example, exploring the world with a curious mind through interacting with physics, sharpens the attention to the present moment. So the artwork sort of serves as an open invitation to collaborate with the instrument by exploring its capabilities while cultivating focus in a meditative way.

TGAM: Could you describe the process you follow when creating a responsive artwork? How do you incorporate interactivity and adaptability into your pieces?

S: I find that adaptability is fortunately made quite easy with Three.js, so it doesn't require much consideration. The biggest hurdle usually lies in writing a function that allows for zooming the camera in and out, ensuring that the framing looks visually appealing across all aspect ratios.

During the development process, I primarily focus on building interactivity with mouse interactions in mind. However, as the project nears it’s completion, I try and replicate as much of the interactivity as possible using gestures for touchscreens. “Chordal Reveries” was a bit more special as it started with a MIDI controller in mind. This time when approaching the final stages, I created fallback support for computer keyboards, but due to the complexity it would have added, I decided not to include much touchscreen interactivity.

TGAM: How do you see the future of responsive art evolving? Are there any emerging technologies or trends that you're excited about incorporating into your work?

S: I'm particularly excited about delving deeper into MediaPipe by Google. Earlier this year I shared some tests with it on my Twitter feed, and I believe it holds so much potential for creating interactivity within artworks. I’ve got multiple project ideas that would combine well with it that I can’t wait to get started on!

TGAM: Looking ahead, how do you envision further exploring the possibilities of audio in your generative art practice? Are there any new ideas or directions you're excited to explore in this realm?

S: In terms of sound synthesis, the patches I've created so far have been fairly simple, mainly consisting of an ADSR synthesizer with a few effects. So I definitely plan to delve deeper into exploring the boundaries of WebAudio. Additionally, while I find the melodic and rhythmic aspects of my artworks pleasing, they’re certainly somewhat random. Therefore, another avenue I've been thinking to explore is creating generative melodies and harmonies that have more musical quality to them.

TGAM: Lastly, what message or impact do you hope to convey to audiences through your work in responsive generative art, and what legacy would you like to leave in the field?

S: To be honest, as I feel like I've only recently started my artistic journey, the concept of leaving a legacy hasn't crossed my mind. However, in terms of the message I aim to convey, as I mentioned in an earlier reply, playfulness remains at the core!

Shaderism is part of TGAM's Responsive Dreams Festival, the first generative art exhibition in Barcelona dedicated entirely to showcasing art created by code.

"Chordal Reveries" will be minted live during 14/15th June at Nau Bostik, Barcelona. Holders of TGAM's brochures are elegible to mint preferentially.

The Generative Art Museum (TGAM) is a non-profit organization based in Barcelona dedicated to explore, promote, and advance the understanding and appreciation of generative art as a unique form of artistic expression.