Partner / Tool / Canvas: UI for AI Image Generators

“Howl’s Moving Castle, with Solar Panels” – using Stable Diffusion / DreamStudio LIte

Like a lot of folks, I’ve been messing about with the various AI image generators as they open up.

While at Google I got to play with language model work quite a bit, and we worked on a series of projects looking at AI tools as ‘thought partners’ – but mainly in the space of language with some multimodal components.

As a result perhaps – the things I find myself curious about are not so much the models or the outputs – but the interfaces to these generator systems and the way they might inspire different creative processes.

For instance – Midjourney operates through a discord chat interface – reinforcing perhaps the notion that there is a personage at the other end crafting these things and sending them back to you in a chat. I found a turn-taking dynamic underlines play and iteration – creating an initially addictive experience despite the clunkyness of the UI. It feels like an infinite game. You’re also exposed (whether you like it or not…) to what others are producing – and the prompts they are using to do so.

Dall-e and Stable Diffusion via Dreamstudio have more of a ‘traditional’ tool UI, with a canvas where the prompt is rendered, that the user can tweak with various settings and sliders. It feels (to me) less open-ended – but more tunable, more open to ‘mastery’ as a useful tool.

All three to varying extents resurface prompts and output from fellow users – creating a ‘view-source’ loop for newbies and dilettantes like me.

Gerard Serra – who we were lucky to host as an intern while I was at Google AIUX – has been working on perhaps another possibility for ‘co-working with AI’.

While this is back in the realm of LLMs and language rather than image generation, I am a fan of the approach: creating a shared canvas that humans and AI co-work on. How might this extend to image generator UI?

A Manhattan melange of “Macroscopes”

.flickr-photo { border: solid 2px #000000; }
.flickr-yourcomment { }
.flickr-frame { text-align: left; padding: 3px; }
.flickr-caption { font-size: 0.8em; margin-top: 0px; }



Globe of Patents, originally uploaded by blackbeltjones.

By chance this morning found an excellent mini-exhibition in midtown Manhattan.

“Places & Spaces: Mapping Science” has been curated by Dr. Katy Börner and Deborah MacPherson.

From the website:

“Today, the word “science” encompasses myriad arenas of physical and abstract inquiry. This unique exhibition, at the Healy Hall in midtown Manhattan, uses innovative mapping techniques to physically show what and where science is today, how different branches of science relate to each other and where different branches of study are heading, where cutting edge science is erupting as archipelagos in the oceans of the yet unknown – and – how it all relates back to the physical centers of research. The world of science is turned into a navigable landscape.

Modern mapping imagery has come a long way from Ptolemy. In this stimulating show compelling for all ages and backgrounds, audiences will both visually and tactilely uncover how contemporary scientific thought has expanded. Such visualization of scientific progress is approached through computer-generated relationships, featured on large panels as well through the collaboration of New York based artists W. Bradford Paley, Digital Image Design Incorporated and Columbia University and Ingo Gunther with renowned scientist from the field of scientonometrics: Eugene Garfield, Henry Small, André Skupin, Steven A. Morris, Kevin Boyack and Dick Klavans.”

Scientonometrics! Awesome!!!

It’s a concise, enjoyable and clear exhibit showing concrete examples of what John Thackara might call ‘macroscopes’: artworks, mappings and visualisations of complex interconnected systems (in this case science and intellectual property) that help ‘ordinary folk’ examine the choices they make and those being made for them.

Recommended.

Clayton Cubitt interviews Tom Carden on “Generative Art”

Tom and Clayton collaborated on a set of beautiful images this year, and now Clayton has published a short interview with Tom on his site.

Tom discusses with Clayton his reaction to the finished work and the process they shared to create it; but also his route to generative art, it’s history and his influences:

“Before mass access to computers, people used other hardware, tools, toys and rule-sets to make algorithmic and process-driven art – pendulums, spirographs, Indian rangolis, Celtic knots, mandalas and so on – and a lot of the methods people use in computer generated art were investigated by mathematicians by hand before computers were available, such as Fibonacci series and the Golden Ratio. Casey Reas has looked into Kinetic Sculpture in some depth, and that’s something I keep intending to read up on. I’m sure that before computers were around the same things that people like about generative art were satisfied by fireworks, fountains, may poles, crop circles, wax lamps and oscilloscopes. Grid-based games such as Go and Othello are very reminiscent of the patterns created by certain types of Cellular Automata, too. The main advantage with using a computer is speed, such that there is now scope for using any of these systems over long periods of time and with minute variations.”

Beautiful stuff – congratulations to both artists.

Tracks in the city


This is "Ghetto Superstar"
Originally uploaded by blackbeltjones.

This is Pras, ODB and Mya’s “Ghetto Superstar”: a visualisation from Jake Elliot’s PopSketchSeries.

Artists statement:

“this is a series of drawings generated from pop songs. the songs are analyzed note-by-note. at each note, a line is drawn. the angle at which the line is drawn is determined by the pitch of the note and the length of the line is determined by the volume of the note. the result is a series of playful, doodle-like, linear drawings.

Imagine taking music visualisation, mixing in play and embodiment into the mobile realm – mobile music players allowing you to trace your tracks like a demented Logo Turtle through the city.

Joined-up listening – groups and groupies conga together through the streets propelled by a programmatic peer-2-peer pied piper.

Dance Dance Dance Situationist Revolution.

Vib Ribbon Reality…