Yandex Cloud neural network learns to collaborate with a human artist

This project aimed to explore the space between the physical and digital worlds we live in today. Together with the artist, we tried to bring art and technology together and show that machine learning and artificial intelligence can be full-fledged participants in the creative process.

During the development of the project, the machine learning algorithm constantly supplemented and filled in the image alongside the human artist. The participants called this method of digital art the “ping-pong principle.” The pre-trained convolutional neural network GG-19 was deployed in the Yandex DataSphere ML development environment, and it was used to process a rough sketch of the artist’s illustration. Andrey Berger then corrected the resulting image, and the neural network “filled in” the illustration once again. This ordered exchange was repeated more than 15 times. In addition to the sketches, QR codes, the Yandex Cloud logo, Malevich’s “Black Square,” and images of bubbly, layered, and other natural structures were “mixed in” during the performance.

We transferred the image developed using the neural network to a unique collection of phygital blankets. The blankets were given to 1,000 of Yandex Cloud platform services’ largest customers in terms of consumption. In addition, we prepared an exhibition where you can see the art object itself. The event opened at the Yandex Museum on January 28, 2023.

“Now many artists are exploring the territories of machine learning. There are many scenarios for interacting with neural networks. Essentially, the artist uploads his query to neural networks and the algorithm gives him something in return. One of the most common is the so-called text query generation or source image generation, and depending on what the network was trained on, the artist gets a result. When the first such projects appeared, the results made a splash. Today, we can see the growth in popularity of these algorithms’ use for entertainment purposes. However, it was interesting for me to establish a mutual interaction with artificial intelligence. I didn’t just want to train the neural network on my images, but also to look at its work myself, to try to understand its logic, to foster a dialogue of co-creation.”

“Developing a neural network for art science is often a technologically challenging task, one which can require significant computing resources and time from a ML specialist to solve. The intuitive interface and quick setup within Yandex DataSphere helped the specialists create a unique neural network in a short time and make an important contribution to the development of digital art.”