Artificial intelligence art

Artificial intelligence art is any artwork created through the use of artificial intelligence.

Tools and processes

Imagery

There are many mechanisms for creating AI art, including procedural ‘rule-based’ generation of images using mathematical patterns, algorithms which simulate brush strokes and other painted effects, and artificial intelligence or deep learning algorithms such as generative adversarial networks and transformers.

One of the first significant AI art systems is AARON, developed by Harold Cohen beginning in the late 1960s. AARON is the most notable example of AI art in the era of GOFAI programming because of its use of a symbolic rule-based approach to generate technical images. Cohen developed AARON with the goal of being able to code the act of drawing. In its primitive form, AARON created simple black and white drawings. Cohen would later finish the drawings by painting them. Throughout the years, he also began to develop a way for AARON to also paint. Cohen designed AARON to paint using special brushes and dyes that were chosen by the program itself without mediation from Cohen.

Since their design in 2014, generative adversarial networks (GANs) are often used by AI artists. This system uses a “generator” to create new images and a “discriminator” to decide which created images are considered successful. More recent models use Vector Quantized Generative Adversarial Network and Contrastive Language–Image Pre-training (VQGAN+CLIP).

DeepDream, released by Google in 2015, uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating deliberately over-processed images. After DeepDream’s release several companies released apps that can transform photos into art-like images with the style of well-known sets of paintings.

Several programs use text-to-image models to generate a variety of images based on various text prompts. They include OpenAI’s DALL-E which released a series of images in January 2021, Google Brain’s Imagen and Parti which was announced in May 2022 and Microsoft’s NUWA-Infinity. The input can also include images and keywords and/or configurable parameters such as artistic style which is often used via keyphrases like “in the style of {name of an artist}” in the prompt and/or selection of a broad aesthetic/art style.

There are many other AI art generation programs including simple consumer-facing mobile apps and Jupyter notebooks that require powerful GPUs to run effectively. Examples include Midjourney, StyleGAN, and Stable Diffusion, among many others.

On 22 August 2022, Stable Diffusion was released, making the technology much more accessible and free to use on personal hardware as well as extendable by third-parties (i.e. other software projects). This enabled a surge in further innovative applications and extensions from developers around the world – such as plugins for Krita, Photoshop, Blender, and GIMP. The Automatic1111 Stable Diffusion UI is a popular web-based open source user interface for using the tool on one’s own computer including, continuously integrated, new features (such as “Inpainting” or “Textual Inversion”). The web interface by Stability.ai that allows running the software without any new installation is called DreamStudio.

Impact and applications

In September 2022, an expert concluded that “AI art is everywhere right now”, with even experts not knowing what it will mean, a news outlet established that “AI-generated art booms” and reported about issues of copyright and automation of professional artists, a news outlet investigated how online communities (e.g. their rules) confronted with many such artworks react, news outlets raised concerns over deepfakes, a magazine highlighted possibilities of enabling “new forms of artistic expression”, and an editorial noted that it may be seen as a welcome “augmentation of human capability”.

Midjourney image from the prompt “swimming pool filled with a galaxy on a moonlit night”

Examples of such augmentation may include e.g. enabling expansion of noncommercial niche genres (common examples are cyberpunk derivatives like solarpunk) by amateurs, novel entertainment, novel imaginative childhood play, very fast prototyping, increasing art-making accessibility and artistic output per effort and/or expenses and/or time – e.g. via generating drafts, inspirations, draft-refinitions, and image-components (Inpainting).

Synthetic media, which includes AI art, has been described in 2022 as a major technology-driven trend to affect business in the coming years.

Prompt engineering and sharing

There are platforms for sharing, trading, searching, forking/refining and/or collaborating on prompts for generating specific imagery from image generators. Prompts are often shared along with images on image sharing-websites such as reddit and AI art-dedicated websites. They are not the complete input or details used for the generation of images.

Development

Additional functionalities are under development and may improve various applications or enable new ones – such as “Textual Inversion” which refers to enabling the use of user-provided concepts (like an object or a style) learned from few images. With textual inversion, novel personalized art can be generated from the associated word(s) (the keywords that have been assigned to the learned, often abstract, concept) and model extensions/fine-tuning (DreamBooth).

Generated images are sometimes used as sketches or low-cost experimentations or illustration of proof-of-concept-stage ideas – additional functionalities or improvements may also relate to post-generation manual editing (polishing or artistic usage) of prompts-based art (such as subsequent tweaking with an image editor). In the case of Stable Diffusion, the main pre-trained model is shared on the Hugging Face Hub.

Music

A key part of this field is the development of music software programs which use AI to produce music. As with applications in other fields, AI in music also simulates mental tasks. A prominent feature is the capability of an AI algorithm to learn based on past data, such as in computer accompaniment technology, wherein the AI is capable of listening to a human performer and performing accompaniment. Artificial intelligence also drives interactive composition technology, wherein a computer composes music in response to a live performance. There are other AI applications in music that cover not only music composition, production, and performance but also how music is marketed and consumed. Several music player programs have also been developed to use voice recognition and natural language processing technology for music voice control.

For example, AI can be used in the adjustable generation of novel sounds and samples that can be used by artists for music tracks.

Other

Some prototype robots can create what is considered forms of art – such as dynamic cooking robots that can taste and readjust.

There also is AI-assisted writing beyond copy-editing (including support in the generation of fictional stories such as helping with writer’s block or inspiration or rewriting segments).

AI could be and has been used in video game art beyond imagery only, especially for level design (e.g. for custom maps) and creating new content or interactive stories in video games.

Sales

An auction sale of artificial intelligence art was held at Christie’s Auction House in New York in 2018, where the AI artwork Edmond de Belamy sold for $432,500, which was almost 45 times higher than its estimate of $7,000–$10,000. The artwork was created by “Obvious”, a Paris-based collective.

Criticism and issues

An image generated using several artist’s names, including Yoshitaka Amano and Kelly McKernan. The use of other artist’s names in prompts has generated considerable controversy.

Copyright

Ever since artists began using AI to create art in the 20th century, the use of AI-generated art has sparked a number of debates. In the 2020s, some of those debates concerned whether AI art can be defined as art or not and concerning the impact it will have on artists.

In 1985, Pamela Samuelson considered the legal questions surrounding AI art authorship as it relates to copyright: who owns the copyright when the piece of art was created by artificial intelligence? Samuelson’s article, Allocating Ownership Rights in Computer-Generated Works, argued that rights should be allocated to the user of the generator program. Victor Palace has presented three possible choices. First, the artificial intelligence itself becomes the copyright owner. To do this, Section 101 of the Copyright Act would need to be amended to define “author” as a natural person or a computer. Second, following Samuelson’s argument, the user, programmer, or artificial intelligence company is the copyright owner. This would be an expansion of the “work for hire” doctrine, under which ownership of a copyright is transferred to the “employer.” Finally, no one becomes the copyright owner, and the work would automatically enter the public domain. The argument here is that because no person “created” the piece of art, no one should be the copyright owner.

Reema Selhi of the Design and Artists Copyright Society stated in September 2022 that “there are no safeguards for artists to be able to identify works in databases that are being used and opt out.”

Concerns about impact on artists

Some artists in 2022 have raised concerns about the impact AI art could have on their ability to earn money, particularly if AI art is used to replace artists working in illustration and design. In August 2022, an example of text-to-image AI art won the first-place $300 prize in a digital art competition at the Colorado State Fair’s. Digital artist R. J. Palmer said in August 2022 that “I could easily envision a scenario where using AI a single artist or art director could take the place of 5-10 entry level artists… I have seen a lot of self-published authors and such say how great it will be that they don’t have to hire an artist,” adding that “doing that kind of work for small creators is how a lot of us got our start as professional artists.” Polish digital artist Greg Rutkowski said in September 2022 that “it’s starting to look like a threat to our careers,” adding that it has gotten more difficult to search for his work online because many of the images returned by search engines are generated by AI that was prompted to mimic his style.

Issues of deepfakes

As with other types of photo manipulation since the early 19th century, some people in the early 21st century have been concerned that AI could be used to create content that is misleading, known as “deepfakes”.