Let's Make some Art
(The title image is called "AI Brain". It is three images that were generated by the artificial intelligence neural network I trained.
Learning through experience is one of the best ways to learn. So this is how I went about learning about machine learning, artificial intelligence and neural networks. I am an artist and programmer that became interested in how computers could make art. So lets use artificial intelligence to create new artwork.
I decided to use an open source project on GitHub called art-DCGAN by Robbie Barrat. His implementation was used recently in Christie's first Artificial Intelligence art auction. But I first encountered Robbie Barrat's work on superRare.co
To start out we need some data to train the neural network. I wanted to use simple drawings to train my first model. I happen to have a library of art that I personally created. They are line drawings made using sound played through an oscilloscope. It turns out that most of the prebuilt code libraries utilize very small image sizes and you need at least 10,000 images to train them effectively.
Get Lots of Images
I happen to have movies of animated oscilloscope line drawings. Most of them come from this artwork called "Baby's First Mobile", Space Pong, No Image Productions Learn how I make oscilloscope art at this post.
To generate a large number of images I turned the videos into individual frames using ffmpeg. Using commands like:
<code>ffmpeg -i babyMobileVoice.mp4 -vf crop=500:500 thumb%04d.jpg -hide_banner</code>
I was able to crop out the center of the video and produce individual frames. After running ffmpeg on all the oscilloscope videos I have and deleting all the black frames I ended up with over 10,000 images.
Make them Small
The next step is to make the images small. Some tools only accept 64 x 64 pixel images. The art-DCGAN uses 128 x 128 pixel images. I borrowed a python script from this blog at coria.com. To double the number of images I read that someone rotated them by 90 degrees for additional training examples. Using these directions I added a rotate feature and made a copy of each
import numpy as np
from os import walk
# width to resize
width = int(sys.argv)
# height to resize
height = int(sys.argv)
# location of the input dataset
input_dir = sys.argv
# location of the output dataset
out_dir = sys.argv
if len(sys.argv) != 5:
print("Please specify width, height, input directory and output directory.")
# get all the pictures in directory
images = 
ext = (".jpeg", ".jpg", ".png")
for (dirpath, dirnames, filenames) in walk(input_dir):
for filename in filenames:
for image in images:
img = cv2.imread(image, cv2.IMREAD_UNCHANGED)
h, w = img.shape[:2]
pad_bottom, pad_right = 0, 0
if h < w:
ratio = w / h
ratio = h / w
if h > height or w > width:
# shrinking image algorithm
interp = cv2.INTER_AREA
# stretching image algorithm
interp = cv2.INTER_CUBIC
w = width
h = round(w / ratio)
if h > height:
h = int(height)
w = round(h * ratio)
pad_bottom = abs(height - h)
pad_right = abs(width - w)
scaled_img = cv2.resize(img, (w, int(h)), interpolation=interp)
padded_img = cv2.copyMakeBorder(
cv2.imwrite(os.path.join(out_dir, os.path.basename(image)), padded_img)
rotate_img = imutils.rotate(padded_img, 90)
cv2.imwrite(os.path.join(out_dir, '90_'+os.path.basename(image)), rotate_img)
Running this command
python resize_images.py 128 128 /origin /destination
will produce images of the proper size for the training to begin.
Examples from the Training Set
Setup an AWS GPU Instance
The GitHub art-DCGAN project has install directions but I will add to them here. In AWS I selected:
Ubuntu Server 14.04 LTS (HVM), SSD Volume Type - ami-06b5810be11add0e2 with the g3s.xlarge instance type.
Install all the Software
Once the instance started. I followed the directions on art-DCGAN install. Make sure to find the exact packages in the directions. On the Nvidia site you will need to press the legacy releases button to find the right version of the CUDA software. The deb file I used is cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb Newer updates of the software will not work with the art-DCGAN code. Once it has finished building the code clone the art-DCGAN code. Now move your thousands of images into a folder called myimages/images. Make sure to have a folder called images inside the myimages folder as described in the project art-DCGAN was forked from. Now you can run the command below but note the checkpoint options. This will cause it to save checkpoint models as it runs. The code tended to crash on me before it completed.
<code>DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=50 ngf=150 th main.lua</code>
saveIter will tell it to save a model for the discriminator and the generator after every 10 epochs. In my case an epoch was 318 training rounds. But with these settings the training would constantly collapse. If the output of the training starts to show either the discriminator or the generator running at 0% your training has failed. To fix it I lowered the number of filters for the discriminator so it could keep up with the generator. The GAN model is an adversarial training. A generator will create a random image and the discriminator will judge it based on the training source images. It repeats this process until the neural network is wired in such a way to produce images like the ones in the training set.
<code>DATA_ROOT=myimages dataset=folder saveIter=10 name=scope1 ndf=12 ngf=150 th main.lua</code>
is what worked for my particular artwork. After running for five hours and making it to 90 epochs before failing I ended up with some images that look a lot like mine but with many variations.
Now the Artificial Intelligence model will make some new art
To generate a 36 image grid find the newest model file and use the command:
batchSize=36 net=./checkpoints/scope_90_net_G.t7 th generate.lua
^ One of the Artificial Intelligence Outputs From Epoch 90
I also generated output from the fifth generation of training:
You can see how much it learned.
Now it becomes a matter of picking out the best work as an AI art curator.
With all of this work it becomes obvious that this is a very slow way to make art. It is still very much a human endeavor. But allowing a computer to generate lots of ideas for an artist to explore is helpful. Many artists have used randomized systems to generate art ideas. Artists such David Bowie, Brian Eno, Nam June Paik and John Cage have used mechanical and computer generators to automate some of their work and kick start the creative process.
In addition by making this model we have learned how to train other models that can be used for image recognition. You can also try out the other features of art-DCGAN that will download art from a public site and then generate new images based on that genre.
Look for more posts on this generated artwork to come soon. I have submitted one of the AI works to SuperRare.co so people can purchase it. Once it is accepted I will announce it here.
Update: I have been accepted to SuperRare. I will release the first artwork of this series on Nov 20 2018.
Update 2: The artwork Alien Intelligence has been released come take a look at it.
Read my next post for a science fiction backstory. - The SAGE Anomaly