{"id":5010,"date":"2017-04-14T13:27:45","date_gmt":"2017-04-14T18:27:45","guid":{"rendered":"https:\/\/blog.lib.uiowa.edu\/studio\/?p=5010"},"modified":"2018-10-31T13:50:16","modified_gmt":"2018-10-31T18:50:16","slug":"neural-network-poetry","status":"publish","type":"post","link":"https:\/\/blog.lib.uiowa.edu\/studio\/2017\/04\/14\/neural-network-poetry\/","title":{"rendered":"Neural Network Poetry"},"content":{"rendered":"
\"\"<\/a>
An Iowa writer hard at work<\/figcaption><\/figure>\n

As you may know, April is national poetry month<\/a>, an annual series of events by the Academy of American Poets to help support the appreciation of American poetry. If you’re looking for great book-length collections of poems, you might be interested in the Iowa Poetry Prize winners<\/a>. Many of the previous years’ winners are made available in PDF form<\/a> at Iowa Research Online. What you may not know is that April is also National Poetry Generation Month<\/a>, an annual tradition where programmers and creative coders spend the month writing code that generates poetry.<\/p>\n

In honor of this time of year, I thought I’d take a look at the Iowa Poetry Prize winners through code.\u00a0There are many methods for analyzing and generating natural language, but one system that has received a lot of attention recently is neural networks<\/a>. A neural network is a large collection of artificial neurons based very loosely on a biological brain. These neurons exist in layers that perform statistical calculations and affect the state of other connected neurons. It differs from other computational models in that there is no knowledge hard coded and controlled by elaborate conditional statements (if this<\/em> then that<\/em>). Rather, neural networks learn to solve tasks by observing data and producing optimal functions that will produce similar outputs given new data it’s never seen before. The uses for such a system include image and speech recognition, classification problems, and many forms of prediction and decision making. For example, a neural net could be trained<\/em> to detect images of cats\u00a0by observing tens of thousands of labeled images of\u00a0cats. Google has recently launched a new project<\/a> that uses this technique to match your doodles with professional drawings.<\/p>\n

What happens when we train an artificial intelligence to write english language having only read Iowa Poetry Prize winners? Let’s find out!<\/p>\n

To start, I downloaded all of the IPP winners from Iowa Research Online, extracted the poems as plain text, and concatenated them all into a single text file named poems.txt. This served as the training set. Next, I set up this Torch<\/a>-based Docker container<\/a>\u00a0implementation of a recurrent neural network<\/a> based on work by Andrej Karpathy<\/a> and Justin Johnson<\/a>. It was tempting to spin up the Google cloud\u00a0VM with an attached GPU, since these types of machine learning tasks are sped up greatly running on a graphics processing unit with CUDA<\/a>, but it’s also quite expensive at 75 cents-per-hour. Once I had it working, I started the preprocessing and training, which took about 16 hours to complete.<\/p>\n

After a lot of experimentation to create some useful training models and keep the network from overfitting and underfitting<\/a> the data, I had something that was acceptable and so began sampling output. One parameter of sampling that was fun to play with was the “temperature” of the sample. A lower temperature produced output that was much more predictable and less error prone while a higher temperature was much more inventive but riddled with mistakes. I decided to split the difference and start at 0.5. Here’s the first poem.<\/p>\n

Speritas Of The Stars<\/p>\n

Morning comes of the sun
\nto the thin world is a star of her light.<\/p>\n

The sheet and the body of parts
\nof the flame is a light, the body
\nsees of the wars beautiful on the street.
\nThe sun, the stars of the sound, and desire,
\nand a man could love the streets.<\/p>\n

The single shiller of light,
\nand the single stranger falls countal.
\nFather and she were the sutters of the body
\ninstraining to the complete
\nwindow of light, still.<\/p>\n

You’ll notice a few words in this poem that don’t actually exist in english. That’s because this RNN operates at the character level, not the word level. It has to learn, from scratch, how to write english. It starts with random strings of letters and slowly, after many iterations, learns about spaces, proper punctuation, and finally readable words. The higher the sampling temperature, the more invented words. Let’s look at a “hot” poem.<\/p>\n

Pelies, One Yighter<\/p>\n

The shadows just plance croved
\nI am one
\nits funlet from the wind
\nstaskaccus, gring of detches of hearts face eashog
\nwhat wing to the streed in the resert of change, a glince
\nthe life.
\nShe read on his fill bathered, a hand the
\nmarks
\nwith beautiful, casty, stery, kooms, in one father<\/p>\n

something the mouth cold leaves.
\nA night and no one is a woman; you green her<\/p>\n

My spere would must not the look teering mower
\nI see itselfor.
\nAt that sign they thought the remelled the mum,
\nbut like an wait they mite of ammiral
\nafter things of the body
\nwhich children would love
\nnow, not
\nthe forest flowers and hark a path.
\nThe shawr rate in a ruched parts in humstily
\nhis poom her as of the trabs conterlity.<\/p>\n

Much more Jabberwockyesque. If we ease up just a little on this we get<\/p>\n

A Badicar Flower<\/p>\n

The watcher blue says
\nthey would have shapes,
\nthe night dreaming,
\na painted nother
\ntricks me, the wind,
\nthe dayed from the boging feeling
\nof the histance in his everyness.<\/p>\n

What do you think — poetry prize worthy? While writing poetry is fun, there are, of course, practical applications too. I’m currently working with faculty member\u00a0Mariola Espinosa<\/a>\u00a0on a HathiTrust\u00a0project called\u00a0Fighting Fever in the Caribbean: Medicine and Empire, 1650-1902<\/a>. We have 9.3 million pages of medical journals and need to find references to yellow fever in multiple languages. A trained neural network could look through these quickly and find references that a human might miss. I’m also working on another project with\u00a0Heidi Renee Aijala<\/a> looking for references to coal smoke in Victorian literature. Perhaps a neural net could be trained to look for non-keyword references.<\/p>\n

While I’m probably not going to put a poet out of work any time soon, you can imagine many real-world uses. There is a tremendous potential for neural networks and other types of machine learning to caption images, transcribe handwriting, translate\u00a0documents, understand the spoken word, and play chess at the international master level<\/a>. Perhaps someday it might also write a meaningful poem<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

As you may know, April is national poetry month, an annual series of events by the Academy of American Poets to help support the appreciation of American poetry. If you’re looking for great book-length collections of poems, you might be interested in the Iowa Poetry Prize winners. Many of the previous years’ winners are madeContinue reading “Neural Network Poetry”<\/span><\/a><\/p>\n","protected":false},"author":153,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"syndication":[],"_links":{"self":[{"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/posts\/5010"}],"collection":[{"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/users\/153"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/comments?post=5010"}],"version-history":[{"count":17,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/posts\/5010\/revisions"}],"predecessor-version":[{"id":5032,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/posts\/5010\/revisions\/5032"}],"wp:attachment":[{"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/media?parent=5010"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/categories?post=5010"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/tags?post=5010"},{"taxonomy":"syndication","embeddable":true,"href":"https:\/\/blog.lib.uiowa.edu\/studio\/wp-json\/wp\/v2\/syndication?post=5010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}