- A new deep learning model writes poems that outperform human-written poems in rhyme and meter.
- Some people find it hard to distinguish between the two, but according to experts AI falls short on emotion and readability.
Can deep learning methods be harnessed for creative tasks? The answer is yes. We have been using these machine learning techniques in creative applications like composing music, designing sculptures and automatic choreography.
Now researchers at IBM, University of Toronto and the University of Melbourne have focused on a creative textual task: automatic poetry composition. They have developed a deep learning model, named Deep-speare that captures the language, rhyme, meter for sonnets, and generates poems.
It’s quite fascinating that these AI-generated poems resemble ones most popularly written by an English poet, William Shakespeare. The model works very well: it creates sonnet quatrains with rhyme and stress patterns, which are sometimes indistinguishable from poems written by humans.
A sonnet is a short lyric poem, popularized by Shakespeare. Usually, it contains 14 lines bundled as 3 quatrains (four lines) and a couplet (two lines).
How Did They Do This?
Researchers focused on sonnets and produced quatrains in iambic pentameter based on an unsupervised model of rhyme, language and meter trained on a novel corpus of sonnets. They trained a vanilla language model on sonnet corpus, which captures meter implicitly at human-level performance.
In short, they proposed modeling both poetry content and forms with a neural architecture, which contains three key components
- Language model
- Pentameter model to capture iambic pentameter
- Rhyme model to learn rhyming words.
To predict the next word in a particular sonnet line, the language model utilizes standard categorical cross entropy. Similarly, the pentameter model is trained to learn the alternating patterns of iambic stress. Finally, the rhyme model separates rhyming word pairs from non-rhyming ones in a quatrain, using a margin-based loss.
Reference: arXiv:1807.03491
The language model generates one word at a time, while the pentameter model samples meter-conforming sentences and the rhyme model enforces rhyme. All these components are trained together.
Image credit: The Daily Dot
The neural network is trained on 2,685 sonnets containing about 367,000 words, using NVIDIA Tesla GPUs with TensorFlow powered by CUDA deep learning framework.
Researchers even took the help of an English literature expert, Adam Hammond, to rate 4 aspects of the machine generated poems – readability, rhyme, meter and emotion. Hammond was unaware of poems’ source. The results revealed that the AI outperforms human’s work in rhyme and meter, but falls short on emotion and readability.
Although the model may not seem directly relevant to practical applications, it shares the same core algorithm that powers other generation-problems like summarization, translation and chatbots.
Read: A Psychopath AI That Sees Death In Everything
Prior to this, researchers had attempted to mimic creativity using neural networks, including a bot-written script for ‘Scrubs’ (a TV comedy series) and Google’s DeepDream image generation project. They say that to achieve impressive poetry, they will look beyond content and forms in future research.
