So we needed more words, but how many?
Sherjil Ozair, a computer scientist at Baidu Research’s Silicon Valley AI lab, told me I would probably need more text than I’d ever written. Doubling or tripling the existing sample size wouldn’t be enough. “You’ll be able to generate something, but the story won’t be legible,” Ozair told me. “You really need an order of magnitude like 100 megabytes—but that data doesn’t exist, and that’s the fundamental problem. We have these methods that could work really well, but only if you have a tremendous amount of data.”
Increasing my sample size to 100 megabytes would mean going from 500,000 words to something like 18 million—or the equivalent of War and Peace 30 times in a row. That wasn’t going to happen, but Sloan was still willing to experiment with a much (much) leaner sample. After hours of piling on the megabytes—adding text from stories I’d written for The New York Times, The Washington Post, Slate, Gawker, Honolulu Civil Beat, and elsewhere dating back to around 2012—I was only up to about 725,000 words.
Sloan ran the experiment again, this time using the prompt, “She told me…”
She told me and more like modernings in our computer. Of course, this is not a human work. That's the web that's selfies that would be the moon is that they'll make it online. That's not only more importantly changed and most of them in all of those and questions about their factors. For example, as far several cases, all this kind of regulations for information—that's the person who painted itself that's the technological change of human process. And it's going to be the first electric companies that can make it into their context in places—is not a technological computer. Scientists reported her everything that's because there's also an interview that there's something like a story of the findings of their work. But that's what does it reported by their work. This is that Internet connections can complete the way that these people are searching to their cars and the passage of itself. And that's the situation of the library's first computer that was more likely to have all this person betwe…
There was a slight difference, but the output still wasn’t exactly publishable, not even close. So Sloan encouraged the model to be less conservative, riskier in its predictions about which letter to generate next in sequence. “Those predictions are hedged and probabilistic, not definite: at every step, the net establishes a probability for every possible next character,” Sloan explained.
When a model is less risky, it’s more likely to be repetitive. (That’s why, on the most conservative setting, the Adrienne Corpus produced sentences like this: “The most people who are all the story of the company and the story of the story of the story of the company and the first place.”) But too risky can predict its own breed of nonsense. Here’s what the riskier model based on my larger sample of writing produced:
She told me that the first # years ago in a decade from the Times—even as well as much them. "The background change was created the world that was focused on a time that the pool can make the world." The format points that they weren't always "demonstrating and being solved with organizations over time. (Researchers have been spuns from the property source of the town of a being print people who are likely to try on the platform, and the Internet age is perfectly happening a diversity of a screen. In an excerpt, like himself, the people so we're really been right at the past year. "The world is long and browse discourse," George We said. "It was also a significant model that makes it interesting." March #, you can see the algorithms. The currency of the site servers shifted the first time when the GoPro does it is an infectious earthquakes and technology in the film in a myth. And the sweet was the biggest project, scientists were built before the internet, it was probably looking for this tech...
And here’s another example from a riskier model:
She told me to any new assault. "I was recorded in the world's book," said Marty McBring, a historian of Security Trust and Army CH for earlier his research in #. "The appeal of a little idea was when and what it's hard to release the most politicism they didn't use the brand of same-time. And they take a process of magnitude to the ground it was like before. I would come out with the classification of engineering, the same emergency and the design we didn't interact themselves so that a sort of three poys material. The larger device is more more than the error and the process of what we go online. That's why you can do something about the person more than the moon were encourages that Catholic levels of the U.S. Geological Handburg. But a colleague is about amerscare structures. They probably be able to only use much of the technical cortain-partace group. "We have a mere of earthquake and main the smoke -- which could be a health point and information to the computer behaviors of what he was now"
“In general, I think these big blocks of text don’t show the RNNs at their best—because, again, they have no sense of the larger structure of a piece of writing,” Sloan said. “It's the shorter ‘completions’ that are more compelling.” (You can see some of the autocompletes from Sloan’s sci-fi experiment on his website.)