As books from major publishers get bigger and more expensive, smaller houses are taking risks on more creative, original works.