Joe Wiesenthal reads a book on mortgage fraud from 2005, and concludes that the biggest problem was the low-to-no downpayment loans:
The author, Michael Richardson, owned a Colorado mortgage company that was busted by HUD for processing too much fraudulent paperwork. This caused him to discover that unbeknownst to him his employees were (on their own) engaging in mortgage fraud, prompting him to write this book and try to warn the industry.
This alone is interesting -- that even on the small-time level, there was an information problem (bosses not knowing what the underlings were doing) -- and the book is rich with details about the nuts and bolts of mortgage fraud.
But beyond that, one point he makes clear -- and remember, this is before 2005, so before the crash and before conservatives blamed government intervention in the housing market for the crash -- is that the FHA's subsidization of $0-down loans made it all possible.
If you make someone pay 10% or 20% of a house's cost upfront, then there's no way you can alter the paperwork enough to make an ineligible buyer buy a house for an inflated price. But once you drop that requirement, everything goes. You can sell any house to any buyer for any price as long as you put in the effort to falsify documents and go through the cumbersome legwork.
As far as I know, by the end, private lenders were also doing a fairly booming business in low downpayment loans. But it's probably true that they thought it was safe because, after all, FHA loans were doing okay. And the broader point stands: the low downpayments, not the adjustable interest rate or other exotica, were probably the biggest single problem in the market.
That's true for a number of reasons. As Joe points out, when people have to put a big chunk of the purchase price down, this serves as a limit on your ability to commit fraud; you're no longer gambling with just your credit score, but also with tangible money that has other immediate uses. It also serves as a limit on house price appreciation; if the prices rise too fast, they will outstrip the resources of new buyers.