What we need is pension reform, and re-structured compensation.
No, seriously. One of the barriers to effective discipline on most police forces is that when cops sign up for the job, they believe they're making a bargain - twenty years of mediocre pay, long hours, and often dangerous working conditions, in exchange for a pension while they're still middle-aged. So it's not just about the thin blue line, about solidarity and camaraderie, about the brotherhood of people under fire - though all of those things are crucial. It's also that taking an officer, particularly a veteran officer getting close to that pension, and throwing him out onto civvie street taps into every officer's worst fear. "I could screw up, make a single mistake, and throw away everything I've worked for," they think.
This is bad news, all around. It means that instead of a force comprised of people doing their jobs because it's what they want to be doing, or at least the best available option, we often end up with police forces stocked with officers trying to "put in their twenty." It means that people who decide they're unsuited for the life, or simply can't take it any more, still try to stick it out. And it means that even when supervisors decide that an officer shouldn't be out on the streets, that they don't have the temperament or judgment to be a good cop, they're more likely to try to transfer them to some other unit than to see them dismissed from the force. It's not a problem that's unique to police forces, either. Other public-sector employees suffer from similarly warped incentives - public school teachers leap to mind as the prime example. Trapping people in their jobs, and raising the perceived costs of dismissing them from their jobs, is bad policy.
Which is not to downplay the importance of good oversight, of community review, of reform, of internal affairs, or of any of the myriad other checks and balances for which advocates continually press. But the simplest way to encourage the dismissal of those poorly-equipped to handle the responsibilities of police work from the force is to lower the perceived cost of that dismissal. That means paying cops more up front, and less later on. Or, if that's too tough to put into place, shifting them from pensions over to 457 plans, in which they receive annual retirement payments from the city that are fully portable, and can be taken with them and funneled into a 401(k) after a short vesting period. Little changes, big impacts.