The Wire notes the risk of replacing Fred Armisen with Jay Pharoah right before election season. But I am going with Christopher John Farley:
I think it's written right in the U.S. Constitution somewhere that Fred Armisen's impression of President Barack Obama isn't very funny.
When Armisen first came on for Obama there was some disgruntlement because he wasn't black. But I always thought Darrell Hammond's Jesse Jackson was awesome. Armisen's impressions just weren't funny. Here's Pharoah's Will Smith.