Increases in average levels of education, levels of disposable income, gender equality, and access to birth control -- that is, increases in the ability of people (and especially women) to deliberately control the conditions of their own lives -- generally lead people to choose a smaller rather than larger number of children. As far as I can tell, Bryan's response is that it "lacks perspective" to take at face value this truly striking tendency of choice under conditions of increasing personal control. If Bryan really thinks rising education, wealth, and gender equality have somehow made us worse at evaluating the costs and benefits of children, he probably ought to turn in his economist card.
None of this is to say that there aren't excellent reasons to have families larger than the relatively small rich-country norm. It's just that these tend not to be the kinds of reasons economists consider "selfish."