Wikipedia claims that
There is no legal requirement for individuals to join the Social Security program. The Social Security Act does not require a person to have a Social Security Number (SSN) to live and work in the United States. Any "duty" to apply for and obtain a Social Security number can be summarised in this way: you get it if you need it or request it. There is no legal compulsion to do so. However, once joined there is no general provision for individuals to opt out of or quit the program.
My reading of the citations is that you don't have to join the program . . . but you still gotta pay the tax, so it's a distinction without a difference. Do any readers have thoughts?