It's impossible to logically oppose the Employment Non-Discrimination Act, which protects orientation and gender-identity rights, without opposing the Civil Rights Act
Does federal civil-rights law unduly infringe upon First Amendment freedoms of speech, association, and religion? The officially settled answer to that question (the only politic answer) is "no," as Rand Paul discovered when he tactlessly revealed his opposition to regulating private-sector discrimination at the outset of his 2010 senate campaign. Within 24 hours he took it back, stressing his unqualified support for the 1964 Civil Rights Act, citing its prohibition of the "abhorrent practice of segregation and Jim Crow laws." (Paul may have forgotten or maybe didn't know that this seminal statute also prohibited sex discrimination in employment.)
Of course, precisely because the Act barred race discrimination in public accommodations and the workplace, it was vehemently opposed by Southern Democrats, who mounted a two-month filibuster against it. Barry Goldwater also opposed it, out of the concern for individual freedom briefly espoused by Rand Paul. But toward the end of his life, in 1994, Goldwater was advocating for federal legislation prohibiting employment discrimination against gay people, who have "a constitutional right to be gay."
Goldwater would not be considered a conservative today (as he observed, he was eventually condemned as a liberal for supporting abortion rights), and relatively few Republicans would join him in supporting the Employment Non-Discrimination Act (ENDA), barring discrimination on the basis of sexual orientation or gender identity. Recently reintroduced in Congress, with a little bipartisan Senate support, ENDA has virtually no chance of passing the ultra-conservative Republican House. Today's conservatives are selectively resurrecting libertarian arguments against civil-rights laws in their fight against the dreaded "homosexual agenda."
Bans on racial, religious, and sexual discrimination, in general, and the 1964 Civil Rights Act, in particular, still require political obeisance, but bans on discrimination based on sexual orientation or gender identity are condemned by the religious right as flagrant violations of religious freedom. ENDA "wages war on freedom of religion in the workplace," Congressman Mike Pence declares. Right-wing advocacy groups agree: "ENDA is a dangerous, blatantly unconstitutional bill that would pit the government directly against the free exercise of religion," according to Alliance Defense Fund President, Alan Sears.
There are principled, moral, and pragmatic libertarian arguments for unregulated markets unhampered by civil-rights laws. The laws do infringe on freedom of association, and if you consider the associational rights of corporate employers equal to the associational rights essential to private groups and intimate relations (which I do not), then you agree with Rand Paul's 2002 statement that "a free society will abide unofficial, private discrimination." That's the moral argument against civil-rights laws. The pragmatic argument rests on the view that free markets are rational and discrimination irrational. Free markets are more effective than civil-rights laws in "promoting tolerance and reducing bigotry," Boston Globe columnist Jeff Jacoby asserts. But if this were true, then the 1964 Civil Rights Act would have been redundant. If this were true, then the end of segregation in public accommodations and decline of gross employment discrimination against women and minorities that followed passage of landmark civil-rights legislation would have been mere coincidence. Libertarian arguments against civil-rights laws are, in my view, unpersuasive and ahistorical, but they are not unprincipled.
There are, however, no consistent, principled libertarian distinctions between "good" anti-discrimination laws that protect racial or religious minorities and women and "bad" laws that protect gay and transgendered people. If civil-rights laws unconstitutionally restrict individual freedom to discriminate, they restrict it regardless of the group they seek to protect. If religious freedom includes a right to discriminate against gay people, why, for example, doesn't it include a right to discriminate against women? Why shouldn't an employer who deems it sinful or a violation of some divine order for women to work outside the home or in traditionally male jobs have the same First Amendment right to discriminate as an employer who considers homosexuality a sin?
If your answer is "religious beliefs that require women to stay home are wrong and held only by small minorities of employers while belief in the sinfulness of homosexuality is right," then you're the one advocating religious discrimination -- insisting that laws should respect your beliefs or beliefs favored by the majority while disrespecting minority beliefs or beliefs you disdain. Religious opponents of ENDA are apt to attack it for extending "special rights," but if there's nothing special about protecting women and racial minorities from discrimination, then there's nothing special about protecting gay people. Enacting ENDA would not extend any special equality rights to gay and transgendered people. Instead, religious opposition to its enactment demands special rights to discriminate against them.