Bryan Caplan claims that
prohibiting someone from immigrating to the United States … violates private property rights and freedom of association. An American’s freedom to hire an immigrant to work in the business she owns is protected by her private property rights as well as her freedom to associate with the immigrant and the immigrant’s freedom to associate with her. The same goes for decisions to reside or congregate with people from other countries.
The fundamental right of freedom of association necessarily includes the freedom not to associate with a certain person or persons. A group of like-minded individuals may therefore band together and declare that they will not associate with, say, illegal immigrants.
Further, that group may hold joint title to “public” property (e.g., hospitals, schools, and other “public” services) by virtue of paying taxes to maintain the property and to support the operations conducted on it. The group therefore has the right to forbid certain persons (outside the group) from using that property, both as a matter of property rights and freedom of association.
Caplan’s kind of libertarianism is in fact destructive of property rights and freedom of association.