Except the census also counts where the people live to set the districts. Also SS numbers are given to anyone staying in the US on a visa. student visa, temp work visa, etc...