You cannot seperate "gender" and "sex", as people are a tryptych of gender expression, gender identity, and biology. You do not walk through the wold as your sexed self, you move through the world as the expression you feel most comfortable conforming to. Sexed bodies do matter for medical purposes, but using sex to exclude runs the risk of becoming controlling and sadistic. Seperating the two also risks creating a Cartesean dualism of body and mind, which is patently false for all people.