You pass on the prestigious projects and focus on your classes..
Your physics professors are flabbergasted that you would pass up such an opportunity. They no longer think of your interest in linguistics as “fascinating,” but more as “distracting from your real work.” Your relationship with the physics department chills, and while they let you graduate with your degree, it becomes clear that they have discussed you with their colleagues elsewhere, and grad school in physics is not a possibility.
Go to grad school in linguistics.
Just try to get a job.