According to the Journal of Neuroscience, tone deafness could be caused by a very particular brain defect.

This region, a neural ‘highway’ called the arcuate fasciculus, is known to be involved in linking music and language perception with vocal production.The arcuate fasciculus was smaller in volume and had a lower fiber count in the tone-deaf individuals. More notably, the superior branch of the arcuate fasciculus in the right hemisphere could not be detected in the tone-deaf individuals. The researchers speculated that this could mean the branch is missing entirely, or is so abnormally deformed that it appears invisible to even the most advanced neuroimaging methods.