You are only partially right. Some thousands of years ago, certain races domesticated animals such as goats and cows for their meat. However, until these were slaughtered, the females of these species gave off milk, which those people realized could be given to their children instead of thrown away. Back then, only children could digest milk, but the naturally some children could digest it until a later age, and some stopped being able at an earlier age. The ones who could digest milk for longer got the benefit of the extra food (something hard to come by at the time), and thus had higher chances of spreading their milk digestion gene.

Most of the descendants of these races can digest milk until old age today, because milk gave them a higher chance of spreading their genes. If milk is unhealthy, they would have a LOWER chance of spreading the gene, and only children would be able to digest it today.

Ungoliant MMDCCLXIV (talk)15:24, 11 April 2011