A baby would not be allergic to breast milk, it is mostly things that are consumed by the Mother whilst breastfeeding in fairness here.  For one babies should not be exposed to things like peanuts in the first year or so due to allergies, but if the mother consumes a known allergen such as peanuts or soy prducts and the baby is allergic, it is the allergen not the breast milk.  That in itself occurs less than allergies to say cows milk and soy based milk etc because it is not always that a mother is consuming that particular food, unless she over indulges every day--then that would naturally cause a problem.   If you remove the allergen--the baby is fine.