Basically, it's take your pick. I've listed some below, with the most improbable first.
My own favourite is that the name comes from a connection with water. The yeomen were farmers and would come from irrigated or watered land.
The Old English for a river or stream is ea. This may be from the same base root as l'eau, the French for water.
This is consistent with the practice of naming after occupation (such as Cooper, Carter, Smith, Chandler etc).
However, Sheila Yeo, who has undertaken a lot of research into the name (far more than me), has this to say:
"The first mention I can find of the name is William atte Yeo in 1346. I have been trying to research to which Yeo this alludes.
However, in 1066 in the Domesday Book reports a lot of rivers in Devon were called Nymet, an Anglo Saxon word for water/rivers, the Normans changed these rivers to Yeo, French for the water Y'eau. All my early research makes me think that the Yeo we originate from is a place rather than a river, although the place could have been named Yeo because it is on a river.
The other possibility is that it links to the Count of Eau who was related to William the Conqueror and certainly he held land at Yeovil in Somerset."