Breasts don’t need to be large throughout adult life for the purpose of nursing. Look at the non-human apes and other primates. The hypothesis that large adult human breasts outside of nursing evolved due to sexual selection is completely reasonable.

Sexual selection is not inherently good or bad. It just is. If that theory is correct, then breasts which are (on many women) large before and between nursing stages came about due to sexual attraction. There was selection for women with larger breasts and selection for men attracted to them. It’s not the responsibility or fault of either sex, and the genes increasing both the attraction in males and the breast size in females are passed through both sexes. Fathers and mothers who have daughters with larger breasts and sons attracted to them will have more grandkids.

None of this comes with the baggage of how we should set up our society. We can suppress sexual activity in public, demand consent, be free to cover or not cover regardless of how it was we got to where we are today.