The "but" caveat is our allowance to whine, to complain, to make sure it's known that we are willing to do something but we are not happy about it. There's a "poor me" echo in "yes, but." It's dishonest in a way. It's not a real yes.
The world is confusing because we are all so frequently unclear about our meaning. We tell people what we think they want to hear instead of telling them the truth. Let's bring awareness and care to what we are saying. Let's say "yes" when we mean "yes," and "no" when we mean "no." It sounds simple but it's not. We confuse the two all the time, and end up bitter and resentful. We feel used and misunderstood, but the error is in us. We have to be careful not to send mixed messages.
I avoid saying "yes, but." I say "yes" instead, or "no," or "maybe." I bring awareness to the words I use and I say what I mean.