Of course it could be a bust, but that's life. There are no guarantees either way, so you've just got to go with the way that you believe is best for you, taking all your circumstances into consideration.
As for the "whinge" part, I think you worry far too much about that. Again, that's life, not everyone is going to be happy all the time, or agree with the direction you're taking, so you've just got to do what you believe is best and let come what may. There's no point worrying about that shit or it will do your head in and you'll never take a risk and life will pass you by as you sit in the corner, scared to do anything, because of "what if's".