I'm big, ugly, and yes sweaty during setup, but cool down and wash off when changing for my event - and my clients love me.
I used to offer a tux when I first started, but not any more. I'll still wear it if they insist, but I no longer off it as an option up-front, I simply say "semi-formal attire okay? Button-down shirt, dress slacks, dress shoes..." They say yes every time.
Of course there are extremes where appearance would/could be a deal killer, but it's really about a total package, and I don't believe that certain things matter to clients as much as some people think/claim they do. As long as your appearance isn't a distraction, and your attitude, mannerisms, and conduct are appropriate for the event or situation at hand, that's all that matters.
Perhaps if you're in the worther bracket and attracting the kind of client that is looking for or swayed by those kinds of things they are more prone to be judgmental based on appearances (ya think...?). At the end of the day those aren't the kind of clients I would want any way.