Visible vs. Invisible disabilities

The funny thing about Down Syndrome is that it is pretty self-evident. You can see it on my little guy's face. My husband and I sometimes talk about how Y's features are not so pronounced but the truth is that most people can tell right away. While this summer, people recognizing Y on the street earned us lots of brachos, that may not always be the case.
There are both advantages and disadvantages to having a visible disability. The advantage is that people may be more tolerant and patient with the child, because they see he is not "typical". There are the rare times when Y acts up in public and I have a hard time controlling him. It helps to know that people are not judging him or me, or at least I hope they aren't.
The disadvantage is that people may be too tolerant of Y's shenanigans or have very low expectations of him. Even though Y is developmentally delayed, we still expect him to learn how to behave properly. We were recently in a store and Y decided to sit down in the middle of a busy aisle. I immediately told him he had to get up, while the store clerk kept saying, "Don't worry. Let him stay there." While, it was nice of her to say that, Y knows better and this is not behavior I want to encourage. So I thanked her, but still insisted that he stand up right away.
I also do not want people to see my son and automatically assume that he can't do something because of his disablity- can't talk/play/behave/understand or otherwise. Because the truth of the matter is that very often he can and will. People often express surprise at how well Y can navigate the playground and/or children's museum by himself and we definitely like to encourage that independence.
Parents of children with autism or other so-called invisible disabilities often complain about the opposite problem. Their children often look typical and whenthey act up in public, people are not always understanding and will judge the parents for not controling their child better. They sometimes have their children wear cute shirts that say "I have autism, what's your excuse?" or something similar, so that others will be more tolerant.
I guess I want it both ways- for people not to make assumptions right away when they see Y, but also to get some leeway in the instances when he does need some extra help, patience or understanding.

In an ideal world I wouldn't care what others thought and they in turn would not judge people by their visible disabilities or other external appearances. All I really can do, though, is to try my best to help my little man learn how to act like a "mentch" and not worry so much about what others think when things don't always go according to plan.