Of course it is, thats the whole point isnt it?
You are saying he is something less than he is.

Have you never said to someone "....is a big girl" and someone has replied, "thats insulting to girls?"
The comeback is meant as a joke, but there is truth in it as well.