The last thing you would ever say is, "something something annoys me."
That's a guarantee that the thing that annoys you is going to be hung around your neck like a millstone. Sam Smith is annoyed that people think he's the male equivalent of Adele (he's not; he's the male equivalent of Rick Astley).
In the British music press, they must be chortling (if there remains a British music press, obviously) at the idea that Sam Smith is receiving torrents of abuse from Americans like Howard Stern. They have seized on his weight (he's not fat by any standard other than that of a heroin addict with bare cupboards) and his success and now they're going to start beating the drum over his annoyance with the Adele comparisons.
If Smith truly doesn't care, and if this is a feint to get them to stop making fun of his chubbiness, well played. The problem is, he can never reap the benefits of an Adele duet. That cudgel has been snatched from his hands.