If the theory is that the Phelpses were disturbing the peace/disorderly conduct, they could have been charged with that, the argument being that somebody shouting something noncontroversial -- "I think apple pie is wonderful, if eaten in moderation!" -- in the same circumstances would also be a discon, and obviating the Constitutional question.

(I don't think it would fly, mind you, but . . . )

Again: their opinion is, well, nutty 'way past disgusting, but it's not defamatory. (And no, speech is not libel; speech can be slander, but the essence of libel, other than its defamatory nature, is that it's written.)