ISP's blocking VoIP transmissions?

VoIP provider Vonage, through its CEO, has announced that certain ISP's have blocked users of the over-the-internet phone service. His initial contention is that this constitutes a form of prior restraint, which if true would be unconstitutional censorship by the ISP. This raises a host of interesting issues, starting with whether the packets transmitting data (whether voice or email) constitute protected speech as contemplated by our Constitution. Why wouldn't blocking spam be equally wrong (although I doubt even the ACLU would take up such an argument)?