A colleague just asked me this question. I had to confess that I wasn't sure of the answer.
It appears that a range of species from a number of different families eat sponges. These include some species of angelfishes (Pomacanthidae), wrasses (Labridae), leatherjackets (Monacanthidae), boxfishes (Ostraciidae) and pufferfishes (Tetraodontidae) For some species, sponges comprise over 70% of the diet. The references below provide additional details.