A colleague just asked me this question. I had to confess that I wasn't sure of the answer.
It appears that a range of species from a number of different families eat sponges. These include some species of angelfishes (Pomacanthidae), wrasses (Labridae), leatherjackets (Monacanthidae), boxfishes (Ostraciidae) and pufferfishes (Tetraodontidae) For some species, sponges comprise over 70% of the diet. The references below provide additional details.
- Randall, J.E. & W.D. Hartman. 1968. Sponge-feeding fishes of the West Indies. Mar Biol 1:216–225.
- Ruzicka, R. & D.F. Gleason. 2008. Latitudinal variation in spongivorous fishes and the effectiveness of sponge chemical defenses. Oecologia. 154:785–794.