Went to the Fossil Fest in Round Rock, and I came home with a kit to clean off a fossil fish. Though it'd be worth a try. The slab is from Wyoming, about 3" tall and 5" long, and they provided a tool that was really just a paintbrush with a steel nail attached to the other end.
It's a bit beat up. Some of that was me and my inexperience, some of it was already like that. A lot of fin rays were already broken, and I'm pretty sure some of the fins were outright missing their tips even before I got into here. Maybe something nibbled on the edges a bit?
I'm looking for some advice on how to do better, basically. Most of the skin came off, for one thing. There were especially loose flakes of matrix that looked like I could remove them with my fingertips, and whenever I touched one with the nail to take it off, it took the skin with it. I'm not sure if that was my fault or not. It also looks sort of like some of the skin was pulled towards the tail somehow.
Damage or not, I think it worked out pretty well for my first try, at least it's mostly intact.