For some strange reason the term 'small flake of bone' sticks in my mind as the likely evidence to be found of early fish. It came to mind when I noticed this shape under the loupe.
My dating of Ordovician is based on the fact that I find the cross sectional shapes of Prismostylus in other of the distinctive rocks. I'm fairly sure they are actually crumbs that rolled of onto the lake shore during construction of a rail bed.
It's so small I had a hard time getting a clear shot of it, but do you think there is a chance ?