Pennsylvania is one of the few states, along with Delaware, Maryland and Virginia, that are cool with people collecting fossils in state parks. In fact, at Beltzville, if you go in the visitor center, they will be quite happy to hand you a map and an ID sheet, and circle the sweetest spot on the map for you. They were thrilled last year when they heard that word was getting out of their existence!
The fossils there are all Devonian marine species from the Mahatango Formation. They can be found easily enough by exploring the gravel along the lake banks or wading in the shallows, where they appear as casts and molds in various stages of weathering. Fresh breakage will reveal highly detailed impressions!