I wonder if the erratic timing could be caused by harmonics. The RPM you experience it at is around the range it could happen at. Every engine is different.
Maybe get a different balancer and try it. A SBC will fit for testing on the dyno.
Inliner Member 1716 65 Chevelle Wagon and 41 Hudson Pickup Information and parts www.12bolt.com