I am a bit worried about your engine.
I have seen the cam bearings fail on new rebuilds. What brand bearing did you use?
They will mash out on the bottom and smear babbit material over the oil holes that lube the cam.
You say there is little oil flow up above. What is the idle oil pressure? Hot.