Your lifters were new until you ran them on a bad camshaft. Now your new lifters are bad too.
Noise possibly comes and goes as the lifter rotates from a more to less worn position.
Not saying this is your problem for sure, but I'm placing my bet here.
