That doesn't sound right to me at all, the Vref shouldn't be involved at all. To control timing with an HEI four things need to be wired/set.
1) Inside the cap. Pickup coil in the distributor needs one wire going to the "TACH" port on the relay board, and the other wire from the pickup coil just needs to be grounded.
2) Outside the cap. Relayed** +12v needs to go to the power side of the distributor, essentially like normal with the 4-pin, then "S5" from the relay board goes to the negative terminal on the HEI cap.
3) In the Megasquirt. On the 3.0 board make sure the VR sensor jumpers are in place for VR IN. You may need to adjust the R55/R56(?) pots to clean the tach signal up. I think turning them all the way counter-clockwise will get it started up, but double check the directions.***
4) In the software. Set the settings for Basic trigger wheel up in TunerStudio, make sure you have the right dwell settings for one coil. 2.5-3.0ms I think? Expect to adjust the trigger offset angle to get the timing light to match the megasquirt.
When it's all wired up, I'd unplug the +12v to the HEI cap and crank the engine and look for rpm in TunerStudio. You may need to adjust the R55/56 pots to get tach signal. Do not bother hooking up the +12v to the HEI cap until you get a good rpm reading of at least 100rpm while cranking. My other tip at this point would be, set your timing to "fixed" under ignition settings (instead of table) to 8º or 10º, trigger angle to 5º and it should start right up. Then adjust your trigger angle until the timing light shows the same timing you are commanding "fixed".
**You can leave the factory +12v wiring in place and it'll start fine and such, BUT it could overheat the ignition coil as the 4-pin had a protection circuit in it. It's recommended the distributor coil(s) get their power from the fuel pump relay. So if the fuel pump isn't running, the coils don't have power either.
***Once running, I've left the cover off the megasquirt and adjusted the pots while the engine was running. I believe the base setting will get it idling and a couple thousand rpm, but if you see it loosing tach signal at higher rpm, you can adjust it, rev it, adjust it, rev it, etc until it revs up the full rpm range and picks up the tach signal at low rpm too. Then put the lid back on it.
