You can switch to the smaller diameter 251 damper without any problems. Some 251's didn't even use dampers. I am running the larger diameter damper on my 265. As a matter of fact, I spent over a year looking for the larger damper and then had to send it to Damper Dudes to be rebuilt. I'd like to see a photo of your dampers to compare with mine.
A problem I have with the larger damper is that the stock fan and pulley cause the blades to hit the larger damper. From what I can tell, the larger damper was used on 265's that ran the dual carb and exhaust setup. Some of those 265's also used the sodium filled valves.