Backfires only happen from lean conditions. You have a vacuum leak.
Mine was a hard one to find. It was the vac line to the trans. The rubber hose down right at the trans.
Another time it was the grommet on the brake booster. Another time it was the booster itself. Yet another time it was the vac advance canister.
The best way to find it is to disconnect each vacuum hose one at a time and plug it. Eventually you'll find the culprit. If it's not a hose try the old carb cleaner trick.
I had the same problem. Took me months to find it.
