Edelbrock AVS carb getting mounted to a factory quadra-jet intake manifold would definitely require an adapter
These in order of appearance, came off the Parts Avatar site, the Amazon site and the Summit Racing site , all the same, Edelbrock P/N 2696, and the prices are close within a range of 50-60 dollars.
The why of the adapter is that your manifold is drilled for the Q-jet meaning that it has a pair of small primary ports and a bigger set for the secondaries. The AVS is a square bore meaning that all four holes are identical for diameter. The AVS is a direct fit to the Edelbrock Performer series of intakes. (or Holley, or a Chinesium knockoff).
At idle with the old adapter, did you happen to hear any whistling or air sucking noises? Mismatched gasket/adapter mounting faces would definitely be a cause for that to occur.