In my opinion, if they can't be the same size, you are much better with a larger header primary opening bolting to a smaller head exhaust port.
The extra size of the header opening will slow exhaust gas velocity slightly but the extra space inside the header tube should still allow for laminar exhaust gas flow and not create (or increase) any additional back pressure.
The problem with a primary header tube smaller than the head exhaust port is not so much a matter of flow restriction, although there is that and it does have some degree of negative effect on hp & torque.
The real issue is that the interference caused by the smaller header tube against the larger exhaust port either causes or increases exhaust gas turbulence in the header. This affects exhaust gas speed all the way from the point where the turbulence is created (where the header tube blocks some of the head's exhaust port opening) to the far end of the exhaust system. Instead of having a relatively clear path for the gas to escape while moving in a single direction, the turbulence forces the gas to tumble (not move in a single direction) and push its way out of the exhaust. That means slower exhaust gas velocity and increased back pressure.
One of the main purposes of port matching and gasket trimming is to promote laminar exhaust gas flow coming out of the heads, through the manifold/headers and down the exhaust system. Laminar flow can dramatically increase exhaust gas velocity and this promotes more efficient combustion chamber exhaust scavenging, increasing the suction needed to pull in the maximum air/fuel charge for the next firing cycle. It also helps reduce or minimize back pressure.
There is also the issue of reverse gas pulses traveling back up the exhaust system to the combustion chamber but that is a bedtime story for another time.
