Okay, we're talking about two separate things, then.
Yes, the pointers themselves will always be larger on a 64-bit system, but the objects they point to are not necessarily larger if they're made up of atomic types.
The real question is... how much is the difference, really? If it's 10%, I don't see that as a real concern, especially since there's going to be some overhead in the WOW32 subsystem. But if the 64-bit version uses twice as much RAM...that would be something that would concern me.