Can I use the same .config file to compile 32-bit and 64-bit kernel?

Hi, I'm currently configuring a custom kernel on Fedora x86_64. The first option on the configuration menu is "64-bit kernel" which I think is the only option controlling the architecture of kernel. Copying this .config file to a 32-bit system directly and turning off this option, can I compile a 32-bit kernel which doesn't have any hidden problems? If this way is feasible, I think the only difference between these two kernels is architecture. Regards, Hao Lee