If you are encountering this error installing the x64 version of VirtualBox on Windows 7, then there is a good chance it is because the MSI installer service is trying to execute a custom action in 32-bit mode. The error message about the Windows Installer is somewhat misleading in this case.
To verify this is the case, run the VirtualBox executable from a command prompt with the /logging parameter and review the logfile created. If this is the cause of the problem, you will see an error like this in the log:
MSI (s) (DC:98) [12:00:00:000]: Custom Action Server rejected – Wrong Context
MSI (s) (DC:B8) [12:00:00:001]: CA Server Process has terminated.
The fix is a very simple Windows Registry change using to set the WOW64 entry to 0 (zero) for the msiserver i.e.
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\msiserver]
WOW64=0
Note: For the change to take effect, the Windows Installer Service must be restarted under Services.
There is a more detailed explanation by Aaron Stebner over on MSDN: http://blogs.msdn.com/b/astebner/archive/2011/05/16/10165211.aspx