Tuesday, June 15, 2010

VMware vmmon compilation issues on 64-bit kernel 2.6.35

The vmmon module in VMware Player 3.x and Workstation 7.x fails to compile on the latest 64-bit linux kernel 2.6.35:

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmmon-only'
make -C /lib/modules/2.6.35-2-generic/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.35-2-generic'
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/driver.o
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/driverLog.o
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/hostif.o
/tmp/vmware-root/modules/vmmon-only/linux/hostif.c: In function ‘HostIFReadUptimeWork’:
/tmp/vmware-root/modules/vmmon-only/linux/hostif.c:2004: warning: ‘newUpBase’ may be used uninitialized in this function
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/iommu.o
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function ‘IOMMU_SetupMMU’:
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:156: error: implicit declaration of function ‘iommu_map_range’
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function ‘IOMMU_VMCleanup’:
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:403: error: implicit declaration of function ‘iommu_unmap_range’
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/iommu.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.35-2-generic'
make: *** [vmmon.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmmon-only'

In my case this is on 64-bit Ubuntu Lucid with the linux-image-2.6.35-2-generic image from the kernel-ppa's: https://edge.launchpad.net/~kernel-ppa/+archive/ppa

Using VMware-Player-3.1.0-261024.x86_64.bundle from here: https://www.vmware.com/products/player/

Within the vmmon sources, the iommu.c file has function calls to iommu_map_range and iommu_unmap_range. After some poking around I figured these calls need the '_range' suffix removed. Update: these kernel changes are documented here.

After installing VMware, the following commands patches the vmmon sources:

cd /tmp
tar xvf /usr/lib/vmware/modules/source/vmmon.tar -C /tmp
perl -pi -e 's,_range,,' vmmon-only/linux/iommu.c
tar cvf /usr/lib/vmware/modules/source/vmmon.tar vmmon-only

To recompile the kernel modules, run 'vmplayer'.

If anyone knows how to stop VMware from recompiling the modules every time please let me know.

Update: 

The recompiling on each load is likely something to do with the "vsock" kernel module. The default installation enables it.

The vmware start-up output looks like this:

~# /etc/init.d/vmware start
Starting VMware services:
VMware USB Arbitrator done
Virtual machine monitor done
Virtual machine communication interface done
VM communication interface socket family failed
Blocking file system done
Virtual ethernet done
Shared Memory Available done

The module failing to load is most likely causing the need to recompile. Also confirmed during the (re)compilation:

ERROR: modinfo: could not find module vsock

The easiest workaround I came up with is to disable the vsock module:

~# nano /etc/vmware/config

And change the vsock configuration line to:

VSOCK_CONFED = "no"

47 comments:

  1. Thanks A lot for this great howto. I have the same problem after move from 2.6.32 to 2.6.35 but now everything works great with Vmware Player 3.1 and the new Intel Video drivers. Thank you !

    ReplyDelete
  2. Sorry for the double post, but I investigated a bit. I found a patch for the vsock module, so it will compile right (tested on brian rogers 2.6.35):

    http://www.debuntu.org/how-wmware-workstation-7.1-ubuntu-maverick-meerkat-10.10

    ReplyDelete
  3. Thanks a bunch... I'm using 2.6.35-rc6, custom compiled, on Ubuntu 10.04 with vmplayer 3.1.0 build-261024.

    Alan

    ReplyDelete
  4. Hey! Thank you! Works for VMware Workstation 7 installs on 2.6.35 as well. I'd have saved myself $70 US if I'd have found your posting first. However, you got VMware Workstation 7 up and running for me. Good luck with the research and the dissertation!

    Sincerely,

    John Moore, PhD

    ReplyDelete
  5. Thank you so much!!!!
    You solved my problems with my vmware-player in Ubuntu Lucid (10.04) amd64.

    ReplyDelete
  6. Wohoo thanks a lot ... It's a shame, vmware hasn't fixed this with 7.1.1!!

    ReplyDelete
  7. Great.. thanks a million! Just installed Maverick, and was struggling to get VMWare up and running... lovely post!!!

    ReplyDelete
  8. Nice work! That's a very elegant fix. Much obliged :)

    ReplyDelete
  9. I'm just repeating what everyone else said. Thank you so much!!!

    ReplyDelete
  10. saved by this post. a heartfelt thank you.

    ReplyDelete
  11. Thanks for the post it works very well for the vmon module in Ubuntu 10.10 x86_64 and VMware Workstation 7.1.2 build-301548, for vsock I rather the patch proposed by shiba89, works well too. Thanks again.

    ReplyDelete
  12. Thanks a lot, this seems to work for me (Ubuntu/Maverick 64)

    ReplyDelete
  13. Dude, YOU ARE AWESOME!!!!!!!!!!!!

    (really, i mean it.) thanks a million.

    ReplyDelete
  14. for vsock I have found that:
    http://www.linuxinsight.com/persuading-vmware-workstation-7.1-to-cooperate-with-linux-kernel-2.6.35.html

    ReplyDelete
  15. for vsock error I have found :
    http://www.linuxinsight.com/persuading-vmware-workstation-7.1-to-cooperate-with-linux-kernel-2.6.35.html

    ReplyDelete
  16. Fantastic! This is good stuff. Impossible for me to figure out on my own, thanks very much!

    ReplyDelete
  17. Your fix worked just great. I've just upgraded to Ubuntu 10.10 and without your fix I would had no choice but to go back to an older version of Ubuntu just to have VM Player running.
    Thank you for the great post.

    ReplyDelete
  18. Thank you! works also with 32 bit and VMWare Workstation 7.1.2

    ReplyDelete
  19. Thank you very much.
    It works for my Ubuntu 10.10

    ReplyDelete
  20. Thanks for the info. After upgrading ubuntu to 10.10, I was worried that vmware player wasn't compiling, but I was back up and running after a couple of minutes of reading this.

    ReplyDelete
  21. Thanks for the info, this helped me compile the modules. The command to recompile modules is

    'vmware-modconfig --console --install-all'

    ReplyDelete
  22. Hi there,

    I had the same issue - the player wanted every time that the modules were compiled after program start.

    VSOCK_CONFED = "no"

    That helped me a lot.

    Best wishes for 2011 !!!

    ReplyDelete
  23. cd /tmp
    tar xf /usr/lib/vmware/modules/source/vsock.tar
    perl -pi -e 's,(\w+)->compat_sk_sleep,compat_sk_sleep(\1),' vsock-only/linux/af_vsock.c
    cat <<EOF| patch vsock-only/shared/compat_sock.h
    @@ -204,4 +204,9 @@
    # define compat_sock_create_kern sock_create_kern
    #endif

    +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 34)
    +# undef compat_sk_sleep
    +# define compat_sk_sleep(sk) sk_sleep(sk)
    +#endif
    +
    #endif /* __COMPAT_SOCK_H__ */
    EOF

    tar cf /usr/lib/vmware/modules/source/vsock.tar vsock-only

    ReplyDelete