There are three types of server virtualization, namely operating system virtualization, hardware emulation and paravirtualization. Each one of the three has their own disadvantages.
One serious drawback of operating system virtualization is that your choice of operating system is limited depending on the host operating system.
The major disadvantage of hardware emulation is that the hypervisor (virtualization software) hurts performance, and you will often find that applications run slower on virtualized systems.
One other drawback is that since the VMM (virtual machine monitor) acts as an interpreter between the hypervisor and the physical machine, there is need for device drivers to be installed between the hypervisor and the VMM, however you also have to update these drivers from time to time. But an ordinary user is unable to install these devices. This might lead to a situation where some resources may not run on virtualized environment when there are hypervisor drivers for them.
The main disadvantage of paravirtualization is that it is lightweight and needs that the guest operating system be modified so it to interact with paravirtualization Interfaces. This is only possible if you have access to the source code of the OS. This is only possible with open source OSs for example Linux.