« .NET framework libraries source code to be released | Home | Singularity source code released »
Virtual machines performance comparison
Wednesday, October 24, 2007
I have always been a big fan of virtualization. I started using virtualization with a demo of Virtual PC in 2001 as a testing platform while developing a Windows tweaker called SysTuner. After Microsoft bought Virtual PC from Connectix in 2003 and released Virtual PC and Virtual Server for free, I was really happy that I finally had a great testing platform for all my operating system experiments.
Today, there are many free virtualization programs. Even though I am a (mostly) happy user of Microsoft’s Virtual PC 2007, I started to have concerns about it’s performance. To find out which virtualization program would fit my needs better, I made a little performance comparison.
Testing environment
For the test I used a physical system with Intel Core 2 Duo processor @ 2.4 GHz with 2 GB of RAM. The host operating system was Windows XP Professional. The guest systems were configured to use 256 MB of RAM and only one processor core. Operating system used was again Windows XP Professional.
The test was done with Virtual PC 2007, VMWare Server 1.0.3 and VirtualBox 1.5.0. I also wanted to test VMWare Player, but it showed up that I am not allowed to install VMWare Server and Player on the same computer (and I need the Server version for some work related stuff).
The test
I used two synthetic benchmarks to test the virtual machine performance: SiSoftware Sandra Lite 2008.1.12.34 and Super PI 1.1.
Following results were obtained without virtual machine additions installed (virtual machine additions are a set of drivers providing optimized communication between guest operating systems and virtualization software):
| VMWare | Virtual PC | VirtualBox | |
| Processor Arithmetic | |||
| MIPS | 12254 | 10121 | 33534 |
| MFLOPS | 8587 | 6766 | 22411 |
| Processor Multi-Media | |||
| Multi-Media Int x4 iSSE | N/A | 26972 iit/s | 89424 iit/s |
| Multi-Media Float x4 iSSE2 | N/A | 33096 fit/s | 109290 iit/s |
| Physical Disks | |||
| Drive index | 40 MB/s | 44 MB/s | 424 MB/s |
| Random Access | 8 ms | 10 ms | 7 ms |
| Memory Bandwidth | |||
| Int Buff’d iSSE2 Memory Bandwidth | 6307 MB/s | 1402 MB/s | 30709 MB/s |
| Float Buff’d iSSE2 Memory Bandwidth | 6319 MB/s | 1325 MB/s | 35093 MB/s |
| Memory Latency | |||
| Memory (Random Access) Latency | 105 ns | N/A | 33 ns |
| Speed Factor | 85 | N/A | 82.6 |
The VirtualBox results are the best in every category. But as you can notice, the numbers are really strange (the Drive index of 424 MB/s is even better than the drive index of the host computer!). My theory is that VirtualBox has some serious problems with hardware clock emulation and this makes the results incomparable. To make some use of the data I collected, I made also a second test with virtual machine additions installed. I think it’s safe to compare the numbers obtained with and without additions on each product.
| VMWare | Virtual PC | VirtualBox | |
| Processor Arithmetic | |||
| MIPS | 13990 | 10489 | 55000 |
| MFLOPS | 9669 | 6973 | 36674 |
| Processor Multi-Media | |||
| Multi-Media Int x4 iSSE | 83863 iit/s | 27971 iit/s | 137573 iit/s |
| Multi-Media Float x4 iSSE2 | 45656 fit/s | 34202 fit/s | 180852 iit/s |
| Physical Disks | |||
| Drive index | 40 MB/s | 44 MB/s | 498 MB/s |
| Random Access | 9 ms | 10 ms | 8 ms |
| Memory Bandwidth | |||
| Int Buff’d iSSE2 Memory Bandwidth | 7717 MB/s | 4002 MB/s | 43293 MB/s |
| Float Buff’d iSSE2 Memory Bandwidth | 7675 MB/s | 4073 MB/s | 40207 MB/s |
| Memory Latency | |||
| Memory (Random Access) Latency | 94 ns | N/A | 20 ns |
| Speed Factor | 84.4 | N/A | 71 |
The performance gain is visible in almost every area. It gives us a really good reason not to forget to install the virtual machine additions.
I was somehow disappointed with the results I got. I wanted to see the performance differences between virtualization software but the only thing I got was a bunch of numbers with no possibility of objective comparison between categories. I had to do a second test, which would allow me to do a somehow more objective comparison. So I made several runs of Super PI test:
| VMWare | Virtual PC | VirtualBox | |
| Super PI | |||
| Super PI 4 MB | 2:00 | 2:02 | 1:59 |
I also used a physical stopwatch to ensure that the time is measured properly. The results were quite surprising. I did not expect any time differences at all because user-level code should run at native speed on any virtualization software. The three-second interval was probably a product of context switching and scheduling in the kernel mode. I finally got the numbers I was looking for.
The results seen here are the results of synthetic benchmarks. Real-world results may be completely different.
Topics: Performance | 2 Comments »
November 1st, 2007 at 2:52
[...] up my data nice enough to present. So hats off to Michal, and check out his post "Virtual machines performance comparison" and a snippet is [...]
November 1st, 2007 at 8:13
[...] – virtualbox 1.5 vs. vmware server 1.0.3 vs. virtualpc 2007 [...]