虚拟机的基准测试，分析

Question

On many different sources you can read about time keeping issues in virtual machines. As every benchmark relies on time keeping i am not sure how to interpret e.g. apache benchmark or xdebug profiler results on vmware and how credible they are.

http://communities.vmware.com/docs/DOC-5581

VMware suggests to build a special vm for performance testing. There are many tips in addition which seems to be workaround solutions (install VMware Tools, special configuration and so on).

Especially when it comes to disk i/o performance issues i´m very concerned about how accurate the results are. Rasmus Lerdorf talked about it at drupalcon, saying that some of his performance improvements regarding stat calls aren´t visible in his VMware benchmarks because VMware has special disk caching you can´t compare to physical disks:

http://www.nowpublic.com/tech-biz/rasmus-lerdorf-simple-hard-drupalcon-2008-key-note

After all this I´m not sure if VMware is capable for any benchmarking at all.

Answer 1

A lot depends on what you are trying to do.

1. Just keep measurements to look for regressions
2. Find time-drains that you can fix to improve performance

Many people think these are the same, but I think that comes from limited experience actually doing (2).

The way I do (2) is not measuring, but asking, at a few random points in time, what is happening and why. For that, it doesn't matter very much if it's on a virtual machine or not. If it's doing something sneaky that takes a large fraction of time, you will see that whether or not it's on a virtual machine. The exact fraction doesn't matter. Exposing the sneaky activity is what matters.

Here's an example in python, but you can do it in any language.