More to the point, has there been any progress on the subject since last year? For example, has any serious scientific paper been published on the comparative performance issues of the Mac OS X and GNU/Linux system calls?

Some testing I and some others did pointed in the direction that the Apachebench bug is in the client side. I don’t have an opinion if it is in ab itself or OS X. Under OpenBSD ab runs also stall in a similar way, I assume it to be the same problem.

I’m glad to hear that you are developing better ways to test web server performance because the one you are using now is probably the most non-realworld test there can be. Web servers in a production environment are seldom used from localhost only, and requesting one page only. Also in making good tests there should be as little changing variables as possible. When testing a server should differ make sure that the clients, pages served and apache configurations are identical.

Using one separate client for the apache testing would probably have done a great difference. If your better tests are not ready before the next article please at least use one invariant client. It would show if the problem is really in client or server side.

But I couldn’t help but react to stingmerman:

“The new Anandtech article still contains the AB flaws and tries to use their flawed benchmark to support their MySQL findings, which to me still indicates that they do not know what they are doing”

First of all, read what Dominik Wagner says:
“My recent investigations in the MySQL source code shows that the F_FULLFSYNC only is in place in the InnoDB table handlers, not in the MyISAM, which makes this argument quite unimportant”

We use MyISAM and I clearly indicated that in the article. No Fsync problem.

About Apachebench: “Why exactly does the client stall? Is it really a bug or is it running out of some resources? We didn’t delve deeper, as we are developing a less synthetic, closer to the real world benchmark to test web servers.”

We posted those benchmarks to show that the apachebench problem does not exist in linux and that the G5 does well there. I clearly indicated that you should take the apachebench results with a bit of salt. So attacking the article on that is a bit lame.

Is it a configuration issue, is it a 4.1 issue resolved in 4.2? Is it a bug in the testing methodology like the AB bug? I just feel dirty after reading their article, like I have been abused.

Linux 2.2 and 2.4 had the same kind of performance issues, overall performance increased significantly with fine-grained locking in 2.6.

Maybe apple will finally fix it in 10.5.