NetBSD (and others) and the Ruby Benchmark
Jamie 'ober' Fournier has wondered what the performance of
the Ruby language on NetBSD is, in comparison to other
operating system platforms. To answer the question,
he ran the Ruby benchmark
on a number of operating systems:
FreeBSD, OpenBSD, Linux, OpenSolaris ... and NetBSD, of course.
describes the list of operating systems (and versions!) tested,
a bit of configuration for each operating system, and the hardware
and software versions used.
There are two charts available,
the first one
showing results for Ruby version 1.8.7,
the second one
with results for Ruby version 1.9.
Please note that you can click on the graphs to get the corresponding
ruby code - which is not commented too good, and thus leaves it to
own interpretation on what exactly the test actually benchmarks.
Here's an example graph:
From a first look, NetBSD seems to be good for CPU-intensive
applications (i.e. it has less overhead), but performance
with virtual memory, regular expressions and threading
can still be improved.
From looking at all the tests, it can be said that NetBSD makes
a pretty good platform for Ruby deployment.
For more, follow
the discussion on the tech-perform list.
[Tags: benchmark, ruby]
NetBSD 5.0: Overview and Benchmarks
Andrew Doran has made an overview of NetBSD 5.0,
available as HTML and
which includes a general description of what
NetBSD is, what's new in 5.0 and what is important
for users of workstations, servers and embedded apps as well
as for developers and hobbyists. Besires the lists of features,
the most interesting part is a number of benchmarks
that show that NetBSD can compare well to FreeBSD
and Linux (by usually beating them 8-).
A list of possible features for NetBSD 6.0 concludes.
Some details on the benchmarks:
- hackbench IRC server simulation:
- sysbench: MySQL OLTP simluation:
- build.sh: Compile benchmark:
[Tags: benchmark, hackbench, releases, sysbench]
More kernel works: audio, benchmarks, modules
In the past few weeks, Andrew Doran has made another bunch of
changes to NetBSD's kernel area, including interrupts in NetBSD's
audio framework, benchmarks of the system, and the handling of
SMP & audio:
One area that hasn't been changed
for moving towards fine-grained kernel locking was NetBSD's audio
subsystem. As audio recording and playback is mostly done via
interrupts, and as latency in those is critical, the audio
subsystem was moved to the new interrupt handling system.
The work can be found on the ad-audiomp branch, more information
is available in Andrew's posting about the MP safe
audio framework and drivers.
Changing a system from inside out is a huge technical task.
On the way, performance measurements and tuning are needed to
make sure that the previous performance is still achieved while
getting better performance in the desired development area.
As a result,
benchmarks results from Sun's
posted, which allow comparison not only against Linux and FreeBSD, but
also between NetBSD-current and NetBSD 4.0, in order to identify if any bad
effects were added. All performance tests were made on a machine with 8
CPUs, and the areas tested cover "small" (micro) areas like various system calls. Of course this doesn't lead to a 1:1 statement on how
the systems will perform in a real-life scenario like e.g. in a
database performance test, but it still help identifying
problems and gives better hints where tuning can be done.
Another benchmark that was also made in that regard comes
from Gregory McGarry, who has published
performance measurements previously.
This time, Gregory
has run the lmbench 3.0 benchmark on
recent NetBSD and FreeBSD systems as well as a number of previous
NetBSD releases - useful for identifying performance degradation, too!
One other benchmark on dispatch latency
run was made by Andrew Doran: on a machine
that was (CPU-wise) loaded by some compile jobs, he started a
two threads on a CPU that wasn't distracted by device interrupts,
and measured how fast the scheduler reacted when one thread
woke up the other one. The resulting graph
shows that the scheduler handles the majority of requests in less than
10us - good enough for some realtime applications?
are another area that's under heavy change right now, and after
recent changes to load modules from the bootloader and the kernel,
the kernel build process was now changed
so that pre-built kernel
modules can be linked into a new kernel binary, resulting in a
non-modular kernel. Eventually, this could mean that
src/sys is built into separate modules, and that the (many) existing
kernels that are present for each individual platform -- GENERIC,
INSTALL is already gone, ALL, etc. etc. -- can be simply linked from
pre-compiled modules, without recompiling things over again for each
kernel. Of course the overal goal here is to speed up the system (and
kernel!) build time, while maintaining maximum flexibility between
modules and non-modular kernels.
With the progress in kernel modules, it is a question of time
when the new kernel module handling supercedes the existing
loadable kernel modules to such an extent that the latter will
be completely removed from the system -- at least the
was alredy proposed, but I'd prefer to
see some documentation of the new system first. We'll see
what comes first! (Documentation writers are always welcome! :-)
[Tags: benchmark, kernel, smp]
Performance improvements: numbers and pictures
Andrew Doran, who was recently hired by the NetBSD project to
work on NetBSD's SMP implementation, has done a lot of good work,
and he has
some of his work from the vmlocking-branch into NetBSD-current.
Effects of this are that
time for build.sh on a quad-Opteron went down by ~10%.
Andrew also updated
his previous benchmarks,
about his recent results:
``Most of the sysbench runs that I've seen to date have sysbench running on
the same machine as the database. That's a good test but with the exception
of small installations and out-of-band activity, production setups rarely
look like that. So I ran sysbench itself on a seperate dual core system.''
There are images that compare
NetBSD 3 with NetBSD-current
(where most of Andrew's changes are now), and
NetBSD-current compared to Linux, FreeBSD, and OpenBSD:
The original benchmarks didn't include Solaris/x86, so Jaime Fournier
sat down and
repeated the test
(on a single system).
that NetBSD beats Solaris by ~25% in the ReadOnly test,
and that they're about on par in the ReadWrite test,
with NetBSD kicking in earlier WRT the number of client threads,
but Solaris keeping up longer before they both degrade.
The courves are quite similar, and my guess is that there is some
room for finetuning there:
[Tags: benchmark, mysql, performance]
Application-level benchmarking NetBSD and FreeBSD
There was some public outcry some time ago when
Gregory McGarry did a micro-benchmark
Comparing of NetBSD 2.0 and FreeBSD 5.3 - microbenchmarks are
useless, only application-level benchmarks show real differences, etc.
Leaving aside all the arguments, Thor Lancelot Simon did some musing
on the subject, and taking two identical MacMinis, he ran them with an
application that's common to many of us: compiling stuff. "Stuff" in
that case was building a release of the NetBSD 4.0 release branch, using
NetBSD-current (as of may 2007), and FreeBSD 6.2RC2. The results?
I.e. 97 minutes build time on NetBSD, and 115 minutes on FreeBSD.
Juggling numbers, one could say that NetBSD was about 20% faster
in this case. Wow!
Build platform: NetBSD 4.99.19 i386
build.sh started: Tue May 15 18:24:20 EDT 2007
build.sh ended: Tue May 15 20:01:29 EDT 2007
Build platform: FreeBSD 6.2-RC2 i386
build.sh started: Tue May 15 18:25:28 EDT 2007
build.sh ended: Tue May 15 20:20:47 EDT 2007
Maybe it should be noted that the building was done with 4 concurrent
processes, to keep both cores of the MacMinis warm.
On the NetBSD side, the SMP implementation is also the new "newlock2"
based code by Andrew Doran that removes the kernel big lock for some
paths inside the kernel, which may be responsible for the boost in performance.
for more details.
(P.S.: This isn't intended as FreeBSD bashing, in case it's not obvious.
Just a few facts...)
[Tags: benchmark, freebsd]
Article: Linux versus Windows: OS impact on uptime and speed
From the article
"Linux versus Windows: OS impact on uptime and speed":
``There are many factors which affect Website availability and performance from end user perspective, namely ISP Internet connection, server location, server parameters, programming language, application architecture and implementation. One of the critical parameters is a selected Operational System (OS). Most users often need to select between Linux and Windows, two popular choices for web servers. By providing free monitoring service, we at mon.itor.us collected large amount of data to perform a unique analytical research examining OS correlation with uptime and performance. ''
There are three results of interest here:
Citing the article again: ``Summarizing, statistically Linux based servers provide better availability and response speed than Windows servers. Absolute leader is NetBSD in both categories; OpenBSD and Solaris also demonstrate good results.
- Amount of monitored websites by detected operating system:
So out of ~13000 websites, it's only 238 (2%) that run NetBSD.
Not terribly high, esp. when 774 sites run Minix (wtf?!), but well
- it's lonely at the top. 8-)
- Analyzing wesite performance statistcs by operating systems
sorted by uptime:
So over a period of at least three weeks, NetBSD
showed best uptime results. Yai!
- Analyzing websites performance statistics by operational systems
sorted by response time:
Again NetBSD scores the top position over three weeks when
looking at the response times.
We hope this data will be valuable for our community in their hosting selection process.''
[Tags: benchmark, uptime]
Catching up: events, articles, benchmarks, summer of code...
So I was away for a few days, being sick and then giving a talk
at the Chemnitz Linuxdays and then off for a few days in
visiting Vienna & Zotter,
and there's a backlog of stuff that happened in
NetBSD's madhouse^Wwonderful world. Here's a quick run-down of things
that I'm too lazy to post single items on:
So much for now. Enjoy!
I was only there for my presentation on sunday, due to not feeling
too well the days before. Still, Stefan, Jörg, Charlie and many
others staffed the booth just fine, and I think every single household
in and around Chemnitz has a NetBSD install and/or Live CD now. :)
Related talks to mention are
Stefan Schumacher's talk on
hardening systems with systrace
My own talk was not too NetBSD specific, showing an application on
how to implement dynamic DNS with some retail web/domain hoster.
Slides for my talks are available
as OpenOffice .ODP
and as PDF.
(I'll reconsider the move from TeX/prosper to OpenOffice after it was
NOT as easy as I expected to find a machine running OOo for presentation
purpose, after my laptop's harddisk crashed on the way to Chemnitz!)
- While at roadshows:
Stefan Schumacher has made DIN A4 pkgsrc flyers in
- NetBSD's puff-based FUSE implementation "refuse" is now in a state
to also run the
filesystem, which offers read/write support for NTFS.
It's available from
- Google News found me an article that
NetBSD stack supports Geode NAS design:
``Wasabi Systems Inc.'s BSD-based NAS (network attached storage) software stack now supports a Geode-based reference design from AMD. Wasabi Storage Builder for NAS, combined with AMD's Geode LX NAS RDK (reference design kit), provides a secure, reliable platform for the development of NAS devices, according to Wasabi. ''
While that's all fine for Wasabi, it should be noted that
whatever the company Wasabi offers is not automatically available in the freely
available operating system called NetBSD. Integration efforts
would have to happen first, so the headline of that article
is unfortunately misleading if not to say plain wrong!
- Another article that's more to the point:
Julio M. Merino Vidal has worked on getting
multiboot support into NetBSD, and in his article
``Making NetBSD Multiboot-Compatible''
he talks more about it.
- Andrew Doran has done lots of work on NetBSD's thread and SMP
implementation recently, and he has made a comparison between
performance of the Scheduler-Activations-based code in
NetBSD 4 and the one that will be in NetBSD 5 (AKA NetBSD-current,
currently numbered as 4.99.13). See
his mail to tech-kern
or watch the images for
'make cleandir' on an empty source tree
the MySQL supersmack benchmark.
- Google runs another Summer of Code,
and this year it's not clear upfront who will be allowed as mentoring
organizations. NetBSD is ready to participate again, and there's
official announcement from NetBSD
about this, including pointers to
our suggested/wanted list of projects
the project application HowTo.
People interested in submitting a project proposal (via google!)
are encouraged to use the remaining time until the deadline to
discuss their proposals on the public NetBSD tech-* lists!
(Personally I'll try to stay out of GSoC this year to finish some
reallife work. At least that's the plan so far ...)
- Three new security advisories were released:
- Another article that doesn't mention NetBSD but g4u:
``How to Install a New Hard Drive: Tech Clinic''
by Joel Johnson. From the article:
`` To make your new drive work like your old drive, you'll need a disk
"cloner." There are a myriad of options, from commercial solutions such as
the old favorite Ghost from Symantec ($70; symantec.com) and Copy
Commander from VCom ($35; v-com.com) to free applications, such as
MaxBlast from Maxtor, that come bundled with hard drives. If you're
comfortable mucking around with Linux/BSD, I've had great luck with the
free g4u application. If you have a local file server, you can even send
the disk image from your laptop to an FTP site, install the larger drive,
then FTP it back to your laptop, obviating the need for a drive enclosure''.
[Tags: Advocacy, Articles, benchmark, clt, Events, g4u, google-soc, hubertf, pkgsrc, Security]
Overhead of vnd(4) as disk device in Xen
When you run a Xen domU, the question is what to use as a
harddisk to run the operating system from. Options are
a dedicated harddisk partition, or using a vnd(4) disk-image.
While disk-images are a lot more flexible the manage than
partitions, there's a drawback because access to anything
on the image goes through the dom0's filesystem, and is
How much slower exactly using a disk image instead of a raw
was asked on tech-perform,
and Greg Troxel gave
some interesting numbers.
Now what I wonder is
what impact the filesystem type has...
[Tags: benchmark, xen]
Experimental benchmark result of recent PC UNIX file systems
I haven't seen
before, and it's a bite dated (from 2003,
speaking about NetBSD 1.6ZC, FreeBSD 5.1 and RedHat 9), but I
think the results are not uninteresting, citing from the
Seems there's more than 1-2 operating systems that are "good" on PCs.
(Oh wait, NetBSD isn't strictly for PCs... but I guess adding all those
abstraction layers that make NetBSD portable did not hurt performance
too hard; I'd be interested to see a 2006 edition of that benchmark)
- Linux software RAID5 25MB/s
- NetBSD software RAID5 10MB/s
- FreeBSD software RAID5 5MB/s
- Linux hardware RAID5 10MB/s
- NetBSD hardware RAID5 25MB/s
- FreeBSD hardware RAID5 20MB/s
- Linux software RAID5 45MB/s
- NetBSD software RAID5 33MB/s
- FreeBSD software RAID5 25MB/s
- Linux hardware RAID5 25MB/s
- NetBSD hardware RAID5 30MB/s
- FreeBSD hardware RAID5 24MB/s
[Tags: benchmark, filesystem, raid]
Ideas for 2nd Fefe benchmarks?
Reading Fefe's (german language) weblog he writes:
``I'm working on new benchmarks similar to my
old ones, but I want to
measure more this time. I have taken about 30GB of image data from a
customer's webserver, and logged about 50.000 web requests, and wanted
to replay them as fast as possibl. To measure the filesystem I want to
keep 10% of the 30GB images seperately, and then use tar to unpack and
measure. But the tests so far are very negative for BSD (10 minutes
unpacking on Linux with ext3 against 26 minutes under FreeBSD 6 with
UFS2. I'm therefore looking for benchmark ideas in this category (so not
"install postgres and shove in the following selects"), which are
friendly to BSD and unfriendly to Linux. There has to be something?!
Please mail your hints to me!''
Given the last round of benchmarks and the
that were made to
NetBSD after them, this sounds very interesting. Ideas? What is it that
NetBSD can do better than Linux (and FreeBSD :-). Voice your opinion
on the tech-perform mailing list!
[Tags: benchmark, fefe]
Grab the RSS-feed,
or go back to my regular NetBSD page
Disclaimer: All opinion expressed here is purely my own.
No responsibility is taken for anything.