NetBSD vs. Microkernels - From Kernel to Userspace and Abroad
A few weeks ago
I made a comment about NetBSD not being a "microkernel" despite
it's move to kernel modules. Antti Kantee wrote back to me (Hi Antti!)
reminding me of his work on RUMP, PUFFS and the like, which I think
does definitely deserve mentioning in that context:
With RUMP, PUFFS and the like, Antti is set on a mission to define
interfaces between kernel components, and move those kernel
components from inside the kernel to the userland. As a result,
the (once) kernel-code is now running as
processes that have all the benefits like memory protection
and not taking down the whole system in case of an error.
Examples that Antti has been bashing on started with
moving file systems to the userland, which resulted in PUFFS and
ReFUSE, an implementation of the Linux inferface of
File systems in USErland (FUSE). Not stopping at file systems
(which are traditionally used for accessing data on storage media,
like e.g. for ntfs-3g), Antti went further and moved into
getting network services like SSHFS going - in userland,
based on FUSE.
Not stopping there, more recent works include:
Remember when NetBSD got (re)FUSE? All of a sudden we got a
bazillion of filesystems back then. So why not turn this development around, and
make RUMP available as an interface for all sort of drivers
to other operating systems. That way, they can get our
file systems, usb stack and drivers, etc., and run them
in userspace as well. Developing a driver on one operating
system, and using it on many - a wet dream would come true!
Sounds impossible? Ye fear not, it's been done!
Arnaud 'stacktic' Ysmal already has ported RUMP to FreeBSD and Linux,
the work is available via
and there is also
Arnaud's page on
Rump on non-NetBSD Operating Systems for more information.
Whew... lots of research and development going on in this area,
and - getting back to the initial topic - we may well see an
operating system in the future that moves from the monolithic
to a microkernel approach, and it may or may not be called NetBSD.
Fact is, that a lot of research is going on in that area, on NetBSD,
here and now. Hats off, Antti!
[Tags: fuse, microkernel, nfs, ntfs-3f, puffs, refuse, rump, sshfs, tcp/ip, ukfs, usb]
Google and NetBSD Summer of Code Projects in 2008 - Midterm status reports (Updated)
Google's Summer of Code has passed the midterm date. With it, students
and their mentors were asked to give internal status reports of thei
works. While the internal reports themselves are not public, many of
our students have sent mail to NetBSD's public lists, giving details
on their status. Let's give a summary of the state of
affairs. Projects were students have posted reports come first:
No status report was sent by the following students (or at least I
haven't seem one). Data given here is from the project pages, mostly:
- wscons: Expansion for wstablet in NetBSD
Student: Jason W. Beaudoin
To support tables for the wscons console driver, a number of
changes to the wscons API have been proposed in the student's
status report. While there are a number of similarities with
the wsmouse interface, there are also a number of differences
that need to be worked around, e.g. absolute vs. relative
coordinates. Many of the proposed changes are implemented, and
we're looking forward to complete this project successfully
- subfiles: Subfile Support for NetBSD
Student: Adam Burkepile
Subfile allow to associate data with a regular file, just like
regular allow to associate data with a directory. New internal
data structures were defined to identify subfiles within the
file system, and tools like newfs and dumpfs were adjusted. An
API is being designed to access subfiles, and work to realize
the assorted functions is being approached.
Status report part
- hurdt: Hurd translators
Student: Marek Dopiera
Translators are programs which provide filesystems in user
space functionality. This is provided via NetBSD's RUMP
interface, and additional system calls and file system
operations have been defined to activate the server processes
when access to such a "translator" is made. Currently, the
translators are only implemented in NetBSD's ext2 file system,
as this allows testing of interaction with Hurd - Hurd's
support for FFS seems non-working, unfortunately. The project's
under busy development, and we're looking forward to see the
- lvm: Write and improve NetBSD LVM driver
Student: Adam Hamsik
This project implements the Linux LVM API (libdevmapper) on
NetBSD, to allow using Linux' lvm2tools for logical volume
management. The project's making excellent progress, there is an
ISO image (see URL in status report) as well as a
image available for testing,
reports show that linear volumes can be configured and mounted
- uvc: Add support for UVC devices (USB web-cams)
Student: Patrick Mahoney
A kernel driver was developed to read data from webcams using a
custom API, work to implement the video2linux API is under
way. Current challenges lie in NetBSD's USB stack, which lacks
support for isochronous transfers, which is used by many (but
not all) webcams. The project has made excellent progress so
far, and we're positive that the project will be a success.
- dvb: DVB drivers and kernel framework
Student: Jeremy Morse
This project implements a driver for Digital Video Broadcasting
to supplement bktr(4)'s TV card support. So far, a driver and
an API to transport data from the kernel to userland was
implemented, matching LinuxTV. The project's making good
progress, even due to conflicts with the academic schedule.
- install-tool: Customizable install tool for NetBSD
Student: Zachary Wegner
NetBSD's current installer, sysinst, is being split into
frontend and backend parts, with a configuration file building
the interface between the two parts. Untangling the current
mix of user interaction and install operation are ongoing, with
challenges like request of install media (think floppy #42) and
network configuration. Also, a parser for the configuration
file was written, and work not started yet is the frontend
creating the configuration file for the backend.
- fs-utils: File system access utilities
Student: Ysmal Arnaud
This project is using NetBSD's RUMP and the ukfs library to
access a file system image from userland. So far, makefs(1) can
generate a file system image, and it can now be manipulated as
well. Both a "file system console" as shell to operate on the
image as well as separate tools for single operations have been
designed. Many of the "normal" userland tools' functionality
like ls(1), cp(1) and rm(1) were implemented. This project has
made excellent progress so far, see the status report and
- cwrapper: pkgsrc: rewrite wrapper framework in C
Student: Amitai Schlair
The student was distracted by
travel obligations for some time, but work has started in
pkgsrc now. Current work includes a set of ATF tests to cover
the usage of the existing pkgsrc wrapper framework and a design
plan for the new wrapper implementation. The actual wrapper
implementation remains to be written, and we're looking forward
to see the it happen, including integration into pkgsrc plus
benchmarks on the increase in speed.
- atfify: Converting remaining regression tests to the Automatic Testing Framework
Student: Lukasz Strzygowski
The Automated Testing Framework was added to NetBSD as a result
of last year's Summer of Code. This year, all the remaining
regression tests from src/regress are being converted to
ATF. Test suites for tools (awk, grep, make, ...) and libraries
(libm, libpthread, ...) were converted so far. Tests for libc
are currently being converted, and kernel tests are next. We're
looking forward to get all of src/regress changed to ATF!
So much for now. We wish all our students good luck in the second part
of this year's Google Summer of Code, and are looking forward to see
your final results, which are due in 3-4 weeks from now. Keep on
- ext3: Implement Ext3 file system support
Student: Rus-Rebreanu Alin-Florin
This project intends to implement journaling in the file system
by reusing Wasbi's wapbl code. Unfortunately, little has
happend to reach this goal (to say the least), and the student
got AWOL. Interested
parties are welcome to try
(which also does ext3, despite the name) for now.
- packet-classes: Create an in-kernel API for "packet classes"
Student: Anish Babu
Nothing has happened in this project as well. I hear about
communication problems, and it remains to see if things move
forward here. :-(
- teredo: Implementation of RFC4380 (Teredo) in NetBSD
Student: Arnaud Lacombe
The project aims at creating a Teredo client, server and relay
via a kernel pseudo device and assorted userland tools.
The project is currently still in the analysis and design
phase, which will give it little time for completion -- as for
other students, this is due to a clash with the academic
schedule of the student. We're holding up hopes that the set
goals can be met at least partially.
- syslogd: Improve syslogd
Student: Martin Schütte
The project aims at implementing the upcoming IETF standards
for syslog transport over TLS (instead of UDP), a more formal
format of the messages themselv for easier automatic parsing,
and signing of messages to assert authentication, integrity and
correct sequencing of syslog messages. The first part of this
is already working, the second part is development. We're
looking forward to see this project succeed, as it will be of
benefit not only to NetBSD.
Fixed a typo (thanks tron!), got an update on the ext3 project,
and added a link to the status report of the
cwrapper project, which I've missed (sorry!).
[Tags: atf, dvb, ext3, google-soc, hurd, lvm, pkgsrc, rump, subfile, sysinst, syslog, teredo, ukfs, usb, uvc, wscons, wstablet]
RUMP - Userspace kernel file system framework
One of this year's summer-of-Code projects is Antti Kantee's
Userspace Kernel File System framework,
which has the goal of creating an interface to run file system code
in userspace. Antti did a lot of work in his previous
Userspace filesystem hooks
which also resulted in
puffs and (re)fuse.
This year's project goes one step further as not only the
are passed from kernel to userspace, but the full filesystem code
is ran in userspace, including memory management (buffer cache!).
This allows developing filesystem code in userland with the "normal"
kernel interfaces, but no panics if something goes wrong -- a simple
core dump, and that's it. To cite from Antti's mail:
``If you want to give it a whirl, upgrade, go to sys/rump, type "make", go
to fs/bin/$foofs and run the file system. It should work to a certain
degree provided your system comes with puffs. Does it do anything a
kernel file system wouldn't do? No .. except the core dump may be smaller
and faster and less intrusive. Oh, actually it does one cool trick: you
can run it on non-block devices, so you don't need to vnconfig the file
system image first (provided, of course, that the file system starts right
from the beginning of the image and that you wouldn't need a disklabel).''
For more information, see
email and his
Summer-of-Code project page.
[Tags: google-soc, puffs, rump, ukfs]
Grab the RSS-feed,
or go back to my regular NetBSD page
Disclaimer: All opinion expressed here is purely my own.
No responsibility is taken for anything.