hubertf's NetBSD Blog
Send interesting links to hubert at feyrer dot de!
[20080428] Recent development related to puffs, ReFUSE, rump, and more (Updated)
NetBSD's kernel is under very active development these days, and while many changes are related to improve SMP, it's not the only area. An area where very interesting and unique work is being done is the filesystem interfaces that Antti Kantee is working. Things started out as a past year's Google "userfs" SoC project to implement an interface for running filesystem code in userland. The project was imported into NetBSD some time ago. On top of that, a library that mimics the Linux interface for filesystems in userland. Following the Linux name FUSE, the re-implementation is called ReFUSE (pun intended :). See the webpage about puffs, refuse, FUSE on the NetBSD website for more information.

Another project that was started by Antti after his work to run filesystem code in userland is "rump". The project allows to use "ordinary" filesystems that usually run inside the kernel, and mimic an environment similar to what's available inside the kernel, and move the whole filesystem into userland - verbatime, with no code changes! This allows to develop filesystem code in userland, and later on move it inside the kernel with no further changes - a bit step forward for filesystem development!

This all sounds rather easy, but as filesystems need to move data between storage and memory, a big issue in filesystems is interfacing with the virtual memory subsystem, and adding interfaces like puffs and ReFUSE also needs to consider VM for efficient transfers and caching.

Work in this area is still ongoing, and I've asked Antti about his recent achievements in this area[1]. While the only user-visible change is caching and performance improvements in the Secure Shell filesystem's handler "mount_psshfs", most of the changes are on the inside. Antti wrote me: ``The interesting ones from a programmer's perspective are probably:

  • Splitting userspace transport out of puffs in the kernel (putter)
  • Using putter to implement support for userspace block/char device drivers (pud). pud does still not have a userspace library similar to libpuffs. libpuffs needs to become libputter and lib{puffs,pud}.

  • Removing special case handling for the puffs user/kernel protocol transport. This means that file system requests can now be read/written like any other protocol. This is covered in the AsiaBSDCon 2008 paper "Send and Receive of File System Protocols: Userspace Approach With puffs"

    With some minor work in libpuffs, it possible to e.g. do an ffs mount from a remote site with the help of rump.

Finally, while not really useful for anything except puffs development, I think the following is cool from the perspective of completeness:

  • Add support to rump to be able to run the puffs kernel module in userspace. This means that that *any* puffs file system (incl. rump ones) can be mounted so that requests pass once through the puffs kernel module running in the kernel and once through the puffs kernel module running in userspace before being delivered to the file system driver. Example:
      sys/rump/fs/bin/syspuffs> ./syspuffs mount_psshfs server.address /path 

With puffs and rump, there are two very interesting and active projects doing research in filesystems on NetBSD, which may lead to changes in the way filesystems are understood in the Unix world. While there, a third project that may be worth watching in this regards is this year's Google hurdt Summer of Code project by Marek Dopiera, which aims at implementing Hurd translators for NetBSD,

Update: Antti dropped me a note that another project related to filesystems is this year's "fs-utils" SoC project. The goal is to create a userland tool to manipulate filesystem images, and the idea is to reuse kernel code with the ukfs library. That way, no redundancy between kernel sources and userland sources are created, and both areas benefit from mutual testing and code maturity.

[Tags: , , , , , ]

Disclaimer: All opinion expressed here is purely my own. No responsibility is taken for anything.

Access count: 24529538
Copyright (c) Hubert Feyrer