• Google Summer of Code is over for some time, but apparantly no final report has emerged so far (shame!). Still, a number of individual status reports came by on the official lists:
  • Zach Wegner's sysinst project
  • Adam Burkepile's subfile project (PDF)
  • Martin Schütte's syslogd project
  • Jeremy Morse's DVB project
  • Mark Dopiera's hurd translators project
  • Adam Hamsik's lvm project

  I know of at least one other project (uvc) that has completed but that I didn't see a report here - maybe I've missed it. Anyways, GSoC was another big success this year. Thanks, Google!

• Speaking of Adam Hamsik and Logical Volume Management (LVM), Adam has continued his work in that are, and he has written a device mapping library that interacts with his kernel driver. This allows to interact with his GSoC project without using any GPL code! See Adam's posting for more details.

• Force 10 Networks, producer of 10gbit switches that use an operating system based on NetBSD, have added a new feature as part of their FTOS operating system: VirtualView, which provides virtualization of Force 10 based equipment. From the xchange article: ``Force10 Networks Inc. this week introduced VirtualView software for benchmaking, troubleshooting and managing virtualized environments based on Force10 gear.''

  More information is available from the Force 10 Networks homepage, plus in articles by fibresystems.org, light reading, Zycko, and SmartBrief.

• Following the latest hype in portable computers, NetBSD has created a netbook page that intends to list models and the extent to which they are supported. Your contributions are most welcome here! (Contact me for sending updates and hardware :-)

• Zafer Aydogan has made RSS feeds available for CVS commits to single files - see his mail to netbsd-users for more details.

• New security advisory were released that I've missed in my last update:
  • NetBSD Security Advisory 2008-011: ICMPv6 MLD query
  • NetBSD Security Advisory 2008-012: Denial of service issues in racoon(8)

• A project that's been ongoing for quite some time is the move from "old-school" loadable kernel modules (LKMs) to new-style kernel modules. Important changes include the fact that modules can be either linked into the kernel at build time, or loaded into the kernel at runtime from the same file. Also, the bootloader was modified to load modules after the kernel, e.g. for a RAM-disk like the one that is used by the INSTALL kernel.

  In the same line, some parts are starting to be moved out of the GENERIC kernel, and installed as modules that can be loaded by the new framework then. The start is made by POSIX semaphores as a first step and proof-of concept, even if some details are still under hot debate, e.g. what the file system layout for modules is, and if the belong to the kernel and its build process, or to the userland.

• While talking about splitting the kernel into modules, Antti Kantee has continued his work to move parts of the kernel into userspace, in particular running file system code as userland in his RUMP, and puffs and (Re)FUSE works. The idea is to provide the interfaces that file systems need in the userland, and the result is that you can run code that used to run inside the kernel in userland now.

  Another subsystem running in the kernel that could be moved to userland by providing appropriate interfaces with the rest of the kernel is the network stack, and Antti has moved just that to the userland. See Antti's mail to tech-net@ for more information on this impressive work.

• NetBSD has shipped XFree in previous releases, and people who wanted to use X.org had to install it from pkgsrc. That's all fine, but to get a modern X, one had to compile things, as no precompiled binary packages are made available for many platforms. This is changing now, and NetBSD is getting X.org integrated via a reachover infrastructure which is also enabled for crosscompiling.

  The "user interface" for this is still in flux, but after some detour ("build.sh -V MKXORG=yes", without -x), "build.sh -x" now builds whatever X is considered the default for the platform. Some platforms already default to use X.org as X, and more will come, as changes that were made to NetBSD's copy of XFree are adopted to X.org.

  Platforms that use X.org by default now are macppc (see here and here), sparc sparc64, shark, amd64 and i386.

  As X.org is at Revision 7 now, it's installed in /usr/X11R7, which will lead to a lot of interesting effects. pkgsrc is already prepared for the new layout, but there are still many minor details that will need adjusting to the new directory. If you find one, post your patches to tech-x11.

• Besides the GNU C compiler, there's the BSD-licensed Portable C Compiler around for some time now. It doesn't offer the same support as its GNU cousin yet, but this may change now: The BSD Fund is currently doing a fund drive to get money to enhance PCC. The goal is to raise $12,000US to improve support for core compiler functionality as well as support for C99, gcc compatibility and the amd64 architecture. See the project page for further details.

• The NetBSD 5.0 release cycle has started! There's a netbsd-5 branch in CVS, daily binaries are available for testing, and some of the highlights of the upcoming release include file system journalling for FFS via WAPBL, and X.org.

  To help testing of NetBSD on Cobalt machines, Izumi Tsutsui has made a NetBSD 5.0_BETA based version of the Cobalt restore CD available. Enjoy!

• As the final point today, a word on NUMA support from Christoph Egger. Non-Uniform Memory Access is needed in massive parallel systems where some nodes have RAM more tightly associated than others, where the RAM is further away, resulting in different access times for different regions of memory. In order to support this, Christop Egger has made first steps.

  His example implementation uses information from ACPI, and shows some heavy dmesg-pr0n from a 16-core machine with four sockets. Yumm!

• In order to re-initialize x86 machines' video/VGA state after suspend and resume, some BIOS functions can be used. This needs to be done in real mode(?), which is a bit hard to do from an operating system kernel that runs in protected mode. To help doing so, a x86 CPU emulator was added to NetBSD some time ago, to help run VGA bios for ACPI resume. Now Joerg has added a sysctl that does just this, assuming your kernel has the VGA_POST options -- set machdep.acpi_vbios_reset=2

• Inside the kernel, data sent/received through the network stack is stored in chains of mbufs. So far, the mbufs were also used to store socket options, i.e. data describing further how the sending/receiving is done. This was split out into a separate struct sockopt by Ian 'plunky' Hibbert now. For more information, see sockopt(9).

• Hans 'woodstock' Rosenfeld has added a new accalerated driver for SPX graphics boards found in some VAXstations, which replaces the old and broken lcspx driver. The work is based on work by Blaz Antonic.

• The simonb-wapbl branch was merged: ``Add Wasabi System's WAPBL (Write Ahead Physical Block Logging) journaling code. Originally written by Darrin B. Jewell while at Wasabi and updated to -current by Antti Kantee, Andy Doran, Greg Oster and Simon Burge.'' This makes NetBSD the first second (see update below) BSD operating system that has a working file system with journaling (not counting LFS, which again and again has issues). Mmm, no more fsck! :-) See my other posts for more on journaling / wapbl.

  Update: James Mansion wrote me to that NetBSD's not the first BSD to have journaling, and I think he's right: DragonflyBSD's HAMMER file system apparently offers similar functionality: ``HAMMER implement an instant-mount capability and will recover information on a cluster-by-cluster basis as it is being accessed.''

• Accept filters were ported from FreeBSD by Coyote Point Systems, and integrated into NetBSD by Thor Lancelot Simon. What are accept filters? According to the accept_filter(9) manpage, they ``allow an application to request that the kernel pre-process incoming connections.'' Pre-defined filters are available with accf_data(9) and accf_http(9). The latter makes sure that the application's accept(2) call only sees the connection if there's a valid HTTP header, moving parts of the parsing from userland (httpd) to the kernel.

• Work is underway for crossbuilds of modular X.org. This is done via src/external/mit/xorg, which needs xsrc/external/mit. The results will be installed in /usr/X11R7(!). (XXX Where can I find more about this?)

• Gregory McGarry is working to get the tree compiled with PCC instead of GCC. This is still ongoing.

• nvi was updated from version 1.79 to 1.81. The most important part of this update is that internationalization is now handled by default.

• Following a bigger masterplan, new 3rd party software packages are now imported into src/external/${license}, which will replace src/dist, src/crypto/dist and src/gnu/dist in the long run. Packages will be moved on upgrades only, existing packages are not being moved just for the sake of moving them.

• Adam Hamsik is working on getting Logical Volume Management (LVM) going in NetBSD. He has adapted Linux' "device mapper" kernel-interface as part of his Google Summer-of-Code project, and with the help of the (GPL'd) Linux tools, things are looking pretty good. More on this in a separate post. This work is currently happening on the haad-dm branch.

• In the context of his work on UDF, Reinoud has added routines for speeding up directory handling by using hash gables. Lookup of files was O(n*n) and is now O(1) even for file creation. See my other blog posting for details and impressive numbers.

• Perry Metzger is working to make binary builds identical. This is useful for binary diffs between releases/builds, e.g. when providing binary patches for updates and security fixes. Areas where this had an impact on are C++ programs and various bootloaders (which had a builder, build date, etc. in it so far).

• EHCI (USB) can now do high speed isochronous support. This was developed by Jeremy Morse as part of his Google Summer-of-Code "dvb" project this year, it is useful for fast transfer of data that comes in steady streams, e.g. from video cards.

• fsck_ffs(8) now has options -x and -X (just like dump) that create a file system snapshot via fss(4), and then operates on the snapshot. This allows "fsck_ffs -n" to work on a snapshot of a read/write mounted file system, and avoid errors related to file system activity. Can be made permanent for the nightly script by setting run_fsck_flags="-X" in /etc/daily.conf. This was brought to you by our Xen-hacker Manuel Bouyer. :-)

• Support C99 complex arithmetic was added by importing the "cephes" math library
• POSIX Message queues were added
• bozohttpd was added as httpd.
• the x86 bootloader now reads /boot.cfg to configure banner text, console device, timeout etc. - see boot.cfg(5)
• ifconfig(8) now has a "list scan" command to scan for access points
• SMP (multiprocessor) support is now enabled in i386 and amd64 GENERIC kernels
• Processor-sets, affinity and POSIX real-time extensions were added, along with the schedctl(8) program to control scheduling of processes and threads.
• systrace was removed, due to security concerns
• the refuse-based Internet Access Node file system was committed, which provides a filesystem interface to FTP and HTTP, similar to the old alex file system, see http://mail-index.netbsd.org/source-changes/2007/08/28/0081.html
• LKMs don't care for options MULTIPROCESSOR and LOCKDEBUG, i.e. it's easier to reuse LKMs between debugging/SMP and non-debugging/SMP kernels now.
• PCC, the Portable C Compiler that originates in the very beginnings of Unix, was added to NetBSD. The idea is that it is used as alternative to the GNU C Compiler in the long run.
• In addition to the iSCSI target (server) code that is already in NetBSD 4.0, there'a also a refuse-based iSCSI initiator (client) now, see http://mail-index.netbsd.org/source-changes/2007/11/08/0038.html

• Many driver updates and new drivers, see your nearest GENERIC kernel config file
• Many security updates, see list of security advisories
• Many 3rd software packages that NetBSD ships with were updated: ipsec-tools (racoon), GCC 4.1, Automated Testing Framework 0.4, OpenSSH 4.7, wpa_supplicant and hostapd 0.6.2, OpenPAM Hydrangea

So much on this subject for now. If someone's willing to help out with continuing Mark Kirby's NetBSD CVS Digest either using his software-setup or by simply reading the list and writing a monthly/weekly digest of the "interesting" changes, I'd appreciate this very much. Put me on CC: for your postings! :)

[Tags: alex, bozohttpd, c99, cephes, cvs, cvs-digest, digest, ian, iscsi, lkm, pcc, refuse, smp, systrace]

Competition is usually a good thing; in open source projects, where ideas (and, ideally, code) can flow freely between the projects, everybody wins in the end. The more advanced architecture of LLVM makes it a good long-term bet for future systems, and the simple codebase of PCC makes it a good systems compiler. Both are likely to carve out a significant portion of market share in the next few years, giving developers more of choices for compiling their code.''

Those interested in trying PCC with pkgsrc can set "PKGSRC_COMPILER=pcc", see Jeremy Reed's blog entry and the the CVS commit log for more information. Also, see (hear :) the bsdtalk interview with Anders "Ragge" Magnusson.

[Tags: gcc, pcc]

The compiler aims at implementing the C99 standard right now, and produces code for x86. Other platforms may be added eventually, other languages like C++ are rather unlikely to be added.

For more information on the project, see the homepage.

Update: Jeremy Reed has taught pkgsrc how to use PCC: Get the latest pkgsrc, and set PKGSRC_COMPILER=pcc. Voila!

[Tags: pcc]