Click on the above link for more information on GSoC in general, there is also a list of proposed projects for this year in NetBSD.

Next steps are:

• make up your mind if you want to participate, either as student or mentor

• hash out details of project proposals and possible implementation details, and also see how much time you can devote as possible mentor. Also, think about choosing criteria for students and how to communicate with them on a daily basis and also in cases where problems arise (reallife on either side, student going AWOL, ...)

• go over the list of project proposals, esp. the ones with the right dimension for GSoC, and suggest changes and additions of new projects

• make yourself familiar with NetBSD from a user/admin viewpoint and esp. from a developer point if you intend to apply as student.

  A (slightly dated?) tour through the NetBSD source tree is available for userland, libraries, and the kernel. Also of interest: a guide on NetBSD internals.

• review our guidelines for applying for a project if you're an interested student. We get many really bad, dull and uninformed project proposals each year, and we wish more students would be as serious as YOU probably are (as you're already here :-).

See our list of suggested SoC projects if you need inspiration (other project suggestions are welcome!), and please have a look at our proposal HowTo for a bunch of questions we want to have answered so that we can properly judge your proposal. Get going!

[Tags: google-soc]

Prospective students that are interested in working on a project are recommended to have a look at the Project Application / Proposal HowTo!

[Tags: google-soc]

I've been thinking of a neat-o project for this year's Google Summer of Code:

Abstract: The goal of this project is to provide crypto acceleration by utilizing multiple CPU cores. The work is to extend the existing software-only "swcrypto" crypto driver and hook that up with NetBSD's OpenCrypto framework.

Overview of operation

The following image illustrates the components and their interaction.

Limitations

As examples, while a hifn(4) chip can provide worthwhile speedups on 500MHz and 1GHz CPUs, no performance win is experienced on a 2.4GHz CPU.

Proposal

The following image illustrates the idea of interoperation between a CPU core that runs the kernel and application codes and three cores that are dedicated to crypto code.

Implementation Roadmap

Requirements

• Know how to build and install a kernel
• Understanding of fine grained SMP and locking
• How to use NetBSD's kernel threads, code-wise
• How to interact with NetBSD's scheduler, code-wise
• Tell the scheduler to pin a specific kernel thread to a specific CPU
• Interaction between applications (IPsec, OpenSSL) with opencrypto(9), code-wise
• Interaction of crypto providers with opencrypto(9), code-wise
• Hardware! You won't be able to do this without at least two CPU cores in your machine. The more the better.
• Benchmarking & a test setup for it

Project Applications

If you have any questions let me know, public discussion should be led on the tech-crypto@ list.

Update: There was some discussion. In particular, my understanding of the interaction of the various layers as outlined above is not 100% accurate, and userland applications using opencrypto already seem to benefit from multiple kernel threads. In-kernel applications apparently do not, and before providing multiple crypto-servers in kernel (as suggested), work should probably done first to make sure such applications exist. Examples of this are IPsec (and the whole network stack), but also others like cgd (which AFAIU currently does not use opencrypto(9)).

[Tags: google-soc, opencrypto, swcrypto]

• People owning a Marvell PCMCIA WiFi card can how have a look at the malo(4) driver. If you use this on a gumstix ARM board, have a look as some fiddling with drvctl(8) is needed to get the card recognized properly.

• Speaking about Gumstix, KIYOHARA Takashi has declared the porting effort as finished. Support for the baex, connex, verdex and verdex-pro modules is available, as is support for the support modules.

• Staying in the "embedded" corner, KIYOHARA Takashi has announced that Plathome's OpenBlockS600 (AMCC 405EX) can now boot NetBSD via NFS. See the posting for dmesg output. The OpenBlockS600 comes with a AMCC 405EX PowerPC CPU, two GigE ethernet ports and a bunch of other goodies for a price of about $600US.

• Coming from hardware to software, pkgsrc-2009Q4 was released some time ago, and of course binary packages are available for a number of platforms: 5.0/macppc, 4.0.1/sparc and 5.0.1/sparc, 4.0.1/i386, 5.0.1/i386 and the same for 4.0.1/amd64 5.0.1/amd64. Also, binaries of pkgsrc-2009Q3 are available for 5.0/shark.

• Google's Summer of Code was a big success for NetBSD and all of the Open Source community, and it seems there will be one again this summer.

  Preparations are in an early stage, but there are already a FAQ and a timeline as well as the Program Terms of Service. From the NetBSD side, we're always happy for project suggestions (please use our mailing lists for discussions), and in NetBSD, we are currently working on out projects page. If you plan to submit a proposal for a project with NetBSD, please see our project application/proposal form If you plan to submit a proposal for a project with NetBSD, please see our Project Application/Proposal HowTo.

• Getting back to the NetBSD code, a number of interesting changes were made in the previous weeks. The first to mention is that David Young has continued is work on the new shutdown order for device drivers: ``cgd, dk, dm, md, raid, and vnd gracefully detach from the device tree during shutdown. I believe that ccd is the only virtual disk that does not detach.'' This allows having arbitrary stack of file systems, and still have them unconfigured properly in the right order on system shutdown.

• Another major change that went into NetBSD recently is that terminfo was imported into NetBSD-current. Terminfo replaces termcap, but provides a backward compatible termcap interface. This move follows discussion from last summer, and docs by The Open Group (the people who make things like the POSIX standard and the Single Unix Specification), which indicates that the termcap specification will be withdrawn in the future.

• Moving from userland inside the kernel, David Holland has proceeded with work to unhook LFS from UFS. Historically, the Log structured File System was written after the Berkeley Fast File System. With the idea of sharing the core "Unix File System" code for both file systems, this resulted in a strong relationship between LFS and FFS, which was/is not always the best for the advantage and stability of either one: ``sharing ufs between both ffs and lfs has made all three entities (but particularly lfs) gross. ffs and lfs are not similar enough structurally for this sharing to really be a good design.''

• Another major addition to the NetBSD kernel was made recently by Darren Hunt: ``Courtesy of CoyotePoint Systems, I've been working on a port of DTrace [...] to NetBSD for i386.'' Citing Wikipedia, ``DTrace is a comprehensive dynamic tracing framework created by Sun Microsystems for troubleshooting kernel and application problems on production systems in real time. Originally developed for Solaris, it has since been released under the free Common Development and Distribution License (CDDL) and has been ported to several other Unix-like systems.

  DTrace can be used to get a global overview of a running system, such as the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file. ''

  The code is available in NetBSD-current. I haven't looked into this yet, but I'm looking forward of reports and blog postings if the wikipedia command line examples work.

  (I think like with ZFS, Dtrace could use a hand with documenting the NetBSD side of things. Any takers?)

• The last kernel change to mention is related to security: mapping the address 0 from userland was disabled. This issue went through the press late last year, and it this is now addressed in NetBSD, too. Those that still beed to map address 0 can do so via the USER_VA0_DISABLED_DEFAULT kernel option or the vm.user_va0_disable sysctl.

• So much about about the NetBSD code for now. Of course having all those fine features added screams for an immediate (*cough*) release, which brings me to the fact that NetBSD 5.0.2 has been released: `` NetBSD 5.0.2 is the second critical/security update of the NetBSD 5.0 release branch. It represents a selected subset of fixes deemed critical for security or stability reasons.

  Please note that all fixes in critical/security updates (i.e., NetBSD 5.0.1, 5.0.2, etc.) are cumulative, so the latest update contains all such fixes since the corresponding minor release. These fixes will also appear in future minor releases (i.e., NetBSD 5.1, 5.2, etc.), together with other less-critical fixes and feature enhancements. ''

• There's a hackathon going this weekend (Feb 20/21 2010)
• Volunteers are wanted to setup & man a NetBSD booth at FrOSCamp 2010 Zurich, Switzerland, on Sep 17/18 2010
• pkgsrcCon 2010 will be held in Basel, Switzerland, from May 28ths to 30ths 2010.

• Initial support for the FriendlyArm Mini2440 board has been announced by Paul Fleischer. In a later update, most of the hardware is reported working, and the patch is available for review & comments. Furthermore, the touch screen is usable, and Qt/Embedded was built on top of wscons.

• Force10 Networks Receives Common Criteria Security Certification for Its High-Performance Ethernet Switch/Router Products. According to the article, ``Common Criteria evaluations entail formal rigorous analysis and testing to examine security aspects of a product or system. Extensive testing activities involve a comprehensive and formally repeatable process, confirming that the security product functions as claimed by the manufacturer. Security weaknesses and potential vulnerabilities are specifically examined during wide-ranging evaluation and testing.

  FTOS is the operating system software that runs on Force10 switch/router product lines, including the E-Series, C-Series and S-Series platforms. Based on NetBSD, FTOS leverages a distributed, multiprocessor architecture that delivers highly scalable protocols and reliability. By delivering the same OS across its entire switch/router line, Force10 ensures that customers benefit from stable code, a consistent configuration environment and simpler software management. ''

• While there: Force10 Networks Delivers Ethernet-Optimized Platform for MPLS Core Networks: ``Force10 Networks, Inc. [...] announced the immediate availability of MPLS (multi-protocol label switching) functionality for its ExaScale E-Series core switch/routers. [...] The ExaScale platform combines high-density, non-blocking, line-rate 10 GbE switching and routing with robust MPLS LSR support at 1/5th of the cost of a traditional core router, enabling carriers to fully capitalize on the economic advantages of Ethernet.''

• Create Bootable Live Linux USB Drives with UNetbootin: ``UNetbootin allows you to create bootable Live USB drives for a variety of Linux distributions from Windows or Linux, without requiring you to burn a CD. You can either let it download one of the many distributions supported out-of-the-box for you, or supply your own Linux .iso file if you've already downloaded one or your preferred distribution isn't on the list.''

  And of course we all know that NetBSD is Linux, right? See the list of supported distributions:

  

  The homepage mentions that NetBSD 4.0 is supported, maybe someone wants to give them an update on what's up with NetBSD 5.0? Would be nice to see that on the list!

• Ever wondered what happened with the BSD Certification recently? There's a video from the talk BSD Certification Group: A Case Study in Open Source Certification available that talks about the goal of the project, the two exams (BSD Associate, BSD Professional), and what's going on behind the scenes.

• With the move from XFree to X.org, the X server for the DNARD Shark's NetBSD/shark lost support for accelerated X. Thanks to Michael 'macallan' Lorenz, hardware-accelerated X for NetBSD/shark is back now: ``I finally got around to start working on an Xorg driver for the IGS CyberPro 20x0 family found in rev. 5 Sharks, Netwinder etc. - currently the driver is built only on shark and supports only the VL variant found there. Adding support for PCI chips is trivial though, just needs extra probing. The driver supports autoconfiguration ( X -configure should yield something almost usable, only DefaultDepth needs to be adjusted).''

• Staying with cool platforms, here's a quick procedure to run NetBSD/sun2 5.0.1 on The Machine Emulator (TME) (see pkgsrc/emulators/tme), compiled by Izumi Tsutsui. Who's first do get a pkgsrc bulk build done? :-)

• Jed Davis has committed the RAIDframe parity Summer-of-Code project. See his posting for the details. The project ``drastically reduces the time RAIDframe spends rewriting parity after an unclean shutdown by keeping better track of outstanding writes (thus, "parity map"). The tech-kern archives have more details [...]

  This feature is enabled by default on all sets (other than RAID 0). It can be administratively disabled with the new "raidctl -M" flag, which is described in the changes to the raidctl(8) man page; however, the I/O overhead for updating the parity map is expected to be unnoticeable in practice.''

Before I go into the details, let me add a hint on the "source-changes-full" mailing list, which is like source-changes, but the mails there also contain diffs of the changes (as long as they are below 1MB in size). The -full list is not archived on mail-index.netbsd.org, and I must have missed the announcement. It's definitely useful!

So, what interesting changes have there been to NetBSD-current recently? Here we go:

• New / updated drivers:
  • sdtemp(4) reads the on-DIMM temperature sensors following the JEDEC Standard No. 21-C Section 4-7.
  • gpio(4) got support for Intel ICH southbridges as found on Intel SS4220-E (ICH7) and Acorp 6A815EPD (ICH2) motherboards
  • lom(4) supports the LOMlite "Lights Out Management" boards found on Sun hardware like the LOMlite found on the Sun Netra t1 and the LOMlite2 found on Sun Netra T1/X1 and Sun Fire V100/V120.
  • An initial version of a SD/MMC driver for the Winbond W83L518D (and probably W83L519D) Integrated Media Reader with PNP bus attachment glue was added.
  • acpiwmi(4) was added to added, to further support ACPI development. See the specs for more details.
  • piixide(4) now supports Intel 3400 SATA
• New kernel interfaces and library functions include:
  • getdate(3) and getdate_err(3) were added according to POSIX.1 and the Open Group
  • RAIDframe was sped up massively when checking parity after a system crash with the help of a parity map. This work was done during this year's Google Summer-of-Code.
  • A "netbsd.inet.icmp.bmcastecho" sysctl was added to disable replies to the broadcast address.
  • Floating point functions f{min,max,dim}{,l,f}(3) were added
  • udl(4) adds support for DisplayLink DL-1x0/1x5 based USB LCDs and USB-VGA adaptors
• Random other additions and changes include:
  • hdaudioctl(8) can be used to manipulate hdaudio(4) devices
  • The NetBSD/playstation2 port was removed due to a lack of developer support. It's still available in CVS.
  • The NetBSD/i386 bootloader now knows a "dev" command to list available boot devices and their size.
• Newly imported and updated 3rd party software includes:
  • pfsync from OpenBSD 4.2, adopted in this year's Google Summer of Code.
  • PF from OpenBSD 4.2 was imported.
  • Multicast DNS ("Bonjour") support was added, based on Apple's mDNSResponder. Enable with "dns: mdns" in /etc/nsswitch.conf.
  • GNU binutils were updated to version 2.19
  • Many of the bazillion of X.org components were updated - ways too many to note in detail.
  • BIND was updated to 9.7.0b1
  • Flex 2.5.35 was imported
  • A new m4(1) was imported

• Christopher Berardi: Improve and Extend resize_ffs [Proposal / Project page]

• Jed Davis: Improving RAIDframe parity handling [Proposal / Project page]

• Arnaud Degroote: A tool to dump / restore pf state table [Proposal / Project page]

• Adam Hamsik: The NetBSD ZFS port [Proposal / Project page]

• Nhat Minh Lê: XML command-line utilities for NetBSD [Proposal / Project page]

• Jeremy Morse: Minimalistic X Server - NetBSD port [Proposal / Project page]

• Matthias-Christian Ott: Implementing efficient wide character regular expressions [Proposal / Project page]

• Lloyd Parkes: Miniaturise NetBSD [Proposal / Project page]

• Mike Volokhov: GPT aware boot loader support [Proposal / Project page]

• Maxwell Winderbaum: PXE Bulk Install [Proposal / Project page]

• Arnaud Ysmal: Generic file system mounting [Proposal / Project page]

Good luck to all the students, and may your project be a success!

[Tags: google-soc]

Here's a request for students who have sent in applications: Please have a close look on your applications! Any updates and comments made by our team of mentors will not be sent out automatically, unfortunately, we've learned. So check by repeatedly at your application in the next few days. A good place to further discuss any applications are out tech-* mailing lists.

Finally, the list of accepted students will be announced on April 20th, 2009. For more information, see the Google Summer of Code 2009 site.

[Tags: google-soc]

For more information see

• Our NetBSD list of project ideas
• Our NetBSD application guidelines
• Our SourceForge page with information about current and past projects.

As part of our preparation for the Summer of Code 2009, we have begun reviewing and updating our list of suggested projects and would like toinvite all interested students to likewise begin their research and start discussions with the possible mentors as well as on our public mailing lists.''

While talking about Google's Summer of Code - I'm a bit embarassed to announce this that late, but it seems everyone dropped the ball on what happened in the 2008 Google-SoC WRT NetBSD, there's a status report now, finally. Go have a look and see what stories of success (and failure...) we've seen!

[Tags: google-soc]

• In preparation of the NetBSD 5.0 release, a lot of documentation updates were made, esp. in the release notes. Also, many manual pages were added to the system, documenting existing userland tools, library, system and internal interfaces.

• Following some re-organization of binary packages on ftp.NetBSD.org some time ago, the official URLs are now:
  • ftp://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/{MACHINE}/{VERSION}/All
  • ftp://ftp.NetBSD.org/pub/NetBSD/packages/current-packages/NetBSD/{MACHINE}/{VERSION}/All
  Both should have the same results, the latter is more safe on mirrors that don't carry /pub/pkgsrc. Adjust your PKG_PATHs!

• Syntax for /etc/rc.conf's ifconfig_xxN variables and /etc/ifconfig.xxN was changed to also allow line breaks via ';'s. This allows something like ifconfig_wi0="ssid 'my network'; dhcp"

• Martin Schuette's work on syslogNG from Google Summer of Code 2008 is now available in NetBSD's syslog

• X.org integration is advancing in big steps. It's on by default on a number of platforms (including alpha, i386, macppc, shark, sparc and sparc64), and instead of using the (now obsolete) MKXORG build variable it can be build with "build.sh -x".

• Old-style LKMs are dead, welcome to the new module framework! (XXX Documentation???) In the process, more and more kernel subsystems are being changed to be loadable as a module, e.g. POSIX AIO and semaphores, File System Snapshots, emulations, exec formats, coredump, NFS client and server, http and data accept filters, ppp compressors, and others.

  Hooks into UVM have been added to unload unused kernel modules if memory is scarce.

• MAKEVERBOSE now has two new levels, 3 and 4. The complete list is now:
  • 0 Minimal output ("quiet")
  • 1 Describe what is occurring
  • 2 Describe what is occurring and echo the actual command
  • 3 Ignore the effect of the "@" prefix in make commands
  • 4 Trace shell commands using the shell's -x flag
  The default remains MAKEVERBOSE=2, you can also set this via build.sh's -N switch.

• A POSIX conformant tabs(1) utility was added

• The haad-dm branch was merged to NetBSD-current. This adds Logical Volumen Management (LVM) functionality to the base NetBSD system. It uses Linux LVM2 tools and our BSD licensed device-mapper driver.

• The wrstuden-revivesa branch was merged into NetBSD-current, bringing Scheduler Activation based threading back to NetBSD, and giving NetBSD 5.0 and up both SA and 1:1 threads.

• Support for the ARM-based Cortina Systems SL3516 eval board was added to NetBSD/evbarm

• patch(1) got a major overhaul, based on DragonflyBSD and OpenBSD. There's better detection of double applied patches, rejected diffs remain in unified diff format, and and less limitation e.g. on line length.

• pxeboot now understands boot.cfg

• Boot CD ISO creation has been greatly overhauled, accomodating changes in boot.cfg, and moving away from a ramdisk-based system to using a file system on the cd-rom, which helps reduce RAM usage. Also, the GENERIC kernel can be used there.

• makefs(8)'s ISO-9660 (cdrom) support was enhanced to write RISC OS data. This allows to make bootable CDs for acorn{26,32} directly, without copying the bootloader to a native file system.

• The christos-time_t branch has been merged into NetBSD-current. This gives 64bit time_t and dev_t types (no more year 2038-problem!!!).

  Many related places like timeval and timespec were adjusted, kernel and userland APIs were touched, and shared library major versions (including libc) were bumped for this fairly exhaustive change.

  See src/UPDATING's entry on 20090110 for the full update path!

• New/updated drivers:
  • jme(4) for JMicron Technologies JME250 Gigabit Ethernet and JME260 Fast Ethernet PCI Express controllers
  • u3g(4) provides better support for 3G datacards than ugensa
  • dbcool(4) for dbCool(tm) family of Thermal Monitor and Fan Controller
  • ataraid(4) now supports Intel MatrixRAID and JMicron RAID
  • bwi(4) for Broadcom BCM4302 wlan controllers, otherwise known as Airport Extreme
  • alipm(4) for the Acer Labs M7101 Power Manage- ment controller
  • admtemp(4) for the Analog Devices ADM1021, Analog Devices ADM1023, Analog Devices ADM1032, Genesys Logic GL523SM, Global Mixed-mode Technology G781, Maxim 1617, and Xeon embedded temperature sensors
  • ipw(4),iwi(4),wpi(4),iwn(4): We ship the firmware now, but users have to accept the Intel license manually by setting sysctls like hw.ipw.accept_eula=1. The latter is also offered by sysinst.
  • nsp(4) adds support for the NSP2000 cryptographic processor which does crypto, hashing and arbitrary precision arithmetics in hardware, and which hooks into the opencrypto(9) interface.
  • pseye(4) makes the Sony Playstation Eye USB webcam usable with the new video(4) framework
  • ath(4) now uses the recently-released source-based version of the Atheros HAL, no more binary blob!

• Google Summer of Code is over for some time, but apparantly no final report has emerged so far (shame!). Still, a number of individual status reports came by on the official lists:
  • Zach Wegner's sysinst project
  • Adam Burkepile's subfile project (PDF)
  • Martin Schütte's syslogd project
  • Jeremy Morse's DVB project
  • Mark Dopiera's hurd translators project
  • Adam Hamsik's lvm project

  I know of at least one other project (uvc) that has completed but that I didn't see a report here - maybe I've missed it. Anyways, GSoC was another big success this year. Thanks, Google!

• Speaking of Adam Hamsik and Logical Volume Management (LVM), Adam has continued his work in that are, and he has written a device mapping library that interacts with his kernel driver. This allows to interact with his GSoC project without using any GPL code! See Adam's posting for more details.

• Force 10 Networks, producer of 10gbit switches that use an operating system based on NetBSD, have added a new feature as part of their FTOS operating system: VirtualView, which provides virtualization of Force 10 based equipment. From the xchange article: ``Force10 Networks Inc. this week introduced VirtualView software for benchmaking, troubleshooting and managing virtualized environments based on Force10 gear.''

  More information is available from the Force 10 Networks homepage, plus in articles by fibresystems.org, light reading, Zycko, and SmartBrief.

• Following the latest hype in portable computers, NetBSD has created a netbook page that intends to list models and the extent to which they are supported. Your contributions are most welcome here! (Contact me for sending updates and hardware :-)

• Zafer Aydogan has made RSS feeds available for CVS commits to single files - see his mail to netbsd-users for more details.

• New security advisory were released that I've missed in my last update:
  • NetBSD Security Advisory 2008-011: ICMPv6 MLD query
  • NetBSD Security Advisory 2008-012: Denial of service issues in racoon(8)

• A project that's been ongoing for quite some time is the move from "old-school" loadable kernel modules (LKMs) to new-style kernel modules. Important changes include the fact that modules can be either linked into the kernel at build time, or loaded into the kernel at runtime from the same file. Also, the bootloader was modified to load modules after the kernel, e.g. for a RAM-disk like the one that is used by the INSTALL kernel.

  In the same line, some parts are starting to be moved out of the GENERIC kernel, and installed as modules that can be loaded by the new framework then. The start is made by POSIX semaphores as a first step and proof-of concept, even if some details are still under hot debate, e.g. what the file system layout for modules is, and if the belong to the kernel and its build process, or to the userland.

• While talking about splitting the kernel into modules, Antti Kantee has continued his work to move parts of the kernel into userspace, in particular running file system code as userland in his RUMP, and puffs and (Re)FUSE works. The idea is to provide the interfaces that file systems need in the userland, and the result is that you can run code that used to run inside the kernel in userland now.

  Another subsystem running in the kernel that could be moved to userland by providing appropriate interfaces with the rest of the kernel is the network stack, and Antti has moved just that to the userland. See Antti's mail to tech-net@ for more information on this impressive work.

• NetBSD has shipped XFree in previous releases, and people who wanted to use X.org had to install it from pkgsrc. That's all fine, but to get a modern X, one had to compile things, as no precompiled binary packages are made available for many platforms. This is changing now, and NetBSD is getting X.org integrated via a reachover infrastructure which is also enabled for crosscompiling.

  The "user interface" for this is still in flux, but after some detour ("build.sh -V MKXORG=yes", without -x), "build.sh -x" now builds whatever X is considered the default for the platform. Some platforms already default to use X.org as X, and more will come, as changes that were made to NetBSD's copy of XFree are adopted to X.org.

  Platforms that use X.org by default now are macppc (see here and here), sparc sparc64, shark, amd64 and i386.

  As X.org is at Revision 7 now, it's installed in /usr/X11R7, which will lead to a lot of interesting effects. pkgsrc is already prepared for the new layout, but there are still many minor details that will need adjusting to the new directory. If you find one, post your patches to tech-x11.

• Besides the GNU C compiler, there's the BSD-licensed Portable C Compiler around for some time now. It doesn't offer the same support as its GNU cousin yet, but this may change now: The BSD Fund is currently doing a fund drive to get money to enhance PCC. The goal is to raise $12,000US to improve support for core compiler functionality as well as support for C99, gcc compatibility and the amd64 architecture. See the project page for further details.

• The NetBSD 5.0 release cycle has started! There's a netbsd-5 branch in CVS, daily binaries are available for testing, and some of the highlights of the upcoming release include file system journalling for FFS via WAPBL, and X.org.

  To help testing of NetBSD on Cobalt machines, Izumi Tsutsui has made a NetBSD 5.0_BETA based version of the Cobalt restore CD available. Enjoy!

• As the final point today, a word on NUMA support from Christoph Egger. Non-Uniform Memory Access is needed in massive parallel systems where some nodes have RAM more tightly associated than others, where the RAM is further away, resulting in different access times for different regions of memory. In order to support this, Christop Egger has made first steps.

  His example implementation uses information from ACPI, and shows some heavy dmesg-pr0n from a 16-core machine with four sockets. Yumm!

• wscons: Expansion for wstablet in NetBSD
  Student: Jason W. Beaudoin

  To support tables for the wscons console driver, a number of changes to the wscons API have been proposed in the student's status report. While there are a number of similarities with the wsmouse interface, there are also a number of differences that need to be worked around, e.g. absolute vs. relative coordinates. Many of the proposed changes are implemented, and we're looking forward to complete this project successfully within time.

  Status report, project page, project proposal

• subfiles: Subfile Support for NetBSD
  Student: Adam Burkepile

  Subfile allow to associate data with a regular file, just like regular allow to associate data with a directory. New internal data structures were defined to identify subfiles within the file system, and tools like newfs and dumpfs were adjusted. An API is being designed to access subfiles, and work to realize the assorted functions is being approached.

  Status report part 1 / 2, project page, project proposal

• hurdt: Hurd translators
  Student: Marek Dopiera

  Translators are programs which provide filesystems in user space functionality. This is provided via NetBSD's RUMP interface, and additional system calls and file system operations have been defined to activate the server processes when access to such a "translator" is made. Currently, the translators are only implemented in NetBSD's ext2 file system, as this allows testing of interaction with Hurd - Hurd's support for FFS seems non-working, unfortunately. The project's under busy development, and we're looking forward to see the final results.

  Status report, project page, project proposal

• lvm: Write and improve NetBSD LVM driver
  Student: Adam Hamsik

  This project implements the Linux LVM API (libdevmapper) on NetBSD, to allow using Linux' lvm2tools for logical volume management. The project's making excellent progress, there is an ISO image (see URL in status report) as well as a qemu image available for testing, and latest reports show that linear volumes can be configured and mounted already.

  Status report, project page, project proposal

• uvc: Add support for UVC devices (USB web-cams)
  Student: Patrick Mahoney

  A kernel driver was developed to read data from webcams using a custom API, work to implement the video2linux API is under way. Current challenges lie in NetBSD's USB stack, which lacks support for isochronous transfers, which is used by many (but not all) webcams. The project has made excellent progress so far, and we're positive that the project will be a success.

  Status report, project page, project proposal

• dvb: DVB drivers and kernel framework
  Student: Jeremy Morse

  This project implements a driver for Digital Video Broadcasting to supplement bktr(4)'s TV card support. So far, a driver and an API to transport data from the kernel to userland was implemented, matching LinuxTV. The project's making good progress, even due to conflicts with the academic schedule.

  Status report, project page, project proposal

• install-tool: Customizable install tool for NetBSD
  Student: Zachary Wegner

  NetBSD's current installer, sysinst, is being split into frontend and backend parts, with a configuration file building the interface between the two parts. Untangling the current mix of user interaction and install operation are ongoing, with challenges like request of install media (think floppy #42) and network configuration. Also, a parser for the configuration file was written, and work not started yet is the frontend creating the configuration file for the backend.

  Status report, project page, project proposal

• fs-utils: File system access utilities
  Student: Ysmal Arnaud

  This project is using NetBSD's RUMP and the ukfs library to access a file system image from userland. So far, makefs(1) can generate a file system image, and it can now be manipulated as well. Both a "file system console" as shell to operate on the image as well as separate tools for single operations have been designed. Many of the "normal" userland tools' functionality like ls(1), cp(1) and rm(1) were implemented. This project has made excellent progress so far, see the status report and project page.

  Status report, project page, project proposal

• cwrapper: pkgsrc: rewrite wrapper framework in C
  Student: Amitai Schlair

  The student was distracted by travel obligations for some time, but work has started in pkgsrc now. Current work includes a set of ATF tests to cover the usage of the existing pkgsrc wrapper framework and a design plan for the new wrapper implementation. The actual wrapper implementation remains to be written, and we're looking forward to see the it happen, including integration into pkgsrc plus benchmarks on the increase in speed.

  Status report, project page, project proposal

• atfify: Converting remaining regression tests to the Automatic Testing Framework
  Student: Lukasz Strzygowski

  The Automated Testing Framework was added to NetBSD as a result of last year's Summer of Code. This year, all the remaining regression tests from src/regress are being converted to ATF. Test suites for tools (awk, grep, make, ...) and libraries (libm, libpthread, ...) were converted so far. Tests for libc are currently being converted, and kernel tests are next. We're looking forward to get all of src/regress changed to ATF!

  Status report, project page, project proposal

• ext3: Implement Ext3 file system support
  Student: Rus-Rebreanu Alin-Florin

  This project intends to implement journaling in the file system by reusing Wasbi's wapbl code. Unfortunately, little has happend to reach this goal (to say the least), and the student got AWOL. Interested parties are welcome to try out ext2fuse (which also does ext3, despite the name) for now.

  Project page, project proposal

• packet-classes: Create an in-kernel API for "packet classes"
  Student: Anish Babu

  Nothing has happened in this project as well. I hear about communication problems, and it remains to see if things move forward here. :-(

  Project page, project proposal

• teredo: Implementation of RFC4380 (Teredo) in NetBSD
  Student: Arnaud Lacombe

  The project aims at creating a Teredo client, server and relay via a kernel pseudo device and assorted userland tools. The project is currently still in the analysis and design phase, which will give it little time for completion -- as for other students, this is due to a clash with the academic schedule of the student. We're holding up hopes that the set goals can be met at least partially.

  Project page, project proposal

• syslogd: Improve syslogd
  Student: Martin Schütte

  The project aims at implementing the upcoming IETF standards for syslog transport over TLS (instead of UDP), a more formal format of the messages themselv for easier automatic parsing, and signing of messages to assert authentication, integrity and correct sequencing of syslog messages. The first part of this is already working, the second part is development. We're looking forward to see this project succeed, as it will be of benefit not only to NetBSD.

  Project page, project proposal

Update: Fixed a typo (thanks tron!), got an update on the ext3 project, and added a link to the status report of the cwrapper project, which I've missed (sorry!).

[Tags: atf, dvb, ext3, google-soc, hurd, lvm, pkgsrc, rump, subfile, sysinst, syslog, teredo, ukfs, usb, uvc, wscons, wstablet]

Another project that was started by Antti after his work to run filesystem code in userland is "rump". The project allows to use "ordinary" filesystems that usually run inside the kernel, and mimic an environment similar to what's available inside the kernel, and move the whole filesystem into userland - verbatime, with no code changes! This allows to develop filesystem code in userland, and later on move it inside the kernel with no further changes - a bit step forward for filesystem development!

This all sounds rather easy, but as filesystems need to move data between storage and memory, a big issue in filesystems is interfacing with the virtual memory subsystem, and adding interfaces like puffs and ReFUSE also needs to consider VM for efficient transfers and caching.

Work in this area is still ongoing, and I've asked Antti about his recent achievements in this area[1]. While the only user-visible change is caching and performance improvements in the Secure Shell filesystem's handler "mount_psshfs", most of the changes are on the inside. Antti wrote me: ``The interesting ones from a programmer's perspective are probably:

• Splitting userspace transport out of puffs in the kernel (putter)
• Using putter to implement support for userspace block/char device drivers (pud). pud does still not have a userspace library similar to libpuffs. libpuffs needs to become libputter and lib{puffs,pud}.

• Removing special case handling for the puffs user/kernel protocol transport. This means that file system requests can now be read/written like any other protocol. This is covered in the AsiaBSDCon 2008 paper "Send and Receive of File System Protocols: Userspace Approach With puffs"

  With some minor work in libpuffs, it possible to e.g. do an ffs mount from a remote site with the help of rump.

With puffs and rump, there are two very interesting and active projects doing research in filesystems on NetBSD, which may lead to changes in the way filesystems are understood in the Unix world. While there, a third project that may be worth watching in this regards is this year's Google hurdt Summer of Code project by Marek Dopiera, which aims at implementing Hurd translators for NetBSD,

Update: Antti dropped me a note that another project related to filesystems is this year's "fs-utils" SoC project. The goal is to create a userland tool to manipulate filesystem images, and the idea is to reuse kernel code with the ukfs library. That way, no redundancy between kernel sources and userland sources are created, and both areas benefit from mutual testing and code maturity.

[Tags: filesystem, fuse, google-soc, kernel, puffs, refuse]

• This year's Google Summer of Code projects have been chosen! After 7 (2005), 8 (2006) and 6 (2007) projects, this year brings a whopping 14 projects for students to work on for NetBSD! Rumours say that the reason for the big increase can be found in the high rate of projects that were both finished successfully as well as imported into NetBSD's main source tree. For more information on the projects, see the NetBSD-SoC homepage.

  Next steps for students are to get familiar with NetBSD's community and developers in the so-called "community bonding period", and coding will start late in May. See the Google pages for exact dates.

• Three new security advisories have been published:
  • NetBSD-SA2008-004: bzip2(1) Multiple issues
  • NetBSD-SA2008-005: OpenSSH Multiple issues
  • NetBSD-SA2008-006: Integer overflow in strfmon(3) function
  Mmm, the joys of importing 3rd party sources into your codebase.

Right now we have a number of interesting applications, but there are also a bunch of applications for the same projects. Students who sent in applications for something related to the ext2/ext3 file systems and DHCP may consider their project choice, and see if there are other projects they are interested in -- it is unlikely that we pick more than one project from these areas.

While there, let me add that for successful acceptance, students are welcome to submit a project that has some actual relevance to the NetBSD project, i.e. either the NetBSD operating system or its packages collection. While we are willing to mentor random projects that can be included into our operating system to some extent, it lowers chances for projects to get accepted, and students may consider our other projects that we have to do.

Last, we're serious about the questions in our Application HowTo, given our experiences from the past years. What we are looking for are students who either used NetBSD (to some extent), and who can identify and understand the parts of our source code they will be working with. We're talking mostly C here, and you should be familiar with reading and extending it.

[Tags: google-soc]

As a reminder, there are suggested NetBSD projects for the Summer of Code and harder ones for beyond (or if you feel bored :-). Please take note that we expect to have a few questions answered in the student application. Another interesting link may be Google's Application page, see the help page.

Those who want to do advertizing for the Summer of Code may find localized flyers and videos useful.

[Tags: google-soc]

Students, start thinking about your NetBSD projects!

P.S.: Anyone wondering what's up next on the timeline, see Leslie's blog posting.

[Tags: google-soc]

Prospective students that are interested in persuing projects for NetBSD are welcome to start thinking about what projects they'd like to do (assuming NetBSD gets accepted again), and discuss things on our mailing lists. We have lists of suggest GSoC projects and (harder!) general projects which can serve as inspiration. Also, please see our NetBSD Project Application/Proposal HowTo for information that we'd like to see answered in the eventual project proposals sent to Google -- this information helps us to rank the project proposals we'll get, and to put your project on the best spot!

[Tags: google-soc]

• Introduction
• Release engineering, Sendmail, and kauth
• PaX, fileassoc, and Veriexec
• Linux compat, XFree86, pkgsrc, proplib, and Xen
• Filesystems
• iSCSI and optical disc support
• Bluetooth, mobile devices, agr
• Google Summer of Code
• Hackathons and funding

There's also a report from Pawel in Google's Summer of Code Blog.

[Tags: google-soc, meetbsd]

The full report with details on the projects and their outcome can be seen on the NetBSD website. Also note the hint about the NetBSD Fundraising Campaign which is intended to allow us to fund more projects in the future.

[Tags: google-soc]

Joerg also posted a first set of instructions on how to cross compile modular X.org on NetBSD yourself, from latest pkgsrc.

Feedback is very welcome on both the binary packages and the build process. Please post your findings to the tech-pkg mailing list!

[Tags: google-soc]

Currently, NetBSD's X is built using a framework of reachover-Makefiles that mirror the existing Imakefiles, and use NetBSD's cross-toolchain. With X.org moving towards a heavily autoconf-based framework, the idea is to use the 'native' X.org build environment (configure, ...), and use NetBSD's wrapper-framework in pkgsrc to use its crosscompiler instead of a native compiler.

[Tags: google-soc, sparc]

This year's project goes one step further as not only the VFS operations are passed from kernel to userspace, but the full filesystem code is ran in userspace, including memory management (buffer cache!). This allows developing filesystem code in userland with the "normal" kernel interfaces, but no panics if something goes wrong -- a simple core dump, and that's it. To cite from Antti's mail: ``If you want to give it a whirl, upgrade, go to sys/rump, type "make", go to fs/bin/$foofs and run the file system. It should work to a certain degree provided your system comes with puffs. Does it do anything a kernel file system wouldn't do? No .. except the core dump may be smaller and faster and less intrusive. Oh, actually it does one cool trick: you can run it on non-block devices, so you don't need to vnconfig the file system image first (provided, of course, that the file system starts right from the beginning of the image and that you wouldn't need a disklabel).''

For more information, see Antti's first and second email and his Summer-of-Code project page.

[Tags: google-soc, puffs, rump, ukfs]

• Julio M. Merino Vidal: Automated Testing Framework
• Oliver Gould: Porting ZFS
• Sumantra R. Kundu: A Framework For Enforcing QoS Inside the NetBSD UVM
• Antti Kantee: Running Kernel File Systems in Userspace
• Joerg Sonnenberger: Improving the pkgsrc Build System
• Jachym Holecek: Hardware Monitoring and HAL Port

Maybe a word from me on the number of accepted projects: Learning from the past two years in Google's SoC where we got many suboptimal project proposals, we tried to maintain a rather high bar to ensure quality of submissions this year. The result is that there were less projects submitted, and as a consequence less projects selected. I guess in future years we'll still try to maintain quality, but also suggest more "simple" projects. :)

[Tags: google-soc]

Update: Jan got the word out!

[Tags: google-soc]

• Linuxdays Chemnitz: I was only there for my presentation on sunday, due to not feeling too well the days before. Still, Stefan, Jörg, Charlie and many others staffed the booth just fine, and I think every single household in and around Chemnitz has a NetBSD install and/or Live CD now. :)

  Related talks to mention are Stefan Schumacher's talk on hardening systems with systrace and deleting data. My own talk was not too NetBSD specific, showing an application on how to implement dynamic DNS with some retail web/domain hoster. Slides for my talks are available as OpenOffice .ODP and as PDF. (I'll reconsider the move from TeX/prosper to OpenOffice after it was NOT as easy as I expected to find a machine running OOo for presentation purpose, after my laptop's harddisk crashed on the way to Chemnitz!)

• While at roadshows: Stefan Schumacher has made DIN A4 pkgsrc flyers in english and german language.

• NetBSD's puff-based FUSE implementation "refuse" is now in a state to also run the NTFS-g3 filesystem, which offers read/write support for NTFS. It's available from pkgsrc/filesystems/fuse-ntfs-3g.

• Google News found me an article that NetBSD stack supports Geode NAS design: ``Wasabi Systems Inc.'s BSD-based NAS (network attached storage) software stack now supports a Geode-based reference design from AMD. Wasabi Storage Builder for NAS, combined with AMD's Geode LX NAS RDK (reference design kit), provides a secure, reliable platform for the development of NAS devices, according to Wasabi. ''

  While that's all fine for Wasabi, it should be noted that whatever the company Wasabi offers is not automatically available in the freely available operating system called NetBSD. Integration efforts would have to happen first, so the headline of that article is unfortunately misleading if not to say plain wrong!

• Another article that's more to the point: Julio M. Merino Vidal has worked on getting multiboot support into NetBSD, and in his article ``Making NetBSD Multiboot-Compatible'' he talks more about it.

• Andrew Doran has done lots of work on NetBSD's thread and SMP implementation recently, and he has made a comparison between performance of the Scheduler-Activations-based code in NetBSD 4 and the one that will be in NetBSD 5 (AKA NetBSD-current, currently numbered as 4.99.13). See his mail to tech-kern or watch the images for 'make cleandir' on an empty source tree and the MySQL supersmack benchmark.

• Google runs another Summer of Code, and this year it's not clear upfront who will be allowed as mentoring organizations. NetBSD is ready to participate again, and there's an official announcement from NetBSD about this, including pointers to our suggested/wanted list of projects and the project application HowTo. People interested in submitting a project proposal (via google!) are encouraged to use the remaining time until the deadline to discuss their proposals on the public NetBSD tech-* lists! (Personally I'll try to stay out of GSoC this year to finish some reallife work. At least that's the plan so far ...)

• Three new security advisories were released:
  • NetBSD Security Advisory 2007-001: Integer overflow in ktruser()
  • NetBSD Security Advisory 2007-002: Integer overflows in Render and DBE extensions
  • NetBSD Security Advisory 2007-003: BIND multiple denial of service vulnerabilities

• Another article that doesn't mention NetBSD but g4u: ``How to Install a New Hard Drive: Tech Clinic'' by Joel Johnson. From the article: `` To make your new drive work like your old drive, you'll need a disk "cloner." There are a myriad of options, from commercial solutions such as the old favorite Ghost from Symantec ($70; symantec.com) and Copy Commander from VCom ($35; v-com.com) to free applications, such as MaxBlast from Maxtor, that come bundled with hard drives. If you're comfortable mucking around with Linux/BSD, I've had great luck with the free g4u application. If you have a local file server, you can even send the disk image from your laptop to an FTP site, install the larger drive, then FTP it back to your laptop, obviating the need for a drive enclosure''.

There will be another Google Summer of Code in 2007, and Google has put up a poster for everyone (that's you!) to print and hang, so that we can attract people to get into OpenSource (that's your friends, and you :). From the poster: ``Last year Google spent $3,000,000 to fund 630 student projects across 90 countries with the help of over 100 mentoring organizations. Google Summer of CodeTM is back again in 2007! Google Summer of CodeTM is expressly designed to get the brightest minds on campus contributing code to open source initiatives and inventing new open source programs. After all, while altruistic motives are praiseworthy and the rewards of peer recognition sweet, the idea of a financial prize for stellar work predates the Nobel Commission and who really needs money more than broke college students?

Visit code.google.com/soc/ to learn more and submit your idea for a project. The deadline for student applications is March 24, 2007.''

I'm not trying to do much for GSoC 2007, someone else will do most of the work. :) We're still assembling suggestions for projects, but I think it's fine if you already start thinking about and discussing your proposals on the tech-* mailing lists. When you eventually submit your paper, please don't forget to have a look at our guidelines to make things easier for you to sort out.

Update: I linked to the wrong projects page: there's a separate NetBSD Summer-of-Code Projects page with suggestions that are more likely to be doable within the SoC timeframe. But of course that should not stop anyone from doing the other projects, too, either within SoC our outside. :)

[Tags: google-soc]

With the success of the ipsec6 Google SoC project, the code for hardware-accelerated IPsec with IPv6 (and IPv4) is now ripe for testing, and the author has sent a second mail asking for code review and testing: ``It would be nice too if some people can test the code in more real cases and report success and failure.'' The pointer to the latest patch is in his second mail.

Happy testing!

[Tags: google-soc, ipsec, ipv6]

Can you spot Alistair and me? :-)

[Tags: google-soc]

For those that haven't followed things too closely, there was a Mentor Summit for Google's "Summer of Code" last saturday in Mountain View, California. Google invited two mentors/admins from every participating GSoC project, and for NetBSD, Alistair Crooks (agc) and I (hubertf) were there to chat with people, listen to talks and share our experiences.

I've posted my session notes and references to the summit's wiki to the netbsd-users list, so have a look there if you're interested of what may come up in case there's another "Summer of Code" next year, and what experiences students, mentors and admins made in the past years and what they may learn from for the future.

Now given that this is *my* (hubertf's) weblog, I thought I'd put some of the (more or less) funny but off-topic things in here, in case anyone cares what else happened on the trip to Mountain View and back.

Friday, Oct 13th: Travel & arrival

• Security checks at Munich airport (MUC) suck: they want my hand lotion (two small tubes) in a plastic bag. Both my deodorant and my tooth paste (which was almost empty!) were confiscated as they're security threats. Yeah, right - I'll bring you down with my toothpaste.

  On a related note and in retrospect, I find it interesting that the security checks in Munich were by far the most strict ones. None of the checks in San Francisco and Chicago could compare to those.

• After many hours of flight, taking BART and Caltrain to Mountain View from SFO was ok, even if I had to discover that the schedule on the web isn't upto date. Walking from the train station to the hotel was a lesser fun - I just remembered that you were expected to have a car in this country. (No cabs at ~midnite there, either... doh!)

• I've posted my session notes etc. to the netbsd-users list for further discussing.
• In the evening, a group of NetBSD people had food & drinks in the 21st Amendment brewery club/restaurant in San Francisco, including Alistair Crooks, Darren Reed, Jef Rizzo, Jan Schauman, Jan's wife and me.

• Find a hotel to spend the night from sunday to monday. While I have a very nice offer to use mrg@'s couch, I realized that doing the commute to SFO in the morning would be too time consuming. Also, as I wanted to do some sightseeing in San Francisco today, not having to go back all the way back to Mountain View in the evening was a good decision. (Remember for next time: rent a car!!!)
• I've walked around Market Street and the area surrounding it most of the day, freshing up memories from my last stay there 13 years ago
• Surfing the web back in my hotel room, which had wireless network access, I found that MoinMoin Wiki (which was used for documenting some of the GSoC Mentor Summit's activities) can export as DocBook/XML. Maybe NetBSD make use of that, at least for htdocs/Documentation. (sure not for htdocs/guide, no idea for htdocs/Ports and I don't know what else we have in htdocs... probably ways too much!)

• Not much to say, 4+9 hours of flight (via Chicago) in economy class sucked. Home tuesday Oct 17th at ~2pm, and first thing to do at home is get a new cell phone as my old Siemens A55 kicked the bucket on the flight to the US. Doh!

See Antti's posting for all the details, plus his second posting about his future plans and interfacing with the FUSE API.

Now who's the first to get us a SSH/scp/sftp based filesystem? I offer a BSD Daemon pin as price! :-)

[Tags: fuse, google-soc, puffs]

The agenda's still building up, but my personal point of interest is how much "research" to have in the actual project proposals - I'm under the impression that the NetBSD project proposals could/should be a lot clearer, and leave less decisions or open questions to the student working on a project. Of course it means a lot more work and to at least halfways think through a project before proposing it, but at the end it will mean a clearer defined goal for the student towards which to work. Depending on the actual project, of course.

[Tags: google-soc, hubertf]