Ansible & EC2 - Playbooks for orchestrating NetBSD into the cloud
As follower of my blog you have
seenthestepstowards
getting NetBSD instances started in Amazon's EC2 cloud
with a simple web application deployed on one EC2 instance
and the database on another one.
These blog articles were very detailed on purpose, to have full
logfiles available just in case needed. I have used these logs to
prepare my
pkgsrcCon 2013
talk about Ansible and Amazon's EC2, so things can be looked at
without actually running anything. As it turns out this was good,
because the 32bit NetBSD instances that I've used during my
pkgsrcCon demonstration actually decided to do a kernel panic, and the presentation
was a bit more on the theoretical side than I originally planned.
Managing NetBSD with Ansible, First Steps Background: Plan, Build, Run
I've mused a lot about the transition from
system planning over system setup to system operation recently.
Different kinds of databases may be involved, that serve
different purposes: On the planning end, topics like
version management, license management, life cycle management
and enterprise architecture management come into play.
When this transits into operation, other aspects like
system and network configuration get more important,
while others fade away.
At the end, someone or something has
to make adjustments to systems in order to get them
into service, and keep them there.
This is either done with a lot of manual labor
to keep documentation (CMDB, or whatever serves)
and the systems in sync, or - more likely - documentation
will grow stale quickly and rot away.
With a growing set of hosts, the latter is not an option,
and transparency becomes increasingly important.
This is where system "orchestration" tools like cfengine,
puppet, chef and ansible come into play.
You write a system definition there, and the machine is
configured according to this definition automatically:
Configuration files can be adjusted, accounts created,
software packages installed, services enabled, etc.
System Orchestration Alternatives
Looking at the state of affairs, many people seem to be
fond of Puppet these days. This comes with a pretty long list
of dependencies (and thus complexity to maintain), and
configuration files have to be written in Ruby (which I don't know).
So I chose Ansible as alternative to investigate - it comes with very little overhead
(no separate daemon on neither the central machine nor any
of the configured systems, just using SSH) and its own
"Playbook" language seems easy enough to start with, yet
complete enough to be used in large environments (as the
list of customers listed on the homepage shows).
Packaging Ansible, First Try
My first try to get going with Ansible on NetBSD was to use
the version 0.9 archive available, instead of using a GIT checkout.
The included (GNU) Makefile
tries to determine the version from the git checkout.
Which the archive is not. Using information not available in the archive either.
Checking if the source code was checked out from GIT by looking
if there's a git binary available is not the best idea here,
and so I dropped the ball, cursing the fact that not all
the world is Linux. Sort of.
Looking briefly at a fresh git checkout that this worked there,
but as NetBSD's pkgsrc doesn't support git checkouts (yet, as far
as I know?), I moved back to the 0.9 archive.
Packaging Ansible, Second Try
Leaving out the git-based version games, the next fun was
to get the Python setup script to install the files in the right
place. Having missed much pkgsrc development recently, and not being into
Python either way, this proved unneccessary - Joerg Sonnenberg
pointed me at existing pkgsrc infrastructure for Python
programs, and with them the program was packaged successfully
in no time.
For now, the package is available
on my website,
I'll look into review and import into pkgsrc next.
Ansible on NetBSD - What Works, And What Needs More Work
Modules tested successfully on NetBSD 6:
ping
command
copy
facter
group add/delete
mysql_db add/delete
user add/delete (without system=true)
Modules tested unsuccessfully on NetBSD 6:
user add/delete (with system=true)
service (needs work)
The git version (and the upcoming 1.0 release) also include
a "pkgin" module that can be used to manage packages.
Unfortunately the git-version of the module cannot easily be used
with the 0.9 version, so this has to wait.
Here's an example showing creation and deletion of a user account:
% finger testuser
finger: testuser: no such user
% ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=present"
127.0.0.1 | success >> {
"changed": true,
"comment": "",
"createhome": true,
"group": 100,
"home": "/home/testuser",
"name": "testuser",
"shell": "/bin/sh",
"state": "present",
"system": false,
"uid": 1005
}
% finger testuser
Login: testuser Name:
Directory: /home/testuser Shell: /bin/sh
Never logged in.
No Mail.
No Plan.
% ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=absent"
127.0.0.1 | success >> {
"changed": true,
"force": false,
"name": "testuser",
"remove": false,
"state": "absent"
}
% finger testuser
finger: testuser: no such user
So much for a quick start into Ansible. Do you use NetBSD in a large-scale
environment that went beyond manual systemconfiguration?
Write a blog entry or an email to the NetBSD lists,
and let us know!
FOSDEM talks/slides: pkgsrc on MirBSD, pkgin (Updated)
Last weekend the Free and Open Software Developer Meeting (FOSDEM)
happened. As in previous years, there was a booth manned by
people from various BSD-projects, and there were also
two(?) talks related to NetBSD and its related project pkgsrc:
NetBSD and MirBSD developer Benny Siegert gave a talk titled
"pkgsrc on MirBSD" -
see his slides! pkgsrc is a framework for packaging and building 3rd party applications from source. Besides MirBSD, it runs on many
other platforms like Linux and Mac OS X.
While building from source is fine, it costs a lot of time.
pkgsrc can also create binary packages, and to manage those,
there is "pkgin", a binary package manager.
Its developer, Emile 'iMil' Heitor
introduced it in a talk -
see the slides (PDF)!
Update:
Thomas 'wiz' Klausner pointed out that there was
actually a whole "BSD Devroom", and there were
many BSD-talks there, including:
MINIX3 and BSD, by Arun Thomas
The Lua Scripting Language in the NetBSD Kernel, by Marc Balmer
Touch your NetBSD - towards tablet integration, by Pierre Pronchery
Introduction to pkgsrc, and to package creation in NetBSD, by Noud de Brouwer
pkgin, a binary package manager for pkgsrc, also by Emile Heitor
(slides)
Automated package building, by Nicolas Thauvin
Besides other talks that were less focused on NetBSD/pkgsrc,
this event shows that there's a pretty active group of
BSD advocates in Europe that manage to advocate "BSD"
in its entirety pretty well. Well done, guys!
NetBSD ketchup - news from my mailbox
Here's another bunch of NetBSD-related news that has
been lingering in my inbox for far too long:
Izumi Tsutsui's
NetBSD/cobalt
restore CD is available based on NetBSD versions
5.0.25.1_RC2.
See the
for information on what it is and how to use it.
A negative symbol lookup cache was added
to NetBSD's loader
for shared libraries and shared objects, ld.so_elf, by
Roy Marples:
``I've been researching why Evolution from GNOME takes over 5 minutes to load on my quad core amd64 beast. It boils down to dlsym looking for a symbol that does not exist directly and as such examining every needed library. However, the current implementation does not remember what libraries it as already checked. Normally this isn't a problem, but with the way Evolution is built the search chain is massive.
[...]
With this patch, Evolution (without the patches to and a glib I added to pkgsrc a few days ago) loads in under 2 seconds (5 seconds with initial disk thrashing). ''
The NetBSD Logo
is available in many variants, but a new variant was submitted
via www@ these days by "Tim" - which is actually plain HTML,
no image:
⚑NetBSD Powered!
SafeNet's ProtectDrive is
``a full disk encryption solution that encrypts the entire hard drive of laptops, workstations and servers, as well as USB flash drives, to protect data in the case of the theft or loss of a hardware device.''
How do you implement such preboot authentication and
harddisk encryption software,
esp. if you want to provide thinks like LDAP integration for
the user/key handling and two-factor authentication?
Little is known, but rumors say the 32bit version of the software
is based on NetBSD, as is backed by
this worker bio info:
``Duties: Working on pre-boot restricted environment with loads before operation system and implemented on NetBSD.
Ported and optimized the KDrive X server to NetBSD.
Developed and implemented user secure authentication interface with smart card support.
Environment and tools : NetBSD (3.0), C/C++, FLTK''
A german-language introduction of pkgsrc on OpenSolaris
was given by Michael 'kvedulv' Moll at the Munich
OpenSolaris User Group back in march.
Slides
and a
video
are available.
Are you still looking for a nice small
ARM-based board to start hacking on NetBSD/arm?
The http://www.friendlyarm.net/products/mini2440
may be a good start, esp. after
Paul Fleischer is reaching completion of NetBSD support
for the board. Citing from
his mail to port-arm:
``I have now fairly good (i.e., it works for me) support for the
MINI2440 on NetBSD with support for the following:
- S3C2440 UART
- DM9000 (MAC+PHY)
- S3C2440 SD Controller
- S3C2440 DMA Controller
- S3C2440 IIS Controller
- FriendlyArm 3,5" LCD Display
- S3C2440 USB Host Controller (OHCI)
- S3C2440 Touch Screen
- UDA1341TS audio codec
Currently, support for three things on the S3C2440 are missing:
- S3C2440 NAND Controller
- S3C2440 USB Device Controller
- S3C2440 RTC
I've also created a stage2 bootloader for use with u-boot, which
ensures that the value of bootargs is passed to the NetBSD kernel.
At this point I have only tested the code with the 64Mb version of the
FriendlyArm MINI2440.
While talking about NetBSD on cool hardware:
How about NetBSD/hpcarm on
WILLCOM | W-ZERO3 (WS004SH) mobile devices?
Here is a screenshot of Ebihara-san's WS011SH with CCW screen,
and there is also a video "booting NetBSD/hpcarm on WILLCOM | W-ZERO3(WS004SH)"
posted on YouTube:
More news from NetBSD land: inside and outside the kernel
My inbox is still overflowing with NetBSD related news,
so here is the next chunk for you:
People owning a Marvell PCMCIA WiFi card can how have
a look at
the malo(4) driver.
If you use this on a gumstix ARM board, have a look
as some fiddling with drvctl(8) is needed to get the
card recognized properly.
Speaking about
Gumstix,
KIYOHARA Takashi has declared the
porting effort as finished.
Support for the baex, connex, verdex and verdex-pro
modules is available, as is support for the support modules.
Staying in the "embedded" corner, KIYOHARA Takashi has
announced
that Plathome's OpenBlockS600 (AMCC 405EX) can now
boot NetBSD via NFS. See the posting for dmesg output.
The
OpenBlockS600
comes with a AMCC 405EX PowerPC CPU,
two GigE ethernet ports and a bunch of other
goodies for a price of about $600US.
Preparations are in an early stage, but there are already
a FAQ and
a timeline
as well as the
Program Terms of Service.
From the NetBSD side, we're always happy for project suggestions
(please use our mailing lists for discussions),
and in NetBSD, we are currently working on out
projects page.
If you plan to submit a proposal for a project
with NetBSD, please see
our project application/proposal form
If you plan to submit a proposal for a project
with NetBSD, please see
our Project Application/Proposal HowTo.
Getting back to the NetBSD code, a number of interesting changes
were made in the previous weeks. The first to mention is that
David Young has continued is work on
the new shutdown order
for device drivers:
``cgd, dk, dm, md, raid, and vnd gracefully detach from the device
tree during shutdown. I believe that ccd is the only virtual disk that
does not detach.''
This allows having arbitrary stack of file systems, and still
have them unconfigured properly in the right order on system
shutdown.
Another major change that went into NetBSD recently is that
terminfo was imported into NetBSD-current.
Terminfo replaces termcap, but provides a
backward compatible termcap interface.
This move follows
discussion from last summer,
and
docs by The Open Group (the people who make
things like the POSIX standard and the Single Unix Specification),
which indicates that the termcap specification will be
withdrawn in the future.
Moving from userland inside the kernel, David Holland has
proceeded with work to
unhook LFS from UFS.
Historically, the Log structured File System was written after
the Berkeley Fast File System. With the idea of sharing
the core "Unix File System" code for both file systems,
this resulted in a strong relationship between LFS and FFS,
which was/is not always the best for the advantage and
stability of either one:
``sharing ufs between both ffs and lfs has
made all three entities (but particularly lfs) gross. ffs and lfs are
not similar enough structurally for this sharing to really be a good
design.''
Another major addition to the NetBSD kernel was made recently
by Darren Hunt:
``Courtesy of CoyotePoint Systems, I've been working on a port of
DTrace [...] to NetBSD for i386.''
Citing Wikipedia,
``DTrace is a comprehensive dynamic tracing framework created by Sun Microsystems for troubleshooting kernel and application problems on production systems in real time. Originally developed for Solaris, it has since been released under the free Common Development and Distribution License (CDDL) and has been ported to several other Unix-like systems.
DTrace can be used to get a global overview of a running system, such as the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file. ''
The code is available in NetBSD-current. I haven't looked
into this yet, but I'm looking forward of reports and blog
postings if
the wikipedia command line examples work.
(I think like with ZFS, Dtrace could use a hand with
documenting the NetBSD side of things. Any takers?)
The last kernel change to mention is related to security:
mapping the address 0 from userland was disabled.
This issue
went throughthe press
late last year, and
it this is now addressed in NetBSD, too.
Those that still beed to map address 0 can do so via
the USER_VA0_DISABLED_DEFAULT kernel option
or the vm.user_va0_disable sysctl.
So much about about the NetBSD code for now. Of course having
all those fine features added screams for an immediate (*cough*)
release, which brings me to the fact that
NetBSD 5.0.2 has been released:
`` NetBSD 5.0.2 is the second critical/security update of the NetBSD 5.0 release branch. It represents a selected subset of fixes deemed critical for security or stability reasons.
Please note that all fixes in critical/security updates (i.e., NetBSD 5.0.1, 5.0.2, etc.) are cumulative, so the latest update contains all such fixes since the corresponding minor release. These fixes will also appear in future minor releases (i.e., NetBSD 5.1, 5.2, etc.), together with other less-critical fixes and feature enhancements. ''
I'm closing for today by pointing to three NetBSD-related events:
There's a
hackathon
going this weekend (Feb 20/21 2010)
Volunteers are wanted to setup & man a NetBSD booth at
FrOSCamp 2010 Zurich, Switzerland, on Sep 17/18 2010
pkgsrcCon 2010 will be held in Basel, Switzerland, from
May 28ths to 30ths 2010.
pkgsrc via git
Did you always feel like
exposing your soul to the dark side
trying out
git,
but couldn't find a good reason for?
Maybe here's one for you, found via
The DragonFly BSD Digest:
Matthew Dillon hass etup a GIT pkgsrc repository
to give DragonFly users and developers a more reliable(?) way
to track the pkgsrc tree.
Right now this is just a copy from cvs every 15 minutes, so it won't
allow changes back to pkgsrc, but apparently it's much faster to download
via git than it is via cvs.
For a quick start, run "git clone git://avalon.dragonflybsd.org/pkgsrc.git".
See
Matt Dillon's mail
for a bit more information.
Catching up - various items (and not source-changes, this time)
Many things have happened in NetBSD-land in the past few weeks,
and as I've been slacking^Wbusy again, here's just a digest
of things that I haven't seen mentioned
elsewhere so far, in
random order:
BSD-related radio-show "bsdtalk" has published an interview
with NetBSD's Andrew Doran in its
March 2009 issue. Besides covering Andrews work,
the upcoming NetBSD 5.0 release is also discussed.
Available as
mp3
and
ogg.
Cross-compiling pkgsrc packages is a long-standing dream, and it's
yet waiting for someone to do it. For the time being, Jared McNeill
has come up with an
HowTo on how to build 32bit packages on amd64
(and probably other 64bit systems).
Jared McNeill's been hacking on more stuff recently, and one thing
includes changes to the framebuffer console support on x86 (i.e.
both i386 and amd64). In short,
the recent changes
are just a stop on the way to move the splashscreen code and
esp. image data from the kernel to userland. I.e. that you can put
something like
into your /boot.conf in the future. But we'll see a separate
announcement when that part is done. Let's stay tuned! :)
Martti Kumparinen has tackled generating a UFS file system on
a "large" (~5.5TB) disk. As the process is not straight forward,
he has
posted a howto that may help in the future.
Any takers for adding comments and integrating this into
The NetBSD Guide? :)
Manpages are a major component of every Unix system. If you have ever
tried to write such a manpage, you 'll have learned that they are
in a funny text-based format similar to LaTeX and HTML, with its
own processor - *roff. There are several *roff implementations, and
the one used in NetBSD currently is the GNU implementation. To provide
an alternative here is good for both removing GPL'd code from the
NetBSD codebase, and also because groff is written in C++, which
is slow to compile, and - well - requires a C++ compiler.
A change for that situation may arise eventually, as Kristaps
Dzonsons has been working on a groff replacement to format
Unix manpages recently. See
his posting
and
his homepage for further information.
I've talked about Xen support for PCI passthrough
recently,
and Manuel Bouyer has finished his work to get full support
for passing in access to specific PCI devices from the Xen
Dom0 to DomUs. See
his posting to port-xen
for more details!
Staying at Xen for a moment, David Brownlee has written
instructions on
Installing Windows XP in Xen under NetBSD.
Just in case anyone needs to run a legacy system... :)
The NetBSD operating system supports many different hardware
and CPU platforms. For a specific platform, binaries are
compiled with a specific compiler, and there is a set of
binaries for each platform. This results in a rather big number
of different sets of binaries - currently about 50.
A different approach
with historic precedence
is to have one binary work on may hardware platforms,
so-called "fat" binaries.
Gregory McGarry has posted
suggestions on how to modify NetBSD's toolchain
to produce fat binaries. An interesting concept which would
solve a number of problems (think: support, updates, pkgsrc!)
Qt is a user-interface library found in widespread use in the
Unix/Linux world. It's not exactly small, and its prerequirement
of the X Window System doesn't it make a #1 choice for embedded
systems at the first look. A Qt variant - Qt/Embedded - can be
ran without X, though, and which thus avoids all the configuration
and hardware support trouble of X in one go.
On NetBSD, Qt/Embedded could talk to the wscons driver directly,
and Valeriy 'uwe' Ushakov has posted about his work on
patches to adopt Qt/Embedded to wscons.
Who's first to post some screenshots?
Yet another pkgsrc-based system: BlackMouse Linux
From the
BlackMouse Linux homepage:
``BlackMouse is Linux distribution based on Slackware Linux and pkgsrc package system. Pkgsrc system is used in NetBSD unix system and other modifications in other systems, for example in FreeBSD. Main base packages is from slackware with some modification for pkgsrc and other applications packages are compiled from pkgsrc tree.
Prefer gui is GTK2, so prefer desktop is Xfce and GNOME, but KDE is compiled too. Any others BlackMouse tools is/will be programmed in Python, GTK and Bash. In fact, we could say, that BlackMouse Linux want be BSD distribution /with same clean/. In another view to BSD, it's better, and more freedom licence that GPL. So this distribution as complet (new thinks to GNU Slackware Linux) is BSD. We can call BSD BlackMouse Linux as BSD distribtion with GNU kernel :) Many thinks which they are only modifed are still under GPL licence ! Only some new scripts or separate files, code or thinks are under BSD licence!
At this moment, there are 3 hardware versions of BlackMouse: i586, i686 and for x86_64.''
Catching up on NetBSD source changes - Sep'08 to early Jan'09
OK, I'll try to catch up source-changes a bit more frequently
in the future (new years resolutions... don't we all have some?),
but here's what I've missed by now, from between September 2008
until now (early January 2009):
In preparation of the NetBSD 5.0 release, a lot of documentation
updates were made, esp. in the release notes. Also, many manual
pages were added to the system, documenting existing userland
tools, library, system and internal interfaces.
Following some re-organization of binary packages on ftp.NetBSD.org
some time ago, the official URLs are now:
Both should have the same results, the latter is more safe
on mirrors that don't carry /pub/pkgsrc. Adjust your PKG_PATHs!
Syntax for /etc/rc.conf's ifconfig_xxN variables and /etc/ifconfig.xxN
was changed to also allow line breaks via ';'s. This allows
something like ifconfig_wi0="ssid 'my network'; dhcp"
Martin Schuette's work on syslogNG from Google Summer of Code 2008
is now available in NetBSD's syslog
X.org integration is advancing in big steps. It's on by default on
a number of platforms (including alpha, i386, macppc, shark, sparc and sparc64),
and instead of using the (now obsolete) MKXORG build variable
it can be build with "build.sh -x".
Old-style LKMs are dead, welcome to the new module framework!
(XXX Documentation???) In the process, more and more kernel
subsystems are being changed to be loadable as a module, e.g.
POSIX AIO and semaphores, File System Snapshots,
emulations, exec formats, coredump, NFS client and server,
http and data accept filters, ppp compressors,
and others.
Hooks into UVM have been added to unload unused kernel modules
if memory is scarce.
MAKEVERBOSE now has two new levels, 3 and 4. The complete list
is now:
0 Minimal output ("quiet")
1 Describe what is occurring
2 Describe what is occurring and echo the actual command
3 Ignore the effect of the "@" prefix in make commands
4 Trace shell commands using the shell's -x flag
The default remains MAKEVERBOSE=2, you can also set this via
build.sh's -N switch.
A POSIX conformant tabs(1) utility was added
The haad-dm branch was merged to NetBSD-current. This
adds Logical Volumen Management (LVM) functionality to
the base NetBSD system. It uses Linux LVM2 tools and our BSD licensed
device-mapper driver.
The wrstuden-revivesa branch was merged into NetBSD-current,
bringing Scheduler Activation based threading back to NetBSD,
and giving NetBSD 5.0 and up both SA and 1:1 threads.
Support for the ARM-based Cortina Systems SL3516 eval board was added
to NetBSD/evbarm
patch(1) got a major overhaul, based on DragonflyBSD and OpenBSD.
There's better detection of double applied patches, rejected diffs
remain in unified diff format, and and less limitation e.g. on line
length.
pxeboot now understands boot.cfg
Boot CD ISO creation has been greatly overhauled, accomodating
changes in boot.cfg, and moving away from a ramdisk-based system
to using a file system on the cd-rom, which helps reduce RAM usage.
Also, the GENERIC kernel can be used there.
makefs(8)'s ISO-9660 (cdrom) support was enhanced to write
RISC OS data. This allows to make bootable CDs for acorn{26,32}
directly, without copying the bootloader to a native file system.
The christos-time_t branch has been merged into NetBSD-current.
This gives 64bit time_t and dev_t types (no more
year 2038-problem!!!).
Many related places like timeval and timespec were adjusted,
kernel and userland APIs were touched, and shared library
major versions (including libc) were bumped for this fairly
exhaustive change.
See src/UPDATING's entry on 20090110 for the full update path!
New/updated drivers:
jme(4) for JMicron Technologies JME250 Gigabit Ethernet and JME260 Fast Ethernet PCI Express controllers
u3g(4) provides better support for 3G datacards than ugensa
dbcool(4) for dbCool(tm) family of Thermal Monitor and Fan Controller
ataraid(4) now supports Intel MatrixRAID and JMicron RAID
bwi(4) for Broadcom BCM4302 wlan controllers, otherwise known as Airport Extreme
alipm(4) for the Acer Labs M7101 Power Manage- ment controller
admtemp(4) for the Analog Devices ADM1021, Analog Devices ADM1023, Analog Devices ADM1032, Genesys Logic GL523SM, Global Mixed-mode Technology G781, Maxim 1617, and Xeon embedded temperature sensors
ipw(4),iwi(4),wpi(4),iwn(4): We ship the firmware now, but
users have to accept the Intel license manually by setting
sysctls like hw.ipw.accept_eula=1. The latter is also offered
by sysinst.
nsp(4) adds support for the NSP2000 cryptographic processor
which does crypto, hashing and arbitrary precision arithmetics
in hardware, and which hooks into the opencrypto(9) interface.
pseye(4) makes the Sony Playstation Eye USB webcam usable with
the new video(4) framework
ath(4) now uses the recently-released source-based version
of the Atheros HAL, no more binary blob!
Whee... I should really do this more often to cut things down.
Releasing pkgsrc-2008Q4
From
the release announcement:
`` The pkgsrc developers are very proud to announce the new pkgsrc-2008Q4
release, which has support for even more packages than previous releases.
As well as updated versions of many packages, the infrastructure of
pkgsrc itself has been improved for better platform and compiler
support.
At the same time, the pkgsrc-2008Q3 release has been deprecated, and
continuing engineering starts on the pkgsrc-2008Q4 release.
The pkgsrc-2008Q4 release celebrates 5 years of quarterly releases
within pkgsrc, and we would like to thank all of our users and
developers for using the world's most portable packaging system - to
all of the users, developers and supporters a very large "Thank you"
from all of us.
Some highlights of the new pkgsrc-2008Q4 release are:
Jared McNeill has introduced pulseaudio to pkgsrc, which is a huge
boost, giving pkgsrc the benefits of one of the best audio systems
our GNOME packages have been updated by Thomas Klausner, and much
work has been done on the HAL layer within GNOME by Jared McNeill. We
also now have improved zeroconf support through the avahi package -
our thanks to Adam Hoka for that.
more packages have been moved to install into a staging directory,
thanks to Joerg Sonnenberger
improved support for AIX, again, from Joerg Sonnenberger
many, many packages have been updated to newer versions, to take
advantage of fixes and improved functionality. [...]
other notable changes include
Kouichirou Hiratsuka has added Openoffice 3 to pkgsrc
Stoned Elipot and Havard Eidnes have made it their personal
goal to incorporate all the CPAN packages into pkgsrc. They
have recently been joined in their quest by Ulrich Habel.
the vlc package continues to be updated, again by Jared
McNeill - it is now at version 0.9.8a
we bid a fond thanks, and farewell, to some old favourites,
such as python 1.5, nail, bidwatcher, jssi, jsdk20, grail,
and zope-2.5
the perl package has been upgraded to version 5.10 - a
side effect of this is that binary packages of perl modules
made with perl-5.8 and earlier versions are incompatible
with perl-5.10
the addition of some interesting, pertinent, and shiny
packages such as parpd, openoffice3, twitux, consolekit,
policykit, hal, sslproxy, diffuse, gstfs, openresolv,
and pulseaudio and related packages. ''
Getting pkgsrc going on Solaris 10/x86
The other day I've setup Solaris 10/x86 (not OpenSolaris)
to play with ZFS. That aside, I wanted to get pkgsrc going.
Here's a bit of how I got things going:
There is no precompiled bootstrap kit available for Solaris 10/x86,
only one for some beta version of Solaris 11/x86. Trying that lead
to problems finding the right symbols at runtime and whatnot, and
did not score success. Likewise, installing the precompiled binary
packages that are available for 11/x86 are expected not to work.
(I'll get back to that in a second ;)
As using precompiled binaries wasn't an option, compiling myself
was the only way. Compiling with pkgsrc requires a C compiler, and
pkgsrc.
The C compiler was installed with Solaris 10/x86, I only had to
find it in /usr/sfw.
For getting pkgsrc, I'd have preferred to use (anon)cvs, but
Solaris doesn't ship a binary for that. So,
grabbing a recent pkgsrc tarball, extracting it (all as non-root -
I do not want to mess up the system by accident!),
things went mostly smooth. To make sure I really have the latest
pkgsrc, I grabbed the cvs binary package for Solaris 11/x86,
untarred it, and used the binary in there to do a "cvs update".
Successfully, I should add.
Next step: boot strap pkgsrc. For future upgrades, I may want to add
other versions of pkgsrc-compiled software, so I'll add simple
versioning and put things into $HOME/pkg1 for now. Bootstrapping this
with the latest pkgsrc was easy:
% set pkgbase=$HOME/pkg1
% sh bootstrap/bootstrap \
? --prefix=${pkgbase} \
? --pkgdbdir=${pkgbase}/var/db/pkg \
? --varbase ${pkgbase}/var \
? --ignore-user-check
This went fine, and I put the following into my .cshrc
to have pkgsrc easily available at any time:
set pkgbase=$HOME/pkg1
set path=($pkgbase/bin $pkgbase/sbin $path)
rehash
alias make bmake
alias pi pkg_info
Now, I was able to see that everything's installed properly:
% pkg_info
bootstrap-mk-files-20080808 *.mk files for the bootstrap bmake utility
pdksh-5.2.14nb3 Free clone of the AT&T Korn shell
bmake-20080215nb1 Portable (autoconf) version of NetBSD 'make' utility
nawk-20050424nb3 Brian Kernighan's pattern-directed scanning and processing language
nbsed-20040821nb1 NetBSD-current's sed(1)
tnftp-20070806 The enhanced FTP client in NetBSD
pax-20080110 POSIX standard archiver with many extensions
pkg_install-20081013 Package management and administration tools for pkgsrc
digest-20080510 Message digest wrapper utility
Also, installing my favourite test package worked fine
(note that 'make' is aliased to 'bmake', the Solaris /usr/ccs/bin/make
command won't work here, of course):
The next test package included the X window system.
Building pkgsrc/x11/xteddy went fine, and starting
it actually game a sweet cuddly teddy bear.
Those two test packages indicate that integral parts of pkgsrc
and its integration into the system work, and I consider it
worth as a base for further platform tests. (Last time I used
pkgsrc on Solaris, it gave me a sweet Firefox binary :)
NetBSD 4.x binary packages
John Klos and Havard Eidnes have made a bunch of precompiled binary
packages available, See
John's
and Havard's
arm
and
alpha
email.
John built packages from the pkgsrc 2008Q2
branch, for the following platforms:
The packages can be found on
ftp.NetBSD.org.
Havard has uploaded a set of preliminary results from a build
of pkgsrc-2008Q1 on NetBSD 4.0/arm. He has a total of 3.5GB
in a total of 4578 packages, which can also be found on
ftp.NetBSD.org (different directory...).
Havard has also uploaded bulk build results from NetBSD 4.0/alpha and
pkgsrc-2008Q2. A total of 6.7GB of packages were uploaded, for a total of 6270
packages. The packages can be found on
ftp.NetBSD.org as well.
GNOME (and Firefox) fonts vs. X11R7 vs. pkgsrc
So I have upgraded my NetBSD 4.0 installation with a -current build with
MKXORG=yes, and then built pkgsrc/meta-pkgs/gnome for the past
few days. Starting up gnome-session just gave me little rectangles
instead of fonts. GRMBL!!!1!
With some investigation, GNOME used pkgsrc's fontconfig goo, which
doesn't include any fonts - which is about what it displays. To fix,
/usr/pkg/etc/fontconfig/fonts.conf can be adjusted to also look
in /usr/X11R6 (which I still have from my NetBSD 4.0 installation)
and /usr/X11R7 (thanks to MKXORG=yes):
I hope this helps anyone encountering a similar situation (including
future incarnations of myself ;). (And if you ask "WTF GNOME?" - I
wanted to have another look, after my last attempt at using GNOME
is now several years ago :-)
NetBSD 5.0 preview: User visible changes in NetBSD-current
I've found a bit of spare time upgrade a NetBSD 4.0 system
to NetBSD-current (4.99.69), and during the usual update procedure
(boot new kernel; build.sh install=/; etcupdate) I found
a number of user-visible changes over NetBSD 4.0 that I'd like
to spotlight here:
audit-packages and download-vulnerability-list are now part
of the NetBSD base system, there's no longer a need to install
them via pkgsrc. I haven't found any hooks to run them automatically
every night, but that can be done easily via cron(8). The commands
in question are:
# download-vulnerability-list
# audit-packages
Per-user-tmp: Currently, /tmp is shared by all users. In order to avoid name
clashes (example: I use /tmp/foo for temporar data as both root
and "normal" user), the /tmp directory can be made to be unique
for each user now, much the same way as this is done e.g. on
Mac Os X. To do so, set
per_user_tmp=yes
in /etc/rc.conf and reboot. In effect, /tmp is unique for
each user then. The implementation is done via magic symlinks:
httpd: NetBSD now ships with a web server in base. Seriously, I have no
idea why this is, but it can be enabled by removing the comment
signs of the 'httpd' service in /etc/inetd.conf, and by populating
/var/www. After that access by both IPv4 and IPv6 is possible.
dhcpcd: NetBSD has shipped with ISC's DHCP client so far, which uses quite
a bit or memory. As a supplement, Roy Marples' dhcpcd has been added
to the base system. In order to use it for interface xx0, put
ifconfig_xx0="dhcp"
into your /etc/rc.conf file (replacing the old "dhclient=yes"
setting).
On a test system, both dhclient and dhcpcd had ps(1) display a
VSZ (virtual size in Kbyte) of ~770, but while dhclient had a RSS
(real memory / resident set size, in Kbyte) of 1068, dhcpcd only
had a RSS of 548, i.e. the new DHCP client daemon uses about half
as much RAM as the ISC dhclient.
/boot.cfg: Last, NetBSD's second stage bootloader can now load a configuration
file on the i386 and amd64 platforms. Assuming the bootloader was
updated after the upgrade ("cp /usr/mdec/boot /boot"), the config
file /boot.cfg is used to print the boot menu and offer appripriate
actions:
# cat /boot.cfg
menu=Boot normally:boot netbsd
menu=Boot single user:boot netbsd -s
menu=Disable ACPI:boot netbsd -2
menu=Disable ACPI and SMP:boot netbsd -12
menu=Drop to boot prompt:prompt
default=1
timeout=5
So much for a first adventure into NetBSD-current. Feel free to point
out more goodies in NetBSD-current that will be new in NetBSD 5.0
over NetBSD 4.0.
The pkgsrc-2008Q2 Release
NetBSD's pkgsrc crew has put great efforts into stabilizing
pkgsrc, and releasing another "stable" version. Alistair
Crooks, co-founder of pkgsrc and long-time manager of
pkgsrc announces:
``The pkgsrc developers are very proud to announce the new pkgsrc-2008Q2
release, which has support for more packages than previous releases.
As well as updated versions of many packages, the infrastructure of
pkgsrc itself has been improved for better platform and compiler
support. [...]
Some highlights of the new pkgsrc-2008Q2 release are:
a new ruby gems framework, from Stoned Elipot and Johnny Lam
many more packages have been moved to install into a staging directory -
the DESTDIR work that Joerg Sonnenberger has done almost singlehandedly
many, many packages have been updated to newer versions, to take
advantage of fixes and improved functionality. The following versions
of packages are included in the pkgsrc-2008Q2 release:
Jared Mcneill has re-worked the compiz window manager
packages
the new ruby gems framework is easy to use, scalable, and
very effective
Eric Gillespie has updated the subversion package to 1.5.0,
and reworked part of the additional language support
thanks to Jared Mcneill, David Holland and Reinoud Zandijk,
wine-1.0 works well on NetBSD
the addition of some interesting, pertinent, and shiny
packages such as acroread8, bind95, blame, boxbackup (client
and server), compiz-fusion, drupal6, firefox3, fltk2,
freeradius2, ftmenu, gambc, gvfs, java-subversion,
mediatomb, mono-tools, mowgli, msel, mtftpd, odt2text,
pkg_leaves, qrencode, ruby-snmp, smbldap-tools, stegtunnel,
torrentzip, unbound, and xsel. ''
I've rebuilt the packages that I've installed on Mac OS X and
Debian Linux into my $HOME using this release the other day,
and things worked really great. Mmm, pkgsrc!
Google and NetBSD Summer of Code Projects in 2008 - Midterm status reports (Updated)
Google's Summer of Code has passed the midterm date. With it, students
and their mentors were asked to give internal status reports of thei
works. While the internal reports themselves are not public, many of
our students have sent mail to NetBSD's public lists, giving details
on their status. Let's give a summary of the state of
affairs. Projects were students have posted reports come first:
wscons: Expansion for wstablet in NetBSD
Student: Jason W. Beaudoin
To support tables for the wscons console driver, a number of
changes to the wscons API have been proposed in the student's
status report. While there are a number of similarities with
the wsmouse interface, there are also a number of differences
that need to be worked around, e.g. absolute vs. relative
coordinates. Many of the proposed changes are implemented, and
we're looking forward to complete this project successfully
within time.
subfiles: Subfile Support for NetBSD
Student: Adam Burkepile
Subfile allow to associate data with a regular file, just like
regular allow to associate data with a directory. New internal
data structures were defined to identify subfiles within the
file system, and tools like newfs and dumpfs were adjusted. An
API is being designed to access subfiles, and work to realize
the assorted functions is being approached.
Translators are programs which provide filesystems in user
space functionality. This is provided via NetBSD's RUMP
interface, and additional system calls and file system
operations have been defined to activate the server processes
when access to such a "translator" is made. Currently, the
translators are only implemented in NetBSD's ext2 file system,
as this allows testing of interaction with Hurd - Hurd's
support for FFS seems non-working, unfortunately. The project's
under busy development, and we're looking forward to see the
final results.
lvm: Write and improve NetBSD LVM driver
Student: Adam Hamsik
This project implements the Linux LVM API (libdevmapper) on
NetBSD, to allow using Linux' lvm2tools for logical volume
management. The project's making excellent progress, there is an
ISO image (see URL in status report) as well as a
qemu
image available for testing,
and latest
reports show that linear volumes can be configured and mounted
already.
uvc: Add support for UVC devices (USB web-cams)
Student: Patrick Mahoney
A kernel driver was developed to read data from webcams using a
custom API, work to implement the video2linux API is under
way. Current challenges lie in NetBSD's USB stack, which lacks
support for isochronous transfers, which is used by many (but
not all) webcams. The project has made excellent progress so
far, and we're positive that the project will be a success.
dvb: DVB drivers and kernel framework
Student: Jeremy Morse
This project implements a driver for Digital Video Broadcasting
to supplement bktr(4)'s TV card support. So far, a driver and
an API to transport data from the kernel to userland was
implemented, matching LinuxTV. The project's making good
progress, even due to conflicts with the academic schedule.
install-tool: Customizable install tool for NetBSD
Student: Zachary Wegner
NetBSD's current installer, sysinst, is being split into
frontend and backend parts, with a configuration file building
the interface between the two parts. Untangling the current
mix of user interaction and install operation are ongoing, with
challenges like request of install media (think floppy #42) and
network configuration. Also, a parser for the configuration
file was written, and work not started yet is the frontend
creating the configuration file for the backend.
fs-utils: File system access utilities
Student: Ysmal Arnaud
This project is using NetBSD's RUMP and the ukfs library to
access a file system image from userland. So far, makefs(1) can
generate a file system image, and it can now be manipulated as
well. Both a "file system console" as shell to operate on the
image as well as separate tools for single operations have been
designed. Many of the "normal" userland tools' functionality
like ls(1), cp(1) and rm(1) were implemented. This project has
made excellent progress so far, see the status report and
project page.
cwrapper: pkgsrc: rewrite wrapper framework in C
Student: Amitai Schlair
The student was distracted by
travel obligations for some time, but work has started in
pkgsrc now. Current work includes a set of ATF tests to cover
the usage of the existing pkgsrc wrapper framework and a design
plan for the new wrapper implementation. The actual wrapper
implementation remains to be written, and we're looking forward
to see the it happen, including integration into pkgsrc plus
benchmarks on the increase in speed.
atfify: Converting remaining regression tests to the Automatic Testing Framework
Student: Lukasz Strzygowski
The Automated Testing Framework was added to NetBSD as a result
of last year's Summer of Code. This year, all the remaining
regression tests from src/regress are being converted to
ATF. Test suites for tools (awk, grep, make, ...) and libraries
(libm, libpthread, ...) were converted so far. Tests for libc
are currently being converted, and kernel tests are next. We're
looking forward to get all of src/regress changed to ATF!
No status report was sent by the following students (or at least I
haven't seem one). Data given here is from the project pages, mostly:
ext3: Implement Ext3 file system support
Student: Rus-Rebreanu Alin-Florin
This project intends to implement journaling in the file system
by reusing Wasbi's wapbl code. Unfortunately, little has
happend to reach this goal (to say the least), and the student
got AWOL. Interested
parties are welcome to try
out ext2fuse
(which also does ext3, despite the name) for now.
teredo: Implementation of RFC4380 (Teredo) in NetBSD
Student: Arnaud Lacombe
The project aims at creating a Teredo client, server and relay
via a kernel pseudo device and assorted userland tools.
The project is currently still in the analysis and design
phase, which will give it little time for completion -- as for
other students, this is due to a clash with the academic
schedule of the student. We're holding up hopes that the set
goals can be met at least partially.
The project aims at implementing the upcoming IETF standards
for syslog transport over TLS (instead of UDP), a more formal
format of the messages themselv for easier automatic parsing,
and signing of messages to assert authentication, integrity and
correct sequencing of syslog messages. The first part of this
is already working, the second part is development. We're
looking forward to see this project succeed, as it will be of
benefit not only to NetBSD.
So much for now. We wish all our students good luck in the second part
of this year's Google Summer of Code, and are looking forward to see
your final results, which are due in 3-4 weeks from now. Keep on
hacking!
Update:
Fixed a typo (thanks tron!), got an update on the ext3 project,
and added a link to the status report of the
cwrapper project, which I've missed (sorry!).
Voltalinux 2.1: Slackware + NetBSD's pkgsrc
Here's another one in the series of
Linux distributions
based on NetBSD's pkgsrc, quoting from the
e-linux.it announcement:
``Matteo Garofano has announced the release of Voltalinux 2.1, a server-oriented distribution based on Slackware Linux and using NetBSD's pkgsrc package management system.
As expected, after the release of Slackware Linux 12.1,
Voltalinux 2.1 (code name 'Livorno') is out. It benefits from the many new features in Slackware: HTTP and FTP install, new kernel, installation on LVM and RAID, etc. Also includes the benefits of the new features found in the NetBSD's pkgsrc port system, such as upgraded packages and a better management system. As usual, Voltalinux comes with more then 150 packages (compiled with the pkgsrc port system) intended for server use, including Postfix, Exim, Dovecot, ClamAV, Pure-FTPd, Bftpd, Squid, MySQL, PostgreSQL, Bind, MaraDNS, etc."''
DracoPKG: an attempt at merging pkgtools and pkgsrc
According to the
homepage, DracoPKg is
``a humble attempt at merging pkgtools and pkgsrc through a simple wrapper. Hiding the complexity through simple commands.'' In this,
pkgsrc is NetBSD's packages collection/toolset, and pkgtools is
the same from Slackware.
The system comes with a wrapper for running the various commands:
dp install foobar to install a package and its
dependencies
dp options foobar to learn about the options that can
be passed to the USE environment variable
dp upgrade foobar to upgrade na installed package by
recompiling
dp replace foobar to replace a single package, without
touching dependencies.
dp remove foobar deinstall a package and all its
dependencies
dp info foobar to print information
...
There are many more commands documented on the
dracopkg homepage,
go and have a look. Maybe this is the thing that gives pkgsrc
the "product readiness" it's been lacking so far?
distbb - A new tool for distributed bulk builds
NetBSD's packages collection knows the concept of "bulk builds"
since the NetBSD 1.3 days, when I wrote
the bulk build infrastructure
in pkgsrc/mk/bulk to test if all packages build, and to rebuild (only)
things that were changed. The infrastructure grew, Dan McMahill helped
to optimize many points, and it was working for its purpose.
A few drawbacks were inherent, though -- a long phase of scanning
all Makefiles before the build and the inability to build on more
than one machine/CPU were two of the more annoying ones.
In an attempt to solve those problems, and offer other features,
Joerg Sonnenberger has worked on the
"pbulk"
system for some optimizations. The system is still in development, and
documentation is somewhat spread over several places, which makes it
not a first-hand replacement for the first system.
Due to this, Aleksey Cheusov has adapted pbulk and improved it into
distbb, a tool for distributed bulk builds.
Aleksey compares his system to the other ones
in a
separate mail. Key items are the use of many separate tools for
processing bits, with the core in /bin/sh and awk. A major point
for forking pbulk was also the easier maintenance for Aleksey as
his own project, with less communication overhead for coordination
with the pbulk author.
I guess we'll see what system is the better one. For me, I'll define
the winner as the system that gives me binary packages for a NetBSD release
for all platforms that NetBSD supports.
Draco GNU/Linux 0.3.0 - powered by pkgsrc
From the
e-linux.it announcement:
``Draco GNU/Linux is a distribution based on Slackware Linux and
"pkgsrc", a package management system developed by NetBSD.
A new
version, 0.3, was released a few days ago: "Introducing Draco GNU/Linux
0.3.0. Featuring kernel 2.6.23 (with optional 2.6.16), glibc 2.6.1,
GCC 4.1.2, and OSS 4.0. Selected packages from pkgsrc are available
through the repository and on an ISO image. This release also introduces
Draco Desktop. Draco Desktop contains the latest stable Draco release,
bundled with software from the latest pkgsrc branch. Draco Desktop
defaults to Xfce, with Fluxbox as an option." Here is the brief
release
announcement. Draco GNU/Linux 0.3.0 is available for download either as
a minimal base system or a "Desktop" edition, an installation CD with
Xfce and Fluxbox window managers.
Testdrive the new pkgsrc.se site
Leo Lundgren from the swedish
netbsd.se /
pkgsrc.se
crew has notified me that they have massively revamped
their pkgsrc website, and are asking for tester. The new
site can be found at
test.pkgsrc.se.
The list of changes as outlined
in the forum
are:
Database backend has moved from MySQL to PostgreSQL
A more accurate database schema
Support for branches
CVS commits
Search function has been improved
Watch list feature is improved, you can now upload your `pkg_info` and for example have a watch list per server
Word watch: Add custom words to look for in the cvs commits, and get notified when a match is found
Support for virtual categories
Fredrik Carlsson also encourages users in the forum:
``We practically vacuum pkgsrc for information, but not everything is presented on the web pages. If you have ideas on what should be visible or how it should look, put a post here or email us. We are very flexible at this point and would like help presenting the information.''
Report on the first pkgsrc-wip hackathon
Adam Hoka posted about
started about doing another hackathon a few days ago,
with the focus of concentrating on pkgsrc-wip,
the SourceForge spinoff of pkgsrc that has "work in progress"
packages, and where it's easy to get commit access to import
packages for easy testing, before they are moved into pkgsrc.
The
"wipathon" ran last weekend, and now
Adam posted a
report on the results of the hackathon:
``We could tidy up pkgsrc-wip and import some packages to pkgsrc.
There are some other software we have made ready to use, and only needs
some testing before we can import them.
We have also got rid of outdated and long time unmaintained packages.
But the most important is, that we could share and discuss our ideas.''
On a side note, it also seems that one or more people got quite
some pkgsrc karma and possibly a new mail address in the process. :)
Start of the freeze for the pkgsrc-2008Q1 branch
NetBSD's package collection pgksrc will take its next
quarter-yearly release in the pkgsrc-2008Q1 branch soon.
To prepare that, pkgsrc is now frozen with respect
to new functionality and infrastructure changes in preparation
of the release branch. The freeze is expected to take at
most two weeks - see
Alistair Crooks' announcement
for more information.
Article: Waving the flag: NetBSD developers speak about version 4.0
Federico Biancuzzi has collected interviews from more than
twenty NetBSD developers in an
multiple-page article
which talks about what's new in the NetBSD 4.0 release:
pkgsrc hackathon focusing on package options next weekend
Adam Hoka writes that ``There is a pkgsrc hackathon scheduled for 3rd-4th of November.
Our goals
Make options.mk files for packages that don`t already use them, and
improve support for options if the package uses them but more options
could be added.''
Essential NetBSD 4.0_RC3/i386 Binary Packages + Install CD (Updated)
I've made a CD from NetBSD 4.0_RC3/i386 and the binary packages that Manuel
built (and is probably still uploading ;-). It is bootable, has all the i386
install sets, plus a selection of "essential" binary packages.
Ten years of pkgsrc - Interviews!
NetBSD's packages collection - today known as "pkgsrc" -
has silently had its 10th birthday! To document the
system, its history and the overall state of the art,
Mark Weinem has written an article
"10 years of pkgsrc".
Besides talking bit about pkgsrc, in general, Mark did a number of
interviews:
There are too many gems to quote here (and I haven't read the full text yet),
but besides the interviews there is also information on
what Linux distributions use similar approaches than pkgsrc
(besides Gentoo :), other pkgsrc-related interviews, information
about pkgsrcCon, and information about the concept of Application
Directories.
Announcing the pkgsrc-2007Q3 branch
Alistair Crooks
writes:
``The pkgsrc developers are very proud to announce the new pkgsrc-2007Q3
branch, which has support for more packages than previous branches.
As well as updated versions of many packages, the infrastructure of
pkgsrc itself has been improved for better platform and compiler
support.
At the same time, the pkgsrc-2007Q2 branch has been deprecated, and
continuing engineering starts on the pkgsrc-2007Q3 branch.
This branch celebrates the tenth anniversary of pkgsrc, and we would
like to take this opportunity to thank all of the people who have made
pkgsrc the most portable packaging system in the world - to all of the
users, developers and supporters a very large "Thank you" from all of
us.''
See
the announcement
for more information of the branch's hilights,
and where to get it.
Two articles on pkgsrc on Solaris and Linux
From the netbsd-in-the-news-department:
Issue 5/2007
of the German freeX magazine has two articles on
pkgsrc, one focussing on Solaris, the other one on Linux.
Ulrich Habel's article "Der Daemon und die Sonne" talks about pkgsrc
on Solaris. He describes how to bootstrap the environment using a
precompiled binary bootstrap that was made available as Solaris
package, then continues on how to use pkg_add and other tools for
using precompiled binaries that are available via www.sunpkg.de.
Dr. Heiko Herrman's article "Daemonic Tux: Linux mit pkgsrc"
describes the situation where he gets to a new workplace that has
Linux on the desktop, but that calls for some software
maintenance. Instead of hunting down the system administrator,
pkgsrc can be used to install everything pkgsrc offers into his home
directory, and without root privileges. The article gives details on
how to bootstrap pkgsrc by compiling, then explains how to compile
packages via pkgsrc and gives some hints on pkgsrc's internals.
The articles cannot be read online, information about the magazine
and how to get it can be found at
www.cul.de.
Catchup: bootprops, pkgsrc logo and security, Chaos Singularity, ... (Updated)
OK, so I was lazy (busy :) again the past few weeks. Here's another
big catch-up of the miracles that happened in NetBSD and pkgsrc land:
Looking at this from someone that raised and left pkgsrc, and who's doing
public relations these days, I'm looking forward to see an official pick
by the people managing pkgsrc this time!
Package security auditing now part of package tools:
Originally, audit-packages was a script that matched every entry in a file
against all installed packages to indicate known security problems.
After some intermediate speedups, this was finally
integrated into the package tools package, "pkg_install"
now - see the link for more details.
Kudos to Adrian Portelli from the pkgsrc-security team!
pkgsrcCon videos online
Julio dropped me mail, hinting at the latest news item
on the pkgsrcCon homepage:
``As a novelty this year, we recorded all pkgsrcCon 2007 talks on video and,
at last, we have finally made them public. They are all linked from the
presentations page,
except for one that is still not available. Enjoy
watching the talks and see what you missed!''.
Rebuilt Solaris sparc pkgsrc/wip
Ulrich Habel, who's doing a lot of build work on Solaris
using pkgsrc, mailed me that he has finished a bulk build
of pkgsrc-2007Q1, and that he next did a bulk run of pkgsrc-wip,
resulting in 3349 packages that are available for download
on his website, sunpkg.de.
Enjoy, and spread the word!
pkgsrcCon 2007 slides
I may have missed the announcement, but in any case (almost) all the slides
from this year's pkgsrcCon are available
now (in mixed formats, though):
Coverity and pkgsrc
I've found this interesting
report
about Coverity and pkgsrc via the
NetBSD News Beat and the
DragonFly BSD Digest,
telling that
``[...] Coverity has chosen to use pkgsrc (www.pkgsrc.org) as a
framework for building and scanning software. Pkgsrc makes it easier for
anyone, including Coverity, to build free software.''
There's also another interesting part that hints that pkgsrc
could gain more users by persuing that user community:
``It happens that
pkgsrc is running a little behind and still has 0.63 as its basis. I will
try to get a more recent version in pkgsrc (in particular because new code
is more likely to contain error, ego notwithstanding), but from looking at
the scan, it's clear many of these bugs remain in the current codebase.''
pkgsrc on HP-UX - bootstrap and diffs (Updated)
Tobias Nygren has
posted
about his experiences and esp.
a binary
bootstrp kit and patches for running
pkgsrc on HP/UX.
Update:
Florian Heigl also
posted
some interesting details on how to work around
a number of bugs.
Catching up: events, articles, benchmarks, summer of code...
So I was away for a few days, being sick and then giving a talk
at the Chemnitz Linuxdays and then off for a few days in
Austria
visiting Vienna & Zotter,
and there's a backlog of stuff that happened in
NetBSD's madhouse^Wwonderful world. Here's a quick run-down of things
that I'm too lazy to post single items on:
Linuxdays Chemnitz:
I was only there for my presentation on sunday, due to not feeling
too well the days before. Still, Stefan, Jörg, Charlie and many
others staffed the booth just fine, and I think every single household
in and around Chemnitz has a NetBSD install and/or Live CD now. :)
While at roadshows:
Stefan Schumacher has made DIN A4 pkgsrc flyers in
english
and
german
language.
NetBSD's puff-based FUSE implementation "refuse" is now in a state
to also run the
NTFS-g3
filesystem, which offers read/write support for NTFS.
It's available from
pkgsrc/filesystems/fuse-ntfs-3g.
Google News found me an article that
NetBSD stack supports Geode NAS design:
``Wasabi Systems Inc.'s BSD-based NAS (network attached storage) software stack now supports a Geode-based reference design from AMD. Wasabi Storage Builder for NAS, combined with AMD's Geode LX NAS RDK (reference design kit), provides a secure, reliable platform for the development of NAS devices, according to Wasabi. ''
While that's all fine for Wasabi, it should be noted that
whatever the company Wasabi offers is not automatically available in the freely
available operating system called NetBSD. Integration efforts
would have to happen first, so the headline of that article
is unfortunately misleading if not to say plain wrong!
Another article that's more to the point:
Julio M. Merino Vidal has worked on getting
multiboot support into NetBSD, and in his article
``Making NetBSD Multiboot-Compatible''
he talks more about it.
Andrew Doran has done lots of work on NetBSD's thread and SMP
implementation recently, and he has made a comparison between
performance of the Scheduler-Activations-based code in
NetBSD 4 and the one that will be in NetBSD 5 (AKA NetBSD-current,
currently numbered as 4.99.13). See
his mail to tech-kern
or watch the images for
'make cleandir' on an empty source tree
and
the MySQL supersmack benchmark.
Google runs another Summer of Code,
and this year it's not clear upfront who will be allowed as mentoring
organizations. NetBSD is ready to participate again, and there's
an
official announcement from NetBSD
about this, including pointers to
our suggested/wanted list of projects
and
the project application HowTo.
People interested in submitting a project proposal (via google!)
are encouraged to use the remaining time until the deadline to
discuss their proposals on the public NetBSD tech-* lists!
(Personally I'll try to stay out of GSoC this year to finish some
reallife work. At least that's the plan so far ...)
Another article that doesn't mention NetBSD but g4u:
``How to Install a New Hard Drive: Tech Clinic''
by Joel Johnson. From the article:
`` To make your new drive work like your old drive, you'll need a disk
"cloner." There are a myriad of options, from commercial solutions such as
the old favorite Ghost from Symantec ($70; symantec.com) and Copy
Commander from VCom ($35; v-com.com) to free applications, such as
MaxBlast from Maxtor, that come bundled with hard drives. If you're
comfortable mucking around with Linux/BSD, I've had great luck with the
free g4u application. If you have a local file server, you can even send
the disk image from your laptop to an FTP site, install the larger drive,
then FTP it back to your laptop, obviating the need for a drive enclosure''.