``
The pkgsrc developers are proud to announce the 54th quarterly release
of pkgsrc, the cross-platform packaging system.  pkgsrc is available
with more than 17500 packages, running on 23 separate platforms; more
information on pkgsrc itself is available at https://www.pkgsrc.org/
A neutral overview can be found at https://www.openhub.net/p/pkgsrc

For the 2017Q1 release we welcome the following notable package
additions and changes to the pkgsrc collection:

  - python 3.6
  - Nextcloud 11
  - firefox 45.8.0 and 52.0.1
  - gradle 3.4
  - pkg_comp 2.0
  - qmail 1.03nb24 binary packages work, supporting common use cases
  - many additional Python, Perl and Ruby modules
  - many additional TeX packages

The default version of Apache has been changed to 2.4 (from 2.2); set
PKG_APACHE_DEFAULT=apache22 in mk.conf to stay with 2.2.

Package removals include gcc 4.5, 4.6, and 4.7 and Xen 3.1, 3.3 and 4.1;
these are old and notable only because of their stature.

The following infrastructure changes were introduced:
  - mk/curses.mk enabled packages to depend on curses without specifying
      a particular version

In total, 192 packages were added, 25 packages were removed, and 1,458
package updates were processed since the pkgsrc-2016Q4 release.

Instructions on using the binary package manager can be found at
http://pkgin.net, and pkgsrc itself can be retrieved from
https://github.com/jsonn/pkgsrc or via cvs or tar file -- see
https://www.netbsd.org/docs/pkgsrc/getting.html.  The branch name
for the 2017Q1 branch is "pkgsrc-2017Q1".''

Quoting from the NASA page, ``Pleiades, one of the world's most powerful supercomputers, represents NASA's state-of-the-art technology for meeting the agency's supercomputing requirements, enabling NASA scientists and engineers to conduct modeling and simulation for NASA missions. This distributed-memory SGI ICE cluster is connected with InfiniBand in a dual-plane hypercube technology.

The system contains the following types of Intel Xeon processors: E5-2680v3 (Haswell), E5-2680v2 (Ivy Bridge), E5-2670 (Sandy Bridge), and X5670 (Westmere). Pleiades is named after the astronomical open star cluster of the same name. ''

The system architecture consists of 163 racks of machines from SGI (still alive!) with Suse Linux as their operating system. Check their page for all the juicy details!

More information on Using Software in the NetBSD Packages Collection (pkgsrc) is available from the NASA Knowledge base.

And you'd think NASA using NetBSD would be the last cool thing they do. :-)

[Tags: nasa, pkgsrc, pleiades]

There are images available that are based on the latest development snapshot, NetBSD-curent, and with the NetBSD 7.0 release around the corner, there is also an image based on NetBSD 7.0 Release Candidat 1.

See the NetBSD wiki for many more details, and if you use your RPI for any cool hacks, be sure to let us know!

[Tags: arm, pkgsrc, raspberrypi, Releases]

These blog articles were very detailed on purpose, to have full logfiles available just in case needed. I have used these logs to prepare my pkgsrcCon 2013 talk about Ansible and Amazon's EC2, so things can be looked at without actually running anything. As it turns out this was good, because the 32bit NetBSD instances that I've used during my pkgsrcCon demonstration actually decided to do a kernel panic, and the presentation was a bit more on the theoretical side than I originally planned.

Now after pkgsrcCon is over, I would like to publish the presentation slides with all the details, and especially the playbooks and all other files to look at - enjoy!

[Tags: amazon, ansible, aws, ec2, pkgsrc, xen]

I've mused a lot about the transition from system planning over system setup to system operation recently. Different kinds of databases may be involved, that serve different purposes: On the planning end, topics like version management, license management, life cycle management and enterprise architecture management come into play. When this transits into operation, other aspects like system and network configuration get more important, while others fade away.

At the end, someone or something has to make adjustments to systems in order to get them into service, and keep them there. This is either done with a lot of manual labor to keep documentation (CMDB, or whatever serves) and the systems in sync, or - more likely - documentation will grow stale quickly and rot away.

With a growing set of hosts, the latter is not an option, and transparency becomes increasingly important. This is where system "orchestration" tools like cfengine, puppet, chef and ansible come into play. You write a system definition there, and the machine is configured according to this definition automatically: Configuration files can be adjusted, accounts created, software packages installed, services enabled, etc.

System Orchestration Alternatives

Looking at the state of affairs, many people seem to be fond of Puppet these days. This comes with a pretty long list of dependencies (and thus complexity to maintain), and configuration files have to be written in Ruby (which I don't know). So I chose Ansible as alternative to investigate - it comes with very little overhead (no separate daemon on neither the central machine nor any of the configured systems, just using SSH) and its own "Playbook" language seems easy enough to start with, yet complete enough to be used in large environments (as the list of customers listed on the homepage shows).

Packaging Ansible, First Try

My first try to get going with Ansible on NetBSD was to use the version 0.9 archive available, instead of using a GIT checkout. The included (GNU) Makefile tries to determine the version from the git checkout. Which the archive is not. Using information not available in the archive either. Checking if the source code was checked out from GIT by looking if there's a git binary available is not the best idea here, and so I dropped the ball, cursing the fact that not all the world is Linux. Sort of.

Looking briefly at a fresh git checkout that this worked there, but as NetBSD's pkgsrc doesn't support git checkouts (yet, as far as I know?), I moved back to the 0.9 archive.

Packaging Ansible, Second Try

Leaving out the git-based version games, the next fun was to get the Python setup script to install the files in the right place. Having missed much pkgsrc development recently, and not being into Python either way, this proved unneccessary - Joerg Sonnenberg pointed me at existing pkgsrc infrastructure for Python programs, and with them the program was packaged successfully in no time. For now, the package is available on my website, I'll look into review and import into pkgsrc next.

Ansible on NetBSD - What Works, And What Needs More Work

Modules tested successfully on NetBSD 6:

• ping
• command
• copy
• facter
• group add/delete
• mysql_db add/delete
• user add/delete (without system=true)

• user add/delete (with system=true)
• service (needs work)

Examples: Ansible On NetBSD

Here's an example session with basic operation:

% cat work/ansible/hosts.HF
#127.0.0.1 ansible_python_interpreter=/usr/pkg/bin/python2.7
127.0.0.1
    
% ansible -i work/ansible/hosts.HF -s all -m ping
  127.0.0.1 | success >> {  
    "changed": false,
    "ping": "pong"
  }

% finger testuser
finger: testuser: no such user

% ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=present"
127.0.0.1 | success >> {
    "changed": true, 
    "comment": "", 
    "createhome": true, 
    "group": 100, 
    "home": "/home/testuser", 
    "name": "testuser", 
    "shell": "/bin/sh", 
    "state": "present", 
    "system": false, 
    "uid": 1005
}

% finger testuser
Login: testuser                         Name: 
Directory: /home/testuser               Shell: /bin/sh
Never logged in.
No Mail.
No Plan.

% ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=absent"
127.0.0.1 | success >> {
    "changed": true, 
    "force": false, 
    "name": "testuser", 
    "remove": false, 
    "state": "absent"
}

% finger testuser
finger: testuser: no such user

Update: A spanish translation of this article is available now, thanks to Maria Ramos from Webhostinghub.com.

[Tags: ansible, pkgsrc]

• NetBSD and MirBSD developer Benny Siegert gave a talk titled "pkgsrc on MirBSD" - see his slides! pkgsrc is a framework for packaging and building 3rd party applications from source. Besides MirBSD, it runs on many other platforms like Linux and Mac OS X.
• While building from source is fine, it costs a lot of time. pkgsrc can also create binary packages, and to manage those, there is "pkgin", a binary package manager. Its developer, Emile 'iMil' Heitor introduced it in a talk - see the slides (PDF)!

• MINIX3 and BSD, by Arun Thomas
• The Lua Scripting Language in the NetBSD Kernel, by Marc Balmer
• Touch your NetBSD - towards tablet integration, by Pierre Pronchery
• pkgsrc on MirBSD, by Benny Siegert (slides)
• Introduction to pkgsrc, and to package creation in NetBSD, by Noud de Brouwer
• pkgin, a binary package manager for pkgsrc, also by Emile Heitor (slides)
• Automated package building, by Nicolas Thauvin

• Izumi Tsutsui's NetBSD/cobalt restore CD is available based on NetBSD versions 5.0.2 5.1_RC2. See the for information on what it is and how to use it.

• A negative symbol lookup cache was added to NetBSD's loader for shared libraries and shared objects, ld.so_elf, by Roy Marples: ``I've been researching why Evolution from GNOME takes over 5 minutes to load on my quad core amd64 beast. It boils down to dlsym looking for a symbol that does not exist directly and as such examining every needed library. However, the current implementation does not remember what libraries it as already checked. Normally this isn't a problem, but with the way Evolution is built the search chain is massive. [...]

  With this patch, Evolution (without the patches to and a glib I added to pkgsrc a few days ago) loads in under 2 seconds (5 seconds with initial disk thrashing). ''

• The NetBSD Logo is available in many variants, but a new variant was submitted via www@ these days by "Tim" - which is actually plain HTML, no image:

  ⚑ NetBSD Powered!

• SafeNet's ProtectDrive is ``a full disk encryption solution that encrypts the entire hard drive of laptops, workstations and servers, as well as USB flash drives, to protect data in the case of the theft or loss of a hardware device.''

  How do you implement such preboot authentication and harddisk encryption software, esp. if you want to provide thinks like LDAP integration for the user/key handling and two-factor authentication? Little is known, but rumors say the 32bit version of the software is based on NetBSD, as is backed by this worker bio info: ``Duties: Working on pre-boot restricted environment with loads before operation system and implemented on NetBSD. Ported and optimized the KDrive X server to NetBSD. Developed and implemented user secure authentication interface with smart card support.

  Environment and tools : NetBSD (3.0), C/C++, FLTK''

• A german-language introduction of pkgsrc on OpenSolaris was given by Michael 'kvedulv' Moll at the Munich OpenSolaris User Group back in march. Slides and a video are available.

• Running NetBSD on an Oracle Sun Fire X4140 Server? Check out this posting by Ignatios Souvatzis for the full dmesg pr0n of this machine with 12 CPU cores and 32GB RAM!

• Are you still looking for a nice small ARM-based board to start hacking on NetBSD/arm? The http://www.friendlyarm.net/products/mini2440 may be a good start, esp. after Paul Fleischer is reaching completion of NetBSD support for the board. Citing from his mail to port-arm:

  ``I have now fairly good (i.e., it works for me) support for the MINI2440 on NetBSD with support for the following:
  - S3C2440 UART
  - DM9000 (MAC+PHY)
  - S3C2440 SD Controller
  - S3C2440 DMA Controller
  - S3C2440 IIS Controller
  - FriendlyArm 3,5" LCD Display
  - S3C2440 USB Host Controller (OHCI)
  - S3C2440 Touch Screen
  - UDA1341TS audio codec
  
  Currently, support for three things on the S3C2440 are missing:
  - S3C2440 NAND Controller
  - S3C2440 USB Device Controller
  - S3C2440 RTC
  
  I've also created a stage2 bootloader for use with u-boot, which ensures that the value of bootargs is passed to the NetBSD kernel. At this point I have only tested the code with the 64Mb version of the FriendlyArm MINI2440.

  All the code is available in a Git repository[1] and is based on the netbsd-5 code base. Progress can be followed on my webpage[2]. ''

• While talking about NetBSD on cool hardware: How about NetBSD/hpcarm on WILLCOM | W-ZERO3 (WS004SH) mobile devices? Here is a screenshot of Ebihara-san's WS011SH with CCW screen, and there is also a video "booting NetBSD/hpcarm on WILLCOM | W-ZERO3(WS004SH)" posted on YouTube:

  For more details, see Izumi Tsutsui's posting on port-hpcarm.

• People owning a Marvell PCMCIA WiFi card can how have a look at the malo(4) driver. If you use this on a gumstix ARM board, have a look as some fiddling with drvctl(8) is needed to get the card recognized properly.

• Speaking about Gumstix, KIYOHARA Takashi has declared the porting effort as finished. Support for the baex, connex, verdex and verdex-pro modules is available, as is support for the support modules.

• Staying in the "embedded" corner, KIYOHARA Takashi has announced that Plathome's OpenBlockS600 (AMCC 405EX) can now boot NetBSD via NFS. See the posting for dmesg output. The OpenBlockS600 comes with a AMCC 405EX PowerPC CPU, two GigE ethernet ports and a bunch of other goodies for a price of about $600US.

• Coming from hardware to software, pkgsrc-2009Q4 was released some time ago, and of course binary packages are available for a number of platforms: 5.0/macppc, 4.0.1/sparc and 5.0.1/sparc, 4.0.1/i386, 5.0.1/i386 and the same for 4.0.1/amd64 5.0.1/amd64. Also, binaries of pkgsrc-2009Q3 are available for 5.0/shark.

• Google's Summer of Code was a big success for NetBSD and all of the Open Source community, and it seems there will be one again this summer.

  Preparations are in an early stage, but there are already a FAQ and a timeline as well as the Program Terms of Service. From the NetBSD side, we're always happy for project suggestions (please use our mailing lists for discussions), and in NetBSD, we are currently working on out projects page. If you plan to submit a proposal for a project with NetBSD, please see our project application/proposal form If you plan to submit a proposal for a project with NetBSD, please see our Project Application/Proposal HowTo.

• Getting back to the NetBSD code, a number of interesting changes were made in the previous weeks. The first to mention is that David Young has continued is work on the new shutdown order for device drivers: ``cgd, dk, dm, md, raid, and vnd gracefully detach from the device tree during shutdown. I believe that ccd is the only virtual disk that does not detach.'' This allows having arbitrary stack of file systems, and still have them unconfigured properly in the right order on system shutdown.

• Another major change that went into NetBSD recently is that terminfo was imported into NetBSD-current. Terminfo replaces termcap, but provides a backward compatible termcap interface. This move follows discussion from last summer, and docs by The Open Group (the people who make things like the POSIX standard and the Single Unix Specification), which indicates that the termcap specification will be withdrawn in the future.

• Moving from userland inside the kernel, David Holland has proceeded with work to unhook LFS from UFS. Historically, the Log structured File System was written after the Berkeley Fast File System. With the idea of sharing the core "Unix File System" code for both file systems, this resulted in a strong relationship between LFS and FFS, which was/is not always the best for the advantage and stability of either one: ``sharing ufs between both ffs and lfs has made all three entities (but particularly lfs) gross. ffs and lfs are not similar enough structurally for this sharing to really be a good design.''

• Another major addition to the NetBSD kernel was made recently by Darren Hunt: ``Courtesy of CoyotePoint Systems, I've been working on a port of DTrace [...] to NetBSD for i386.'' Citing Wikipedia, ``DTrace is a comprehensive dynamic tracing framework created by Sun Microsystems for troubleshooting kernel and application problems on production systems in real time. Originally developed for Solaris, it has since been released under the free Common Development and Distribution License (CDDL) and has been ported to several other Unix-like systems.

  DTrace can be used to get a global overview of a running system, such as the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file. ''

  The code is available in NetBSD-current. I haven't looked into this yet, but I'm looking forward of reports and blog postings if the wikipedia command line examples work.

  (I think like with ZFS, Dtrace could use a hand with documenting the NetBSD side of things. Any takers?)

• The last kernel change to mention is related to security: mapping the address 0 from userland was disabled. This issue went through the press late last year, and it this is now addressed in NetBSD, too. Those that still beed to map address 0 can do so via the USER_VA0_DISABLED_DEFAULT kernel option or the vm.user_va0_disable sysctl.

• So much about about the NetBSD code for now. Of course having all those fine features added screams for an immediate (*cough*) release, which brings me to the fact that NetBSD 5.0.2 has been released: `` NetBSD 5.0.2 is the second critical/security update of the NetBSD 5.0 release branch. It represents a selected subset of fixes deemed critical for security or stability reasons.

  Please note that all fixes in critical/security updates (i.e., NetBSD 5.0.1, 5.0.2, etc.) are cumulative, so the latest update contains all such fixes since the corresponding minor release. These fixes will also appear in future minor releases (i.e., NetBSD 5.1, 5.2, etc.), together with other less-critical fixes and feature enhancements. ''

• There's a hackathon going this weekend (Feb 20/21 2010)
• Volunteers are wanted to setup & man a NetBSD booth at FrOSCamp 2010 Zurich, Switzerland, on Sep 17/18 2010
• pkgsrcCon 2010 will be held in Basel, Switzerland, from May 28ths to 30ths 2010.

For a quick start, run "git clone git://avalon.dragonflybsd.org/pkgsrc.git". See Matt Dillon's mail for a bit more information.

[Tags: git, pkgsrc]