hubertf's NetBSD blog

[20130528]	NetBSD's projects for Google's Summer of Code 2013 have been chosen In this year's round of Google Summer of Code, we have again received a number of project proposals by interested students. After going into details and finding out what projects have a chance to give both the NetBSD project and the student the most benefit (besides the money), this is settled now, and I'm pleased to announce that the following students and their projects are our prospects for 2013: Julian Fagir: System upgrade (system_upgrade) Haomai Wang: Make NetBSD a supported guest OS under VirtualBox (Virtualbox) Manuel Wiesinger: Defragmentation for FFS in NetBSD (defrag_ffs) Myron Aub: Port Linux's drm/kms/gem/i915 (DRM2) Przemyslaw Sierocinski: Implement file system flags to scrub data blocks before deletion (fs_scrub_flags) Of course all other students who have submitted proposals can feel free to participate in NetBSD outside of the Summer of Code. For those participating, this is the time for the timeline. [Tags: google-soc]
[20130520]	NetBSD 6.1 and 6.0.2 released Following NetBSD's release scheme, two new releases are available now. NetBSD 6.1 is the next release from the netbsd-6 release, and it contains security fixes, bug fixes and some new feature. NetBSD 6.0.2 is the second stability update for NetBSD 6.0, and it also contains bugfixes and security fixes, but no new features. See the release map graph on the NetBSD website for a visual representation of the relationship between releases. Major news in 6.1 include: Security: prevent kernel panics via userland requests from kqueue, a random number generator update to prevent weak cryptographic keys and a vulnerability in grep. Networking: many updates to NetBSD's new packet filter npf, and improved SMP operations. Embedded: Raspberry Pi now has working USB and ethernet, support for the watchdog timer in some Marvell SoCs, fixes to the Kirkwood IRQ code Platforms: device driver for Hydra and ASDG Zorro2 bus network cards on Amiga, x68k's bootloader can now boot from CD and network, and dtrace support on amd64. Drivers: add LSI Thunderbolt (SAS2208) controllers, Apple's Thunderbolt to Gigabit Ethernet adapter, and improve stability with multiple concurrent file system snapshots. ... plus numerous bugfixes. For more details see the release notes of NetBSD 6.1 and NetBSD 6.0.2. NetBSD is a volunteer project ran by a non-profit organization and with no commercial backing. As such, your donations are very important to the project, and can fund developing in various areas, including: Improving network stack concurrency and performance. Development of modern file systems and improvement of existing ones. Features which are useful in embedded environments, such as high resolution timers and execute in place (XIP) support. Automatic testing and quality assurance. For more information about donating, visit http://www.NetBSD.org/donations/ The NetBSD Foundation is a 501(c)(3) organization in the US, and donations may be tax deductible. [Tags: releases]
[20130409]	NetBSD is part of Google's Summer of Code 2013 News is out that NetBSD is part of Google's Summer of Code 2013 (GSoC) again. GSoC is about students doing work for Open Source projects over the summer, and getting paid while doing so. By Google. For projects proposed by both students and the Open Source projects. Click on the above link for more information on GSoC in general, there is also a list of proposed projects for this year in NetBSD. Next steps are: April 9 - 21: Would-be student participants discuss application ideas with mentoring organizations. April 22, 19:00 UTC: Student application period opens. May 3, 19:00 UTC: Student application deadline. Interim Period: Mentoring organizations review and rank student proposals; where necessary, mentoring organizations may request further proposal detail from the student applicant. May 6: Mentoring organizations should have requested slots via their profile in Melange by this point. May 8: Slot allocations published to mentoring organizations From there, students work on their projects with the help of their mentors. There's a "midterm" report due with a first part of the money paid, the rest is paid if the project is finished successfully. During the project, students are encouraged to publish news about their process to the world in blogs and other ways found appropriate by their mentoring organizations. Past NetBSD projects can be found on SourceForce. Interested? Act now! [Tags: google-soc]
[20130324]	Ansible & EC2 - Playbooks for orchestrating NetBSD into the cloud As follower of my blog you have seen the steps towards getting NetBSD instances started in Amazon's EC2 cloud with a simple web application deployed on one EC2 instance and the database on another one. These blog articles were very detailed on purpose, to have full logfiles available just in case needed. I have used these logs to prepare my pkgsrcCon 2013 talk about Ansible and Amazon's EC2, so things can be looked at without actually running anything. As it turns out this was good, because the 32bit NetBSD instances that I've used during my pkgsrcCon demonstration actually decided to do a kernel panic, and the presentation was a bit more on the theoretical side than I originally planned. Now after pkgsrcCon is over, I would like to publish the presentation slides with all the details, and especially the playbooks and all other files to look at - enjoy! [Tags: amazon, ansible, aws, ec2, pkgsrc, xen]
[20130321]	Ansible, EC2 and NetBSD final milestone 4 reached: Web and DB on separate VMs in the cloud In the fourth and last step on my journey to use Ansible to bring a non-trivial system of a Web server and a DB server into Amazon's EC2 cloud, this is the final step. After starting out with a local VMware VM and making first steps with Ansible and EC2, the previous step was to push a single system into the cloud. Now, the final step is to setup two distinct VMs, one for the database and one for the webserver, and then make them known to each other. The single steps are: Prepare the two VMs Basic setup for all systems Install the database server Install the webserver Connect database and webserver Again, here are all the steps in detail: As before, ensure local time is correct when talking to Amazon, and also make sure the SSH agent has the proper key loaded. % date Thu Mar 21 00:45:37 CET 2013 % ssh-add -l 2048 d5:25:19:3d:59:40:35:32:03:f7:c5:83:de:19:b6:d0 ../../euca2ools/key-eucaHF.pem (RSA) Make sure security groups are setup properly. We use one group for the database server, and one for the webserver. This defines the access permissions from the internet, and also allows to identify systems for their individual configuration and also for connecting them in the final step: % euca-describe-groups ... GROUP sg-ae54b3c5 749335780469 ec2-dbservers Database servers PERMISSION 749335780469 ec2-dbservers ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-dbservers ALLOWS tcp 3306 3306 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-dbservers ALLOWS icmp -1 -1 FROM CIDR 0.0.0.0/0 GROUP sg-a854b3c3 749335780469 ec2-webservers Web servers PERMISSION 749335780469 ec2-webservers ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS icmp -1 -1 FROM CIDR 0.0.0.0/0 Now, run our playbook to setup the two VMs. This uses the single playbook from the previous milestone, and just runs it twice with different security groups: % ansible-playbook -i hosts-HF config-ec2-prepare-db+web-vm.yml PLAY [localhost] ******************* TASK: [ec2-webservers \| Launch new EC2 instance] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Give the system 30 seconds to boot up] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Get rid of SSH "Are you sure you want to continue connecting (yes/no)?" query] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Fix /usr/bootstrap.sh to run pkgin with -y] ***************** changed: [127.0.0.1] => (item={'cmd': 'install /usr/bootstrap.sh /usr/bootstrap.sh.orig'}) changed: [127.0.0.1] => (item={'cmd': 'chmod +w /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'sed "s,bin/pkgin update,bin/pkgin -y update," /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'chmod -w /usr/bootstrap.sh'}) TASK: [ec2-webservers \| Install pkgin via /usr/bootstrap.sh] ***************** changed: [127.0.0.1] => (item={'cmd': u'env PATH=/usr/sbin:${PATH} /usr/bootstrap.sh binpkg'}) TASK: [ec2-webservers \| Copy over Ansible binary package] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Install Ansible dependencies] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Install Ansible package (manually)] ***************** changed: [127.0.0.1] TASK: [ec2-webservers \| Setup lame /usr/bin/python symlink] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Launch new EC2 instance] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Give the system 30 seconds to boot up] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Get rid of SSH "Are you sure you want to continue connecting (yes/no)?" query] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Fix /usr/bootstrap.sh to run pkgin with -y] ***************** changed: [127.0.0.1] => (item={'cmd': 'install /usr/bootstrap.sh /usr/bootstrap.sh.orig'}) changed: [127.0.0.1] => (item={'cmd': 'chmod +w /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'sed "s,bin/pkgin update,bin/pkgin -y update," /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'chmod -w /usr/bootstrap.sh'}) TASK: [ec2-dbservers \| Install pkgin via /usr/bootstrap.sh] ***************** changed: [127.0.0.1] => (item={'cmd': u'env PATH=/usr/sbin:${PATH} /usr/bootstrap.sh binpkg'}) TASK: [ec2-dbservers \| Copy over Ansible binary package] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Install Ansible dependencies] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Install Ansible package (manually)] ***************** changed: [127.0.0.1] TASK: [ec2-dbservers \| Setup lame /usr/bin/python symlink] ***************** changed: [127.0.0.1] PLAY RECAP ***************** 127.0.0.1 : ok=18 changed=18 unreachable=0 failed=0 Just to make sure, check that the two instances run properly, and are in the right security groups, ec2-webservers and ec2-dbservers: % euca-describe-instances RESERVATION r-a419f9d9 749335780469 ec2-webservers INSTANCE i-21b7c441 ami-5d0f8034 ... RESERVATION r-641efe19 749335780469 ec2-dbservers INSTANCE i-54a2ab3e ami-5d0f8034 ... Next, bring the two freshly setup systems (which are already capable of acting as ansible targets) up to our basic system setup: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-basic.yml PLAY [security_group_ec2-webservers;security_group_ec2-dbservers] ***************** TASK: [ping] ***************** ok: [ec2-54-235-44-118.compute-1.amazonaws.com] ok: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Install tcsh] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Add user feyrer] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Create ~feyrer/.ssh directory] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable ssh login with ssh-key] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Install sudo] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable PW-less sudo-access for everyone in group 'wheel'] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Disable ssh logins as root] ***************** ok: [ec2-54-235-44-118.compute-1.amazonaws.com] ok: [ec2-54-234-139-151.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-54-234-139-151.compute-1.amazonaws.com : ok=8 changed=6 unreachable=0 failed=0 ec2-54-235-44-118.compute-1.amazonaws.com : ok=8 changed=6 unreachable=0 failed=0 Check: % ssh ec2-54-234-139-151.compute-1.amazonaws.com id uid=1000(feyrer) gid=100(users) groups=100(users),0(wheel) % % ssh ec2-54-235-44-118.compute-1.amazonaws.com id uid=1000(feyrer) gid=100(users) groups=100(users),0(wheel) Now that the two machines run with our basline configuration, install their individual software and settings. First the database server: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-dbserver.yml PLAY [security_group_ec2-dbservers] ***************** TASK: [Install mysql] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Install MySQL rc.d script] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Start MySQL service] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Install python-mysqldb (for mysql_user module)] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Setup DB] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Add db-user] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Copy over DB template] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Import DB data] ***************** changed: [ec2-54-235-44-118.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-54-235-44-118.compute-1.amazonaws.com : ok=8 changed=8 unreachable=0 failed=0 Check and see if the database works as expected: % ssh -t ec2-54-235-44-118.compute-1.amazonaws.com mysql -u webapp -p webapp Enter password: **** ... mysql> show tables; +------------------+ \| Tables_in_webapp \| +------------------+ \| names \| +------------------+ 1 row in set (0.01 sec) mysql> select * from names; +----+--------+------+ \| id \| first \| last \| +----+--------+------+ \| 1 \| Donald \| Duck \| \| 2 \| Daisy \| Duck \| +----+--------+------+ 2 rows in set (0.00 sec) mysql> bye Excellent. Now setup the webserver, too: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-webserver.yml PLAY [security_group_ec2-webservers] ***************** TASK: [Installing ap24-php53 package and dependencies] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Install Apache rc.d script] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable and start Apache service] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable PHP in Apache config file] ******************* changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': 'LoadModule.mod_php5.so', 'l': 'LoadModule php5_module lib/httpd/mod_php5.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': 'AddHandler.x-httpd-php', 'l': 'AddHandler application/x-httpd-php .php'}) TASK: [Make Apache read index.php] ******************* changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Add simple PHP test - see http://10.0.0.181/phptest.php] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Install phpmyadmin] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable phpmyadmin in Apache config] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Fix Apache access control for phpmyadmin] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Enable PHP modules in PHP config file] ******************* changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.zlib.so', 'l': 'extension=zlib.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.zip.so', 'l': 'extension=zip.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.mysqli.so', 'l': 'extension=mysqli.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.mysql.so', 'l': 'extension=mysql.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.mcrypt.so', 'l': 'extension=mcrypt.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.mbstring.so', 'l': 'extension=mbstring.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.json.so', 'l': 'extension=json.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.gd.so', 'l': 'extension=gd.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.gettext.so', 'l': 'extension=gettext.so'}) changed: [ec2-54-234-139-151.compute-1.amazonaws.com] => (item={'re': '^extension.bz2.so', 'l': 'extension=bz2.so'}) TASK: [Create directory for webapp] ******************* changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Deploy example webapp] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] TASK: [Create webapp symlink for easy access] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] NOTIFIED: [restart apache] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-54-234-139-151.compute-1.amazonaws.com : ok=14 changed=14 unreachable=0 failed=0 Again, test: % links -dump ec2-54-234-139-151.compute-1.amazonaws.com/ It works! % % links -dump http://ec2-54-234-139-151.compute-1.amazonaws.com/phptest.php \| head PHP Logo PHP Version 5.3.17 System NetBSD ip-10-80-61-33.ec2.internal 6.0.1 NetBSD 6.0.1 (XEN3PAE_DOMU) i386 Build Date Dec 14 2012 10:31:13 './configure' '--with-config-file-path=/usr/pkg/etc' '--with-config-file-scan-dir=/usr/pkg/etc/php.d' '--sysconfdir=/usr/pkg/etc' '--localstatedir=/var' % % links -dump http://ec2-54-234-139-151.compute-1.amazonaws.com/webapp/ Showing table hf.names: Cannot connect to database: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)(2002) Close to optimum, but the last error is actually expectet: In order for proper operation, the Database needs to grant the webserver access, and the web server needs to know where the database server is. So let's connect them! This step is done by preparing a shell script on both systems, which will then be ran to - depending on the system's security group - perform the proper steps: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-connections.yml PLAY [security_group_ec2-webservers;security_group_ec2-dbservers] ***************** TASK: [Collect EC2 host information] ***************** ok: [ec2-54-234-139-151.compute-1.amazonaws.com] ok: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Prepare connection-script in /tmp/do-connect-vms.sh] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] changed: [ec2-54-235-44-118.compute-1.amazonaws.com] TASK: [Run connection-script] ***************** changed: [ec2-54-234-139-151.compute-1.amazonaws.com] changed: [ec2-54-235-44-118.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-54-234-139-151.compute-1.amazonaws.com : ok=3 changed=2 unreachable=0 failed=0 ec2-54-235-44-118.compute-1.amazonaws.com : ok=3 changed=2 unreachable=0 failed=0 With that final step, our test web application works, and the webserver can access the database properly: % links -dump http://ec2-54-234-139-151.compute-1.amazonaws.com/webapp/** Showing table hf.names: +--------------------+ \| id \| first \| last \| \|----+--------+------\| \| 1 \| Donald \| Duck \| \|----+--------+------\| \| 2 \| Daisy \| Duck \| +--------------------+ ---------------------------------------------------------------------- Enter new values: first: _____________________ last: _____________________ [ Submit ] So much for this exercise. I'll talk about the ansible and euca2ools packages at pkgsrcCon 2013 in Berlin. Join in if you're curious about what the actual playbooks used in the above examples look like, or stay tuned to find my presentation and all the data after pkgsrcCon 2013. [Tags: amazon, ansible, ec2, xen]
[20130321]	Happy 20th Birthday, NetBSD! (Update) 20 years back from today, NetBSD was initially checked into CVS. Revision 1.1 of src/Makefile was committed on March 21st 1993 on 09:45:37 by Chris Demetriou (cgd@): % cvs log -Nr1.1 Makefile ... revision 1.1 date: 1993/03/21 09:45:37; author: cgd; state: Exp; branches: 1.1.1; Initial revision NetBSD was started as successor to the Berkeley System Distribution (BSD) Unix with a focus on multiplatform support. Personally, I've followed NetBSD since the day in 1993 when the Amiga port popped up, which was the first platform that the newly forked operating system was ported to after its separation from BSD. Many things have happened in the past 20 years, and a lot could be shown and told for the history books at this point. But I guess that can be done later - I'd be happy to help out with such a project if someone wants to start it, though :) For today I'm very happy that NetBSD is available on a wide range on platforms, runs the software that I want and gives me the assurrance it will be around tomorrow and hopefully for the next 20 years, too. Cheers, NetBSD! Update: Jeremy Reed pointed me at his BSDnewletter posting, which gives a number of details of NetBSD's history. Recommended reading! [Tags: history, netbsd]
[20130321]	Ansible, EC2 and NetBSD milestone 3 reached: Web and database in the cloud With the previous work on setting up a local VM as database and web server and setting up a Xen VM in Amazon's EC2 cloud combined, it is pretty straight forward to setup a EC2 instance that has all the software to serve a simple web application from the cloud. The single steps are: Prepare the environment with proper time, SSH agent and EC2 firewall groups Setup EC2 instance with pkgin and ansible Do basic preparations to meet our standards for logins, shells and general usability and security Setup database server with DB software, user and import of data Setup web server with all the software and some demo application The following details show all the commands can and their output in more detail: Make sure time is set properly - needed when talking to Amazon EC2: % sudo sh /etc/rc.d/ntpd stop ntpd not running? (check /var/run/ntpd.pid). % sudo sh /etc/rc.d/ntpdate restart Setting date via ntp. % sudo sh /etc/rc.d/ntpd start Starting ntpd. % date Sat Mar 16 16:46:19 CET 2013 Teach our EC2 SSH key to SSH agent, so we don't have to type a password (which we don't know anyways - EC2 only works with SSH keys): % ssh-add -l Could not open a connection to your authentication agent. % % eval `ssh-agent` Agent pid 10467 % ssh-add -l The agent has no identities. % ssh-add ../../euca2ools/key-eucaHF.pem Identity added: ../../euca2ools/key-eucaHF.pem (../../euca2ools/key-eucaHF.pem) % ssh-add -l 2048 d5:25:19:3d:59:40:35:32:03:f7:c5:83:de:19:b6:d0 ../../euca2ools/key-eucaHF.pem (RSA) Check security (firewall) groups - those are stored in EC2, and we have previously set them up: % euca-describe-groups ... GROUP sg-a854b3c3 749335780469 ec2-webservers Web servers PERMISSION 749335780469 ec2-webservers ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS icmp -1 -1 FROM CIDR 0.0.0.0/0 See if there are any EC2 instances running: % euca-describe-instances % No - that's fine, we are about to change that! Run first playbook to launch EC2 instance and prepare it for using with ansible: % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml PLAY [localhost] ******************* TASK: [Launch new EC2 instance] ***************** changed: [127.0.0.1] TASK: [Give the system 30 seconds to boot up] ***************** changed: [127.0.0.1] TASK: [Get rid of SSH "Are you sure you want to continue connecting (yes/no)?" query] ***************** changed: [127.0.0.1] TASK: [Fix /usr/bootstrap.sh to run pkgin with -y] ***************** changed: [127.0.0.1] => (item={'cmd': 'install /usr/bootstrap.sh /usr/bootstrap.sh.orig'}) changed: [127.0.0.1] => (item={'cmd': 'chmod +w /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'sed "s,bin/pkgin update,bin/pkgin -y update," /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'chmod -w /usr/bootstrap.sh'}) TASK: [Install pkgin via /usr/bootstrap.sh] ***************** changed: [127.0.0.1] => (item={'cmd': u'env PATH=/usr/sbin:${PATH} /usr/bootstrap.sh binpkg'}) TASK: [Copy over Ansible binary package] ***************** changed: [127.0.0.1] TASK: [Install Ansible dependencies] ***************** changed: [127.0.0.1] TASK: [Install Ansible package (manually)] ***************** changed: [127.0.0.1] TASK: [Setup lame /usr/bin/python symlink] ***************** changed: [127.0.0.1] PLAY RECAP ***************** 127.0.0.1 : ok=9 changed=9 unreachable=0 failed=0 We now have a EC2 instance running that has Ansible installed: % euca-describe-instances RESERVATION r-d77272ad 749335780469 ec2-webservers INSTANCE i-9fafc2f2 ami-5d0f8034 ec2-107-22-69-112.compute-1.amazonaws.com ... With this EC2 instance, we can do some basic preparations for our standards, e.g. a login without requiring root (and while there, actually disable allowing as root), setup sudo and a proper shell: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-basic.yml PLAY [security_group_ec2-webservers] ***************** TASK: [ping] ***************** ok: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install tcsh] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Add user feyrer] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Create ~feyrer/.ssh directory] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable ssh login with ssh-key] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install sudo] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable PW-less sudo-access for everyone in group 'wheel'] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Disable ssh logins as root] ***************** ok: [ec2-107-22-69-112.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-107-22-69-112.compute-1.amazonaws.com : ok=8 changed=6 unreachable=0 failed=0 Let's have a look if things actually work: % ssh 107.22.69.112 id uid=1000(feyrer) gid=100(users) groups=100(users),0(wheel) % ssh ec2-107-22-69-112.compute-1.amazonaws.com id uid=1000(feyrer) gid=100(users) groups=100(users),0(wheel) % ssh ec2-107-22-69-112.compute-1.amazonaws.com sudo id uid=0(root) gid=0(wheel) groups=0(wheel),2(kmem),3(sys),4(tty),5(operator),20(staff),31(guest) Next, install database software and import our demo database, just as we did in out local VM: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-dbserver.yml PLAY [security_group_ec2-webservers] ***************** TASK: [Install mysql] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install MySQL rc.d script] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Start MySQL service] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install python-mysqldb (for mysql_user module)] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Setup DB] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Add db-user] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Copy over DB template] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Import DB data] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-107-22-69-112.compute-1.amazonaws.com : ok=8 changed=8 unreachable=0 failed=0 Again, let's see if everything works as expected: % ssh ec2-107-22-69-112.compute-1.amazonaws.com ... ip-10-202-65-196: {1} mysql -u webapp -p webapp Enter password: ** ... mysql> show tables; +------------------+ \| Tables_in_webapp \| +------------------+ \| names \| +------------------+ 1 row in set (0.00 sec) mysql> select * from names; +----+--------+------+ \| id \| first \| last \| +----+--------+------+ \| 1 \| Donald \| Duck \| \| 2 \| Daisy \| Duck \| +----+--------+------+ 2 rows in set (0.00 sec) mysql> exit Bye ip-10-202-65-196: {2} exit logout Connection to ec2-107-22-69-112.compute-1.amazonaws.com closed. Last, add Apache+PHP and our small demo web-application: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-webserver.yml PLAY [security_group_ec2-webservers] ***************** TASK: [Installing ap24-php53 package and dependencies] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install Apache rc.d script] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable and start Apache service] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable PHP in Apache config file] ******************* changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': 'LoadModule.mod_php5.so', 'l': 'LoadModule php5_module lib/httpd/mod_php5.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': 'AddHandler.x-httpd-php', 'l': 'AddHandler application/x-httpd-php .php'}) TASK: [Make Apache read index.php] ******************* changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Add simple PHP test - see http://10.0.0.181/phptest.php] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Install phpmyadmin] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable phpmyadmin in Apache config] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Fix Apache access control for phpmyadmin] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Enable PHP modules in PHP config file] ******************* changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.zlib.so', 'l': 'extension=zlib.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.zip.so', 'l': 'extension=zip.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.mysqli.so', 'l': 'extension=mysqli.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.mysql.so', 'l': 'extension=mysql.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.mcrypt.so', 'l': 'extension=mcrypt.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.mbstring.so', 'l': 'extension=mbstring.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.json.so', 'l': 'extension=json.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.gd.so', 'l': 'extension=gd.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.gettext.so', 'l': 'extension=gettext.so'}) changed: [ec2-107-22-69-112.compute-1.amazonaws.com] => (item={'re': '^extension.bz2.so', 'l': 'extension=bz2.so'}) TASK: [Create directory for webapp] ******************* changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Deploy example webapp] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] TASK: [Create webapp symlink for easy access] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] NOTIFIED: [restart apache] ***************** changed: [ec2-107-22-69-112.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-107-22-69-112.compute-1.amazonaws.com : ok=14 changed=14 unreachable=0 failed=0 Test! % links -dump http://ec2-107-22-69-112.compute-1.amazonaws.com/ It works! % links -dump http://ec2-107-22-69-112.compute-1.amazonaws.com/phptest.php PHP Logo PHP Version 5.3.17 System NetBSD ip-10-202-65-196.ec2.internal 6.0.1 NetBSD 6.0.1 (XEN3PAE_DOMU) i386 Build Date Dec 14 2012 10:31:13 ... % links -dump http://ec2-107-22-69-112.compute-1.amazonaws.com/webapp/ Showing table hf.names: +--------------------+ \| id \| first \| last \| \|----+--------+------\| \| 1 \| Donald \| Duck \| \|----+--------+------\| \| 2 \| Daisy \| Duck \| +--------------------+ ---------------------------------------------------------------------- Enter new values: first: _____________________ last: _____________________ [ Submit ] At this point, everything is setup and can be enjoyed. If the instance is needed no longer, it can be terminated: % euca-describe-instances RESERVATION r-d77272ad 749335780469 ec2-webservers INSTANCE i-9fafc2f2 ami-5d0f8034 ec2-107-22-69-112.compute-1.amazonaws.com ... % euca-terminate-instances i-9fafc2f2 INSTANCE i-9fafc2f2 % euca-describe-instances RESERVATION r-d77272ad 749335780469 ec2-webservers INSTANCE i-9fafc2f2 ami-5d0f8034 terminated eucaHF ... What's next on my journey is to put database and webserver into separate VMs. First one of each, and then see if I find the nerve to look into a setup with more redundancy. Shameless plug:** I'll talk about the ansible and euca2ools packages at pkgsrcCon 2013 in Berlin. Join in if you're curious about what the actual playbooks used in the above examples look like! [Tags: amazon, ansible, apache, ec2, mysql, php, xen]
[20130314]	Ansible, EC2 and NetBSD milestone 2 reached: Instance preparation and communication On my quest to use Ansible to get a NetBSD virtual machine into Amazon's EC2 cloud, I've previously described how I use ansible to prepare a local machine. Working from a basic NetBSD setup, the system is setup for basic operation, the configured as both a database server and a Web/PHP server to serve a small demo application. Now the next step is to replace the VM with an Amazon EC2 instance. I have previously written about how to manage Amazon/EC2 NetBSD instances, and here are the steps that I make to first prepare an EC2 instance with NetBSD and Ansible, and then use a regular Ansible playbook to talk to all my EC2 instances. Note that the connection between the machines setup via euca2ools and ansible is in the security group names. In this case, the security group "ec2-webservers" is assumed to exist. Make sure SSH agent runs and has the EC2 SSH-key added: % ssh-add -l Could not open a connection to your authentication agent. % eval `ssh-agent` Agent pid 9304 % ssh-add -l The agent has no identities. % ssh-add .../key-ec2HF.pem Identity added: ../../euca2ools/key-ec2HF.pem (../../euca2ools/key-ec2HF.pem) % ssh-add -l 2048 d5:25:19:3d:59:40:35:32:03:f7:c5:83:de:19:b6:d0 ../../euca2ools/key-ec2HF.pem (RSA) % When using a VM to talk to EC2, pay special attention that it has the correct time, else funny things will happen: % date Sun Mar 10 14:42:33 CET 2013 Setup the ec2-webservers security (firewall) group. This is used both when creating the EC2 instances, and when accessing them. It's the link between EC2 and Ansible's ec2.py script. % euca-add-group -d 'Web servers' ec2-webservers % euca-authorize -P tcp -p 80-80 -s 0.0.0.0/0 ec2-webservers % euca-authorize -P tcp -p 22-22 -s 0.0.0.0/0 ec2-webservers % euca-authorize -P icmp -s 0.0.0.0/0 ec2-webservers % % euca-describe-groups GROUP sg-a854b3c3 749335780469 ec2-webservers Web servers PERMISSION 749335780469 ec2-webservers ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0 PERMISSION 749335780469 ec2-webservers ALLOWS icmp -1 -1 FROM CIDR 0.0.0.0/0 List out EC2 instances: % euca-describe-instances % None so far. Let's use our playbook to prepare our first EC2 instance: % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml PLAY [localhost] ******************* TASK: [Launch new EC2 instance] ***************** changed: [127.0.0.1] TASK: [Give the system 30 seconds to boot up] ***************** changed: [127.0.0.1] TASK: [Get rid of SSH "Are you sure you want to continue connecting (yes/no)?" query] ***************** changed: [127.0.0.1] TASK: [Fix /usr/bootstrap.sh to run pkgin with -y] ***************** changed: [127.0.0.1] => (item={'cmd': 'install /usr/bootstrap.sh /usr/bootstrap.sh.orig'}) changed: [127.0.0.1] => (item={'cmd': 'chmod +w /usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'sed "s,bin/pkgin update,bin/pkgin -y update," </usr/bootstrap.sh.orig >/usr/bootstrap.sh'}) changed: [127.0.0.1] => (item={'cmd': 'chmod -w /usr/bootstrap.sh'}) TASK: [Install pkgin via /usr/bootstrap.sh] ***************** changed: [127.0.0.1] => (item={'cmd': u'env PATH=/usr/sbin:${PATH} /usr/bootstrap.sh binpkg'}) TASK: [Copy over Ansible binary package] ***************** changed: [127.0.0.1] TASK: [Install Ansible dependencies] ***************** changed: [127.0.0.1] TASK: [Install Ansible package (manually)] ***************** changed: [127.0.0.1] TASK: [Setup lame /usr/bin/python symlink] ***************** changed: [127.0.0.1] PLAY RECAP ***************** 127.0.0.1 : ok=9 changed=9 unreachable=0 failed=0 % There we go. Let's list it: % euca-describe-instances RESERVATION r-bb3b6ac1 749335780469 ec2-webservers INSTANCE i-2cb9a45f ami-a754dbce ec2-54-234-59-5.compute-1.amazonaws.com \ ip-10-243-150-74.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:47:32.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 54.234.59.5 10.243.150.74 ebs % That worked - excellent! Let's add a few more, just for kicks: % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml >&/dev/null & % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml >&/dev/null & % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml >&/dev/null & % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml >&/dev/null & % ansible-playbook -i hosts-HF config-ec2-prepare1vm.yml >&/dev/null & % <...wait...> % euca-describe-instances RESERVATION r-bb3b6ac1 749335780469 ec2-webservers INSTANCE i-2cb9a45f ami-a754dbce ec2-54-234-59-5.compute-1.amazonaws.com \ ip-10-243-150-74.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:47:32.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 54.234.59.5 10.243.150.74 ebs RESERVATION r-8b3c6df1 749335780469 ec2-webservers INSTANCE i-7cb5a80f ami-a754dbce ec2-23-20-42-71.compute-1.amazonaws.com \ ip-10-203-73-195.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:50:48.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 23.20.42.71 10.203.73.195 ebs RESERVATION r-733f6e09 749335780469 ec2-webservers INSTANCE i-42b5a831 ami-a754dbce ec2-23-20-87-176.compute-1.amazonaws.com \ ip-10-116-37-145.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:50:54.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 23.20.87.176 10.116.37.145 ebs RESERVATION r-713f6e0b 749335780469 ec2-webservers INSTANCE i-40b5a833 ami-a754dbce ec2-54-242-254-237.compute-1.amazonaws.com \ ip-10-195-47-153.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:50:54.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 54.242.254.237 10.195.47.153 ebs RESERVATION r-773f6e0d 749335780469 ec2-webservers INSTANCE i-46b5a835 ami-a754dbce ec2-54-235-232-227.compute-1.amazonaws.com \ ip-10-194-7-72.ec2.internal running ec2HF 0 t1.micro \ 2013-03-10T13:50:54.000Z us-east-1a aki-825ea7eb \ monitoring-disabled 54.235.232.227 10.194.7.72 ebs RESERVATION r-b72475cd 749335780469 ec2-webservers INSTANCE i-b2adb0c1 ami-a754dbce ec2-50-16-129-62.compute-1.amazonaws.com \ domU-12-31-39-14-C6-CB.compute-1.internal running ec2HF 0 t1.micro \ 2013-03-10T13:55:24.000Z us-east-1d aki-825ea7eb \ monitoring-disabled 50.16.129.62 10.206.197.53 ebs % Let's talk to our EC2 instances now. For that, we use the ec2.py script, which enumerates all instances: % ./ec2.py --list { "i-2cb9a45f": [ "ec2-54-234-59-5.compute-1.amazonaws.com" ], "i-40b5a833": [ "ec2-54-242-254-237.compute-1.amazonaws.com" ], "i-42b5a831": [ "ec2-23-20-87-176.compute-1.amazonaws.com" ], "i-46b5a835": [ "ec2-54-235-232-227.compute-1.amazonaws.com" ], "i-7cb5a80f": [ "ec2-23-20-42-71.compute-1.amazonaws.com" ], "i-b2adb0c1": [ "ec2-50-16-129-62.compute-1.amazonaws.com" ], "key_ec2HF": [ "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2-23-20-42-71.compute-1.amazonaws.com", "ec2-23-20-87-176.compute-1.amazonaws.com", "ec2-54-242-254-237.compute-1.amazonaws.com", "ec2-54-235-232-227.compute-1.amazonaws.com", "ec2-50-16-129-62.compute-1.amazonaws.com" ], "security_group_ec2-webservers": [ "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2-23-20-42-71.compute-1.amazonaws.com", "ec2-23-20-87-176.compute-1.amazonaws.com", "ec2-54-242-254-237.compute-1.amazonaws.com", "ec2-54-235-232-227.compute-1.amazonaws.com", "ec2-50-16-129-62.compute-1.amazonaws.com" ], "type_t1_micro": [ "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2-23-20-42-71.compute-1.amazonaws.com", "ec2-23-20-87-176.compute-1.amazonaws.com", "ec2-54-242-254-237.compute-1.amazonaws.com", "ec2-54-235-232-227.compute-1.amazonaws.com", "ec2-50-16-129-62.compute-1.amazonaws.com" ], "us-east-1": [ "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2-23-20-42-71.compute-1.amazonaws.com", "ec2-23-20-87-176.compute-1.amazonaws.com", "ec2-54-242-254-237.compute-1.amazonaws.com", "ec2-54-235-232-227.compute-1.amazonaws.com", "ec2-50-16-129-62.compute-1.amazonaws.com" ], "us-east-1a": [ "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2-23-20-42-71.compute-1.amazonaws.com", "ec2-23-20-87-176.compute-1.amazonaws.com", "ec2-54-242-254-237.compute-1.amazonaws.com", "ec2-54-235-232-227.compute-1.amazonaws.com" ], "us-east-1d": [ "ec2-50-16-129-62.compute-1.amazonaws.com" ] } ec2.py can also give us information about one instance: % ./ec2.py --host ec2-54-234-59-5.compute-1.amazonaws.com { "ec2__in_monitoring_element": false, "ec2_ami_launch_index": "0", "ec2_architecture": "x86_64", "ec2_client_token": "", "ec2_dns_name": "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2_eventsSet": "", "ec2_group_name": "", "ec2_hypervisor": "xen", "ec2_id": "i-2cb9a45f", "ec2_image_id": "ami-a754dbce", "ec2_instanceState": "", "ec2_instance_type": "t1.micro", "ec2_ip_address": "54.234.59.5", "ec2_item": "", "ec2_kernel": "aki-825ea7eb", "ec2_key_name": "ec2HF", "ec2_launch_time": "2013-03-10T13:47:32.000Z", "ec2_monitored": false, "ec2_monitoring": "", "ec2_networkInterfaceSet": "", "ec2_persistent": false, "ec2_placement": "us-east-1a", "ec2_platform": "", "ec2_previous_state": "", "ec2_private_dns_name": "ip-10-243-150-74.ec2.internal", "ec2_private_ip_address": "10.243.150.74", "ec2_public_dns_name": "ec2-54-234-59-5.compute-1.amazonaws.com", "ec2_ramdisk": "", "ec2_reason": "", "ec2_region": "us-east-1", "ec2_requester_id": "", "ec2_root_device_name": "/dev/sda1", "ec2_root_device_type": "ebs", "ec2_security_group_ids": "sg-a854b3c3", "ec2_security_group_names": "ec2-webservers", "ec2_shutdown_state": "", "ec2_spot_instance_request_id": "", "ec2_state": "running", "ec2_state_code": 16, "ec2_state_reason": "", "ec2_subnet_id": "", "ec2_tenancy": "default", "ec2_virtualization_type": "paravirtual", "ec2_vpc_id": "" } Now let's use a regular playbook with the ec2.py script to get a list of all instances in the 'ec2-webservers' group and then use ansible's ping module on all of them: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-basic.yml PLAY [security_group_ec2-webservers] ***************** GATHERING FACTS ***************** ok: [ec2-50-16-129-62.compute-1.amazonaws.com] ok: [ec2-54-235-232-227.compute-1.amazonaws.com] ok: [ec2-23-20-42-71.compute-1.amazonaws.com] ok: [ec2-23-20-87-176.compute-1.amazonaws.com] ok: [ec2-54-242-254-237.compute-1.amazonaws.com] ok: [ec2-54-234-59-5.compute-1.amazonaws.com] TASK: [ping] ***************** ok: [ec2-54-235-232-227.compute-1.amazonaws.com] ok: [ec2-50-16-129-62.compute-1.amazonaws.com] ok: [ec2-23-20-87-176.compute-1.amazonaws.com] ok: [ec2-23-20-42-71.compute-1.amazonaws.com] ok: [ec2-54-234-59-5.compute-1.amazonaws.com] ok: [ec2-54-242-254-237.compute-1.amazonaws.com] PLAY RECAP ***************** ec2-23-20-42-71.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-23-20-87-176.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-50-16-129-62.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-54-234-59-5.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-54-235-232-227.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-54-242-254-237.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 % Finally, clean up and use euca-terminate-instance to delete all instances: % euca-describe-instances \| grep INSTANCE \| awk '{print $2}' \| xargs -n 1 euca-terminate-instances INSTANCE i-60829f13 INSTANCE i-2cb9a45f INSTANCE i-7cb5a80f INSTANCE i-42b5a831 INSTANCE i-40b5a833 INSTANCE i-46b5a835 INSTANCE i-b2adb0c1 % euca-describe-instances RESERVATION r-bb3b6ac1 749335780469 ec2-webservers INSTANCE i-2cb9a45f ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:47:32.000Z us-east-1a \ aki-825ea7eb monitoring-disabled ebs RESERVATION r-8b3c6df1 749335780469 ec2-webservers INSTANCE i-7cb5a80f ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:50:48.000Z us-east-1a \ aki-825ea7eb monitoring-disabled ebs RESERVATION r-733f6e09 749335780469 ec2-webservers INSTANCE i-42b5a831 ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:50:54.000Z us-east-1a \ aki-825ea7eb monitoring-disabled ebs RESERVATION r-713f6e0b 749335780469 ec2-webservers INSTANCE i-40b5a833 ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:50:54.000Z us-east-1a \ aki-825ea7eb monitoring-disabled ebs RESERVATION r-773f6e0d 749335780469 ec2-webservers INSTANCE i-46b5a835 ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:50:54.000Z us-east-1a \ aki-825ea7eb monitoring-disabled ebs RESERVATION r-b72475cd 749335780469 ec2-webservers INSTANCE i-b2adb0c1 ami-a754dbce terminated ec2HF \ 0 t1.micro 2013-03-10T13:55:24.000Z us-east-1d \ aki-825ea7eb monitoring-disabled ebs % The terminated instances will be removed by EC2 eventually, and you can start all over. With the above steps and the previous work to use Ansible to setup a NetBSD system with basic configuration as database- and webserver the next step is to put those two things together, and get a (single) NetBSD machine into the Amazon cloud that serves as both database and webserver. Let's stay tune for this to happen! Shameless plug:** I'll talk about the ansible and euca2ools packages at pkgsrcCon 2013 in Berlin. Join in if you're curious about what the actual playbooks used in the above examples look like! References: CapsUnlock blog post, CentOS Wiki. [Tags: ansible, ec2, xen]
[20130314]	NPF documentation Mindaugas Rasiukevicius has worked on NetBSD's new packet filter "npf" for quite some time now. In order to make things easier for new users, he now has also put up documentation for it. The documentation covers an introduction with a brief note on NPF's design followed by elements of a NPF configuration file. This is followed by instructions for dynamic rules, stateful filtering and network address translation (NAT). Further paragraphs describe the extension API, troubleshooting and appendixes with more information. [Tags: nat, npf]
[20130310]	Talking to the cloud After some more hacking, I have a basic understanding of how to start Amazon NetBSD EC2 instances using Ansible, fix the instances so they can be used as targets for further Ansible commands, and then actually talking to my herd of happy instances. Here's a teaser: Start EC2 instances, put them into ec2-webservers group. Repeat the following command for more than one instance: % ansible -i hosts-HF localhost -m ec2 -a 'image=ami-a754dbce instance_type=t1.micro \ key_name=eucaHF group=ec2-webservers' Prepare instances for Ansible (omitted - needs cleanup & automation) Use Ansible to ping all servers in the ec2-webservers group: % env ANSIBLE_HOSTS=./ec2.py ansible-playbook config-ec2-basic.yml PLAY [security_group_ec2-webservers] ******************* GATHERING FACTS ***************** ok: [ec2-23-23-15-202.compute-1.amazonaws.com] ok: [ec2-54-235-230-206.compute-1.amazonaws.com] TASK: [ping] ***************** ok: [ec2-23-23-15-202.compute-1.amazonaws.com] ok: [ec2-54-235-230-206.compute-1.amazonaws.com] PLAY RECAP ******************* ec2-23-23-15-202.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 ec2-54-235-230-206.compute-1.amazonaws.com : ok=2 changed=0 unreachable=0 failed=0 [Tags: ansible, ec2, xen]
[20130309]	pkgsrcCon 2013 schedule Julian Fagir has posted pkgsrcCon 2013's schedule to the pkgsrc-users list. The event is on March 23rd in Berlin Moabit. Here are a bunch of reasons to get there: pkgsrc release engineering pkgsrc on SmartOS Mancoosi tools for the analysis and quality assurance of FOSS distributions Go On NetBSD Rehabilitating pkglint DeforaOS and pkgsrc (presentation with workshop) Register now! [Tags: Events, pkgsrcCon]
[20130228]	Ansible and NetBSD milestone 1 reached: playbooks for system config, web+db servers In my quest to play with Ansible, I've reached my first milestone: I now have playbooks that take a basic NetBSD installation, configure it into a usable base installation, and then add a MySQL database, Apache and PHP to use it as webserver, and then deploy a simple web application. The playbooks are too emberassing to publish, but here are the steps to get things going: Setup NetBSD 6.0 with "base" and "etc" set, also add "pkgin" from menu Allow root logins via ssh (for a start, will be changed later) Install ansible-1.0nb1 binary package with all its depends From a management station, run: ansible-playbook -k -i hosts-HF config-netbsd-basic.yml Then, run: ansible-playbook -i hosts-HF config-netbsd-dbserver.yml Last, run: ansible-playbook -i hosts-HF config-netbsd-webserver.yml After that, a simple "phptest()" page, phpmyadmin and my simple PHP-based web application can be run. Administration of the system is via SSH and sudo, root logins were disabled in the first ansible playbook. Now to tweak the ansible playbooks to look less ugly, use variables, and then separate database and webserver into two separate machines - all in preparation to move them into the Amazon EC2 cloud. Stay tuned! For the record, here's a log of the three ansible playbooks above, starting from my basic NetBSD installation that already has pkgin and ansible: % ansible-playbook -k -i hosts-HF config-netbsd-basic.yml SSH password: PLAY [netbsd] ******************* GATHERING FACTS ***************** ok: [10.0.0.181] TASK: [Install tcsh] ***************** changed: [10.0.0.181] TASK: [Add user feyrer] ***************** changed: [10.0.0.181] TASK: [Create ~feyrer/.ssh directory] ***************** changed: [10.0.0.181] TASK: [Enable ssh login with ssh-key] ***************** changed: [10.0.0.181] TASK: [Install sudo] ***************** changed: [10.0.0.181] TASK: [Enable PW-less sudo-access for everyone in group 'wheel'] ***************** changed: [10.0.0.181] TASK: [Disable ssh logins as root] ***************** changed: [10.0.0.181] NOTIFIED: [restart sshd] ***************** changed: [10.0.0.181] PLAY RECAP ***************** 10.0.0.181 : ok=9 changed=8 unreachable=0 failed=0 % ansible-playbook -i hosts-HF config-netbsd-dbserver.yml PLAY [dbservers] ***************** GATHERING FACTS ***************** ok: [10.0.0.181] TASK: [Install mysql] ***************** changed: [10.0.0.181] TASK: [Install MySQL rc.d script] ***************** changed: [10.0.0.181] TASK: [Start MySQL service] ***************** changed: [10.0.0.181] TASK: [Install python-mysqldb (for mysql_user module)] ***************** changed: [10.0.0.181] TASK: [Setup DB] ***************** changed: [10.0.0.181] TASK: [Add db-user] ***************** changed: [10.0.0.181] TASK: [Copy over DB template] ***************** changed: [10.0.0.181] TASK: [Import DB data] ***************** changed: [10.0.0.181] PLAY RECAP ***************** 10.0.0.181 : ok=9 changed=8 unreachable=0 failed=0 %% ansible-playbook -i hosts-HF config-netbsd-webserver.yml PLAY [webservers] ***************** GATHERING FACTS ***************** ok: [10.0.0.181] TASK: [Installing ap24-php53 package and dependencies] ***************** changed: [10.0.0.181] TASK: [Install Apache rc.d script] ***************** changed: [10.0.0.181] TASK: [Enable and start Apache service] ***************** changed: [10.0.0.181] TASK: [Enable PHP in Apache config file] ******************* changed: [10.0.0.181] => (item={'re': 'LoadModule.mod_php5.so', 'l': 'LoadModule php5_module lib/httpd/mod_php5.so'}) changed: [10.0.0.181] => (item={'re': 'AddHandler.x-httpd-php', 'l': 'AddHandler application/x-httpd-php .php'}) TASK: [Make Apache read index.php] ******************* changed: [10.0.0.181] TASK: [Add simple PHP test - see http://10.0.0.181/phptest.php] ***************** changed: [10.0.0.181] TASK: [Install phpmyadmin] ***************** changed: [10.0.0.181] TASK: [Enable phpmyadmin in Apache config] ***************** changed: [10.0.0.181] TASK: [Enable PHP modules in PHP config file] ******************* changed: [10.0.0.181] => (item={'re': '^extension.zlib.so', 'l': 'extension=zlib.so'}) changed: [10.0.0.181] => (item={'re': '^extension.zip.so', 'l': 'extension=zip.so'}) changed: [10.0.0.181] => (item={'re': '^extension.mysqli.so', 'l': 'extension=mysqli.so'}) changed: [10.0.0.181] => (item={'re': '^extension.mysql.so', 'l': 'extension=mysql.so'}) changed: [10.0.0.181] => (item={'re': '^extension.mcrypt.so', 'l': 'extension=mcrypt.so'}) changed: [10.0.0.181] => (item={'re': '^extension.mbstring.so', 'l': 'extension=mbstring.so'}) changed: [10.0.0.181] => (item={'re': '^extension.json.so', 'l': 'extension=json.so'}) changed: [10.0.0.181] => (item={'re': '^extension.gd.so', 'l': 'extension=gd.so'}) changed: [10.0.0.181] => (item={'re': '^extension.gettext.so', 'l': 'extension=gettext.so'}) changed: [10.0.0.181] => (item={'re': '^extension.bz2.so', 'l': 'extension=bz2.so'}) TASK: [Fix Apache access control] ******************* changed: [10.0.0.181] TASK: [Create directory for webapp] ***************** changed: [10.0.0.181] TASK: [Deploy example webapp] ***************** changed: [10.0.0.181] TASK: [Create webapp symlink for easy access] ***************** changed: [10.0.0.181] NOTIFIED: [restart apache] ***************** changed: [10.0.0.181] PLAY RECAP ***************** 10.0.0.181 : ok=15 changed=14 unreachable=0 failed=0 % links -dump http://10.0.0.181/webapp/** Showing table hf.names: +--------------------+ \| id \| first \| last \| \|----+--------+------\| \| 1 \| Donald \| Duck \| \|----+--------+------\| \| 2 \| Daisy \| Duck \| +--------------------+ ---------------------------------------------------------------------- Enter new values: first: _____________________ last: _____________________ [ Submit ] % [Tags: amazon, ansible, apache, ec2, mysql, php]
[20130226]	ansible and euca2ools committed to pkgsrc Finally. Find them in pkgsrc/sysutils/ansible and pkgsrc/sysutils/euca2ools, and enjoy! [Tags: ansible, euca2ools, pkgsrc]
[20130216]	Playing with ansible playbooks on NetBSD ______________________ < TASK: [Install tcsh] > ---------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ \|\|----w \| \|\| \|\| changed: [10.0.0.181] [Tags: ansible, pkgin, tcsh]
[20130203]	Managing Amazon/EC2 NetBSD instances with euca2ools What and Why Playing with ansible, its "ec2" module came to my attention: it is intended to manage virtual machines in Amazon's EC2 cloud. The idea is that you describe a system with the property "needs to run in Amazon's cloud", and ansible then starts the machine if it isn't there already. In order to get to the point where this can be played with, a working version of the euca2ools package was required first. Packaging was mostly a no-brainer, and a package is currently under review and will end up in pkgsrc eventually. The more interesting part was to verify if the pkg actually worked as expected. This proved tricky for two reasons: 1) my overall lack of how to use the Amazon AWS command line tools (ec2-ami-tools, ec2-api-tools), and 2) the fact that euca2ools is mostly written for the Eucalyptus Cloud infrastructure, which just happens to be compatible with Amazon AWS. To give future parties something to google, here are the steps that to fire up a NetBSD machine in the Amazon cloud. How - Prerequirements A login for Amazon Web Services (AWS) is required, of which the Elastic Cloud Computing (EC2) Xen infrastructure is a part of. I won't go into details of this, please see the NetBSD wiki or my article ``NetBSD in der Cloud'' in the German FreeX 5/2012 magazine, pages 58-63, for details. Before starting, a few environment variables have to be filled with authentication information. Log into the Amazon AWS Console, click on your name in the upper right corner to get to the "Security Credentials" page, and create an access key if not already present. Get the acces key ID and the secret key, and put them into environment variables EC2_ACCESS_KEY and EC2_SECRET_KEY: % setenv EC2_ACCESS_KEY "AKxxxxxxxxxxxxxxxxxx" % setenv EC2_SECRET_KEY "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" Next create and download a X.509 certificate - make sure to get both the file with the private key (pk-XXXX.pem) as well as the file with the public key (cert-XXX.pem). Set the environment variables EC2_CERT and EC2_PRIVATE_KEY to thos files, respectively: % setenv EC2_CERT .../cert-XXX.pem % setenv EC2_PRIVATE_KEY .../pk-XXX.pem Last, euca2ools want to know what cloud infrastructure to use for virtual machines (EC2) and storage (S3). Coming from the Eucalyptus project, the tools can talk to cloud servers running Eucalyptus, OpenStack and Amazon AWS. Communication is via HTTP, and the environment variables EC2_URL and S3_URL have to be set accordingly: % setenv EC2_URL http://ec2.amazonaws.com % setenv S3_URL http://s3.amazonaws.com Last, make sure your system's time is somewhat in sync with reality, else you will get funny error messages! So much for the preparations, let's dive into euca2ools. List Regions and Availability Zones Amazon's service offers is spread across many data centers across different regions of the world. The list of regions is available via the "euca-describe-regions" command: % euca-describe-regions REGION eu-west-1 ec2.eu-west-1.amazonaws.com REGION sa-east-1 ec2.sa-east-1.amazonaws.com REGION us-east-1 ec2.us-east-1.amazonaws.com REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com REGION us-west-2 ec2.us-west-2.amazonaws.com REGION us-west-1 ec2.us-west-1.amazonaws.com REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com REGION ap-southeast-2 ec2.ap-southeast-2.amazonaws.com Inside one region, systems are grouped together in "availability zones" - usually data centers or separate security zones within (refer to the Amazon documentation for details). To list the availability zones in one region, use the "euca-describe-availability-zones" command: % euca-describe-availability-zones AVAILABILITYZONE us-east-1a available AVAILABILITYZONE us-east-1b available AVAILABILITYZONE us-east-1c available AVAILABILITYZONE us-east-1d available To specify what region to talk to there are two ways. The first is to specify the region name on any of the following commands with the "--region" option (yuck): % euca-describe-availability-zones --region eu-west-1 AVAILABILITYZONE eu-west-1a available AVAILABILITYZONE eu-west-1b available AVAILABILITYZONE eu-west-1c available The second one is to adjust your EC2_URL to point to your preferred region directly: % setenv EC2_URL http://ec2.eu-west-1.amazonaws.com % euca-describe-availability-zones --region eu-west-1 AVAILABILITYZONE eu-west-1a available AVAILABILITYZONE eu-west-1b available AVAILABILITYZONE eu-west-1c available Systems and Listing Available Machine Images Now that we have a basic overview of the cloud infrastructure with its regions and availability zones, the next questions are what hardware is available for running virtual machine instances on, and what operating systems can be put on. Amazon lists available hardware configurations on their "instance types" web sites. Sizes range from Micro Instances with 613MB RAM, up to two CPU cores and no local harddisk (t1.micro) to Extra Large (XL) Instances with 15GB RAM, 8 CPU cores and 1.690 GB local harddisk. Many more configurations are available for situations that require much memory, much CPU, much IO, or do cluster computing with CPU and GPU. As for the operating system and software to put on those virtual machine instances, there is a VERY wide choice available. The "euca-describe-images --all" command lists all available optione: % euca-describe-images --all ... IMAGE ami-abd0d0df 101367081206/NetBSD-i386-6.0-20121015-1054 \ 101367081206 available public i386 machine \ aki-64695810 ebs IMAGE ami-7fc3c30b 101367081206/NetBSD-x86_64-6.0-20121014-1007 \ 101367081206 available public x86_64 machine \ aki-62695816 ebs ... In the output, the configuration is identified by the Amazon Machine Identifier (AMI), e.g. "ami-7fc3c30b" for a NetBSD 6.0/amd64 instance. This image ID is required when defining what virtual machine instance to start. Note that the "euca-describe-images" command depends on the region setting, so you will get (and need) different output depending on the region that you intend your instances to run in. Setup SSH Access When starting a NetBSD AMI, access will be via SSH to the root account. For that, a SSH key pair needs to be created with the "euca-add-keypair" command. The command can write the private key to a local file, be sure to protect it properly - it will be the only way of access to the system! Other interesting commands when managing SSH keys are "euca-describe-keypairs" and "euca-delete-keypair": % euca-describe-keypairs % euca-add-keypair -f key-eucaHF.pem eucaHF KEYPAIR eucaHF b8:e9:05:7e:3a:df:c7:8e:eb:6e:8d:72:ff:77:68:01:e2:03:7e:3e % euca-describe-keypairs KEYPAIR eucaHF b8:e9:05:7e:3a:df:c7:8e:eb:6e:8d:72:ff:77:68:01:e2:03:7e:3e % euca-delete-keypair eucaHF KEYPAIR eucaHF % euca-describe-keypairs % Of course we want to keep a key for logging with it, so let's re-run the important part: % euca-add-keypair -f key-eucaHF.pem eucaHF KEYPAIR eucaHF 9b:d4:15:09:bc:51:b1:76:5c:db:a3:93:52:f0:d8:08:87:a4:80:c7 % cat key-eucaHF.pem -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAn8rCLhqLyfke+NqeOkqb6BUIbfwBFm/9ddG8ghVt9CUmyKUMRrKFSyaTRreO ... wA5a3XZuEFw83HdGrhaRgom2ZJ1SEk2889FpAA+yrhveKhDJIe6Zc2rM+crqUWBfnvs= -----END RSA PRIVATE KEY----- Manage Virtual Machine Instances Now that everything is prepared, telling the cloud infrastructure to find physical hardware, put our preferred operating system on it, and start it is done with the "euca-run-instance" command: % euca-run-instances -t t1.micro -k eucaHF ami-7fc3c30b RESERVATION r-2182506a 749335780469 default INSTANCE i-2ed60264 ami-7fc3c30b pending \ eucaHF 0 t1.micro 2013-02-03T15:51:49.000Z \ us-east-1b aki-62695816 monitoring-disabled \ ebs That's actually as complicate as it gets - one command that tells what hardware to use (t1.micro - can be omitted, a useful default will be chosen), what SSH key to use for the root account, and what machine image (AMI) to use are all used here. In return, the command prints a number of information from the freshly created instance. The one used in the following commands is the "instance id", "i-2ed60264" in this example. When the above command was started, this is a good time to go back to the Amazon AWS console and have a look at your instances - you will find the one listed above there now, too! Instead of the web-based console, the "euca-describe-instances" command can be used: % euca-describe-instances RESERVATION r-2182506a 749335780469 default INSTANCE i-2ed60264 ami-7fc3c30b ec2-54-228-22-143.compute.amazonaws.com \ ip-10-226-194-20.compute.internal running eucaHF 0 \ t1.micro 2013-02-03T15:51:49.000Z us-east-1b \ aki-62695816 monitoring-disabled \ 54.228.22.143 10.226.194.20 ebs Now this is all nice and dandy, but we have just created a NetBSD machine in the Amazon cloud. Let's log in!!!1! To do so, we need the private key file created with the "euca-add-keypair" command, and the host name. The latter is available in the list of instances - be sure to use the one within the "compute.anazonaws.com" domain: % ssh -i key-eucaHF.pem -l root ec2-54-228-22-143.compute.amazonaws.com The authenticity of host 'ec2-54-228-22-143.compute.amazonaws.com (54.228.22.143)' can't be established. ECDSA key fingerprint is f7:a9:f6:21:fc:d2:0e:46:03:41:f8:d5:c1:72:92:28. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'ec2-54-228-22-143.compute.amazonaws.com,54.228.22.143' (ECDSA) to the list of known hosts. NetBSD 6.0 (XEN3_DOMU) Welcome to NetBSD - Amazon EC2 image! This system is running a snapshot of a stable branch of the NetBSD operating system, adapted for running on the Amazon EC2 infrastructure. The environment is very similar to one provided within a typical Xen domU installation. It contains a small, autonomous environment (including a compiler toolchain) that you can run to build your own system. The file system is lightly populated so you have plenty of space to play with. Should you need a src or pkgsrc tree, please use the "bootstrap" script found under /usr to download them. You can also use the script to set up binary packages using "pkgin": /usr/bootstrap.sh [src\|pkgsrc\|binpkg] This AMI sends email to the maintainer on first boot, to help get an idea of what is in use at any given time. You are encouraged to test this image as thoroughly as possible. Should you encounter any problem, please report it back to the development team using the send-pr(1) utility (requires a working MTA). If yours is not properly set up, use the web interface at: http://www.NetBSD.org/support/send-pr.html Thank you for helping us test and improve NetBSD's quality! Terminal type is vt220. We recommend that you create a non-root account and use su(1) for root access. ip-10-226-194-20# uname -a NetBSD ip-10-226-194-20.compute.internal 6.0 NetBSD 6.0 (XEN3_DOMU) amd64 ip-10-226-194-20# exit From here, you are on your own - it's a NetBSD machine, after all. One word of warning at this point: Amazon AWS is not for free (as you should be aware from the Preparations step). If you do not need machines any more, be sure to remove them from the cluster, else this may drive up your bill for nothing! You can use the "euca-terminate-instances" command to do just that: % euca-terminate-instances i-2ed60264 INSTANCE i-2ed60264 When you look at the output of "euca-describe-instances" now, you will see that the machine's state goes from "running" first to "shuting-down" then to "terminated" - the cloud infrastructure will eventually be cleaned up to not list the stale machines any more. What's next? As stated above, the whole goal of this exercise is to manage Amazon EC2 images from ansible. Weekend's mostly over and we will see where this journey is going. For the time being, I'm happy to hear about any comments of you using NetBSD on Amazon's EC2, and of my euca2ools package. Appendix: euca2ools Cheat Sheet Environment variables: setenv EC2_ACCESS_KEY "AKxxxxxxxxxxxxxxxxxx" setenv EC2_SECRET_KEY "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" setenv EC2_CERT .../cert-XXX.pem setenv EC2_PRIVATE_KEY .../pk-XXX.pem setenv EC2_URL http://ec2.amazonaws.com setenv S3_URL http://s3.amazonaws.com Regions & availability zones: euca-describe-regions euca-describe-availability-zones euca-describe-availability-zones --region eu-west-1 Change default region: setenv EC2_URL http://ec2.eu-west-1.amazonaws.com AMIs: euca-describe-images --all SSH Keypairs: euca-add-keypair -f key-eucaHF.pem eucaHF euca-describe-keypairs euca-delete-keypair eucaHF Instances: euca-run-instances -k eucaHF ami-7fc3c30b euca-describe-instances euca-describe-instances i-96a773dc ssh -i key-eucaHF.pem ec2-54-328-43-220.compute.amazonaws.com -l root euca-terminate-instances i-96a773dc [Tags: amazon, ansible, ec2, euca2ools, eucalyptus]
[20130203]	Updating hubertf's NetBSD blog for the Social Web (AKA Facebook) Postings on the front page of this blog as well as individual articles had a "Slashdot it" link on the bottom for quite some time. With noone noticing that this wasn't working for some time, apparently. To fix the situation, and bring this blog a bit closer to the Social Web, I've removed the Slashdot-link, and added some Facebook buttons. I'm aware that this may cause some distress, and I'm curious to hear your opinion - mail me at the above address (or drop me a line on FB :-). Also: FB is the only of those apps that I really use these days - I do not currently have plans to add any others. Again, if I'm flooded with pleas, this can be changed. It may actually save me from logging e.g. into Google+ and Twitter to share things there. What do you think? [Tags: facebook, hubertf]
[20130202]	NetBSD Package Management with Ansible 1.0 and pkgin With Murphy's law striking in full force, ansible 1.0 was released only a few hours after I've packaged version 0.9 yesterday. Anyways, I have updated the package to version 1.0, and made it available for review. There are a bunch of new modules in version 1.0, and probably the most exciting one is a "pkgin" module that allows adding and deleting packages: vmnetbsd6% pkg_info -e figlet vmnetbsd6% ansible -soi work/ansible/hosts.HF all -m pkgin -a "name=figlet state=absent" 127.0.0.1 \| success >> {"changed": false, "msg": "package(s) already absent"} vmnetbsd6% pkg_info -e figlet vmnetbsd6% ansible -soi work/ansible/hosts.HF all -m pkgin -a "name=figlet state=present" 127.0.0.1 \| success >> {"changed": true, "msg": "present 1 package(s)"} vmnetbsd6% pkg_info -e figlet figlet-2.2.5 vmnetbsd6% ansible -soi work/ansible/hosts.HF all -m pkgin -a "name=figlet state=absent" 127.0.0.1 \| success >> {"changed": true, "msg": "removed 1 package(s)"} vmnetbsd6% pkg_info -e figlet vmnetbsd6% [Tags: ansible]
[20130202]	Managing NetBSD with Ansible, First Steps Background: Plan, Build, Run I've mused a lot about the transition from system planning over system setup to system operation recently. Different kinds of databases may be involved, that serve different purposes: On the planning end, topics like version management, license management, life cycle management and enterprise architecture management come into play. When this transits into operation, other aspects like system and network configuration get more important, while others fade away. At the end, someone or something has to make adjustments to systems in order to get them into service, and keep them there. This is either done with a lot of manual labor to keep documentation (CMDB, or whatever serves) and the systems in sync, or - more likely - documentation will grow stale quickly and rot away. With a growing set of hosts, the latter is not an option, and transparency becomes increasingly important. This is where system "orchestration" tools like cfengine, puppet, chef and ansible come into play. You write a system definition there, and the machine is configured according to this definition automatically: Configuration files can be adjusted, accounts created, software packages installed, services enabled, etc. System Orchestration Alternatives Looking at the state of affairs, many people seem to be fond of Puppet these days. This comes with a pretty long list of dependencies (and thus complexity to maintain), and configuration files have to be written in Ruby (which I don't know). So I chose Ansible as alternative to investigate - it comes with very little overhead (no separate daemon on neither the central machine nor any of the configured systems, just using SSH) and its own "Playbook" language seems easy enough to start with, yet complete enough to be used in large environments (as the list of customers listed on the homepage shows). Packaging Ansible, First Try My first try to get going with Ansible on NetBSD was to use the version 0.9 archive available, instead of using a GIT checkout. The included (GNU) Makefile tries to determine the version from the git checkout. Which the archive is not. Using information not available in the archive either. Checking if the source code was checked out from GIT by looking if there's a git binary available is not the best idea here, and so I dropped the ball, cursing the fact that not all the world is Linux. Sort of. Looking briefly at a fresh git checkout that this worked there, but as NetBSD's pkgsrc doesn't support git checkouts (yet, as far as I know?), I moved back to the 0.9 archive. Packaging Ansible, Second Try Leaving out the git-based version games, the next fun was to get the Python setup script to install the files in the right place. Having missed much pkgsrc development recently, and not being into Python either way, this proved unneccessary - Joerg Sonnenberg pointed me at existing pkgsrc infrastructure for Python programs, and with them the program was packaged successfully in no time. For now, the package is available on my website, I'll look into review and import into pkgsrc next. Ansible on NetBSD - What Works, And What Needs More Work Modules tested successfully on NetBSD 6: ping command copy facter group add/delete mysql_db add/delete user add/delete (without system=true) Modules tested unsuccessfully on NetBSD 6: user add/delete (with system=true) service (needs work) The git version (and the upcoming 1.0 release) also include a "pkgin" module that can be used to manage packages. Unfortunately the git-version of the module cannot easily be used with the 0.9 version, so this has to wait. Examples: Ansible On NetBSD Here's an example session with basic operation: % cat work/ansible/hosts.HF #127.0.0.1 ansible_python_interpreter=/usr/pkg/bin/python2.7 127.0.0.1 % ansible -i work/ansible/hosts.HF -s all -m ping 127.0.0.1 \| success >> { "changed": false, "ping": "pong" } Here's an example showing creation and deletion of a user account: % finger testuser finger: testuser: no such user % ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=present" 127.0.0.1 \| success >> { "changed": true, "comment": "", "createhome": true, "group": 100, "home": "/home/testuser", "name": "testuser", "shell": "/bin/sh", "state": "present", "system": false, "uid": 1005 } % finger testuser Login: testuser Name: Directory: /home/testuser Shell: /bin/sh Never logged in. No Mail. No Plan. % ansible -i work/ansible/hosts.HF -s all -m user -a "name=testuser state=absent" 127.0.0.1 \| success >> { "changed": true, "force": false, "name": "testuser", "remove": false, "state": "absent" } % finger testuser finger: testuser: no such user So much for a quick start into Ansible. Do you use NetBSD in a large-scale environment that went beyond manual systemconfiguration? Write a blog entry or an email to the NetBSD lists, and let us know! [Tags: ansible, pkgsrc]
[20130115]	Presentation about NetBSD as base for an mobile environment: Touch you NetBSD Pierre Pronchery has posted to the netbsd-advocacy@ list about the presentation "Touch you NetBSD" that he gave at the EHSM conference in December 2012 in Berlin. The conference was about hardware hacking, and the talk shows how to use NetBSD as base for a handheld environment, including hardware platforms as well as drivers for touchscreens, GPS and GSM. An excellent ressource for people interested in NetBSD as base for a mobile platform! [Tags: ehsm, presentations]
[20130114]	Update of NetBSD on the Raspberry Pi Time has passed since the last status update of NetBSD on the Raspberry Pi, and things have evolved: Recent news include drivers for USB with its many possible devices and display, allowing X to be ran - check out the screenshots provided by Jun Ebihara! There is also this posting on the port-arm mailinglist that gives details on an updates kernel image, Xorg.conf file to get X going and more news hidden in that thread. Anyone up for compiling a comprehensive NetBSD/RaspberryPi webpage, maybe on the NetBSD Wiki? [Tags: arm, dmesg, raspberrypi]
[20121227]	NetBSD End-of-2012 Fundraiser: 60.0000 USD by the end of 2012 NetBSD is an volunteer-driven open source project with no funding company behind its back. The project is governed by the NetBSD Foundation as legal entity. Your donation to the NetBSD Foundation allows the project to make major improvements to the code base. With the release of NetBSD 6.0.1, the Fund Drive targets raising 60,000 USD by the end of 2012. We would like to continue funded development in various areas, including: Improving network stack concurrency and performance. Development of modern file systems and improvement of existing ones. Features which are useful in embedded environments, such as high resolution timers and execute in place (XIP) support. Automatic testing and quality assurance. We have recently made some changes to the way we accept and honor your donations. For more information about donating, visit http://www.NetBSD.org/donations/. The NetBSD Foundation is a 501(c)(3) organization in the US, and donations may be tax deductible. [Tags: fundraising]
[20121227]	NetBSD 6.0.1 security/bugfix released The NetBSD Project is pleased to announce NetBSD 6.0.1, the first security/bugfix update of the NetBSD 6.0 release branch. It represents a selected subset of fixes deemed important for security or stability reasons. To save you from searching, here is the list of relevant changes from the release notes: Security Fixes expat: Fix CVE-2012-1147, CVE-2012-1148 and CVE-2012-0876. BIND: Address CVE-2012-5688: Named could die on specific queries with dns64 enabled. General kernel posix_spawn(): Fix processes with attributes. Resolve races between vget() and vrele() resulting in vget() returning dead vnodes. Prevent crash when unsupported fd's are used with kevent. Networking Fix "atomic fragments" for IPv6. ipf: Fix alignment issues in ipmon. (PR#47101) npf: handle delayed checksums in the network stack. (PR#47235) File systems smbfs: Make smbfs actually work on big-endian ports. Drivers ciss(4): don't try to handle sensors if there aren't any. Platforms x86 (i386, amd64): Work around a possible gcc bug generating bad assembler code. (PR#45673) Disable C1E on AMD K8 CPUs, to prevent freeze during boot. xen: Prevent a memory corruption issue that locks up a Xen DomU, and can potentially cause file system corruption. (PR#47056, PR#47057) Fix: Xen Dom0 NetBSD kernel could crash by adding duplicate xenwatches. Userland fixes Update to tzdata2012j. cdb: don't refuse to open databases without entries or keys. Address graphics corruption in recent Cairo, manifested most commonly by certain rendered text sections appearing as solid rectangular blocks of color. The complete list of changes can be found in the CHANGES-6.0.1 file in the top level directory of the NetBSD 6.0.1 release tree. Complete source and binaries for NetBSD 6.0.1 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at http://www.NetBSD.org/mirrors/. P.S.: Don't miss out on the end of NetBSD 2012 fundraise! [Tags: Release, Security]
[20121227]	Blog news: software updated Just FYI, I've upgraded the blog software to blosxom 2.1.2. No bits were harmed in the upgrade process, if you find anything that doesn't work as expected please let me know! P.S.: Any volunteers to put blosxom into pkgsrc? [Tags: blosxom, hubertf]
[20121017]	Introducing NPF in NetBSD 6.0 NetBSD's development version had npf as another packet filter available for quite some time. With the release of NetBSD 6.0, this is now available more widely, and npf author Mindaugas Rasiukevicius has pointed out that there are two PDFs available that explain more about NPF: a presentation an article [Tags: npf]
[20121017]	NetBSD 6.0 Fund Drive Hidden in the NetBSD 6.0 release notes is a call for funds: ``Your donation to the NetBSD Foundation allows the project to make major improvements to the code base. With the release of NetBSD 6.0, the 6.0 Fund Drive targets raising 60,000 USD by the end of 2012. We would like to continue funded development in various areas, including: Improving network stack concurrency and performance. Development of modern file systems and improvement of existing ones. Features which are useful in embedded environments, such as high resolution timers and execute in place (XIP) support. Automatic testing and quality assurance. We have recently made some changes to the way we accept and honor your donations. For more information about donating, visit http://www.NetBSD.org/donations/. The NetBSD Foundation is a 501(c)(3) organization in the US, and donations may be tax deductible. '' Very well seconded! [Tags: Release]
[20121017]	NetBSD 6.0 is here Quoting shamelessly from the release announcement: The NetBSD Project is pleased to announce NetBSD 6.0, the fourteenth major release of the NetBSD operating system. Changes from the previous release include scalability improvements on multi-core systems, many new and updated device drivers, Xen and MIPS port improvements, and brand new features such as a new packet filter. Some NetBSD 6.0 highlights are: support for thread-local storage (TLS), Logical Volume Manager (LVM) functionality, rewritten disk quota subsystem, new subsystems to handle flash devices and NAND controllers, an experimental CHFS file system designed for flash devices, support for Multiprotocol Label Switching (MPLS) protocol, and more. This release also introduces NPF - a new packet filter, designed with multi-core systems in mind, which can do TCP/IP traffic filtering, stateful inspection, and network address translation (NAT). In addition to many other features, NetBSD 6.0 includes significant developments in various ports. Some highlights: o SMP support for Xen domU kernels, initial suspend/resume support for Xen domU, PCI pass-through support for Xen3, and addition of the balloon driver. o Major rework of MIPS port adding support for SMP and 64-bit (O32, N32, N64 ABIs are supported) processors, DSP v2 ASE extension, various NetLogic/RMI processor models, Loongson family processors, and new SoC boards. o Improved SMP on PowerPC port and added support for Book E Freescale MPC85xx (e500 core) processors. o ARM has gained support for Cortex-A8 processors, various new SoCs, and initial support for Raspberry Pi. Full support for Raspberry Pi and major ARM improvements to come in a future NetBSD release. o time_t is now a 64-bit quantity on all NetBSD ports. This means that the NetBSD world no longer ends in 2037. Please read the release notes for a full list of changes in NetBSD 6.0: http://www.NetBSD.org/releases/formal-6/NetBSD-6.0.html The generous donations of companies and individuals to the NetBSD Foundation in previous years has enabled TNF to sponsor some exciting developments in NetBSD 6.0, including the Xen DOMU multiprocessor support. See our donations page for information about how you or your company can donate to help sponsor future projects! Complete source and binaries for NetBSD 6.0 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at: http://www.NetBSD.org/mirrors/ We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 6.0 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: ftp://ftp.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-6.0_hashes.asc NetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website: http://www.NetBSD.org/ Dedication ---------- NetBSD 6.0 is dedicated to the memory of Allen Briggs, who passed away in March of 2012. Allen's technical contributions to NetBSD were significant, and many. He was a NetBSD developer from the very beginning of the project, and was the main driving force behind the initial import of some of our hardware ports. He also served on NetBSD's core team from 2003 until 2006. More than that, however, he was a mentor to many on the project, and always willing to help when he could. Even for those he didn't mentor, his civilized example was often a guiding influence. He worked with many of us on the project, and in a field where prickly personalities are common, he was always pleasant and kind regardless of your status or technical expertise. He will be sorely missed. Acknowledgments --------------- The NetBSD Foundation would like to thank all those who have contributed code, hardware, documentation, funds, colocation for our servers, web pages and other documentation, release engineering, and other resources over the years. More information on the people who make NetBSD happen is available at: http://www.NetBSD.org/people/ We would like to especially thank the University of California at Berkeley and the GNU Project for particularly large subsets of code that we use. We would also like to thank the Internet Systems Consortium Inc., the Network Security Lab at Columbia University's Computer Science Department, and Ludd (Lulea Academic Computer Society) computer society at Lulea University of Technology for current colocation services. [Tags: Release]
[20120925]	MaheshaNetBSD Live USB I haven't seen too much progress on the NetBSD-Live-CD front recently, but who uses CDs these days, anyways? Right, a Live USB-stick image is much cooler, and MaheshaNetBSD Live USB is just that! Juraj Sipos started his project originally on FreeBSD (and still continues that), but there's a variant based on NetBSD 5.1.2 now, too. The system comes up read-write, so you can make changes to it - as Juraj writes on the webpage: ``With this USB image I can now use NetBSD the same way as if it was installed on a hard drive.'' Oh, and for those impatient to try, the root password is "netbsd5". [Tags: live-cd, mahesha, usb]
[20120716]	Announcing g4u v2.5 After an extended time for beta testing, I'm pushing out g4u V2.5 now, with no functional changes between 2.5beta1 and the final release. Of course full release testing was done on the final release. G4u 2.5 is mainly a maintenance release that brings in commands to upload and restore partition tables with the MBR, has driver updates from NetBSD, and some minor enhancements like (finally!) enabling command line history. See the g4u homepage for more details. [Tags: g4u]
[20120714]	NetBSD on the Raspberry Pi The Raspberry Pi is a pretty recent, cheap ARM-based board, or as the webpage says: ``An ARM GNU/Linux box for $25''. Shipping with today's Windows-for-embedde-boards operating system (AKA Linux), there's also a port of NetBSD on its way. Nick Hudson is at it, and he has posted first dmesg output now, showing the machine going to multiuser mode. The code's not integrated into mainline NetBSD-current yet, but rest assured that that will happen when the code is ripe. Good work, Nick! [Tags: arm, dmesg, raspberrypi]
[20120603]	SMP-ready USB stack on its way for NetBSD - testers welcome! Matt Green has picked up Jared McNeill's work on making the NetBSD USB stack SMP-ready. Besides the USB framework itself, this is also relevant for all the various drivers that can attach to USB - starting form audio drivers over SCSI to serial (ucom) drivers. While the work is far from complete, it is in a shape where users are welcome to start testing, and where developers are also welcome to help in converting more drivers! Please join in and help test the code, and send your feedback to the lists. If no serious issues come up, the code will be merged within a week. See Matt's posting to tech-kern for more details, inclusing diffs and links for amd64 and i386 GENERIC (+usbmp) kernels. Further information on the state of the code - what is and what is not converted yet - can be found in the TODO.usbmp file. [Tags: smp, usb]
[20120420]	More dmesg pr0n: NetBDS/Xen with 128 (virtual) CPUs There was discussion about raising the number of CPU(core)s supported by NetBSD the other day, as the current limit of 32 isn't the sky any more in 2012. In the process, Xen-hacker Manuel Bouyer suggested using booting NetBSD ins a Xen DomU, as you can assign up to 128 (virtual) cores to a DomU. Here's the dmesg output, and I'm sure this is a lot faster than simulating 128 CPUs in qemu. So, how to go beyone 128 CPUs for testing? Anyone played with Qemu recently, or even have some decent hardware at hand? If so, be sure to post dmesg output (and CC: me)! [Tags: dmesg, xen]
[20120420]	tetris rewrite in sed(1) tetris rewrite in sed(1), includes a bash(:-()script to start. Very impressive! [Tags: bash, sed, tetris]
[20120414]	Playing with Amazon's EC2 and NetBSD - top(1) pr0n I've played with NetBSD on Amazon's EC2 service recently, and here's a small teaser for ressources that one can get with a few mouse clicks - operating system is NetBSD 6.0_BETA/amd64, note number of CPU cores and RAM: [Tags: amazon, ec2, top]
[20120307]	NetBSD/xen available for Multi-Processor machines Manuel Bouyer announces that NetBSD/xen is now available for Multi-Processor machines. Citing from the release announcement: ``The NetBSD Foundation is pleased to announce completion of Multiprocessing Support for the port of its Open Source Operating System to the Xen hypervisor. The NetBSD Fundation started the Xen MP project 8 month ago; the goal was to add SMP support to NetBSD/Xen domU kernels. This project has officially completed, and after a few bug fixes in the pmap(9) code it is now considered stable on both i386 and amd64. NetBSD 6.0 will ship with option MULTIPROCESSOR enabled by default for Xen domU kernels. The availability of Xen MP support in NetBSD allows to run the NetBSD Open Source Operating Systems on a range of available infrastructure providers' systems. Amazon's Web Services with their Elastic Cloud Computing is a prominent examples here. Xen is a virtualization software that enables several independent operating system instances ("domains") to run concurrently on the same computer hardware. The hardware is managed by the first domain (dom0), and further guest/user domains (domU) are spawned and managed by dom0. Operating systems available for running as dom0 and domU guests include Microsoft Windows, Solaris and Linux besides NetBSD. NetBSD is a free, fast, secure, and highly portable Unix-like Open Source operating system. It is available for a wide range of platforms, from large-scale servers and powerful desktop systems to handheld and embedded devices. Its clean design and advanced features make it excellent for use in both production and research environments, and the source code is freely available under a business-friendly license. NetBSD is developed and supported by a large and vivid international community. Many applications are readily available through pkgsrc, the NetBSD Packages Collection. NetBSD has been available for the Xen hypervisor since Xen 1 and NetBSD 2.0, released in 2004 , but until now only a single processor was supported in each NetBSD/xen domain.'' [Tags: amazon, ec2, smp, xen]
[20120229]	Minix 3.2.0 released... with lots of NetBSD code added Version 3.2.0 of Minix, the operating system started by Andrew Tanenbaum, has been released. Started quite some time after BSD and before Linux, its userland grew somewhat outdated. To remedy this, Minix' userland was now updated to large extents with components from NetBSD as result of Minix' 2011 Google Summer of Code project, which was done by student Vivek Prakash and mentored by Gianluca Guida. Components merged from NetBSD include: NetBSD C library NetBSD password file format NetBSD bootloader New NetBSD userland utilities: ext2 fsck&mkfs, gzip, m4, man&tools, mkdep, mkdir, mkfifo, mktemp, rm, rmdir, tic, uniq, libcurses, libcrypt, libprop, libterminfo, libutil, bzip2, date, indent, mdocml (mandoc), sed, zoneinfo ports See the Minix release notes for more details, [Tags: minix]
[20120212]	NetBSD on the FriendlyARM Mini2440 Paul Fleischer has ported NetBSD to the FrienldyARM Mini2440 board. He writes on NetBSD's current-users mailing list: ``The FriendlyARM Mini2440 is an evaluation board based on the Samsung S3C2440 ARM SoC. It comes with a DM9000 Ethernet chip and an UDA1341 audio DAC, on-board NAND and NOR flash, a SD-card slot, and optionally a 3.5" or 7" touch display.'' See the link for dmesg-pr0n. Detailled setup instructions are available on the port-arm mailing list and Paul's homepage. Paul is also looking for feedback on the port, so if you have a Mini2440 board, give it a spin and report back to Paul! [Tags: arm, dmesg, friendlyarm, Hardware, mini2440]
[20120212]	Cobalt RestoreCD/RestoreUSB based on NetBSD 5.1.2 Right after NetBSD 5.1.2 is out, Izumi Tsutsui has updated his NetBSD-based Restore CD/USB-image for the Cobalt machines. The files are available at http://ftp.netbsd.org/pub/NetBSD/arch/cobalt/restore-cd/5.1.2/. Citing from the announcement: ``The only changes from 5.1.1 version are CHANGES file and 5.1.2 binaries, so the following instructions are same as 5.1.1 ones: restorecd-5.1.2-20120205.iso.gz is a gzipped RestoreCD ISO9660 image as prior releases. restoreusb-5.1.2-20120205.img.gz is a new "RestoreUSB" image which has almost identical functions with RestoreCD but is intended to be burned into USB memory sticks for USB bootable PCs. You can write the image using gzip(1) + dd(1) on Unix like OSes, or you can also use "Rawrite32" utility on MS Windows: http://www.NetBSD.org/~martin/rawrite32/index.html To use the RestoreUSB for cobalt installation, write the image into >=512MB USB memory stick (or USB HDD etc.) and boot your PC from it, then all other procedures are same as RestoreCD. You no longer have to burn a coaster for every installation ;-) See also "Restore CD Howto" for actual installation procedures: http://www.NetBSD.org/ports/cobalt/restorecd-howto.html: (though RestoreUSB is not mentioned yet) and see files in .tar.gz archive for more details.'' [Tags: cobalt, usb]
[20120211]	NetBSD 5.1.2 released Soren Jacobsen from NetBSD's release engineering team announced on the NetBSD blog: ``On behalf of the NetBSD developers, I am pleased to announce that NetBSD 5.1.2 is now available for download. NetBSD 5.1.2 is the second critical/security update of the NetBSD 5.1 release branch. It represents a selected subset of fixes deemed critical for security or stability reasons. All users are encouraged to upgrade. For full details, please see the 5.1.2 release notes. To download 5.1.2, see http://www.NetBSD.org/mirrors/'' Among the changes are: Fixes from ten Security Advisories: openssl, bind, kernel memory exhaustion, IPCOMP, dhclient, LZW, openpan More security fixes for 3rd party Products: libtelnet, openssl, postfix, dhcpcd, xrdb, glob(3), openssh Improvements of logging in wapbl(4) Improved handling of ECN, NFS Various minor updates and bugfixes Note that the NetBSD x.x.x releases only contain security and critical bug fixes over the last full release (5.0). The number is kept down to improve stability. More updates and changes are available on the NetBSD x.x releases and their related branches, e.g. netbsd-5. (see also) [Tags: releases]
[20120208]	FOSDEM talks/slides: pkgsrc on MirBSD, pkgin (Updated) Last weekend the Free and Open Software Developer Meeting (FOSDEM) happened. As in previous years, there was a booth manned by people from various BSD-projects, and there were also two(?) talks related to NetBSD and its related project pkgsrc: NetBSD and MirBSD developer Benny Siegert gave a talk titled "pkgsrc on MirBSD" - see his slides! pkgsrc is a framework for packaging and building 3rd party applications from source. Besides MirBSD, it runs on many other platforms like Linux and Mac OS X. While building from source is fine, it costs a lot of time. pkgsrc can also create binary packages, and to manage those, there is "pkgin", a binary package manager. Its developer, Emile 'iMil' Heitor introduced it in a talk - see the slides (PDF)! Update: Thomas 'wiz' Klausner pointed out that there was actually a whole "BSD Devroom", and there were many BSD-talks there, including: MINIX3 and BSD, by Arun Thomas The Lua Scripting Language in the NetBSD Kernel, by Marc Balmer Touch your NetBSD - towards tablet integration, by Pierre Pronchery pkgsrc on MirBSD, by Benny Siegert (slides) Introduction to pkgsrc, and to package creation in NetBSD, by Noud de Brouwer pkgin, a binary package manager for pkgsrc, also by Emile Heitor (slides) Automated package building, by Nicolas Thauvin Besides other talks that were less focused on NetBSD/pkgsrc, this event shows that there's a pretty active group of BSD advocates in Europe that manage to advocate "BSD" in its entirety pretty well. Well done, guys! [Tags: Events, fosdem, mirbsd, pkgin, pkgsrc]
[20120205]	Automatic building of Amazon EC2 images from NetBSD NetBSD/xen is available for some time now to work on Amazon's Xen-based Elastic Cloud Computing (EC2) platform, as previously announced in the NetBSD blog, and the NetBSD wiki has instructions on how to subscribe to Amazon, launching and rebuilding the "AMI" images. This work is continued by Jean-Yves Migeon, who is working on build scripts for Amazon EC2, so the "AMI" images can be provided easily, with the eventual goal to include them into the NetBSD build process by Jeff Rizzo, so EC2 images can be automatically generated easily, e.g. by NetBSD's build cluster. [Tags: amazon, ec2, xen]

Security Fixes

General kernel

Networking

File systems

Drivers

Platforms

Userland fixes