hubertf's NetBSD blog

Harddisk image cloning for Unix - g4u 2.4 released

Sun, 25 Oct 2009 23:05:00 GMT

g4u ("ghosting for unix") version 2.4 has been released. g4u is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common set up on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

Three years of time have passed since the last full release of g4u. Here's a list of what's new / changes in g4u 2.4:

Major new supported device types include bluetooth keyboards and SD/MMC cards - feedback highly appreciated!
Lots of new drivers. Too many to list, please see the g4u section of my blog at http://www.feyrer.de/NetBSD/blog.html?-tags=g4u for details.
Based on the NetBSD development version from Sep 2009
Source builds native and without root privileges on NetBSD 5.0 and crossbuilds also without root privileges from Mac OS X (tested) and probably others (untested; expected: Solaris, Linux).

The g4u 2.4 release is available on the g4u homeage at

http://www.feyrer.de/g4u/

g4u 2.4alpha4 is available for testing

Mon, 31 Aug 2009 00:37:00 GMT

Again much more time has passed than I expected, but over the past few days I've made sure that g4u compiles against the latest NetBSD-current sources, and so I'm making g4u 2.4alpha4 available for resting.

What is g4u? ``g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.''

Get it now:

What's new in g4u 2.4alpha4:

Make this build with NetBSD-current as of 2009-08-30
Trim kernel some more (NFS server, quotas)
Put only on the CD what's really needed (31MB->5MB)
Drivers for:
- Marvell Hercules-I/II SATA controller
- SiI SteelVine SATA controllers
- Attansic/Atheros L1 Gigabit Ethernet cards
- Attansic/Atheros L1E Ethernet cards & PHY
- SD/MMC cards as media - feedback highly appreciated!

Again, I'd like to hear any reports if this version works better or worse than any previous alpha version or release, esp. under the light that this version is (another...) attempt to switch to ACPI, which is on by default in NetBSD now. Also, I'd appreciate any reports if using SD/MMC cards on internal card readers work as media - I do not have any hardware to test this. Let me hear if it works for you! If things go well, I want to put this out as 2.5 before it gets old again. ;-)

g4u 2.4alpha3 is available for testing

Sat, 22 Nov 2008 17:53:00 GMT

After a lot of time, a lot of distractions, and a lot of changes to NetBSD's install system, I've finally found time again to adjust g4u to the current world order, and I'm happy to finally have g4u 2.4alpha3 available for testing.

Get it now:

What's new in g4u 2.4alpha3:

Support Bluetooth keyboard and USB hubs
ACPI support is enabled by default. SMP is, too, but that's not really used in g4u; They can both be disabled form the bootloader in case of problems.
Moved away from GENERIC kernel back to customized kernel, to avoid bloat through too many drivers that aren't needed in g4u (audio, video, ...)
The build structure was adjusted to NetBSD's latest framework, which loads the kernel from CD, then loads the ramdisk, too, before starting the kernel. No more kernel+ramdisk for now. (Need to re-evaluate situation for PXE)
Drivers for:
- AMD CS5535/CS5536 and Intel PIIX4 PCI bridges
- Areca SATA PCI RAID controllers
- Workbit NinjaATA-32 CardBus IDE controllers
- Realtek 8139C+/8169/8169S/8110S cardbus wlan cards
- Geode CSC and ScanLogic SL811HS USB controllers
- Intel PRO/Wireless 4965AGN
- JMicron JMC2[56]0 ethernet
- Atheros L2 Fast-Ethernet
- Realtek 8139C+/8169/8169S/8110S cardbus network cards
... and any driver updates, bug fixes and other enhancements from NetBSD-current
This version of g4u was again cross-compiled on Mac OS X. To get your own build, grap NetBSD-current sources from yesterday (20081122), unpack the g4u source .tgz then "cd .../src ; sh g4u-build" and report what happens.

I'd like to hear any reports if this version works better or worse than any previous alpha version or release, esp. under the light that this version is (another...) attempt to switch to ACPI, which is on by default in NetBSD now. Let me hear if it works, or if it hangs for you!

Article: The Top 50 Proprietary Programs that Drive You Crazy - and Their Open Source Alternatives

Sat, 09 Feb 2008 19:46:00 GMT

I'm happy to hear that -- according to this WHDb article -- the NetBSD-based harddisk image cloning suite g4u is one of the cures to mankinds most annoying problems (sort of :-): ``42. Norton Ghost to Partition Image: Norton Ghost isn?t a shabby backup system, as it?s a complete tool that backs up everything but the kitchen sink. If you have a complete disk failure, Norton Ghost can bring it back to life on a new hard disk (although you don?t need to make a complete backup every time). You can take this backup to external drives, CDs or DVDs. If you?re an open source advocate, however, Norton Ghost doesn?t cut the cake. You?ll want something like Partition Image (for Linux) or Ghost for Unix (G4U) for Windows or Unix users. Both tools are disk cloners that act differently, but they?re as robust as Norton Ghost. Read more at their respective Web sites before you make the jump. (For a complete rescue disk including Partition Image see SystemRescueCd).''

g4u 2.4alpha2 ready for testing

Sun, 20 Jan 2008 05:55:00 GMT

I was asked to update g4u to a more recent version of NetBSD-current and here is g4u 2.4alpha2!

Get it now:

Changes in alpha2:

ACPI Support enabled
Support Bluetooth keyboard and USB hubs
Drivers for AMD CS5535/CS5536 and Intel PIIX4 PCI bridges
Driver for the Areca SATA PCI RAID controllers
Driver for Workbit NinjaATA-32 CardBus IDE controllers
Driver for Realtek 8139C+/8169/8169S/8110S cardbus wlan cards
Driver for Geode CSC and ScanLogic SL811HS USB controllers
... and any driver updates, bug fixes and other enhancements from NetBSD-current

BTW, as a funny side note, this version was not built on NetBSD but on Mac OS X - the procedure should be the same on both systems (and maybe other Unix systems, like e.g. Linux): Get NetBSD-current sources as of today (20080120), run "cd .../src ; sh g4u-build" and report what happens.

g4u 2.4alpha1 ready for testing

Wed, 08 Aug 2007 14:34:00 GMT

Please try:

Caveats: This version doesn't have working keyboard for me in Parallels. Please tell me if that's the case for you, too!

Users of PERC- and Broadcom-equipped Dell-Servers, please test and report back success or failure. Include dmesg snipped if failure!

A small g4u status update

Mon, 16 Jul 2007 00:44:00 GMT

I've bought a new machine for desktop and development a few months ago, switching from a 800MHz PC running NetBSD to a MacMini. I haven't found much time to do any development work in the mean time, and as a result haven't transferred the corresponding files. Trying to do just that the other day, I found that the PC doesn't power up any more - either the power supply is dead, or it is just too jealous. :)

Either way, I've finally setup a working development environment for g4u again using NetBSD in Parallels and the g4u sources from the g4u 2.3 release. After the neccessary adjustments to match g4u against latest g4u, I have a working build environment again.

Things to look forward to include a 2.4alpha1 release that has latest sources (for backup reasons :-), and support for Bluetooth keyboards as well as latest drivers, esp. ones for the PERC RAID controllers and Broadcom ethernet controllers in Dell servers that people requested repeatedly, and that have finally showed up in NetBSD. Furthermore, there may be kernels with and without ACPI support. Last, I'm wondering if anyone is still using the floppy version of g4u... Anyone? Send mail!

Guest appearance of G4U in OpenSolaris Screencast

Mon, 28 May 2007 14:06:00 GMT

I thought I'd have mentioned this before but cannot find the link now <insert mumbling about efficience of tag cloud>, but apparently not. So: check out This OpenSolaris Screencast that mentions g4u in a few places for backing (and restoring) up data. Requires flash, though.

(Thanks to "skrueger" for pointing me at this!)

Updated conditions for commercial g4u licenses

Mon, 07 May 2007 12:15:00 GMT

To allow easier for companies to support g4u, I've added voluntary commercial licenses some time ago. Given the success of the concept, I'd like to allow more & smaller companies to give back to g4u, and I've thus adjusted pricing to

$70US / 50EUR for up to 5 machines

Multiple license can be purchased for more machines, I'm open for discussing volume discounts. I will of course send you an invoice for every purchase! Please contact me by email if you're interested.

Thank you for your support of g4u!

Catching up: events, articles, benchmarks, summer of code...

Fri, 09 Mar 2007 15:37:00 GMT

So I was away for a few days, being sick and then giving a talk at the Chemnitz Linuxdays and then off for a few days in Austria visiting Vienna & Zotter, and there's a backlog of stuff that happened in NetBSD's madhouse^Wwonderful world. Here's a quick run-down of things that I'm too lazy to post single items on:

Linuxdays Chemnitz: I was only there for my presentation on sunday, due to not feeling too well the days before. Still, Stefan, Jörg, Charlie and many others staffed the booth just fine, and I think every single household in and around Chemnitz has a NetBSD install and/or Live CD now. :)
Related talks to mention are Stefan Schumacher's talk on hardening systems with systrace and deleting data. My own talk was not too NetBSD specific, showing an application on how to implement dynamic DNS with some retail web/domain hoster. Slides for my talks are available as OpenOffice .ODP and as PDF. (I'll reconsider the move from TeX/prosper to OpenOffice after it was NOT as easy as I expected to find a machine running OOo for presentation purpose, after my laptop's harddisk crashed on the way to Chemnitz!)
While at roadshows: Stefan Schumacher has made DIN A4 pkgsrc flyers in english and german language.
NetBSD's puff-based FUSE implementation "refuse" is now in a state to also run the NTFS-g3 filesystem, which offers read/write support for NTFS. It's available from pkgsrc/filesystems/fuse-ntfs-3g.
Google News found me an article that NetBSD stack supports Geode NAS design: ``Wasabi Systems Inc.'s BSD-based NAS (network attached storage) software stack now supports a Geode-based reference design from AMD. Wasabi Storage Builder for NAS, combined with AMD's Geode LX NAS RDK (reference design kit), provides a secure, reliable platform for the development of NAS devices, according to Wasabi. ''
While that's all fine for Wasabi, it should be noted that whatever the company Wasabi offers is not automatically available in the freely available operating system called NetBSD. Integration efforts would have to happen first, so the headline of that article is unfortunately misleading if not to say plain wrong!
Another article that's more to the point: Julio M. Merino Vidal has worked on getting multiboot support into NetBSD, and in his article ``Making NetBSD Multiboot-Compatible'' he talks more about it.
Andrew Doran has done lots of work on NetBSD's thread and SMP implementation recently, and he has made a comparison between performance of the Scheduler-Activations-based code in NetBSD 4 and the one that will be in NetBSD 5 (AKA NetBSD-current, currently numbered as 4.99.13). See his mail to tech-kern or watch the images for 'make cleandir' on an empty source tree and the MySQL supersmack benchmark.
Google runs another Summer of Code, and this year it's not clear upfront who will be allowed as mentoring organizations. NetBSD is ready to participate again, and there's an official announcement from NetBSD about this, including pointers to our suggested/wanted list of projects and the project application HowTo. People interested in submitting a project proposal (via google!) are encouraged to use the remaining time until the deadline to discuss their proposals on the public NetBSD tech-* lists! (Personally I'll try to stay out of GSoC this year to finish some reallife work. At least that's the plan so far ...)
Three new security advisories were released:
Another article that doesn't mention NetBSD but g4u: ``How to Install a New Hard Drive: Tech Clinic'' by Joel Johnson. From the article: `` To make your new drive work like your old drive, you'll need a disk "cloner." There are a myriad of options, from commercial solutions such as the old favorite Ghost from Symantec ($70; symantec.com) and Copy Commander from VCom ($35; v-com.com) to free applications, such as MaxBlast from Maxtor, that come bundled with hard drives. If you're comfortable mucking around with Linux/BSD, I've had great luck with the free g4u application. If you have a local file server, you can even send the disk image from your laptop to an FTP site, install the larger drive, then FTP it back to your laptop, obviating the need for a drive enclosure''.

So much for now. Enjoy!

Parallels Desktop supports Leopard, Vista ... and g4u

Wed, 28 Feb 2007 11:09:00 GMT

I think I've noted about this here before, but since the new version came out and emphasizes this (again):

``Parallels today released a free update to Parallels Desktop for Mac to enhance its virtualization software with a new installation assistant tool, shared networking support, and official compatibility for Mac OS X 10.5 Leopard. The latest release of Parallels Desktop for Mac also supports Microsoft's recently-released Windows Vista operating system, adds several additional features, and repairs numerous bugs. Parallels Desktop for Mac runs 'guest' operating systems inside a Mac by allowing that operating system to run alongside Mac OS X, managing system resources as the two systems co-exist side-by-side. The latest revision ensures that Solaris guest operating systems don't hang after suspending or resuming, and includes an improved Parallels Tools package. Parallels Desktop now offers full support for OpenBSD 3.8 as a guest operating system, and ensures that the G4U hard disk cloning tool works in virtual machines.''

Someone please go and tell them that they also support NetBSD with that move, and tell the world about that! :-)

g4u 2.3 has been released

Thu, 25 Jan 2007 17:25:00 GMT

This release of the harddisk image cloning software g4u 2.3 includes updated drivers based on the latest development version of NetBSD, a complete overhaul of the build system to remove the 2.88MB size limit, and availability of contracts for technical support over the previous release. g4u ("ghost for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported. Some links for downloading:

CDROM ISOs:
Compressed -> http://www.feyrer.de/g4u/g4u-2.3.iso.zip
Uncompressed -> http://www.feyrer.de/g4u/g4u-2.3.iso
Floppies:
All of them -> http://www.feyrer.de/g4u/g4u-2.3.fs.zip
First one -> http://www.feyrer.de/g4u/g4u-2.3-1.fs
Second one -> http://www.feyrer.de/g4u/g4u-2.3-2.fs
Source -> http://www.feyrer.de/g4u/g4u-2.3.tgz

Please remember to include g4u version, dmesg output and other relevant data when sending bug reports, see http://www.feyrer.de/g4u/#bugreporting. More information is available on the g4u homepage, see http://www.feyrer.de/g4u/#history and http://www.feyrer.de/g4u/.

Book Review: IPv6 Network Programming, by Jun-ichiro itojun Hagino

Thu, 11 Jan 2007 14:14:00 GMT

Thanks to a kind donor of the g4u project, I found Jun-ichiro itojun Hagino's book "IPv6 Network Programming" in the mail today.

The book consists of two parts: the first one talks about IPv6 programming, and the second one lists a number of relevant RFCs. The Programming part includes an overview of the IPv6 APIs, puts emphasis on writing address-independent programming and gives a fair number of guidelines in that area. It then describes details for porting applications to support IPv6, with details on what to look for in client and server code, followed by a number of tips for IPv6 programming. A practical example that shows how existing client and server applications (nail and popa3d) can be changed to support IPv6.

The second part of the book consists of 270 (of ~360!) pages of printed RFCs, which is a bit over the top IMHO - if I'd wanted to read them I would know where to find them. On the other hand I have them all in one place, so I guess that can be excused.

Overall the book is rather technical and very much on the spot, written for people with programming and Unix background to learn on how to get the job done without much fluff. Minor nits are the long appendix and that images appear a bit blurry, but that shouldn't stop anyone from reading the book - go for it!

Request for input: Non-standard g4u applications

Tue, 09 Jan 2007 14:56:00 GMT

g4u was originally made to setup a cluster of PCs at a school. Since then, it has been used for several other types of hardware and application areas. I'd like to collect some of them on the g4u homepage.

Fun stuff so far includes

Copied a dual drive Tivo
Saved Novell NetWare server disks
Copied a Nokia IP330 Checkpoint Firewall 1 boxes
Install several clusters of firewalls, compute machines, school workstations, etc.

Please send me mail if you've used g4u to clone something funny, cool, unusal, geeky, etc.!

g4u 2.3beta3 released

Wed, 20 Dec 2006 02:08:00 GMT

I haven't published floppy images in the past few alpha releases, but someone asked. Also, I think I should push out another release for updated drivers, as the last release is quite some time ago. While speaking about drivers, I've also remove drivers for audio and whatnot from the kernel, which is not needed in g4u. That plus the changes in the build infrastructure should be be enough of a reason, so skipping two internal betas, here here's a 2.3beta3 before we go gold! Some links for downloading:

  * Floppies:
    All of them -> http://www.feyrer.de/g4u/g4u-2.3beta3.fs.zip
    First one -> http://www.feyrer.de/g4u/g4u-2.3beta3-1.fs
    Second one -> http://www.feyrer.de/g4u/g4u-2.3beta3-2.fs
    Third one -> http://www.feyrer.de/g4u/g4u-2.3beta3-3.fs

  * CDROMs:
    Compressed -> http://www.feyrer.de/g4u/g4u-2.3beta3.iso.zip
    Uncompressed -> http://www.feyrer.de/g4u/g4u-2.3beta3.iso

  * Source:
    http://www.feyrer.de/g4u/g4u-2.3beta3.tgz
    (There's a g4u-build script in there that should do all the work...
    I'll update the build docs for the final release)

Please remember to include g4u version, dmesg output and other relevant data when sending bug reports, see http://www.feyrer.de/g4u/#bugreporting.

More information is available on the g4u homepage, see http://www.feyrer.de/g4u/#history and http://www.feyrer.de/g4u/.

Enjoy!

Fixed broken paypal links on g4u page

Sat, 25 Nov 2006 11:39:00 GMT

I wondered why paypal returns for g4u went pretty much down recently, until someone pointed out that the links were broken - seems that Paypal changed something on their site that broke them. I've fixed that on the g4u page now, esp. in the Copying, licenses & donations section.

Now with g4u support: Parallels Desktop for Mac Pro and Leopard

Wed, 22 Nov 2006 01:39:00 GMT

The world's best harddisk imaging software (*cough* g4u *cough*) is supported in the latest release of the VMware-clone for Mac OS X, Parallels Desktop, as several news sites confirm. Oh, and of course the What's new in the Official Update of Parallels Desktop for Mac page: too: ``G4U hard disk cloning tool now works in virtual machines''.

I haven't heared of anyone cloning an Intel-Mac with g4u natively (via Bootcamp, of course) yet, but I guess there's a known-working way to this now. Anyone got some experiences about g4u on an Intel-Mac? Please post to the g4u-help list!

Pushing out g4u 2.3alpha6

Wed, 01 Nov 2006 13:17:00 GMT

After many reports on the g4u-help list about recent hardware not being supported, I've pushed out g4u-2.3alpha6 based on the few actually useful bugreports (i.e. the ones that had dmesg output) included. Maybe this helps a bit...

g4u mentioned in 10/2006 SysAdmin magazine

Sat, 28 Oct 2006 18:34:00 GMT

The 10/2006 issue of SysAdmin Magazine has an article "Multi-Platform Image Backups with Bootable, Open Source Distros" by Bill Pierce, Jon Pomeroy, and Alan Lavitt. The article mentions g4u as a prominent tool to do the job.

Citing from the article, ``The authors show how to use image backups to return a system to a known state with a few commands. [...] g4u [...] is a simple, elegant and well-documented tool that boots a NetBSD kernel''

For those that forgot, g4u ("ghost for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

Status of Google AdWords on my webpages

Fri, 27 Oct 2006 20:43:00 GMT

I'm running Google AdWords for ~2 months on two of my webpages, and given the feedback I got from you guys, maybe it's time for a little report on how things worked out so far:

Bad news first: I haven't seen any money from Google so far. This is due to their policy that they start paying only after you hit $100US one month, which I didn't do the first month (and that mostly because I enabled this in mid-september). I expect this to change in the future, though.

So what happened so far? The two websites I have Google AdWords on are the g4u homepage and my NetBSD blog. In the past ~2 months, the g4u homepage got about 42.000 hits and the blog got 9.100 hits. On the g4u page, 539 people were interested in the ads displayed, while ads on the blogs were of interest to 180 people. I admit I have no idea how Google determines payment per hit/click, but the money they show I've "earned" is $129 for the g4u homepage and $71 for the blog.

So far that's nothing that will pay my bills, but at least it will pay some food every now and then, or some stuff I could use for future NetBSD booths...

(BTW, if you want to help out in that regard directly: I'm VERY MUCH appreciating more help from the NetBSD community at conferences etc. where NetBSD booths are, after manning the Systems booth alone for three days last week; for the monetary help you can speed that up with paypal ;-)