g4u 2.6beta2 has been released - Happy 18th Birthday, g4u!
Just right in time for its 18th birthday, I have released
the 2nd beta version for g4u 2.6 (2.6beta2).
It took some time to get to this point, and I want to move
to 2.6 soon - please take your time to test and get back to me
soon, as I want to push out g4u version 2.6 in june 2017.
g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.
For more information, see
Changes in 2.6beta2 include:
- Make this build with NetBSD-current sources as of 2017-04-17,
binaries were cross-compiled from Mac OS X 10.10
- Go back to keeping the disk image inside the kernel as ramdisk,
do not load it as separate module. Less error prone, and allows
to boot the g4u (NetBSD) kernel from a single file e.g. via PXE
(Testing and documentation updates welcome!)
- Actually DO provide the g4u (NetBSD) kernel with the embedded g4u disk
image from now on, as separate file, g4u-kernel.gz
- Put all object files into one object directory. This may need more cleanup
in the future. Feedback from people building g4u welcome!
- Disable verbose device messages on Microchannel (MCA) machines
- New drivers:
- Toshiba Dynabook hotkeys
- Intel S1200,C2000 (non-pch) SMBus storage controller
- Non-Volatile Memory Express (NVMe) storage controllers and devices
- Intel Wireless WiFi Link 7xxx PCI network
- Intel 8259x 10 gigabit PCI network
- Realtek 8188CE/8192CE 802.11b/g/n PCI network
- VMware VMXNET3
- Marvell Libertas PCMCIA network
- ASIX AX88178a/AX88179 based USB network adapters
- Realtek RTS5209/RTS5229 Card Reader
- Happy 18th Birthday, g4u!
[Tags: g4u, Releases]
g4u 2.6beta1 released
I have release g4u version 2.6beta1. Important changes are an update
to recent NetBSD codebase, and moving the ramdisk from a separate
file back into the kernel. This allows easy netbooting - at least
I hope so, feedback is welcome here.
I'd like to push out version 2.6 within the next few weeks.
Please test and let me know if there are any showstoppers!
Full list of news in g4u 2.6beta1:
- Make this build with NetBSD-current sources as of 2013-10-20
- Move back from a ramdisk that's loaded from a separate file back to a
ramdisk that's part of the kernel image. This allows easier netbooting
for those people who want it - added back by popular demand :-)
- Added more kernel buffer space, to hold all kernel messages for dmesg,
even on machines with large ACPI tables (Hello VMware Fusion!)
- New drivers:
- LSILogic 9x9 and 53c1030 (Fusion-MPT) PCI SCSI
- LSI Logic Fusion-MPT II PCI SCSI
- Atheros AR9k (802.11a/g/n) PCI Wireless
- Marvell PCI Libertas Wireless
- Atheros AR9k (802.11a/g/n) PC-Card Wireless
- Broadcom BCM43xx PC-Card Wireless
- Atheros AR9002U USB Wireless
- Ralink Technology RT2500USB 802.11a/b/g USB Wireless
- Ralink Technology RT(2|30)00 802.11a/b/g/n USB Wireless
- Realtek RTL8187/RTL8187B 802.11b/g USB Wireless
- Realtek RTL8188CU/RTL8192CU 802.11b/g/n USB Wireless
- Intel Atom E6xx PCI-LPC
[Tags: g4u, Releases]
G4U Opinion Time: kernel with embedded RAMdisk vs. miniroot?
my own mail to the g4u-help mailing list: ``I've found little time to hack on
g4u in the recent past. Yet, I've
managed to setup my development and test environment for g4u
(crosscompiling NetBSD from Mac OS X, getting recent Qemu to compile), and
also got g4u built from recent NetBSD-current sources. As such, take this
as small sign of life.
Now, while I don't have any plans for large changes, I'd like to bring an
update with latest drivers and bugfixes from NetBSD.
There's one change that I'm pondering, though: g4u originally came as one
kernel-file that had an embedded RAM-disk. This was changed in the last
release to reflect NetBSD's ability to load a RAM-disk from a separate
file. This change broke the ability to netboot g4u from a single file, and
required some more effort. There were no real wins for g4u as such.
So, opinion time: keep the RAM-disk as separate file, or move it back into
Looking forward for your opinions!''
Announcing g4u v2.5
After an extended time for beta testing, I'm pushing out g4u V2.5 now, with
no functional changes between 2.5beta1 and the final release. Of course full
release testing was done on the final release. G4u 2.5 is mainly a
maintenance release that brings in commands to upload and restore partition
tables with the MBR, has driver updates from NetBSD, and some minor
enhancements like (finally!) enabling command line history.
for more details.
NetBSD vs. disk transfer speeds vs. BIOS settings
A few days ago, Brian Hoard made
an interesting finding about performance
a NetBSD/i386-based disk cloning system.
``First, my problem was I had just replaced my motherboard on my custom
Once I got Windows 7 64-bit loaded and everything working, I sat up to
clone my system drive. The drive is a 500GB Seagate Barracude, SATA 2
Cloning locally to an identical drive.
When booting into g4u, my transfer speeds were extremely slow.
Normally, my 500 Gb clones take only about 90 minutes.
But this was still working after over 6 hours.
The g4u transfer speed was reporting only 1.5 Mb/sec.
I shut things down, and went into my system BIOS. I noticed that the
SATA mode was set to "IDE Mode" for my drives.
I changed this to "AHCI Mode" and continued to boot into g4u.
This worked to fix the transfer speeds, and my clone finished normally.
Getting 83 Mb/sec.
Once the drive was finished, I attempted to boot into Windows, but it
would not boot.
I had to change my BIOS back to "IDE Mode", then Windows behaved normally.
Upon researching this, I am now learning that you should enable AHCI
Mode BEFORE installing Windows for it to work.
Apparently, if Windows is not installed while using AHCI Mode, it
disables the drivers for AHCI on the system drive. So if you later
enable AHCI in your BIOS as I did, Windows will not have the driver loaded.
I saw there is a fix on the Microsoft web site, but I haven't attempted
to try it yet.
If someone else runs into a similar problem, hopefully this will help you.''
FWIW, g4u-2.5beta1 is based on NetBSD-current from January 2012,
so checking your BIOS may help anyone seeing bad disk performance out there.
(Emphasizes in the text added by me)
[Tags: g4u, performance]
g4u 2.5beta1 supports handling of partition tables and bad disk sectors
After some absence (job-related) and technical problems
(building of NetBSD failing for me from Mac OS X),
I'm very happy to release a beta version of g4u
with some long-overdue changes. Those include being
able to backup/restore the MBR, which includes the
partition table - needed when recovering single partitions
to a new disk. Also, the various commands reading disks
are now adjusted to not abort when a disk sector cannot
be used. Instead, the bad bytes are skipped and the
rest of the disk is recovered. Please give me feedback on
this feature as I didn't have a bad disk to test this!
Other news include a command to wipe a disk by completely
overwriting it with 0-bytes (once).
Last, command line editing was enabled - finally!
Remember that this is a test release, so your feedback
is wanted - either to me in person, or to the
g4u-help mailing list. Thanks!
Here's a full list of changes:
Get g4u 2.5beta1:
- New commands "uploadmbr" and "slurpmbr" to backup and restore the master
boot record, which includes the partition table. Required to restore a
partition to an empty disk.
- New command "copymbr" to copy the MBR from one disk to another, similar
- New command "wipedisk" to write the disk full with 0-bytes once from
start (sector 0) to end (last sector)
- Enable command line history/editing by forcing /bin/sh to be built
without -DSMALL (ugly hack... there be lots of dragons!)
- When setting up a fresh compile tree, g4u patches are now applied
automagically without aborting the build
- Error detection was now enabled in the dd(1) command, which is the core
of g4u (surprise!). With that, disks with broken/unreadable sectors
should now be copied, skipping the unreadable sectors and copying the
rest. This affects a number of programs: copydisk, copypart, uploaddisk,
uploadpart. BEWARE: I wasn't able to actually test this as I do not have
a disk with bad sectors here. Please report back your experiences!!!
- Make this build with NetBSD-current sources as of 2012-01-12
- New drivers added to the kernel:
- RDC PMX-1000 IDE controllers
- Intel SCH IDE controllers
- TOSHIBA PICCOLO controllers
- Attansic/Atheros L1C/L2C Ethernet
- Broadcom BCM43xx wireless
- Agere/LSI ET1310/ET1301 Gigabit Ethernet
- RDC R6040 10/100 Ethernet
- USB LCDs and USB-VGA adaptors, e.g.:
- DisplayLink DL-1x0/1x5
- Option N.V. Wireless USB WAN modems
- Microsoft RNDIS specifications USB ethernet
- Atheros AR9001U USB Wifi
- Intersil PrismGT USB Wifi
- Virtio PCI, memory balloon, disk & network devices
- ... and many more that slipped past QA
- ... and any driver updates, optimizations and bug fixes and other
enhancements from NetBSD-current
Netbooting g4u via PXE
Doing a network based boot with PXE is not exactly hard,
but you need some debugging and the right tools in place.
If you want to netboot g4u, the NetBSD-based tool for harddisk
image cloning via FTP, via PXE, there's a description
on how to do
Netbooting of g4u via PXE
by Mariusz Zynel.
Details include setting up a TFTP server for loading
the bootloader and getting DHCP sending out the right files.
[Tags: g4u, pxe]
Enlarging a (virtual) disk
I've tried to build NetBSD-current at various points in the past few
months, and always hit one of two bugs: -current blows up with
a gcc Internal Compiler Error when crossbuilding on Mac OS X,
and kernel panics with native NetBSD builds with sources on NFS.
This stinks, and I've successfully managed to do a successful
-current build with sources on (local) disk. With NetBSD
running within VMware Fusion on Mac OS X.
To go on from there, I found that my NetBSD VM's only disk
was too small to do anything useful. Options for enlarging
that came to mind:
Option #3 was it, and after removing all VMware snapshots, enlarging the
disk was easy with VMware Fusion, going from 10GB to 20GB.
After growing the disk itself, the next question was how to use the
newly gained disk. Of course some file system needs to use it,
and in theory there are the following options:
- NFS - see 'panic' above, no go.
- Adding another (virtual) disk - easily doable, but I felt like not adding one
- Extending the existing disk - adventure time!
The disk was resized from 10GB to 20GB. The partition table
(disklabel) was created by a standard NetBSD install, and first
had the root file system, followed by the swap partition.
From that, adding 10GB more swap was not useful,
so I've decided to change the disklabel to add the new disk space
as a new partition behind the existing partitions.
This is also an excuse to not frob with
growfs and resize_ffs. (And of course I'm ignoring the option of backing up the full file system, doing a full rebuild of the filesystem
and then doing a restore :-)
- Enlarge the last file system on disk
- Fix the partition table to add another partition for the new space
For those in a similar situation, here are the steps to use
the newly gained space on an enlarged (virtual) disk:
Now let's see if I get things far enough to get a build
of g4u going... wish me luck!
- Prepare: save the old output of "dmesg" (/var/run/dmesg.boot is OK)
- Enlarge - VMware Fusion wants a shutdown for that, you cannot suspend
- After booting, run a diff on the saved "dmesg" output, to learn
what the old and new size of the disk is, in sectors. My diff looks
like this, note the size change in sectors:
-wd0: 10240 MB, 22192 cyl, 15 head, 63 sec, 512 bytes/sect x 20971520 sectors
+wd0: 20480 MB, 44384 cyl, 15 head, 63 sec, 512 bytes/sect x 41943040 sectors
- Backup the existing/old disklabel, just in case: disklabel wd0 >disklabel.BAK
- Edit the disklabel: disklabel -e wd0
- In the editor, adjust the disk size in sectors from 20971520 to 41943040:
total sectors: 41943040
- Partition 'd' is the full disk on i386/amd64, it starts at sector 0
and is 41943040 sectors big
# size offset fstype [fsize bsize cpg/sgs]
d: 41943040 0 unused 0 0 # (Cyl. 0 - 44384*)
- Partition 'c' is the NetBSD part of the disk. As this VM only has NetBSD, all the usable space is used. Note that "usable" space excludes the first 63 sectors of the disk (mbr etc.), i.e. it is 41943040 - 63 = 41942977 sectors:
# size offset fstype [fsize bsize cpg/sgs]
c: 41942977 63 unused 0 0 # (Cyl. 0*- 44384*)
- After this everything is in sync with the new disk again, and the remaining/new space can be used for new partition 'e'. As the new space starts where
the disk used to end, its offset is the old size, 20971520 sectors.
The size of the new partition expands from the offset sector 20971520
to the end of the disk at sector 41943040, i.e. the partition size is:
% expr 41943040 - 20971520
In total, this gives for the new partition:
# size offset fstype [fsize bsize cpg/sgs]
e: 20971520 20971520 4.2BSD 2048 16384 0 # (Cyl. 22192*- 44384*)
- Last, create file system, mount and populate:
# newfs /dev/rwd0e
# mkdir /disk2
# echo '/dev/wd0e /disk2 ffs rw,log 2 2' >>/etc/fstab
# mount /disk2
# cd /usr ; pax -rw -pe -v stuff /disk2
# rm -fr stuff ; ln -s /disk2/stuff .
P.S.: I'm offering choccolate to anyone fixing crossbuilding of
NetBSD-current from Mac OS X. Any takers?
[Tags: disklabel, fusion, g4u, vmware]
Booting g4u from USB
Currently, there is no bootable image available that can be put on an USB stick.
a link to Jared McNeill's blog
which describes how to transmogrify an ISO image into a USB/disk image.
Harddisk image cloning for Unix - g4u 2.4 released
g4u ("ghosting for unix") version 2.4 has been released. g4u is a
NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to
deploy a common set up on a number of PCs using FTP. The floppy/CD offers
two functions. The first is to upload the compressed image of a local
harddisk to a FTP server, the other is to restore that image via FTP,
uncompress it and write it back to disk. Network configuration is fetched
via DHCP. As the harddisk is processed as an image, any filesystem and
operating system can be deployed using g4u. Easy cloning of local disks
as well as partitions is also supported.
Three years of time have passed since the last full release of g4u.
Here's a list of what's new / changes in g4u 2.4:
The g4u 2.4 release is available on the g4u homeage at
- Major new supported device types include bluetooth keyboards and SD/MMC
cards - feedback highly appreciated!
- Lots of new drivers. Too many to list, please see the g4u section
of my blog at http://www.feyrer.de/NetBSD/blog.html?-tags=g4u
- Based on the NetBSD development version from Sep 2009
- Source builds native and without root privileges on NetBSD 5.0 and
crossbuilds also without root privileges from Mac OS X (tested) and
probably others (untested; expected: Solaris, Linux).
Grab the RSS-feed,
or go back to my regular NetBSD page
Disclaimer: All opinion expressed here is purely my own.
No responsibility is taken for anything.