Wednesday, May 5, 2010

EON on a ZFS mirrored boot/root?

A typical operating system requires a boot disk and certain storage requirements to reside, boot and do what it's designed to do. The OS or boot disk where the operating system resides in a conventional install becomes a possible point of failure. The OS disk failing, would cause an outage and cut-off access to your data and storage appliance. RAID 1 or mirroring of the OS disk is often used to address/counter this type of failure. ZFS has the features to make an OS disk redundant by mirror-ing (RAID 1), providing an added layer of insurance to your storage appliance's uptime.

A design decision with EON, was to make it run from a RAM disk, hence it would not have an associated failure attached to the media(USB, CF or hard disk) it boots from or resides on. Size becomes a disadvantage running from a RAM disk but the trade off is an add layer of insurance, resilience and uptime. EON uses the boot/residence media mostly for out of band task that are not tied to performance and are not frequently used, so the typical 5+ MB/s performance of a USB/CF media is not a problem. After EON is booted, the USB media could be pulled (simulating a boot disk failure) and only the services below would be impacted. The important thing to note is that EON would continue to run, along with all the services enabled. The show goes on and facilitates dynamic repair or correction of the disk failure situation. EON uses the USB/CF media to preserve:
  • edits to the configuration files
  • edits to the grub menu entries or updating the image
  • updates to the zpool.cache
  • user optional binaries on the USB/CF media 
A ZFS boot/root was recently requested as a necessary feature/requirement for EON. I explained the resilience already built into the design and the challenges. I however gave it try the other night and succeeded in booting EON on a ZFS boot/root. This would allow the use of redundant USB/CF media. When will it be seen in a release? I'm not sure. It required using steps that were hard coded and needs more testing and refinement before it's ready for general audience and use. So yes, a ZFS boot/root for EON is possible. Here's a ZFS boot/root birth cry screenshot.


Javier said...

A simple question... i have eon+napp-it installed... but each time the system is down... I have to import -f all pools. ¿How can i make the pools "fixed" between reboots???

Andre Lue said...

Hi Javier,

-You have to run setup, this configures your hostid.
-After setup is complete, run update image,
-Then edit /mnt/eon0/.exec entry POOL=abyss, change "abyss" to your pool name.
Now you would create your zpool under this hostid but if it is already created you will have to do zpool import -f poolname one more time to match it with the current hostid.

vertex_vr4 said...

Hi Andre,

Once again - nice work.

I'm not sure if I'm reading it right but what 'services below' do you mean?

Also does this mean that you can boot from either usb device or are you making a RAIDz device of ROOT?


Andre Lue said...

Hi vertex_vr4,

I was referring to the services that need the /mnt/eon0 mount. This mount would be unavailable, so the following would fail:
- edits to the configuration files in /mnt/eon0/
- edits to the grub menu entries or updating the image (
- updates to the zpool.cache
- user optional binaries on the USB/CF media mounted in /mnt/eon0

NOTE: None of the other services that are facilitate storage rely on this mount so they would all still be online and continue to function.

This means,
- You could have 2 redundant USB/CF drives and could boot EON from either one.
- It could be mirrored (RAID 1) via ZFS.
- If one boot media failed a new USB drive could be inserted and restored/synced without halting the storage or interrupting service.

Andre said...

Hi Andre

As you have built the system to be fully redundant would it not be easier to simply implement a 'primary' and 'secondary' boot drive, and on every successfull boot the config from the primary is copied to the secondary? We use a number of enterprise NAS units that do exactly this - it also allows for firmware upgrades to happen on the primary card while leaving the secondary set with the older code release in case of problems,

Andre Lue said...

Hi Andre,

Easier would spark much debate but I would agree with you.

Regarding, updating and testing a new version and rolling back if you do not like it. That is currently available using transporter, if a backup is made before upgrading. Also, you would make sure not to do a zfs upgrade to get the new features until you are sure you like the new version, as there is no way to undo a zfs upgrade without rebuilding the zpool (as far as I know).

Dan said...

I've gotten EON all installed and love it :) Just a problem with iscsi...

eon:63:~#zfs create -V 5G tank/iscsitgt
eon:64:~#zfs set shareiscsi=on tank/iscsitgt
cannot share 'tank/iscsitgt': iscsitgtd failed request to share

Am I missing something? Is ISCSI sharing working in this build?

Andre Lue said...

Hi Dan,

The steps you used is for the older iscsitgtd and does not work with the COMSTAR iscsitgtd which replaced the older iscsitgtd in EON.

Please see the steps to a COMSTAR iscsitgt howto
Howto 1
Howto 2

The service starting steps will not be needed.