Friday, January 15, 2010

EON ZFS Storage min and nginx web server

In a previous post, lighttpd was used as a add in web server for the EON ZFS min storage (NO HTTPD) users. Another web server option for the ZFS storage (NO HTTPD) image is nginx. This post will show how to easily add HTTPD service using the nginx web server. 

Prerequisite(s): A storage pool must be created and a binary kit must be installed. The binary kit is needed because it contains libpcre, a dynamic library needed by nginx.

Let's get started adding a nginx web server, to our ZFS min (NO HTTPD) storage image. First, get the pre-built nginx version 0.8.32 here. I have also included compilation notes under the nginx section in case you wish to compile and add other features on your own. Transfer the nginx-0,8.32,tgz file to the storage pool via CIFS share, sftp or USB. Here, I start with the file located in /tmp and a storage pool named abyss. Substitute your zpool name for abyss.
cd /abyss
gzip -dc /tmp/nginx-0.8.32.tgz | tar -xf -
Now let's add the automation entries to /mnt/eon0/.exec. Also, execute the commands to create the necessary symlinks. These entries align paths entered in nginx.conf
(cd /usr ; ln -s ../abyss/nginx .)
At this stage, the nginx web server is ready to run, but feel free to customize the /abyss/nginx/conf/nginx.conf and generate your own /abyss/nginx/conf/cert.pem, /abyss/nginx/conf/cert.key file. OpenSSL (part of the binary kit which is a pre-requisite for this post) would be needed to generate your own self signed cert file (cert.pem, cert.key). To start and automate the web server, run and add the following to /mnt/eon0/.exec.
/usr/nginx/sbin/nginx -c /usr/nginx/conf/nginx.conf

4 comments:

Peter J. Lu said...

Thanks so much for doing all of this. I finally got everything working: zfs send|receive (though I have to figure out the right options for incremental), and adding NFS, the latest napp-it and the binary kit to the USB image. An amazing piece of work you have done---once you figure out what's going on!

I also did a bit more digging in terms of performance. With the zfs send/receive over SSH, I get around 15-20 MB/sec on average, with some peak transfers of up to 50 MB/sec. This isn't great, but both source and destination were only a single mirrored pair of 2 TB Hitachi SATA 7200 RPM drives. I looked at the output from "iostat -D", and the disk utilization was around 80+%, so it doesn't look like any settings will improve performance substantially. Would you agree that I basically need to add more disks in order to get performance up?

Now on to the http server. I just want to be able to put up files for other people to download. Nothing fancy (I have a real hosting company to take care of my web page), just the directory with the files. What's the easiest way to get this done? Anyone going to the IP address would then just get the directory listing, so I don't need to execute any programs, just make the files available, preferably in a separate zfs filesystem. WHat do you suggest?

Andre Lue said...

Peter,

Good to hear you're making progress. Patience and time will get it refined and performing to peak.

For the napp-it and the web listing you could symlink them to your zpool via /mnt/eon0/.exec. You may have to allow directory indexing via the respective web server configs(apache2, light, nginx).

I think you can squeeze more out of the disks and NFS. Run diskspeed.sh as root it will give you an idea what each disk peak numbers are and help to tell if you're at peak numbers.

There are also tuning for NFS which I will include in the next release for the following /etc/system entries:
nfs:clnt_max_conns=8
nfs:nfs3_nra=16|32
nfs:nfs3_max_threads=32|64

Peter J. Lu said...

Thanks, as always, for your suggestions.

I had a funny experience today. Someone at work was having trouble mounting a drive in Ubuntu, and I realized that I know how to try to figure it out in OpenSolaris (start with 'format', then try and diagnose the problem) but don't really have any special clue in Linux. So I guess that's the first time it's clear that I tend to think from the Osol perspective now...

I am inclined toward staying with Apache because it's already installed; is there a reason to choose the others ones?

Also, will you be able to move EON to OpenSolaris from SXCE, so we can continue to benefit so enormously from your efforts?

Finally, if you could put nano in with the next binary kit, it would make life easier for a lot of us who aren't as familiar with vi and it's way of dealing with the world!

Andre Lue said...

Peter,

Reasons for choices in web servers:
-Some felt Apache2 was more webserver than was needed (big memory footprint).
-Some didn't want a web server in the core storage image.
-The other server options are lighter in memory footprint and offer advantage(s) in some areas over apache2.

I will have one last super release on SXCE 0.60 based on snv 130 as soon as I fix this Active Directory bug. I am also currently researching opensolaris/IPS for the future path and updates.

I will give nano a look when I get some BW as others have asked about it.