Prelude and cleaning drive

I recommend downloading the latest service pack for your version of Windows on another system while you do the rest of this – then you’ll be able to install it quickly once you’re done. Windows 7 SP1 is downloadable as an ISO.

Similarly, if you have another Ubuntu machine, you may wish to update it to the most recent packages during your install, then copy the packages over later.

In this tutorial I’m describing how to build a fresh installation, so first back up any important data on the drive. If your existing drive had any sensitive data on it, use a third-party tool like CMRR’s Secure Erase to do a secure wipe of it. Even if it doesn’t contain sensitive information, if it is an SSD, a secure wipe may be needed if your Windows install freezes unexpectedly.

Install Windows

In the Windows installer choose the option to manually partition the disk. Delete any existing partitions and then create a new one, leaving enough space for your desired Ubuntu partitions. Windows will automatically create a small system partition. Proceed with the rest of the install normally.

Install Ubuntu

The easiest way to get full disk encryption with Ubuntu is using the alternate installer CD, which includes partition encryption options. Follow this guide (“Installing Ubuntu 8.04 with full disk encryption”, yungchin, Learning in Linux) to set it up with encryption for your root partition and swap (it still works in Ubuntu 11.10). This method of encryption covers everything but the boot partition and supports hibernation and resume from an encrypted swap partition, unlike some other methods. When asked to install the bootloader, install it to the MBR.

Test booting into Ubuntu. You should be asked for your passphrase during boot. In my case, I had to boot in recovery mode first to install the restricted NVIDIA drivers, and then could reboot and boot in normal mode. You can also test hibernation and resume under Linux now.

Install Grub into boot partition

This is the trickiest bit – and I’m not even totally sure it’s necessary (because TrueCrypt may chain the old MBR when it’s installed, I’m just not sure).

Boot into Ubuntu. Find your boot partition device with “df”. Install grub into it with: sudo grub-setup –force (your boot partition device, e.g. /dev/sda3) (note: make sure to use grub-setup, not grub-install)

If you did not install grub into the MBR during setup, you’ll get an error saying that /boot/grub/core.img does not exist. You can create it with this command: grub-install –grub-setup=/bin/true /dev/sda (or whatever your boot device is). This will not actually write to the MBR, but will create core.img. Then try grub-setup as above.

TrueCrypt

Reboot into Windows, download the latest version of TrueCrypt, install it, and then open TrueCrypt and do System->Encrypt System Partition/Drive. This will encrypt your entire system partition including your swap and hibernation files, your users directory, and so on. Select to encrypt only the system partition. It will ask if you are multibooting (say yes) and if you have another bootloader loaded into the MBR (lie and say no). You do not need to erase the drive, since it has no private data on it yet.

It will then create a recovery disc and do a test reboot. During the test reboot, make sure you can still boot into Ubuntu normally by pressing ESC, selecting Linux from the grub menu, and entering your passphrase. Reboot, enter your TrueCrypt passphrase, and if the grub menu appears, select your Windows OS. TrueCrypt should verify that the test boot succeeded, and give you the option to being encrypting the drive. Let the drive encryption complete before proceeding (since any disk activity will slow it down). If you downloaded SP1, you may install Virtual CloneDrive while you’re waiting.

You now have Windows and Linux with full disk encryption on a single drive!

Install updates on Windows and Linux

On Windows, use Virtual CloneDrive to mount the Service Pack downloaded earlier and install it. You will have to temporarily set the boot volume created by Windows, a small volume of size 100 MB, to be the active partition in Disk Management (under Control Panel->Administrative tools->Computer Management) in order to install SP1.

Then retrieve any additional updates from Windows Update.

On Linux, install Synaptic and use its Settings->Repositories->Download from->Other->Select Best Server to choose a good package server. Then use the update manager or “sudo apt-get update && sudo apt-get upgrade” to upgrade to the latest versions of packages. You can also do “sudo apt-get dist-upgrade” to upgrade your kernel.

Is not encrypting the boot partition okay?

Most of us don’t keep any kind of secrets in the boot partition, so the only reason to encrypt it would be to make tampering more difficult. However, this isn’t really what encryption is designed for – encryption is not authentication. Moreover, while someone could capture your passphrase by tampering with your boot files, if they had physical access anyway, they could just replace the MBR and capture it anyway. It’s still essential to ensure that you don’t leave your device unattended in a place others can access it.

Cold boot attacks

The primary reason that I wanted so badly to get hibernation working properly in both OSs with a fully encrypted hibernation file is that hibernation and shutting down are essentially the only viable defenses against cold-boot attacks, an attack on disk encryption in which a person with physical access to your machine extracts and reads the contents of your memory modules before the information has faded, including your encryption keys. You should always hibernate your machine if you are leaving it unattended, especially for a long period of time or in an easily-accessible area.

1. Create a new volume of the desired size.
2. Launch another instance (ideally the same distro/version as the server you’re expanding, but doesn’t really matter). This server will be used to migrate the data from the old disk to the new disk.
3. Attach the new volume to the migration server as /dev/sdf and boot it up. Connect to it via SSH. Format the new volume (e.g. with “mkext4 /dev/sdf”), and mount it with “mkdir /mnt/new; mount /dev/sdf /mnt/new”.
4. Stop the instance that you want to expand the root volume for. Detach its volume. To be safe, take a snapshot at this point for backup purposes. Attach it to the migration server as /dev/sdg, and then mount it with “mkdir /mnt/old; mount /dev/sdg /mnt/old”.
5. Do: “cd /mnt/old; cp -a . /mnt/new”. The “-a” switch preserves all file attributes including owner, mode, etc. You can monitor progress easily with “df -h”.
6. Dismount the new volume. Run this command to mark the new volume as a valid root volume: e2label /dev/sdf uec-rootfs
7. Stop the migration server. Detach the new volume from the migration server, and attach it to the original server as “/dev/sda1″ (it suggests /dev/sdf, etc. but you can type “/dev/sda1″ in directly).
8. Start up the original server. Make sure to reassociate the public IP if necessary.

And now I have 60% space free! Let me know if you have trouble with this or need extra steps to get it working in your environment.

Plugin installation:

1. Open Eclipse. Make sure it is either installed in your home directory (user directory on Windows) or that you are running as root/administrator.
2. I read claims that EGit comes with version 3.7.0, but it didn’t come with mine. To see if yours has it, go to Help->About and choose “Installation Details”. If you see “Eclipse EGit”, it is already installed.
3. Go to Help->Install new software. Enter the EGit plugin update URL http://download.eclipse.org/egit/updates and hit “Add…” Enter EGit for the name. Click OK.
4. Open up the items provided and check “Eclipse EGit”, “EGit Project Set Support”, and “Eclipse JGit”. Don’t check the remaining items.
5. Click Next. Continue clicking Next until the install is complete, then Finish.
6. Close Eclipse and re-open it as your normal user.

Setting up keys (must be done only once per user)

1. Go to Window->Preferences. Select General->Network connections->SSH2.
2. If you have already generated a key pair for use with Github, skip to step 4. Otherwise, go to the Key management tab and select Generate RSA Key. Then click Save Private Key to save the private key file, which you will use in step 3. Copy the public key. If you wish to use an existing key pair you generated in the past, instead click Load Existing Key, then copy the public key.
3. Go to Github, log in, click your username in the upper-right, click the “Edit your profile” button, click SSH keys, click Add SSH key, and paste in the public key from step 2.
4. Go back to the Eclipse SSH2 preferences. Select the General tab. Click Browse and select the directory where your private key is stored. Enter its filename in the “Private keys” field.

Putting a new/existing project into a new GitHub repository

1. Go to Github. Click your username in the upper-right. Click New repository.
2. Fill out the fields and click “Create repository”. You will be given an information screen. You’ll need to refer back to this screen.
3. Go to Eclipse. Create/open the project.
4. Right-click the project in the Package Explorer pane and select Team->Share Project. Select Git. Click Next.
5. Click “Create”. Enter a name (the same name as your GitHub repository is fine). Click OK. Click Finish. Your project should now say next to it “NO-HEAD”.
6. Right-click your project, choose Team->Commit. The name should be the name you wish your check-ins to be attributed to, and the e-mail should match the one given on Github’s information page from step 2 (after user.email).
7. Check all the files you want to commit – you should usually include at least any source files, “.classpath”, and “.project”. Enter a commit message. Click OK.
8. Right-click your project, choose Team->Remote->Push. On Github’s information page from step 2, find the line that looks like “git remote add origin git@github.com:username/Blah.git” and copy the part after “origin”. Put it in the “URI” field.
9. For Protocol select SSH. Change nothing else. Click Next. If you are asked to accept Github’s host key, click Yes.
10. For Source ref select from the dropdown “master [branch]“. Destination ref will fill in with the same. Click “Add all branches spec”. Click Finish.
11. Refresh the information screen on Github to verify your check-in completed successfully.

Importing an existing GitHub repository

1. Choose File->Import. Select Git->Projects from Git. Click Next.
2. Choose URI. Click Next.
3. On the GitHub project’s page, click “SSH” (selected by default), then copy the full URI (starting with git@…). Make sure it says “Read+Write access” to the right of it.
4. Enter the URI in Eclipse. Select SSH for protocol. Click Next.
5. Select the branches you want (usually only one, master, will be offered and will be automatically selected). Click Next.
6. Select where the local files and git metadata will be stored. The default should be fine. Click Next.
7. If it is an Eclipse project previously added using EGit, choose Import existing project. Otherwise choose one of the other options.
8. Click Next. Click Finish.

Once you’ve gotten your project set up, try right-clicking on source files and messing around with the options under “Team” to see what they do. “Show in history” is particularly useful, opening the history view which allows you to explore previous versions of files. If you do Team->Commit, it should automatically select previously committed files that have changes, but new files have to be checked to add them.

You can use the “Compare with” menu to view changes to your file over time; “Compare with HEAD revision” shows changes since your last commit. Compare views dynamically update as you make modifications.

There is one enormous problem with this: if your Flickr is set up to allow downloads of original images, those original images are unprocessed/unmodified and contain any EXIF data that was in the original file. If your camera automatically inserts location information using a GPS, or you added it using a third-party tool, most likely it’s still available in there. Flickr, to their credit, has a giant scary warning about this:

Please note: If you upload a photo with geo data, that info will be embedded in the EXIF data of the original file. If you don’t want people to have access to this information, you should restrict who can download your originals.

This is a terrible solution however, since I want people to have the full-size versions of my images. Is there another way?

Turns out yes, but it requires some manual steps and is a bit taxing:

1. Visit http://www.flickr.com/map/ and visit the private location. Zoom until only the area you want to remove location information from is in view (or as close as possible).
2. Use “Search the map” and select “Your photostream” to show only your photos.
3. There is a strip with two arrows listing every photo of yours in the map area. Click each one and, if it is close enough to your center point, open it in a new tab. (Don’t use the pink bubbles on the map itself. The pink bubbles may not be displayed for every photo or may be hard to click on.) Download the original size versions of each of them. Put them all in a single directory.
4. Strip the geographical EXIF data. To do this: Install exiftool. Make sure it’s in your path. Open a command prompt, cd to the directory where you downloaded the files, and type: exiftool -a -gps:all= *
   Note that this will destroy the EXIF data forever, so make a private copy of the images first if you want to preserve the original versions with full EXIF data. Flickr location data, although based on your EXIF data, is stored separately and will still be displayed to you and people permitted to see inside your geofence.
5. Use “Actions->Replace this photo” to re-upload the modified images over the old ones. When replacing, you can easily find the matching image by copying the last bit of the URL after the final “/”, which will be the prefix of the filename.
6. You have to do this again if you upload new photos in this area, preferably before your initial upload for maximum security – if you do it afterwards, someone quick enough may still grab the data.

Turns out I do have .htaccess on this server. However, the access log is not one of the configurable parameters. But through a hackish combination of several tricks, I can log all accesses to my public_html tree. Here’s my .htaccess:

RewriteEngine on
RewriteCond %{REQUEST_URI} !^/hitcounter.php
RewriteRule ^ http%1://www.%{HTTP_HOST}/hitcounter.php?uri=%{REQUEST_URI} [L,R=301]

It redirects all incoming accesses to the “hitcounter.php” script, which logs the access to a file called “log” in the usual Apache access log format:

<?php
if (getenv(HTTP_X_FORWARDED_FOR)) {
    $ip = getenv(HTTP_X_FORWARDED_FOR);
    $vip = getenv(REMOTE_ADDR);
} else {
    $ip = getenv(REMOTE_ADDR);
}
$uri = $_GET['uri'];
$date = date('d/m/Y:H:i:s O');
$fh = fopen('log', 'a');
fwrite($fh, "$ip - - [$date] \"GET $uri HTTP/1.0\" 200 -\n");
fclose($fh);

header("Location: /nocount$uri");
?>

At the end of the script it redirects to the original URL prepended with “/nocount”. The directory “/nocount” simply contains symbolic links to everything in the webroot, along with a new .htaccess that turns rewriting back off again:

RewriteEngine off

One handy feature of this is I can bookmark or give out links containing “nocount” and those won’t be logged, so I can avoid self-logging. Downside is, this particular scheme results in “/nocount” being prepended to all my URLs.

I ended up abandoning this scheme because I was using my web server to web seed torrents and these redirects confused and broke the uTorrent client somehow. It also turned out to be unnecessary because Feral Hosting actually lets me configure my own private Apache server even on shared hosts via the .apache2 directory in my home directory.

Installation part based on this blog entry, with modifications.

Installing Sun JDK

First, assuming you want to use the Sun JDK, we need to get access to the Sun JDK package. Uncomment this in your /etc/apt/sources.list:

deb http://archive.canonical.com/ubuntu natty partner
deb-src http://archive.canonical.com/ubuntu natty partner

And do this (from this post):

sudo add-apt-repository ppa:sun-java-community-team/sun-java6
sudo apt-get update

Install Eclipse and Sun JDK and plug-in:

sudo apt-get install eclipse sun-java6-jdk sun-java6-jre sun-java6-bin sun-java6-plugin

Accept license terms when prompted. After this completes, make Sun’s JDK the default:

sudo update-java-alternatives -s java-6-sun

Installing PyDev

1. Run Eclipse and then go to Help | Install New Software
2. Click Add… next to the “Work with” drop down. Specify the following:
   Name: PyDev
   URL: http://pydev.org/updates
3. Check “PyDev”. Do not check “Pydev Mylyn Integration”. Press Next.
4. Press Next again, accept license terms, click Finish to begin install. Installation may proceed in background – see lower-right of the window for status.
5. If prompted to accept a certificate, click Select All, then OK. If prompted to restart Eclipse, click Yes.
6. Open Windows | Preferences | Pydev | Interpreter – Python.
7. Click Auto Config. A “selection needed” window will appear for the SYSTEM pythonpath. Click Select All, click OK.
8. When Auto Config completes, click OK and wait for Pydev to scan your system Python libraries.

PyDev installation is now complete.

Trying out PyDev

To create a new (fresh) Python project in Eclipse with PyDev:

1. Click File | New | Project | PyDev | PyDev Project, and click Next.
2. Enter a project name. Click Finish.
3. An Open Associated Perspective dialog will appear. Check the checkbox and click Yes.
4. Right-click “src” and choose New | PyDev Module. Enter a Name (you need not enter a Package). Click Finish.
5. Type some Python into the text file, like:
   print(‘hello world’.capitalize())
   You can press ENTER to autocomplete the capitalize() call while typing it.
6. Click Run | Run. Leave set to “Python Run” and click OK. A Console window tab should appear showing the program output.
7. Add the following two lines of source code:
   x = ‘hello there’
   print(x)
   Right-click in the margin to the left of the first source code line and choose “Add Breakpoint”.
8. Choose Run | Debug. When “Confirm Perspective Switch” appears, check the box and click Yes.
9. Execution will break at the first line. Select Run | Step Over, click the “Step Over” toolbar button, or press F6 to step through the source code line by line. After stepping over the line “x = ‘hello there’” the variable x should appear in the “Variables” window (you may have to scroll it down). Use Run | Debug or F11 to start the program over again during debugging.
10. When done debugging, click the “PyDev perspective” in the upper right hand corner to return to the normal source editing view.
11. To use PyDev with an existing Python project you’re working on, see “Creating a Project from existing sources“.

Keyboard shortcuts

• sudo apt-get install ruby1.9.1
• update-alternatives --config ruby
• (select the option reading “/usr/bin/ruby1.9.1″)
• update-alternatives --config gem
• (select the option reading “/usr/bin/gem1.9.1″)

Contrary to their names, /usr/bin/ruby1.9.1 is actually a newer version (1.9.2p290 at the present time) and /usr/bin/gem1.9.1 is actually RubyGems 1.3.7 (there is no RubyGems 1.9.1). These numbers refer to the package name ruby1.9.1.

During a “bundle install”, depending on what gems are being installed, you may encounter other errors. I encountered the following three:

• ERROR: Error installing nokogiri:
  ERROR: Failed to build gem native extension.
  
  /usr/bin/ruby1.9.1 extconf.rb
  <internal:lib/rubygems/custom_require>:29:in `require': no such file or load -- mkmf (LoadError)
  from <internal:lib/rubygems/custom_require>:29:in `require`
  from extconf.rb:5:in `<main>'

  To fix this one do: apt-get install ruby1.9.1-dev

  ERROR: Error installing nokogiri:
  ERROR: Failed to build gem native extension.
  
  /usr/bin/ruby1.9.1 extconf.rb
  checking for libxml/parser.h... no
  -----
  libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
  -----

  To fix this one do: apt-get install libxml2-dev

  ERROR: Error installing nokogiri:
  ERROR: Failed to build gem native extension.
  
  /usr/bin/ruby1.9.1 extconf.rb
  checking for libxml/parser.h... yes
  checking for libxslt/xslt.h... no
  -----
  libxslt is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
  -----

  To fix this one do: apt-get install libxslt1-dev

• Installing sqlite3 (1.3.4) with native extensions
  Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
  /usr/bin/ruby1.8 extconf.rb
  checking for sqlite3.h... no
  sqlite3.h is missing. Try 'port install sqlite3 + universal'
  or 'yum install sqlite3-devel' and check your shared library search path (the
  location where your sqlite3 shared library is located).

  To fix this one do: sudo apt-get install libsqlite3-dev

• Gem::InstallError: hoe requires RubyGems version >= 1.4. Try 'gem update --system' to update RubyGems itself.

  Version 1.9.2 of the “hoe” gem requires a newer version of RubyGems than is presently available via the Ubuntu package manager. You have to force an upgrade with:

  sudo REALLY_GEM_UPDATE_SYSTEM=1 gem update --system

  After doing so, you must redo “gem install bundler” and “bundle install”. Note that this might screw up other things, since you’re upgrading gem outside the package manager interface now.

If these instructions don’t work for you, or you encounter problems with other gems and figure out how to fix them, please post in the comments below!

However out-of-the-box, FreeNAS is a very minimal distribution, missing a lot of kernel modules needed to make this device work. To fix this, I followed these steps:

• Download the installer ISO for the matching version and platform of FreeBSD.
• Mount the ISO and dug into /boot/kernel
• Use “mount -uw /” on the NAS to remount the root as read/write so you can write to /boot and /conf.
• Use scp to upload the following kernel modules from the FreeBSD ISO to your running FreeNAS install under /boot/kernel:
• if_run.ko runfw.ko wlan*.ko
• Edit /boot/loader.conf and add the lines:

  if_run_load="YES"
  runfw_load="YES"

• Go to /conf/base/etc and edit/create the necessary files as described in the FreeBSD Handbook Wireless Networking Guide. I have a WPA-authenticated network so I created wpa_supplicant.conf and edited rc.conf to add:

  wlans_run0="wlan0"
  ifconfig_wlan0="WPA DHCP"

  Note the use of “run0” here, since this device uses the “run” kernel module.

• Copy your modified files from /conf/base/etc to /etc and test with:

  /etc/rc.d/netif start

  If you connect successfully, ping a host like google.com to confirm connectivity.

• Reboot. Log back in. Use /sbin/ifconfig and ping to confirm you are still connected.

Fortunately, the machine is x86 and made entirely of commodity parts, so it’s possible to install any OS I like on it. The catch is, it has no VGA out. However, it does have integrated video on the motherboard – they just didn’t solder a connector onto it. Following the directions here, I disassembled one end of a VGA cable (first I used a flathead screwdriver to push off the plastic cover, then used some needlenose pliers to twist the metal tabs until they broke off, exposing the pins). I then poked the pins through a small piece of paper, for insulation, and inserted them directly into the motherboard. The VGA holes are at the back of the motherboard near the other connectors. I was successful in getting VGA out, although it took quite a bit of fiddling with the angle of the cable to make it touch things just right – in the future I may want to solder a connector onto the board for convenience. Once I’d done this, I could plug in a USB keyboard and hit DEL to enter BIOS at boot and turn on USB booting – note that you do not want to set USB-CDROM/USB-FDD etc. as the boot device, but rather you must plug in a USB flash drive first, which the BIOS will correctly see as a disk device, and then change the priority of boot disk devices. After that, I could set up the USB flash drive with any installer or OS I wanted – I used an 8 GB flash drive. There are alternative ways to getting another OS to run on the board – like replacing the internal 128 MB flash drive – but it’s much easier to diagnose boot issues if you can see the VGA out.

I started out by installing FreeNAS, just to discover it doesn’t support the N7700′s SATA controller. Next I tried Openfiler but couldn’t get it to boot from USB (although this might be possible). Finally I decided to go with the freshly-released Ubuntu Server 11.10 Linux. I used Unetbootin to put its installer ISO onto my USB key, and sure enough it could see all my disks. Catch is, I wanted to install it to the USB key, but I was using that for the installer. The internal flash drive was too small to install to (128 MB). The Thecus N7700 only has two USB ports – if I had a second USB flash drive and a USB hub handy, I would’ve just installed directly onto the second USB key (I recommend trying this), but I didn’t, so instead I installed onto a small root partition on one of my RAID hard drives, sized to match the USB flash drive. It failed to install Grub to the MBR during install, but I ignored this. Afterwards, I attached both the flash drive and the disk to my PC, repartitioned the flash drive in Parted Magic, and then used Clonezilla to copy the installed partition from my hard disk to the flash drive. There was a small catch to this: I had to create a tiny 1 MB partition before my Ubuntu partition and mark it as a BIOS boot partition for Grub. Then, I used the Ubuntu 11.10 Desktop Live CD in live mode to chroot into the Ubuntu filesystem (see this guide to chrooting involving mount -o bind) and run “install-grub /dev/sdb”, where /dev/sdb was my USB flash device, and “update-grub”. At this point the device was bootable.

Once I had booted up the device, I had to fix some annoyances. First of all, device names of disks can change between boots, particularly if you remove or add disks. To deal with this I created /etc/udev/rules.d/nas-disk.rules with the following contents:

SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:04:00.0-sas-0x0000000000000000-lun-0", SYMLINK+="nasa"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:04:00.0-sas-0x0100000000000000-lun-0", SYMLINK+="nasb"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:04:00.0-sas-0x0200000000000000-lun-0", SYMLINK+="nasc"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:04:00.0-sas-0x0300000000000000-lun-0", SYMLINK+="nasd"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:05:00.0-sas-0x0000000000000000-lun-0", SYMLINK+="nase"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:05:00.0-sas-0x0100000000000000-lun-0", SYMLINK+="nasf"
SUBSYSTEM=="block", ENV{ID_TYPE}=="disk", ENV{ID_PATH}=="pci-0000:05:00.0-sas-0x0200000000000000-lun-0", SYMLINK+="nasg"

Now, /dev/nasa thorugh /dev/nasg referred to my seven disks in order from the top down. Warning: if you mount any volumes in your /etc/fstab, make sure you include the “nobootwait” option. Otherwise, if a volume fails to mount for any reason during boot, it will just sit there waiting for you to press S, before even starting the SSH server.

Once I had done this I installed a number of useful packages:

• sshd: Already installed during installation, lets me administer the machine over the network with the box closed and no VGA cable attached.
• mdadm: Needed for software RAID support.
• xfsprogs reiserfsprogs btrfs-tools hfsprogs: Tool support for additional filesystems.
• nfs-kernel-server: Lets me share files with other UNIX machines over NFS. (see this guide – I found it useful to set no_root_squash so my client had free access)
• samba: Installed during installation, lets me share files with Windows clients.
• webmin: Gives me a remote web administration interface on port 10000 that I can use to, among other things, create, modify, and destroy RAID partitions.
• smartmontools: Allows me to regularly check the SMART data of my disks and run self-tests, and report any problems.
• transmission-daemon: Provides a web interface for BitTorrent downloads. Requires some configuration, see this guide.
• sysstat: Includes iostat, which lets me view current disk activity.
• open-iscsi open-iscsi-tools: iSCSI support (not currently using this but I used to use iSCSI exclusively and manage the volume as NTFS on Windows clients).

At first I considered running btrfs on my NAS, which is a very promising up-and-coming competitor to ZFS, but there’s no official stable release quite yet and I’m uncertain about performance. ZFS itself is only available in alpha form on Linux, primarily due to licensing issues. I ended up running XFS because it’s mature and handles volumes with many large files well.

Some notes on RAID: I used software RAID 5 over 5 disks with mdadm. It starts in degraded state, then rebuilds, and will send mail to root about this, so don’t be alarmed. You can use it right away if you want (although this slows down rebuild), and you can monitor the rebuild process via SSH with while true; do cat /proc/mdstat; sleep 5; done.

Note that if you do modify the OS of your Thecus N7700, the disk LEDs and LCD display will no longer work. The LCD display will show “Running self tests” forever, and the blue and red LEDs will also remain on and not flashing indefinitely. I believe a sufficiently clever person could reverse engineer their interface and provide a command-line utility or daemon to control them, which would be quite handy, but I don’t need it. The power and network LEDs should still work fine.

Note that if you insert or remove drives, or boot without the USB flash drive attached, you may accidentally alter the disk boot priority order in BIOS, causing the internal flash drive to gain priority, or causing the flash drive to fall off the list. If this happens, you still need the VGA cable to get back into BIOS to fix it. To mitigate this problem, I recommend completely erasing the internal flash drive by filling it with zeros: dd if=/dev/zero of=/dev/sda bs=128M (make sure it’s /dev/sda on your machine though!)

Warning: When you close back up the case, remember to plug back in the case fans. Seven disks generate a lot of heat and you will fry your machine if you don’t do this. Sadly I destroyed my Thecus this way only a single day after getting my RAID set up, so I can’t say much about long-term use.

Let me know if you have any problems with this, and please share your own experiences! To close this out, here is a bunch of data about the Thecus N7700 system that you may find useful if you’re trying to get another distro or OS running on it:

lspci:

00:00.0 Host bridge: Intel Corporation Mobile 945GME Express Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile 945GME Express Integrated Graphics Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 1 (rev 02)
00:1c.2 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 3 (rev 02)
00:1c.3 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 4 (rev 02)
00:1c.4 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 5 (rev 02)
00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 02)
00:1d.0 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #3 (rev 02)
00:1d.3 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #4 (rev 02)
00:1d.7 USB Controller: Intel Corporation N10/ICH 7 Family USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev e2)
00:1f.0 ISA bridge: Intel Corporation 82801GHM (ICH7-M DH) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801GBM/GHM (ICH7 Family) SATA AHCI Controller (rev 02)
00:1f.3 SMBus: Intel Corporation N10/ICH 7 Family SMBus Controller (rev 02)
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
04:00.0 RAID bus controller: Marvell Technology Group Ltd. 88SE6440 SAS/SATA PCIe controller (rev 02)
05:00.0 RAID bus controller: Marvell Technology Group Ltd. 88SE6440 SAS/SATA PCIe controller (rev 02)

/proc/cpuinfo

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 14
model name      : Intel(R) Celeron(R) M CPU        440  @ 1.86GHz
stepping        : 12
cpu MHz         : 1866.707
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 ss tm pbe nx constant_tsc up arch_perfmon bts aperfmperf pni monitor tm2 xtpr pdcm dts
bogomips        : 3733.41
clflush size    : 64
cache_alignment : 64
address sizes   : 32 bits physical, 32 bits virtual
power management:

/proc/meminfo

MemTotal:        1023884 kB

lsmod

Module                  Size  Used by
serio_raw              12990  0
i915                  505108  1
drm_kms_helper         32889  1 i915
drm                   192226  2 i915,drm_kms_helper
i2c_algo_bit           13199  1 i915
video                  18908  1 i915
lp                     17455  0
parport                40930  1 lp
raid10                 30270  0
usb_storage            44173  1
raid456                61518  1
async_raid6_recov      12906  1 raid456
async_pq               12959  2 raid456,async_raid6_recov
uas                    17699  0
ahci                   21634  0
libahci                25727  1 ahci
mvsas                  51615  8
libsas                 62786  1 mvsas
scsi_transport_sas     34482  2 mvsas,libsas
e1000e                139775  0
raid6_pq               88205  2 async_raid6_recov,async_pq
async_xor              12738  3 raid456,async_raid6_recov,async_pq
xor                    21860  1 async_xor
async_memcpy           12481  2 raid456,async_raid6_recov
async_tx               13123  5 raid456,async_raid6_recov,async_pq,async_xor,async_memcpy
raid1                  26291  0
raid0                  17067  0
multipath              12977  0
linear                 12792  0

smartctl -a /dev/hda (info about the internal flash drive)

Device Model:     128MB ATA Flash Disk
Serial Number:    C191100004A12b39GGwP
Firmware Version: AD B512D
User Capacity:    128,057,344 bytes [128 MB]
Sector Size:      512 bytes logical/physical
ATA Version is:   6
ATA Standard is:  ATA/ATAPI-6 T13 1410D revision 3a
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

/proc/ioports

0000-0cf7 : PCI Bus 0000:00
  0000-001f : dma1
  0020-0021 : pic1
  0040-0043 : timer0
  0050-0053 : timer1
  0060-0060 : keyboard
  0064-0064 : keyboard
  0070-0073 : rtc0
  0080-008f : dma page reg
  00a0-00a1 : pic2
  00c0-00df : dma2
  00f0-00ff : fpu
  0170-0177 : 0000:00:1f.1
    0170-0177 : ata_piix
  01f0-01f7 : 0000:00:1f.1
    01f0-01f7 : ata_piix
  02f8-02ff : serial
  0376-0376 : 0000:00:1f.1
    0376-0376 : ata_piix
  03f6-03f6 : 0000:00:1f.1
    03f6-03f6 : ata_piix
  03f8-03ff : serial
  0400-04bf : pnp 00:08
    0400-0403 : ACPI PM1a_EVT_BLK
    0404-0405 : ACPI PM1a_CNT_BLK
    0408-040b : ACPI PM_TMR
    0428-042f : ACPI GPE0_BLK
  04d0-04d1 : pnp 00:01
  0500-051f : 0000:00:1f.3
  0680-06ff : pnp 00:01
  0880-088f : pnp 00:01
0cf8-0cff : PCI conf1
0d00-ffff : PCI Bus 0000:00
  8000-8fff : PCI Bus 0000:04
    8f00-8f7f : 0000:04:00.0
      8f00-8f7f : mvsas
  9000-9fff : PCI Bus 0000:03
    9f00-9f1f : 0000:03:00.0
  a000-afff : PCI Bus 0000:02
    af00-af1f : 0000:02:00.0
  b000-bfff : PCI Bus 0000:01
  c000-cfff : PCI Bus 0000:05
    cf00-cf7f : 0000:05:00.0
      cf00-cf7f : mvsas
  d000-dfff : PCI Bus 0000:06
  f300-f30f : 0000:00:1f.2
    f300-f30f : ahci
  f400-f403 : 0000:00:1f.2
    f400-f403 : ahci
  f500-f507 : 0000:00:1f.2
    f500-f507 : ahci
  f600-f603 : 0000:00:1f.2
    f600-f603 : ahci
  f700-f707 : 0000:00:1f.2
    f700-f707 : ahci
  f800-f80f : 0000:00:1f.1
    f800-f80f : ata_piix
  fb00-fb1f : 0000:00:1d.3
    fb00-fb1f : uhci_hcd
  fc00-fc1f : 0000:00:1d.2
    fc00-fc1f : uhci_hcd
  fd00-fd1f : 0000:00:1d.1
    fd00-fd1f : uhci_hcd
  fe00-fe1f : 0000:00:1d.0
    fe00-fe1f : uhci_hcd
  ff00-ff07 : 0000:00:02.0

/proc/iomem

00000000-0000ffff : reserved
00010000-0009a7ff : System RAM
0009a800-0009ffff : reserved
000a0000-000bffff : PCI Bus 0000:00
  000a0000-000bffff : Video RAM area
000c0000-000dffff : PCI Bus 0000:00
  000c0000-000c7fff : Video ROM
000e0000-000effff : pnp 00:0b
000f0000-000fffff : reserved
  000f0000-000fffff : System ROM
00100000-3fddffff : System RAM
  01000000-01534783 : Kernel code
  01534784-017bbd7f : Kernel data
  01870000-01946fff : Kernel bss
3fde0000-3fde2fff : ACPI Non-volatile Storage
3fde3000-3fdeffff : ACPI Tables
3fdf0000-3fdfffff : reserved
3fe00000-febfffff : PCI Bus 0000:00
  d0000000-dfffffff : 0000:00:02.0
  e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff]
    e0000000-efffffff : reserved
      e0000000-efffffff : pnp 00:0a
  fd300000-fd3fffff : PCI Bus 0000:02
  fd400000-fd4fffff : PCI Bus 0000:02
    fd4c0000-fd4dffff : 0000:02:00.0
      fd4c0000-fd4dffff : e1000e
    fd4fc000-fd4fffff : 0000:02:00.0
      fd4fc000-fd4fffff : e1000e
  fd500000-fd5fffff : PCI Bus 0000:01
  fd600000-fd6fffff : PCI Bus 0000:06
  fd700000-fd7fffff : PCI Bus 0000:06
  fd800000-fd8fffff : PCI Bus 0000:01
  fd900000-fd9fffff : PCI Bus 0000:05
    fd900000-fd93ffff : 0000:05:00.0
  fda00000-fdafffff : PCI Bus 0000:05
    fdaff000-fdafffff : 0000:05:00.0
      fdaff000-fdafffff : mvsas
  fdb00000-fdbfffff : PCI Bus 0000:04
    fdb00000-fdb3ffff : 0000:04:00.0
  fdc00000-fdcfffff : PCI Bus 0000:04
    fdcff000-fdcfffff : 0000:04:00.0
      fdcff000-fdcfffff : mvsas
  fdd00000-fddfffff : PCI Bus 0000:03
  fde00000-fdefffff : PCI Bus 0000:03
    fdec0000-fdedffff : 0000:03:00.0
      fdec0000-fdedffff : e1000e
    fdefc000-fdefffff : 0000:03:00.0
      fdefc000-fdefffff : e1000e
  fdf00000-fdf7ffff : 0000:00:02.0
  fdf80000-fdfbffff : 0000:00:02.0
  fdffc000-fdffc3ff : 0000:00:1f.2
    fdffc000-fdffc3ff : ahci
  fdfff000-fdfff3ff : 0000:00:1d.7
    fdfff000-fdfff3ff : ehci_hcd
fec00000-ffffffff : reserved
  fec00000-fec003ff : IOAPIC 0
  fed13000-fed1dfff : pnp 00:0b
  fed20000-fed8ffff : pnp 00:0b
  fee00000-fee00fff : Local APIC
    fee00000-fee00fff : pnp 00:0b
  ffb00000-ffb7ffff : pnp 00:0b
  fff00000-ffffffff : pnp 00:0b

/proc/interrupts

           CPU0
  0:        304   IO-APIC-edge      timer
  1:          4   IO-APIC-edge      i8042
  8:          1   IO-APIC-edge      rtc0
  9:          0   IO-APIC-fasteoi   acpi
 14:      23215   IO-APIC-edge      ata_piix
 15:          0   IO-APIC-edge      ata_piix
 16:   88518197   IO-APIC-fasteoi   uhci_hcd:usb5, mvsas, i915
 17:   92740547   IO-APIC-fasteoi   mvsas
 18:          0   IO-APIC-fasteoi   uhci_hcd:usb4
 19:          0   IO-APIC-fasteoi   uhci_hcd:usb3
 23:    1014224   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb2
 48:          0   PCI-MSI-edge      ahci
 49:     745524   PCI-MSI-edge      eth1-rx-0
 50:     667535   PCI-MSI-edge      eth1-tx-0
 51:          3   PCI-MSI-edge      eth1
NMI:          0   Non-maskable interrupts
LOC:   19023450   Local timer interrupts
SPU:          0   Spurious interrupts
PMI:          0   Performance monitoring interrupts
IWI:          0   IRQ work interrupts
RES:          0   Rescheduling interrupts
CAL:          0   Function call interrupts
TLB:          0   TLB shootdowns
TRM:          0   Thermal event interrupts
THR:          0   Threshold APIC interrupts
MCE:          0   Machine check exceptions
MCP:        262   Machine check polls
ERR:          0
MIS:          0

1. Right-click on the download link for the large file and then strip components of the URL to view the directory it’s in. Often that same directory will contain either a hash file for that large file (e.g. blah.iso.md5 or blah.md5), or a hash file for all the files in the directory (MD5SUMS). Similarly, try taking the filename and adding “.md5″ or replacing the extension with “.md5″ (or the same with .sha1) and then Googling that to see if it’s hiding somewhere else.
2. If the file is hosted on SourceForge, SourceForge provides SHA1 and MD5 hashes for every file on their site. Just go to the “Files” section of the project, find the directory the file is in, and then click the “i” information bubble to the right of the filename.
3. If there is a Torrent for the file, either download using that in the first place, or just open the Torrent file and point it at your existing file for seeding. Not only will this verify your file, but any corruption will be detected and fixed at the block level, so you don’t have to re-download the whole file. Once you’ve verified the file with your Torrent client, you can safely use a hash tool like md5sum to compute your own hash file for it.
4. If the file is a compressed archive, such as a ZIP, .gz, .bz2, .xz, or .7z file, attempt to decompress it (when using gunzip/bunzip2/xz, make sure to pass –keep to keep the original input file around, because you might still need its hash later). A ISO may also contain hash files for files inside the image – check for these by using “mount -o loop” (or Daemon Tools/Virtual CloneDrive under Windows). There’s not an fsck for ISO 9660 that I know of, but you can scan through an ISO filesystem by using a command like “grep -r randomstring *” or “findstr /s randomstring *” in the root of the mounted ISO, where “randomstring” is a string that does not occur in any file. If any of these tests fail, the image is corrupt (but if they succeed, it does not necessarily prove that it is valid).
5. If none of the above apply, things get tricky. Compute the MD5 and SHA1 hashes of the file using md5sum and sha1sum, then type them into Google. If you get any search results, it is extremely likely that your file is valid and someone else who downloaded it has posted their hash on a forum or digest listing (any false positives would indicate a hash collision, which these hashes are designed to make very unlikely).
6. Finally, suppose you get no Google results. At this point, your only recourse is to re-download the file and compare the SHA1 of the two files. If they are identical, your file is good (provided of course the server’s copy is good – use an official server for at least one of the downloads). If they are different, redownload it again. Continue to redownload it until you get at least two files of the correct size with identical hashes. At this point you can be reasonably assured the file is valid (since two invalid versions are usually different from one another). If you get this far, be sure to post your MD5, SHA1, and SHA256 hashes, along with the original URL and a description of the file, on a public forum so that others can benefit. You might even create and publish a torrent for the file on a relevant torrent tracking website.