The purpose of this article isn’t to do a backup 101, but rather more of a 201 or 301 perspective. It assumes a general understanding of the user-side of Backups. Basically what we are going to do is take a look at how Microsoft’s Windows Home Server handles backups. This information is specific to Version 1, which was released in 2007. A future article on how WHS V2 codename Vail does backups is planned closer to when the Vail featureset is locked in.

Settings

To begin with, lets go real quick through the setup screens on the WHS. First up is the backup settings:

The first is the settings window specific to backup (it should be pointed out that the WHS defaults are 3 months, 3 week, and 3 days, and the window is 12am to 6am). While this may seem fairly straight-forward, there are couple of items worth nothing. First, any computer that is on during the opening of the backup window will start to backup at that time (or it will wake up the PC’s if you have “wake up to backup” selected). For example, if you have 3 PC’s on, all 3 will start to backup. The backups will actually happen one PC at a time.

The other item to note is what happens with a 4th PC that isn’t on at the beginning of the window. When you turn the PC on (or restore the network connection), the WHS connector on the PC will connect to the server. This checks various things such as password synching and stuff like that. One other thing it checks is to see if it needs to be backed up. If the PC is turned on during the backup window, it should still start the backup up during the window. In my home environment we have 3 home pc’s (a desktop and 2 laptops) and my work laptop. The main PC’s backup at 6pm during the window and my work PC backups up whenever I turn it on every few days to do some work. This is usually around 8pm, and one of the first things the machine does is backup.

I would recommend that you set a timeframe that encompasses all the times that the various PC’s will be on. For example, if you have an HTPC, I’d probably setup the backup window to start with a time of day when the PC isn’t likely to be doing anything. This will make sure that the backup doesn’t interfere with your viewing. You could set the window from 6am to 11pm if you really wanted to.

The second part of those backups settings is the “Automatic Backup Management” section. Here is where you configure how long you want WHS to keep the backups. This should be fairly self-explanatory.

Here is what the window looks like when you click on a PC in the “Computers” tab of the console and select “view backups”:

Semi-technical

Now to begin looking at how the backups work. Let me paraphrase how it works first and then we’ll look at the technical details. Consider the following scenario:

The following file: (C:\Program Files\Zune\Zune.exe) is stored on 2 computers. This is for the Zune software version 4.6. They get backed up on December 31st. On January 1st, Computer A updates to version 4.7 of the Zune software. This means that the zune.exe file changes. On January 2nd, Computer B updates. The backup settings are set to keep 7 daily, 0 weekly, and 0 monthly backups. So what is on the server:

• Dec 30 = 1 copy of zune.exe, which is linked in both Computer A’s and Computer B’s backup.
• Dec 31 = Same as Dec 30
• Jan 01 = 2 copies of zune.exe. The first is version 4.6’s which is linked to all of Computer B’s backups and Computer A’s backups prior to Jan 01. The 2nd is version 4.7’s which is linked to Computer A’s backup on Jan 01.
• Jan 02 = 2 copies of zune.exe. The first is version 4.6’s which is linked to Computer B’s backups prior to Jan 02 and Computer A’s backups prior to Jan 01. The 2nd is version 4.7’s which is linked to Computer A’s backup on and after Jan 01 and Computer B’s backup on and after Jan 02.
• Jan 03 = same as Jan 02
• Jan 04 = same as Jan 02
• Jan 05 = same as Jan 02
• Jan 06 = same as Jan 02
• Jan 07 = 2 copies of zune.exe. The first is version 4.6’s which is linked to Computer B’s backups prior to Jan 02. The 2nd is version 4.7’s which is linked to all of Computer A’s backups and Computer B’s backup on and after Jan 02.
• Jan 08 = 1 copy of zune.exe, which is linked in both Computer A’s  and Computer B’s backups.

So what happened here? Each week (at the beginning of the Sunday backup window to be exact), there are 2 actions that take place: 1/ delete backups 2/ clean up backup database. This first action is just a simple check that the server does to see if any backups are outside of the windows defined in the settings. In this scenario, if there are any backups more than 7 days old, those backups get deleted. The 2nd action is to clean up the database. What this does is to check every file that exists in the backup catalog and see if there are any orphaned files, that is, files that were originally linked to an old backup and are not linked to any current backups.

In our scenario above, the first action started taking place on Jan 7th, by deleting both computers backups of Dec 30. On Jan 8th, the backups from Dec 31 were deleted. For action 2, on January 7th, there were no orphaned files, so no files were actually deleted. However on Jan 8th, the old zune.exe file was no longer referenced anywhere, and so that file was considered an orphan and deleted.

Technical

Now that we’ve looked at the paraphrased version, lets get into the nitty gritty. First off, the backup “database” isn’t a database like you normally think of that is hosted in a registry file or an SQL server. It is a database that stores its data in actual files.

The WHS stores the backups on the server in a non-duplicated area (unless you are using the BDBB to duplicate them). On my WHS, they are in D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4} When you look in that folder, you will see basically 4 types of files. The first corresponds to the computer-specific backups. Take a look at the following:

This is all of the files for one of my PC’s. The explanation I gave above is a simplistic version. What really happens is that instead of storing files as files, it actually stores files in the form of clusters. A little background: the hard drive in your computer is comprised of clusters. When you format the disk, the file system splits the drive up into little chunks called clusters. NTFS, by default, breaks those chunks into clusters that are each 4KB in size. This means that every file in your computer is split up into 4KB chunks. Take a look at this:

What you can see is that while the zune.exe file itself is only 201KB, it actually takes up 204 KB (51 clusters worth) of disk space (any file less than 4KB still takes up a full cluster, as you can’t have multiple files in a single cluster (unless it is inside of a .zip file or the like)).

When you do your initial set up of WHS and attach the first PC, the first backup will take a lot of time. What WHS actually does is copy every cluster to the server. The client computer calculates what are called hashes for each cluster. A hash is basically the output of an algorithm that is performed for a set of data. In the WHS arena, each cluster runs through the algorithm and a hash is generated that (for all intents and purposes) uniquely identifies this cluster. This hash is sent to the WHS along with the cluster for storage.

When you do subsequent backups on that PC or on a different PC, the client connector reads every cluster on the hard drive and looks to see when that cluster was last changed (this is a function of the NTFS file system). If that change date is after the time of last backup, the connector generates a hash for that cluster. It sends the hash to the WHS. The WHS looks up that hash in the backup database and if that hash doesn’t return a result, that cluster is then sent to the WHS for safe keeping. What this practically means is that any time a file changes, the client checks with the WHS to see if the WHS already has that file from another computer and if not, sends the file for storage.

The output from each backup is thus twofold. The first is that each cluster on the client computer will have a copy on the WHS (whether due to being recently copied or from prior existence). The second is that each backup will have a list of every cluster that is on the client hard drive and the hash corresponding to where that cluster is stored in the WHS backup database.

Speaking of which, lets look at where those clusters are stored:

WHS stores the clusters in a .dat file that grows to a maximum of 4GB in size (that would be 2^20  or roughly 1 million clusters). It then rolls over to the .1.dat and then a .2.dat and so on and so forth as far as is needed to store every cluster (I’m up to .152.dat on my WHS). These can be thought of as non-compressed .zip files that are store the clusters in a flat (no subfolders) format. These .dat folders don’t associate the clusters with any specific computer or backup. One other thing to notice from the screenshot above is that .dat files don’t stay at the 4GB file size (of my 153 files, only 38 are still at 4GB). When the backup cleanup is done, any orphaned clusters are deleted, but the .dat files aren’t consolidated.

Misc Notes

A few other things to note: the reason why WHS can’t do simultaneous backups is because the database needs the backups to be serialized in order for the hash database to be accurate. Because the client backup procedures does a lookup of hash values, you wouldn’t get accurate results if it was doing multiple backups at the same time. The second is that if you have systems with different cluster sizes (such as an HTPC with 64KB clusters (like SageTV sometimes recommends)) the cluster hashes will always be different than a system with 4KB clusters and thus there will be duplicate data stored on the WHS. Lastly, WHS will only backup from NTFS shares, which is why it doesn’t support Windows 98 (which uses fat32) or Mac OS (which uses HFS).

Excluded files. I just wanted to point out real quick that there are files that WHS doesn’t back up. These are:

• User temporary files
• System page file
• Hibernation file
• Client-side cache folders
• Shadow volume implementation folders
• Media Center temporary files &  Recorded TV (this can be changed if desired)

Conclusion

I hope this helps to give the reader more information as to how WHS backups work. It truly is a marvelous system and, in my opinion, is one of the most flexible backup systems out there. This is especially true when you consider how it performs differential backups, taking multiple systems into account. You can restore from any backup of any PC onto whatever you want. (You can even backup from/to virtual PC’s which is really nice). You can even mount a backup image as a virtual hard drive and extract any file you want. A truly wonderful  system

Resources:

WHS Backup Technical Doc

Special Thanks to Alex Kuretz for helping ensure the technical accuracy.

Welcome back WGS! If you have not already done so, head over to WeGotServed.Co.UK and change your RSS feeds. As I very well know, a server move means new RSS feeds. Stay up to date with WGS feeds!