SQL Server: Recovering from Disasters Using Backups

Paul S. Randal

There isn't much point taking SQL Server backups unless you know how to restore them. If you have anything more complicated than just a full database backup, you're going to need to know some RESTORE options to be able to successfully restore your database to the desired point in time.
This is even more the case if you have a complicated database layout or a complex backup strategy and you want to be able to restore, for example, a single file or filegroup, or take advantage of partial database availability.

As long as you have an effective backup strategy and your backups are valid, you should be able to recover from a disaster within your Recovery Time Objective (RTO) and to your Recovery Point Objective (RPO). In the first article in this three-part series, I discussed the various types of backups and how to formulate a backup strategy (see "Understanding SQL Server Backups" in the July 2009 issue).
In this article, I'll explain how restore works and how to perform some of the more common restore operations. It will be helpful if you've read the backup article, and the background material I mentioned in that article's introduction. I'm also going to explain a few more tricky operations, such as doing a point-in-time restore and an online piecemeal restore with partial database availability.
Just as in the previous article on BACKUP, I'm not going to explain how the RESTORE syntax works or go through the specific steps of all restore operations. SQL Server Books Online does an excellent job of that. See "RESTORE (Transact-SQL)" for more info, especially the examples spread throughout the topic. There are actually so many options to RESTORE that it's a whole other topic to explain them! "Backing Up and Restoring How-to Topics (SQL Server Management Studio)" explains how to use the tools to perform restores.

The Four Phases of Restore

Let's start with how restore actually works. A restore operation has up to four phases:

1. File creation and initialization
2. Data and/or transaction log copy
3. REDO phase of recovery
4. UNDO phase of recovery

One of the primary goals of disaster recovery is to bring the database online as quickly as possible. If your disaster-recovery plan involves restoring backups (instead of, say, failing over to a database mirror), you're going to want the restore process to be as fast as possible. With each of the four restore steps in mind, is there anything you can do to speed them up?
The first step can be essentially skipped if the data and log files already exist. This means that if you're going to overwrite an existing database, don't drop the database before doing the restore. Instead, use the WITH REPLACE option on the first restore operation to tell SQL Server to use the existing files. If the files don't exist, they will be created and then initialized. Creating the files is very fast, but the process of zero-initializing them can be very slow.
For security reasons, and by default, all database files are zero-initialized. You can enable instant file initialization for the SQL Server instance, which skips the zeroing process for data file create and grow operations, including those required during a restore -- possibly saving hours of downtime, even if the data files are only gigabytes in size. Transaction log files are always zero-initialized because of the circular nature of the transaction log itself.
You can read more about all of this, with references to Books Online, in my "Instant Initialization" blog post category.
You may be wondering about the second phase -- why does the item say "and/or transaction log"? If you read the previous article, you'll remember that all full and differential backups also include some transaction log records, to enable the database to be restored to a transactionally consistent point in time. Phase two is a pure copy operation -- no processing of the data is performed -- so the main way to speed this up is to have a better-performing I/O subsystem. This is one of the few times when it's acceptable to "throw hardware at the problem."
The other way to speed up the copy phase is to use some kind of backup compression technology, either native to SQL Server 2008 or through one of the various third-party solutions. The two parts of the copy phase are reading from the backup media and writing out to the data and/or log files. If you can do fewer reads (using compressed backup media), you can speed up the overall process, at the expense of a small amount of CPU resources.
Phases three and four are about running recovery on the transaction log to bring the database to a transactionally consistent point in time. I explained the details of recovery in the February 2009 article "Understanding Logging and Recovery in SQL Server". Note that phase four is optional, as I'll explain later.
Suffice it to say that the more transaction log that needs to be recovered during a restore, the longer the restore will take. This means that if, for instance, you have a full database backup from a week ago and hourly transaction log backups since then, the restore process will essentially replay all the transactions from the last week before completing. I discussed a solution for this in the previous article -- adding differential backups into a full-plus-log backup strategy.
A differential database backup contains all the datafile pages that have changed since the last full database backup, and can be used in a restore operation to avoid having to replay all the transactions that occurred in the period between the full database backup and the differential database backup. This can vastly reduce the time it takes to restore the database, but at the expense of a slightly more complicated backup strategy.
You can find more information in the Books Online topic "Optimizing Backup and Restore Performance in SQL Server".

What Do You Need to Restore?

When disaster strikes, the first thing you need to do is work out what has been damaged, as this is going to dictate the actions you must take to recover from the disaster. For storage media failure, the possibilities include:

• Damage to the entire database (for instance, whatever was storing the database was destroyed, or the database was a single data file and it was damaged).
• Damage to a single filegroup of a multi-filegroup database.
• Damage to a single file of a multi-file filegroup.
• Damage to a single page in the database.
• Damage spread through the database.

You can ascertain the damage by looking through the SQL Server error log for notifications that file(s) are inaccessible, that page-read errors occurred (for instance, page checksum failures or a torn-page detection error), or that general corruption was encountered. If damage occurred, it is usual practice to run the DBCC CHECKDB consistency-checking operation to get an idea of how pervasive the damage is.
An explanation of consistency checking is beyond the scope of this article, but you can watch a video of a presentation I made at the Tech-Ed IT Forum in November 2008 titled "Corruption Survival Techniques", and listen to a TechNet Radio interview from earlier this year where I discuss database corruption (direct download links are here).
Disasters are not limited to I/O subsystem or server failures -- there's also human error to consider. Database tables (or data from them) are often accidentally deleted by poorly programmed applications or careless Transact-SQL statements (the "I didn't realize I was on the production server" scenario). In such cases, it can be very difficult to figure out what needs to be restored and from what point in time, especially if no one owns up to making the mistake. You might get lucky using standard reports from the default trace where the DDL operation is still available or the DELETE statement was caught by your own logging -- but often there's no record of who did what to the database. I'll discuss recovering from this situation in more detail later. Regardless of who performed the accidental data deletion or when it happened, the longer you wait to recover -- or the more time that passes before you are made aware of the problem -- the more complex it can be to recover.
So, as a first step if the database is running in the FULL recovery model and the transaction log is undamaged, perform a tail-of-the-log backup to ensure that all transactions up to the point of the disaster are backed up. This "final" transaction log backup will have everything up until the time of the disaster and can be used to bring the database being restored as far as possible, possibly up-to-the-minute.
In a nutshell, you need to work out what you have to restore. Then it becomes a question of what you are able to restore.

What Are You Able to Restore?

The aim of any restore operation is to restore the fewest possible backups, so the restore operation is as fast as possible and completes within your RTO, while also allowing you to meet your RPO.
The main question to ask here is "What backups do I have?" If the only backup you have is a full database backup from a week ago and the whole database has been lost, there's only one restore option -- to a point in time a week ago, losing all work since then. Simply put, your backup strategy should always ensure that you are able to restore what you need to in the event of a disaster, as I discussed in the previous article.
So how can you determine what backups you have available? First, you can query that various backup history tables in the msdb database. These tables contain a record of all backups that have been taken in the SQL Server instance since the last time the backup history tables were cleared out.
As far as the backups themselves are concerned, it is a best practice to name backup files to include the database, type of backup, date and time so that the backup can be identified at a glance. If you haven't done this, you can find out what a backup file contains using the RESTORE HEADERONLY command. This will display the contents of the backup file's header, essentially the metadata that describes the backup itself. You can read more in the Books Online topic "Viewing Information About Backups".
Using either method, you are trying to work out the restore sequence to use to restore the damaged or deleted data. A restore sequence is a set of backups that must be restored and the appropriate order in which to restore them. The restore sequence may be as simple as just one full backup (of a database, filegroup or file), or a complicated set of full, differential and transaction log backups.
For instance, imagine a scenario where the backup strategy involves full database, differential database and transaction log backups. If a system crash occurs and the data files are damaged, what is the restore sequence? Figure 1 illustrates this example.
In this case, the shortest and fastest restore sequence is the most recent full database backup (F), the most recent differential database backup (D2), and then all subsequent transaction log backups, up to and including the tail-of-the-log backup (L7 and L8).
One of the tricky problems when planning a restore sequence is finding the earliest required transaction log backup to restore (sometimes called finding the "minimum-LSN," or "minimum-Log Sequence Number"). In the example in Figure 1, only transaction log backups L7 and L8 are required, because the differential database backup D2 brings the database to a more recent point in time than all the previous transaction log backups.

Figure 1: Example Restore Sequence
SQL Server will allow previous, unneeded transaction log backups to be restored, but they will not be used and essentially just waste disaster-recovery time.Continuing my example, what would happen if the differential database backup D2 was damaged or missing? Figure 2 shows this case.

Figure 2: Restore Sequence with a Damage Differential Database BackUp
In this scenario, the shortest and fastest restore sequence is the most recent full database backup (F), the next most recent differential database backup (D1), and then all subsequent transaction log backups (L4, L5, L6, L7 and L8). This is possible only as long as backups D1, L4, L5 and L6 are still available. It is important that you do not delete backups too soon; otherwise you could run into problems during a disaster.
For instance, if the full database backup F is damaged, unless the previous full database backup is still available, the database will not be recoverable. If the differential database backup D1 is deleted as soon as D2 completes, then the scenario in Figure 2 will not be possible, and the restore sequence will involve all transaction log backups since the full database backup -- a potentially very long restore sequence.
This raises the question of "When should you delete your previous backups?" The answer is definitely "It depends!" If you don't have a legal obligation to keep your backups around for a certain length of time, then it's up to you, and depends on the backup strategy you have and how much disk space is required. Regardless, don't immediately delete previous backups as soon as a new one is taken; it's best to keep at least one or two complete cycles of backups before getting rid of older backups. Ideally, you should test your backups before removing older ones.
For transaction log backups, in general you must have all of them since the last full database backup was taken, as that is the final fall-back restore sequence. If a single transaction log backup from the transaction log backup "chain" is missing or damaged, the restore operation can't proceed past the gap. As I mentioned in the previous article, verifying the integrity of your backups is a key part of being able to restore successfully.
You can find more details on figuring out what you're able to restore in the comprehensive Books Online topic "Working with Restore Sequences for SQL Server Databases".

Example Restore Scenarios

The most common restore scenario involves a full database backup and then one or more transaction log backups to bring the database forward in time. You can do this through SQL Server Management Studio (SSMS) or through Transact-SQL, although there is something you need to be aware of if you're going to use RESTORE commands directly.
When a backup is restored, there are three options for how the restore operation completes and they all relate to the UNDO phase of recovery. As each successive backup in the restore sequence is restored, the REDO phase of recovery is always performed, but the UNDO phase cannot be performed until the very last backup in the transaction log backup chain has been restored. This is because as soon as recovery is completed, no further transaction log backups can be applied, so all restores in the restore sequence must specify not to run the UNDO phase of recovery.
The default, unfortunately, is to run the UNDO phase of recovery -- the equivalent of using the WITH RECOVERY option on the RESTORE statement. When restoring multiple backups, you must be careful that each one specifies WITH NORECOVERY. In fact, the safest way is to use the WITH NORECOVERY option on all restores in the restore sequence, and then manually complete recovery afterward. Here is some example Transact-SQL code to restore a full database backup and two transaction log backups, and then manually complete recovery to bring the database online:

RESTORE DATABASE DBMaint2008 FROM
DISK = 'C:\SQLskills\DBMaint2008_Full_051709_0000.bak'
WITH REPLACE, CHECKSUM, NORECOVERY;
GO
RESTORE LOG DBMaint2008 FROM
DISK = 'C:\SQLskills\DBMaint2008_Log_051709_0100.bak'
WITH NORECOVERY;
GO
RESTORE LOG DBMaint2008 FROM
DISK = 'C:\SQLskills\DBMaint2008_Log_051709_0200.bak'
WITH NORECOVERY;
GO
RESTORE DATABASE DBMaint2008 WITH RECOVERY;
GO

Notice that I also used the CHECKSUM option on the restore of the full database backup to ensure that any page checksums present in the database being restored are verified as they are restored.
If WITH NORECOVERY was not specified on the first RESTORE statement, the following error is returned:
Msg 3117, Level 16, State 1, Line 1
The log or differential backup cannot be restored because no files are ready to rollforward.
Msg 3013, Level 16, State 1, Line 1
RESTORE LOG is terminating abnormally.
You must be very careful to use the right option, otherwise you risk having to start a long restore sequence again -- there's no way to undo recovery once it's completed.
There is, however, an interesting option which kind of does that -- the WITH STANDBY option. This is the last of three options I mentioned earlier. It works by running the UNDO phase of recovery, but it keeps a note of what it did (in an "undo" file whose name and path you specify) and then allows read-only access to the database. The database is transactionally consistent, but you have the ability to continue with the restore sequence. If you decide to continue, the UNDO is reversed (using the contents of the undo file) and then the next transaction log file is restored. This is useful in two scenarios: for allowing read-only access to a log-shipping secondary database and for looking at the contents of the database during the restore sequence.
If the disaster you're recovering from involves accidental deletion of a table, for instance, you may want to do a point-in-time restore. There are several ways to do this, but the most common is where you want to restore the database but ensure that recovery does not proceed past a certain time. In that case you can use the WITH STOPAT option to prevent transaction log restore from going past the time you know the table was deleted. For instance, using the Transact-SQL example above, if I wanted to prevent the database from being restored past 1:45 a.m., I could use the following syntax on the second RESTORE LOG statement:

RESTORE LOG DBMaint2008 FROM
DISK = 'C:\SQLskills\DBMaint2008_Log_051709_0200.bak'
WITH NORECOVERY, STOPAT = '2009-05-17 01:45:00.000';
GO

I could even combine STOPAT and STANDBY to see whether that was the correct point in time and then, if not, restore the same transaction log backup with a time a few seconds later and so on. This kind of operation becomes very tedious, but it may be the only solution if you don't know what time an operation took place.
A comprehensive discussion of these and other options for the RESTORE statement can be found in the Books Online topic "RESTORE Arguments (Transact-SQL)".
One of the coolest new features introduced in SQL Server 2005 Enterprise Edition was partial database availability. This feature allows a multi-filegroup database to be online and available as long as at least the primary filegroup is online. Obviously, data in any offline filegroups can't be accessed, but this feature allows a very large database to be split into separate filegroups for easier and faster recoverability. Another Enterprise-only feature that was added is the ability to perform piecemeal restores (for instance, a single filegroup from a multi-filegroup database) online, while the rest of the database is being used for processing.
These two features combined enable some quite sophisticated and efficient restore scenarios, as long as the database has been architected that way and the correct backups exist.
You'll find an excellent, in-depth SQL Server Technical Article titled "Microsoft SQL Server 2005 Partial Database Availability" with some extensive examples available at tinyurl.com/mbpa65. There's also a 75-minute recording of Kimberly L. Tripp delivering a Tech-Ed EMEA session titled "SQL Server 2005 VLDB Availability and Recovery Strategies" that is well worth watching.

Considerations When Restoring to a Different Location

The simplest restore scenario is when the database is being restored on the same SQL Server instance to which it is usually attached, and with the same name. As you move further away from that scenario, the aftermath of the restore operation becomes more complicated.
If the database is being restored on the same instance, but with a different name, you may need to make changes to elements like DTS/SSIS packages, database maintenance plans, application strings and anything that relies on a database name.
If the database is being restored on a different instance on the same server, things get a lot more complicated:

• The SQL Server logins will be different or may not exist.
• SQL Agent jobs and DTS/SSIS packages will be different or may not exist.
• The master database is different, so any user-defined stored procedures may be missing.
• The SQL Server instance name will be different, so there may be client connectivity issues.
• If the database is being restored on an instance on a different server, everything listed applies, but there may be added security issues as Windows accounts may be different, and they may be in a different Windows domain.
• One other consideration is the edition of SQL Server the database is being restored on. There are some features that, if used in the database, make the database "Enterprise-only" -- it cannot be restored on a Standard Edition (or lower) SQL Server instance.
• In SQL Server 2000 and earlier, this is not an issue. In SQL Server 2005, if table or index partitioning is used, the database is "Enterprise-only." In SQL Server 2008, the feature list is:
• Change data capture
• Transparent data encryption
• Data compression
• Partitioning

All of these require sysadmin privileges to enable except data compression, which can be enabled by a table owner, thus potentially breaking a disaster-recovery plan involving restoring to a Standard Edition instance. You can tell whether any of these features are being used in the database using the DMV sys.dm_db_persisted_sku_features and adjust your disaster-recovery plan accordingly.

Dig Deeper

Just as with the first article in the series on backups, there are lots of facets of restore operations that I didn't have space to cover. Now that you know the basics, however, you can dive into some of the Books Online and blog links for deeper information. The best place to start in Books Online is the topic "Restore and Recovery Overview (SQL Server)". You can also find a lot of information on my blog, starting with the Backup/Restore category.
The main takeaway I'd like you to gain from this article is that to successfully recover a database using backups, you need to practice to make sure you know what to do. You don't want to be learning the syntax for the RESTORE command during a high-pressure disaster-recovery situation. You may also find that your backup strategy does not allow you to recover within your business requirements. Maybe the backups take too long to restore, maybe your log backups are accidentally overwriting each other, or maybe you forgot to back up the server certificate used to enable transparent database encryption in SQL Server 2008.
By far the best way to prepare for disaster is to have a restore plan that lists the steps to go through, and to have a set of scripts that will help identify what backups exist and the order in which to restore them. I always like to say that this should be written by the most senior DBA on the team and tested by the most junior DBA -- to ensure that everyone can follow the steps safely. However, if you're an involuntary DBA, you're going to need to put a plan together yourself and make sure you can follow it.
In the next article, I'll explain how to recover from database corruption if you don't have any backups, and why you might choose to run a repair operation even if you do have backups.
In the mean time, and as always, if you have any feedback or questions, drop me a line -- Paul@SQLskills.com.

Thanks to Kimberly L. Tripp for providing a technical review of this article.
Paul S. Randal is the managing director of SQLskills.com, a SQL Server MVP and Microsoft regional director. He worked on the SQL Server Storage Engine team at Microsoft from 1999 to 2007. Randal wrote DBCC CHECKDB/repair for SQL Server 2005 and was responsible for the Core Storage Engine during SQL Server 2008 development. Randal is an expert on disaster recovery, high availability and database maintenance and is a regular presenter at conferences around the world. He blogs at SQLskills.com/blogs/paul and is on Twitter as @PaulRandal.

Writing monitors to target Logical or Physical Disks

This is something a LOT of people make mistakes on – so I wanted to write a post on the correct way to do this properly, using a very common target as an example.
When we write a monitor for something like “Processor\% Processor Time\_Total” and target “Windows Server Operating System”…. everything is very simple.  “Windows Server Operating System” is a single instance target…. meaning there is only ONE “Operating System” instance per agent.  “Processor\% Processor Time\_Total” is also a single instance counter…. using ONLY the “_Total” instance for our measurement.  Therefore – your performance unit monitors for this example work just like you’d think.

However – Logical Disk is very different.  On a given agent – there will often be MULTIPLE instances of “Logical Disk” per agent, such as C:, D:, E:, F:, etc…   We must write our monitors to take this into account. 
For this reason – we cannot monitor a Logical Disk perf counter, and use “Windows Server Operating System” as the target.  The only way this would work, is if we SPECIFICALLY chose the instance in perfmon.  I will explain:

Bad example #1:
I want to monitor for the perf counter Logical Disk\% Free Space\ so that I can get an alert when any logical disk is below 50% in free space.
I create a new monitor > unit monitor > Windows Performance Counters > Static Thresholds > Single Threshold > Simple Threshold. 

I target a generic class, such as “Windows Server Operating System”.
I choose the perf counter I want – and select all instances:

And save my monitor.
The problem with this workflow – is that we targeted a multi-instance perf counter, at a single instance target.  This workflow will load on all Windows Server Operating Systems, and parse through all discovered instances.  If an agent only has ONE instance of “Logical Disk” (C:) then this monitor will work perfectly…. if the C: drive does not have enough free space – no issues.  HOWEVER… if an agent has MULTIPLE instances of logical disks, C:, D:, E:, AND those disks have different threshold results… the monitor will “flip-flop” as it examines each instance of the counter.  For example, if C: is running out of space, but D: is not… the workflow will examine C:, turn red, generate an alert, then immediately examine D:, and turn back to green, closing the alert. 
This is SERIOUS.  This will FLOOD your environment with statechanges, and alerts, every minute, from EVERY Operating System.
A quick review of Health Explorer will show what is happening:
This monitor went “unhealthy” and issued an alert at 10:20:58AM for the C: instance:

Then went “healthy” in the same SECOND from the _Total Instance:

Then flipped back to unhealthy, at the same time – for the D: instance.


I think you can see how bad this is.  I find this condition all the time, even in “mature” SCOM implementations… it just happens when someone creates a simple perf threshold monitor but doesn't understand the class model, or multi-instance perf counters.  In an environment with only 500 monitored agents – I can generate over 100,000 state changes – and 50,000 alerts, in an HOUR!!!!

Ok – lesson learned – DONT target a single-instance class, using a multi-instance perf counter.  So – what should I have used?  Well, in this case – I should use something like “Windows 2008 Logical Disk”  But we can still screw that up!  :-)

Bad example #2:
I want to monitor for the perf counter Logical Disk\% Free Space\ so that I can get an alert when any logical disk is below 20% in free space.
I create a new monitor > Unit monitor > Windows Performance Counters > Static Thresholds > Single Threshold > Simple Threshold.

I have learned from my mistake in Bad Example #1, so I target a more specific class, such as “Windows Server 2008 Logical Disk”.
I choose the perf counter I want – and select all instances:

And save my monitor.
Ack!  The SAME problem!  Why????
The problem is – now, instead of each Operating System instance loading this monitor, and then parsing and measuring each instance, now EACH INSTANCE of logical disk is doing the SAME THING.  This is actually WORSE than before…. because the number of monitors loaded is MUCH higher, and will flood me with even more state changes and alerts than before.
Now if I look at Health Explorer – I will likely see MULTIPLE disks have gone red, and are “flip-flopping” and throwing alerts like never before.


When you dig into Health Explorer – you will see – that they are being turned Unhealthy – and it isn't event their drive letter!  I will examining the F: drive monitor:
I can see it was turned unhealthy because of the free space threshold hit on the D: drive!

and then flipped back to healthy due to the available space on the C: instance:

This is very, very bad.  So – what are we supposed to do???

We need to target the specific class (Windows 2008 Logical Disk) AND then use a Wildcard parameter, to match the INSTANCE name of the perf counter to the INSTANCE name of the “Logical Disk” object.  Make sense?  Such as – match up the “C:” perf counter instance – to the “C:” Device ID of the Logical Disk discovered in SCOM.  This is actually easier than it sounds:

Good example:

I want to monitor for the perf counter Logical Disk\% Free Space\ so that I can get an alert when any logical disk is below 20% in free space.
I create a new monitor > Unit monitor > Windows Performance Counters > Static Thresholds > Single Threshold > Simple Threshold.

I have learned from my mistake in Bad Example #1, so I target a more specific class, such as “Windows Server 2008 Logical Disk”.
I choose the perf counter I want – and INSTEAD of select all instances, I learn from my mistake in Bad Example #2.  Instead – this time I will UNCHECK the “All Instances” box, and use the “fly-out” on the right of the “Instance:” box:


This fly-out will present wildcard options, which are discovered properties of the Windows Server 2008 Logical Disk class.  You can see all of these if you viewed that class in discovered inventory.  What we need to do now – is use discovered inventory to find a property, that matches the perfmon instance name.  In perfmon – we see the instance names are “C:” or “D:”

In Discovered Inventory – looking at the Windows Server 2008 Logical Disk, I can see that “Device ID” is probably a good property to match on:


So – I choose “Device ID” from the fly-out, which inserts this parameter wildcard, so that the monitor on EACH DISK will ONLY examine the perf data from the INSTANCE in perfmon that matches the disk drive letter.


The wildcard parameter is actually something like this:

$Target/Property[Type="MicrosoftWindowsLibrary6172210!Microsoft.Windows.LogicalDevice"]/DeviceID$

This simply is a reference to the MP that defined the “Device ID” property on the class.

Now – no more flip-flopping, no more statechangeevent floods, no more alert storms opening and closing several times per second.


You can use this same process for any multi-instance perf object.  I have a (slightly less verbose) example using SQL server HERE.

To determine if you have already messed up…. you can look at “Top 20 Alerts in an Operational Database, by Alert Count” and “Historical list of state changes by Monitor, by Day:” which are available on my SQL Query List.  These should indicate lots of alerts, and monitor flip-flop, and should be investigated.

Your Windows 7 Deployment Guide

Chris Adams

Depending on the size of your enterprise, the complexity of deploying Windows 7 ranges from fairly simple to insanely complex. For Microsoft's own deployments, the task has been falling closer to the latter description. However, using System
Center, Configuration Manager 2007's Operating System Deployment (OSD) feature and the soon-to-be-released Service Pack 2 (SP2) can greatly simplify the process. No matter where your enterprise falls on the complexity spectrum, you can use the Microsoft blueprint to move your enterprise to Windows 7 sooner rather than later.
This article describes how the Microsoft team approached deploying Windows 7 to our own enterprise. It covers how we developed our Windows 7 deployment solution and how you can apply these same tools to simplify your

Understanding Enterprise Desktop Deployment Scenarios

The first step, of course, is assessing the requirements and scenarios needed to accomplish a wide distribution of a new OS. Unlike applications that are delivered to the desktop, operating systems pose a significant risk to users' productivity and data. Thus, you often spend a great deal of time trying to ascertain the current state of affairs and attempting to minimize risks while finding the sweet spot. That's especially the case at Microsoft, where the target is 280,000-plus desktops.

Figure 1: Enterprise Desktop Disk Configuration Scenarios

When developing an enterprise desktop solution, the primary focus areas are hard-disk configurations, encryption technologies, applications and user data. (While drivers for hardware devices also play a big role in deployment, they're beyond the scope of this discussion. However, the solution we developed to simplify drivers for Microsoft's own hardware is outlined in this TechNet blog post: tinyurl.com/kog748.)
The desktop configuration, whether involving single or multiple disks, is important when deploying Windows 7 using Configuration Manager. The challenge is understanding whether you're deploying Windows 7 only to new machines or via a migration process where an OS already exists. As shown in Figure 1, we had to develop a solution for Microsoft that works with a single disk with a number of partitions or with complex multi-boot and multi-disk configurations.
Many desktops at Microsoft are running Windows Vista, and some business groups are required to use the BitLocker encryption technology introduced with Vista. BitLocker encrypts and protects the system partition in case of theft or loss. When trying to upgrade the OS that's running on an encrypted drive, the task sequence must disable or suspend the encryption. Most mobile environments require encryption, so any solutions developed for them must take encryption scenarios into account.
One absolute, non-negotiable fact in every enterprise is that users have data that can't be lost during migration. For Windows users, Microsoft provides the User State Migration Toolkit (USMT), which is designed to simplify the gathering and restoring of users' data. Windows 7 introduces the next generation of USMT (version 4.0), which offers vast improvements over its predecessor. The primary differences between USMT 3.0 and version 4.0 are the focus of the last part of our scenarios: selecting the right process for gathering the user state and the method for saving that state.
Unlike previous iterations, USMT 4.0 works outside the full operating system. When gathering a user's state for restoration after the upgrade, the full OS poses some challenges because files are often in use or locked; also, other applications, such as anti-virus solutions, can cause failures during attempts to back up their data. The new version's data-gathering process works outside the full OS in environments such as Windows Pre-Execution (PE), greatly reducing the number of running services, applications in use and other scenarios involving open user data. The ability to load a user's state from Windows PE (via offline backup) works nicely with the Configuration Manager OSD task sequence that runs in Windows PE, streamlining the process of backing up the user's data.
After USMT has gathered the necessary user data, it needs somewhere to "store" this data during the migration. Microsoft had numerous choices for where to store its users' data, but that same range of options may not be within many other enterprises' IT budgets. For example, let's assume that the typical Microsoft user state was around 1GB. Options for storing this data for later retrieval included external hard disks, file servers and optical drives such as DVDs. For enterprises using Configuration Manager 2007, the State Migration Point role allows for storing the data on a remote server while the migration takes place, but that still has limitations.
The key reason that an "external" device isn't cost-effective is that it requires you to have physical storage space equal to your users' state. Thus, at Microsoft, we would have needed 280TB space free and available to support our user base. That approach only works if you can accurately gauge the amount of user data. Otherwise, it's a non-scientific process that's likely to produce unpredictable and unsatisfactory results.
The last option is using the machines where the migration is to take place. This may seem like the most logical and cost-effective approach, but it's often a technical challenge. It requires that users' machines have a large amount of free space available for backing up their data; it also involves copying users' data from one place on disk to another. As most IT veterans have learned, it takes awhile to move or copy any file onto a hard disk -- which, in turn, causes deployments to simply take too long. A deployment that takes away a user's productivity for a day (or days) simply isn't feasible -- and it represents a risk that you don't want to take.
USMT 4.0 introduces support for hard links, a feature that wasn't available in previous versions. Hard-link migration allows users' data to get stored locally on the same computer efficiently, requiring little time and disk space.
The only requirement for using hard links is that the user's computer must have 250MB of free space available. Hard-links support offers the ability to back up and restore without moving the physical files on disk. Instead, USMT stores only the pointers to the physical files and uses these pointers at restore time, significantly reducing the amount of time required for Windows 7 migration. (For more information on hard-links, see this TechNet article: tinyurl.com/m76dxv.)
Having an understanding of the various scenarios will help you build an action plan to tackle them when migrating to Windows 7.

Building Your OS Task Sequence for Windows 7

Once you understand the requirements and scenarios, implementing the solution is a straightforward task. The steps for building your solution fall into three primary categories:

1. End-user experience
2. Building the Windows 7 task sequence
3. User-state migration

Microsoft IT developed a solution, code-named "Modena," that enables the ability to support the first category by using a powerful OSD wizard, the second category through an exported task sequence and the third with state-migration scripts. This next section will outline how to use the Modena OSD tools, which include the OSD wizard, an exported task sequence and scripts.

Using the OSD Wizard

Configuration Manager's OSD is designed for IT administrators. For that reason, OSD doesn't offer a built-in wizard, which means that most enterprises must develop their own. In short, there's no out-of-box functionality for gathering end-user input using Configuration Manager -- a challenge that Microsoft itself faced.
Microsoft is a user-driven enterprise; all users operate as administrators.
Many frown on Microsoft IT decisions that affect their ability to use all of an OS's features. As a result, Microsoft had to develop a highly robust user experience that obtains as much data as possible from users without overwhelming them or impeding their ability to work.
The Modena OSD wizard bridged the gap at Microsoft and is available to your enterprise. (For information on obtaining OSD tools, see our blog at blogs.technet.com/osd.)
The Modena OSD wizard has two components: the executable and a configuration file. The executable, OSDSetupWizard.exe, is the self-contained user experience written by Microsoft. It's designed to validate that a computer is ready for the Windows 7 migration and also to gather end-user input. The wizard's final role is taking a user's input and setting OSD task-sequence variables.
A key aspect of the wizard is that it's designed to work much like plug-and-play so that it's useful in as many scenarios as possible. It accomplishes that goal through its configuration file. In fact, in complex environments such as Microsoft, the same executable can be used with various different configuration files by using the switch /xml:{osdconffilename.xml}.
For example, in deployments at Microsoft where support for both Run Advertised Programs (RAP) and Preboot Execution Environments (PXE) are required, the same task sequence is used, but the wizard performs differently, using two different configuration files based on the environment it's running (RAP or PXE). This creates a one-size-fits-all deployment package yet allows for various configurations supporting that single package.
To introduce the OSD wizard, let's start by looking at the available end-user screens (also called pages). Eight pages are available as part of the OSD wizard. We use the term "available" loosely because each page has one of three states: enabled, disabled and silent. If enabled, the page is shown to the end user; if disabled, the page isn't displayed.
Silent is a special case in which the page isn't displayed unless an OSD task-sequence variable is null. If that's the case, the page will prompt the user to provide the input allowing the wizard to continue. (For more information on the OSD wizard's enabled, disabled and silent features, see the TechNet blog post at blogs.technet.com/osd.) In some situations, end-user input is required on a specific page, but other data aren't.
For example, many enterprises allow end users to provide their own machine names, but don't allow them to select their Active Directory (AD) domains or organizational units. The OSD wizard adapts easily, allowing you to display a wizard page, but not allowing users to change the content of a specific input such as domain and organizational unit. This helpful locking functionality is available across most pages in the configuration file.
Beyond disabling and locking, some pages have additional attributes that change the wizard's internal behavior. Wizard functionality can automatically check AD to see whether a computer name is already in use or whether a user's credentials are valid. For pages where additional attributes exist, you can use values to enable or disable these features without re-compiling the wizard.
Each of the OSD wizard's eight pages is named for the function it's supposed to provide: Welcome, Pre-Flight, Computer, Network, Language, Volume, Application and Summary. Let's explore the key functionality that some of these pages provide and discuss how you can use them.
You can uniquely brand your wizard so that it matches your enterprise's typical IT branding. You can also easily accomplish branding through the configuration file by placing the bitmap name in the header attribute. To re-brand the wizard to match your environment, simply create a bitmap image with the size 630x100, add the image to your OSD package and edit the configuration file. (For more information on branding, see the TechNetblog post at tinyurl.com/r7jdve.)
Among the OSD Wizard's most powerful features is the ability to snap in your own pre-flight checks that execute prior to the migration to Windows 7. For example, let's say that your company has a human-resources application that isn't compatible with Windows 7. To minimize impact on user productivity, you create a script and add your OSD wizard pre-flight check on whether this application is installed. Based on the result of that check, you allow the users in question to continue the migration or warn them about the incompatible applications.
The OSD Wizard currently has two built-in pre-flights that you can enable or disable via the configuration file. These particular pre-flights are included because they're applicable in most enterprises. The first pre-flight is a power check that's performed in the full OS (for example, when the user migrates using RAP or Add\Remove programs) and returns an error notification if the users aren't running on AC. If the pre-flight detects that the user isn't plugged in, it will return an error notification asking the user to plug in an AC adaptor. After doing so, the user can click Retry Pre-Flight Checks and proceed, if no other errors occur.
The second built-in pre-flight is the wireless check. OSD is a bandwidth-intensive process and runs best when plugged into an Ethernet adaptor (for example, 802.3 wired connections). When the wireless pre-flight detects that a user isn't plugged in, it returns an error notification until the user establishes a wired connection to the network.
However, the pre-flight isn't constrained to the built-in checks. It supports any executable or Windows Scripting Host scripts, such as Visual Basic scripts. There's no limit to the number of pre-flight checks as long as they can execute and complete in less than five minutes (if they take longer, the OSD Wizard stops executing the script).

Figure 2 OSD Wizard
Every time a script or executable is executed, a code is returned to the OSD Wizard process. Based on the wizard's configuration, it will return a Success, Warning or Error status notification (see Figure 2). When a Success or Warning notification is returned, the user can continue through the rest of the wizard. But when an Error notification is returned, the user will be blocked from continuing. The acceptable codes returned from your pre-flight script, or from the built-in scripts, are configurable in the osdconf.xml and don't require changes to the wizard's executable. Beyond that, the text description of the error is also configurable.
There are two approaches for delivering applications as part of your Windows 7 deployment: as part of the base Windows 7 Windows Installation Image (WIM) or as individual task-sequence steps. Including the applications in the Windows 7 WIM image is often used for applications that most end users require, but that aren't updated often. This approach has two primary deficiencies. First, the image size increases, which often affects the times for downloading to clients; in addition, there's the administration of the image. Each time the applications are updated, the WIM image will require you to create a new installation image and update the appropriate Configuration Manager packages associated with the base image.

Figure 3 OSD Wizard Application Selection
For those reasons, the Modena tools integrate nicely with the Configuration Manager 2007 Install Software application feature to provide end users with the ability to select the applications they'd like to install as part of the OSD process. (For more information, see the TechNet article at tinyurl.com/pdfp5s.) As shown in Figure 3, the applications are listed in a tree-like view, based solely on your design in the OSD Wizard configuration file. For example, you can define the application group by business unit, location or application type, then define all the applications for that group and the default selection. This makes it possible to easily change or add applications, with the only requirement being that the application is packaged and available in the Configuration Manager 2007 database.
Again, the wizard's primary purpose is providing users with the ability to affect the final outcome of their Windows 7 image. For environments where users aren't expected to have much input, the wizard can be configured to ask for only minimal information; the remainder is hard-coded in the task sequence by administrators.

Losing Data: Not an Option

USMT 4.0 includes a base set of configuration files capable of capturing many enterprises' user states. These configuration files, MigApp.xml and MigDocs.xml, cover most scenarios to capture your users' data. (For more information on configuration files, see the TechNet article at tinyurl.com/okfgw4.)
OSD will clean the volume where Windows 7 is being installed. For that reason, it's extremely important that user states are captured properly and accurately every time that OSD is used. In short, data loss is simply not an option.
The best practice -- also used here at Microsoft -- is creating a safe location via a task sequence step that's your target folder for your user's state. Then, using the OSD built-in task sequence variable called OSDStateStorePath, you can exclude this directory from the volume clean done by the Apply OS step in the task sequence. (To learn how to accomplish and fully use this functionality, see the TechNet blog post at blogs.technet.com/osd.)
Building Your Task Sequence for Windows 7 Deployment
The Modena OSD tools include an exported copy of the OSD task sequence used at Microsoft. The exported task sequence is broken down into several groups, starting at the root, with "children" named Master Group and Failover. The Master Group has children associated with each major step used in the Windows 7 deployment. The children (see Figure 4) each have error conditions that return the error to the Master Group, which will then forward to a specialized step called Failover.
Breaking the task sequence into these steps makes the logging, reporting and error-handling processes much easier. As with any such undertaking, errors will occur no matter how much time you spend in preparation -- which is the reason for the Failover Group. This group is designed to gather all the pertinent log files necessary to troubleshoot a failed installation and place them in the OSD safe folder.
To implement some fault tolerance in your task sequence, start at the root level and create the two child groups, Master Group and Failover. Another best practice is having the Master Group include all the working steps required in your deployment and using it as the group designated to continue on error. Each child group -- in Microsoft's case, the five nested groups for the Master Group, shown in Figure 4 -- are needed to continue the task sequence and are set to fail on error.

Figure 4 Task Sequence Grouping
Another best practice is to set any child groups that are non-catastrophic to the deployment to continue on error. The OSD task sequence engine always determines actions to take in case of error by looking to its parent group to see whether it should continue. Because of this behavior, you should define whether each step should halt on error or continue. If a step isn't set to continue on error, the task sequence will go to its parent group to determine what action to execute next. In the design in Figure 4, the Master Group is set to continue on error so that it will execute the Failover Group that is its peer.
For example, let's say that a group is created to capture user state. Because of the importance of the steps in this group, the design for each one ensures, in case of error, deployment doesn't continue further, possibly deleting the volume (assuming that that's the next step in your task sequence). In such a case, the task- sequence engine determines whether the executing step is set to continue on error; if not, it will revert to the step's parent group to determine what action to take next. The Backup State parent group whose continue on error is unchecked will revert to its parent, who in this design is the Master Group.
The purpose of the Failover Group, as mentioned, is to ensure that we capture all the data necessary to troubleshoot in case of failures. This group needs to be set up as a peer to the Master Group to have the Master Group refer to it upon errors (see Figure 4). The Failover Group is never run unless there is a catastrophic failure during deployment. For that reason, it's the last step in the task sequence and is always run unless setup succeeds.
The state of the deployment is based on the value stored in the _SMSTSLastActionSucceeded variable. That's how the task sequence engine is used to transverse the task-sequence "tree" until it ultimately reaches completion -- in this case, meaning that the Failover Group steps are executed, gather all logs and necessary data and then fail. (For more on designing this, read the TechNetblog post at blogs.technet.com/osd.)

Providing User Status Using Bitmaps and BGInfo.exe

At Microsoft, it's important to let users know where in the migration process they are in OSD. By default, all client-based status is communicated through the Configuration Manager OSD features. While these status messages work well for some enterprises, there are other creative methods for communicating to users about the larger steps making up the migration.
The first step is understanding what the larger steps are that accompany the migration process. For example, these steps could be Partition Drive, Install Windows and Last Install applications. The primary reason for doing this is that these steps provide users with a high-level definition of overall progress -- and users enjoy this type of information.
The five main steps used at Microsoft are Backup State, Install Windows, Set Up Windows, Install Applications and Restore State. Users receive communications through the use of static bitmaps that are dynamically rendered using the TechNet Sysinternals tool BGInfo.exe. (You can download BGInfo.exe at tinyurl.com/2nbxmd, but it's included in the Modena OSD tools.) This tool allows the loading of a static bitmap when called, and it will allow you to set the bitmap that reflects the migration's current location.
Modena includes five bitmaps, one for each step, and the images that communicate the status are set through the task sequence. For example, in the task-sequence group Install OS, BGInfo.exe is called to load the bitmap representing this step (see Figure 5).

Figure 5 Displaying Status

The Modena OSD tool provides the framework for deploying this in your enterprise and only requires that you replace the bitmaps. To do this, locate the scripts folder, open BG directory and replace each bitmap image with your updated version. This will inform your end users about the five primary steps used to deploy Windows 7. This is a nice way to keep users in the loop without requiring them to read detailed steps provided by the OSD task-sequence engine.

Putting It All Together

Windows 7 is available to businesses now, but due to the complexity involved, many enterprises haven't yet started their upgrade projects. At Microsoft, this process began nearly a year ago; we've almost mastered it, yet it's never completely refined.
Windows 7 provides your end users maximum productivity and performance. Ultimately, the only barrier is getting the deployment ready. Microsoft offers System Center Configuration Manager 2007 users the Modena OSD tools to decrease deployment preparation time and complexity. Whether your enterprise needs high interaction with its end users or relies on "minimal touch," you can streamline your Windows 7 project using the same process that we used at Microsoft.


Chris Adams (chrad@microsoft.com) is a senior lead program manager in Microsoft's Management and Services Division. He focuses on System Center Configuration Manager and System CenterVirtual Machine Manager.

System Center Operations Manager: Version Control

The current version of System Center Operations Manager 2007 doesn’t use version control for management packs. If you manage complex environments and use lot of custom monitoring, it would be nice to see when there is something changed and even better: easily fall back to your previous version in case an error slipped in. Another benefit of version control is you have exported management packs ready to import in other environments: i.e. between acceptance and production or between customers in case you are managing multiple customers like we do.

I’ve written some scripts to automatically export all your unsealed management packs if there is something changed, write a version number and email the new managementpack with a summary of all differences.
Another script will import your version controlled management packs (if there is something changed).
Scripts are updated on 12-11-2009

To setup version control for Management Packs, you need the following:
- ExportUnsealedMPs.ps1
- ExportUnsealedMPs_Simple.ps1
- ImportMPs.ps1
- SCOMSettings.xml
- A folder structure to save management packs
- SVN and TortoiseSVN
Folder Structure
SVN\SCOM
Scripts
SVNControlledMPs
UnsealedMPs
– diff
– temp
– test1
– test2
SVN
SVN is needed to do commits directly from within the script. TortoiseSVN helps you to commit, update and compare files within the Explorer.
Setup
After you setup SVN and Tortoise SVN and created the folder structure. You can create scheduled tasks to run the import and export jobs. It is a good idea to run the export at the end of the day to capture all changes of that day, I export them at 19:00.
Best time to import the management packs is 13:30, you don’t want to get suprised at night if some import causes pages.
First run ExportUnsealedMPs_Simple.ps1 to get all unsealed management packs without version control.
Commit your Unsealed Folder in SVN (before you do that, make sure you have marked xml files as text:
Edit Tortoise Settings file, add the following lines
Under [miscellany]:
enable-auto-props = yes
Under [auto-props]:
*.xml = svn:mime-type=text/xml;svn:eol-style=native;svn:keywords=Rev Date
Run the export job, it should not export anything, since you are up-to-date.
Change something in a management pack, update a description will do.
Run the export job again, you should see it is updated, committed and an email is sent to the address mentioned in your settings file..
ExportUnsealedMPs.ps1

#########################################################################################
# #
# Author: Jan Jacob Bos #
# Date: 17-01-2009 #
# Version: 1.5 #
# #
# Usage: ExportUnsealedMPs #
# #
# Purpose: Script to export unsealed management packs with version control #
# Only if something is changed the version number is increased and #
# management pack is saved, committed in SVN and emailed #
# #
#########################################################################################
# initialize settings from config file
$xmlSettings=[xml](get-content “.\SCOMSettings.xml”)
$ServerName = $xmlSettings.SCOMSettings.General | %{$_.RootMS} | select-object -unique
$targetFilePath = $xmlSettings.SCOMSettings.General | %{$_.ExportFolder} | select-object -unique
$SMTPServer = $xmlSettings.SCOMSettings.Mail | %{$_.SMTPServer} | select-object -unique
$From = $xmlSettings.SCOMSettings.Mail | %{$_.From} | select-object -unique
$To = $xmlSettings.SCOMSettings.Mail | %{$_.To} | select-object -unique
#Initializing the Ops Mgr 2007 Powershell provider
add-pssnapin “Microsoft.EnterpriseManagement.OperationsManager.Client” -ErrorVariable errSnapin ;
set-location “OperationsManagerMonitoring::” -ErrorVariable errSnapin ;
new-managementGroupConnection -ConnectionString:$Servername -ErrorVariable errSnapin ;
set-location $Servername -ErrorVariable errSnapin ;
# initializing variables
$Differences1 = “”
$Differences2 = “”
$VersionFirstpartCurrent = 0
$RevisionCurrent = 0
$VersionFirstpartFile = 0
$RevisionFile = 0
# Defining Functions
function ExternalCommand([string]$Command, [string]$Parameter)
{ $StartInfo = new-object System.Diagnostics.ProcessStartInfo
$StartInfo.FileName = $Command
$StartInfo.Arguments = $Parameter
$StartInfo.LoadUserProfile = $true
$StartInfo.UseShellExecute = $false
# Execute command
$proc = [System.Diagnostics.Process]::Start($StartInfo)
# Wait for command to finish
$proc.WaitForExit()
}
function GetRevision {param ([string]$VersionNo, [REF]$VersionFirstpartOUT, [REF]$RevisionOUT)
# look for last dot in version number (is always in format: 1.0.0.0) we only change last number (revision)
[int]$count = $VersionNo.length – ($VersionNo.LastIndexOf(”.”) + 1)
[int]$LastDot = $VersionNo.length – $count
$VersionFirstpartOUT.Value = $versionNo.substring(0, $LastDot)
[int]$RevisionOUT.Value = $VersionNo.substring($LastDot , $count)
}
function SendMail {param ([string]$Subject, [string]$Message, $Attachment)
# send mail
$msg = new-object Net.Mail.MailMessage
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$msg.From = $From
$msg.To.Add($To)
$msg.Subject = $Subject
$msg.Body = $Message
if ($Attachment -ne “”)
{ $att = new-object Net.Mail.Attachment($Attachment)
$msg.Attachments.Add($att)
}
$smtp.Send($msg)
if ($Attachment -ne “”) {$att.Dispose()}
}
# get all unsealed management packs
$mps = get-managementpack | where-object {$_.Sealed -eq $false -and $_.ContentReadable -eq $true}
foreach($mp in $mps)
{ $ID = $mp.Name
$FileName = $ID + “.xml”
# Determine Revision Number from current MP
GetRevision $mp.version ([REF]$VersionFirstpartCurrent) ([REF]$RevisionCurrent)
# Retrieve the MP currently stored
# trap {”File not found ” -f $_.Exception.Message}
# $fileExists = [system.IO.File]::Exists($targetFilePath + “\” + $Filename)
# If File doesn’t exist, it is a new management pack
# write-host “Filename: $fileExists ”
if (-not [system.IO.File]::Exists($targetFilePath + “\” + $Filename))
{ # write-host “Filename (within IF) $ID Version: $VersionFirstpartCurrent$RevisionCurrent”
$NewFileName = $targetFilePath + “\” + $FileName
export-managementpack -managementpack $mp -path $targetFilePath
ExternalCommand (”"”C:\Program Files\SVN\bin\svn.exe”"”) (” add “”$NewFilename”"”)
ExternalCommand (”"”C:\Program Files\SVN\bin\svn.exe”"”) (” commit -m “”$ID Version: $VersionFirstpartCurrent$revisionCurrent”" “”$NewFilename”"”)
# Send mail a new managementpack has been exported
write-host “a new managementpack has been exported”
$MailContent = “A new Management Pack: $Filename with Version $VersionFirstpartCurrent$revisionCurrent has been exported”
SendMail “New managementpack $Filename on $Servername in repository” $MailContent “”
}
else
{
$FileSavedMP = [system.IO.Directory]::GetFiles($targetFilePath, $Filename)
# write-host “Filename: $FileSavedMP”
# we need the file in xml format to get version info
$xmlSavedMP=[xml](get-content $FileSavedMP)
# we need the the file in text format to compare it with the current version
$txtSavedMP=(get-content $FileSavedMP)
# get version number from xml file
$xmlSavedMP.selectnodes(”/ManagementPack/Manifest/Identity”)|foreach {
$_.selectnodes(”Version”) | foreach {
$versionFile = $_.get_InnerText()
write-host $ID $versionFile
# Determine Revision Number from file
GetRevision $versionfile ([REF]$VersionFirstpartFile) ([REF]$RevisionFile)
# check to see if version number is greater than the saved file
write-host “Revision Current version ” $RevisionCurrent
write-host “Revision Saved Version ” $RevisionFile
# if ($mp.version -gt $versionFile)
if ($RevisionCurrent -gt $RevisionFile)
# Current version number is higher, probably imported from another customer
{ $NewFileName = $targetFilePath + “\” + $FileName
export-managementpack -managementpack $mp -path $targetFilePath
ExternalCommand (”"”C:\Program Files\SVN\bin\svn.exe”"”) (” commit -m “”$ID Version: $VersionFirstpartFile$revisionFile”" “”$NewFilename”"”)
# Send mail a newer version is exported
write-host “a newer version is exported”
$MailContent = “Management Pack: $Filename with Version $VersionFirstpartCurrent$revisionCurrent has been updated in the repository”
SendMail “Updated $Filename on $Servername in repository” $MailContent “”
}
if ($RevisionCurrent -lt $RevisionFile)
{ # Send mail could not export because saved file has higher revision number
write-host “could not export because saved file has higher revision number”
$MailContent = “Management Pack: $Filename with Version $VersionFirstpartFile$revisionFile could not be exported”
SendMail “coud not export because saved file has higher revision number on $Servername” $MailContent “”
}
# if version number is equal then we need to check if mgmt pack is changed
if ($RevisionCurrent -eq $RevisionFile)
{ write-host “check if MP is changed”
$TargetFilePathTemp = $targetFilePath + “\temp”
export-managementpack -managementpack $mp -path $targetFilePathTemp
$FileCurrentMP = [system.IO.Directory]::GetFiles($targetFilePathTemp, $Filename)
$xmlCurrentMP = [xml](get-content $FileCurrentMP)
$txtCurrentMP = (get-content $FileCurrentMP)
# Check if MP is changed
# write-host “Current MP Size” $FileCurrentMP.Size
# Write-host “Saved Mp Size” $FileSavedMP.Size
$TargetFilePathTest1 = $targetFilePath + “\test1\”
$TargetFilePathTest2 = $targetFilePath + “\test2\”
set-content -Encoding “Unicode” -Path $TargetFilePathTest1$Filename $txtCurrentMP
set-content -Encoding “Unicode” -Path $TargetFilePathTest2$Filename $txtSavedMP
# set-content -Path $TargetFilePathTest1$Filename $txtCurrentMP
# set-content -Path $TargetFilePathTest2$Filename $txtSavedMP
compare-object $txtCurrentMP $txtSavedMP |foreach{
if ($_.SideIndicator -eq “< =") {[string]$Differences1 = $Differences1 + $($_.InputObject)}
if ($_.SideIndicator -eq "=>“) {[string]$Differences2 = $Differences2 + $($_.InputObject)}
}
# if difference is empty, there are no changes
if ($Differences1 -ine “” -or $Differences2 -ine “”)
{ # MP is changed, we need to increment the version number
write-host “Changes found”
# read information from management pack
$xmlCurrentMP.selectnodes(”/ManagementPack/Manifest/Identity”)|foreach {
$_.selectnodes(”Version”) | foreach {
[int]$revision = $revisionFile
$revision++
$NewVersion = $VersionFirstpartFile + $revision
write-host “new version $NewVersion”
# write new version number to file
$_.set_InnerText($NewVersion)
$NewFileName = $targetFilePath + “\” + $FileName
$xmlCurrentMP.save($NewFileName)
# write new version number to managementpack
$mp.set_Version($NewVersion)
$mp.AcceptChanges()
# new-item -ItemType file -force $NewFileName -value $xmlCurrentMP
# $xmlCurrentMP | Export-Clixml $NewFileName
# set-content -Encoding “Unicode” -Path $NewFileName $xmlCurrentMP
# commit changes in SVN
ExternalCommand (”"”C:\Program Files\SVN\bin\svn.exe”"”) (” commit -m “”$ID Version: $VersionFirstpartFile$revision”" “”$NewFilename”"”)
[string]$DiffFile = $targetFilePath + “\diff\$ID.txt”
# Get differences from SVN
ExternalCommand (”cmd.exe”) (”/c “”C:\Program Files\SVN\bin\svn.exe”" diff $NewFilename -r PREV >$DiffFile”)
# get changes since last version stored in SVN
[string]$DiffContentent = (get-content $DiffFile)
# send mail
$MailContent = “Management Pack: $Filename Version MP: $NewVersion $DiffContentent”
SendMail “New updates available on $Servername” $MailContent $NewFileName
# clean up
$Differences1 = “”
$Differences2 = “”
}
}
}
}
}
}
}
}

ExportUnsealedMPs_Simple.ps1

param ($ServerName)
add-pssnapin “Microsoft.EnterpriseManagement.OperationsManager.Client”;
set-location “OperationsManagerMonitoring::”;
new-managementGroupConnection -ConnectionString:$ServerName;
set-location $ServerName;
$mps = get-managementpack | where-object {$_.Sealed -eq $false}
foreach($mp in $mps)
{
export-managementpack -managementpack $mp -path “E:\SVN\SCOM\UnsealedMPs”
}

ImportMPs.ps1

#########################################################################################
# Date: 17-01-2009        �
# Version: 1.0         �
#          �
# Usage:    ImportMPs         �
#         �
# Purpose:  Script to import management packs with version control  �
#     Only if a newer version is available the management pack is  �
#  imported and an email is sent     �
#           �
#########################################################################################
# initialize settings from config file
$xmlSettings=[xml](get-content “.\SCOMSettings.xml”)
$ServerName = $xmlSettings.SCOMSettings.General | %{$_.RootMS} | select-object -unique
$Folder = $xmlSettings.SCOMSettings.General | %{$_.SVNControlledFolder} | select-object -unique
$SMTPServer = $xmlSettings.SCOMSettings.Mail | %{$_.SMTPServer} | select-object -unique
$From = $xmlSettings.SCOMSettings.Mail | %{$_.From} | select-object -unique
$To = $xmlSettings.SCOMSettings.Mail | %{$_.To} | select-object -unique
add-pssnapin “Microsoft.EnterpriseManagement.OperationsManager.Client”;
set-location “OperationsManagerMonitoring::”;
new-managementGroupConnection -ConnectionString:$ServerName;
set-location $ServerName;
$SmtpClient = new-object system.net.mail.smtpClient
$SmtpClient.host =
$SmtpServer
$Files = [system.IO.Directory]::GetFiles($Folder, “*.xml”)
foreach($File in $Files) {
Write-host “Importing ” $File
$xml=[xml](get-content $File)
$xml.selectnodes(”/ManagementPack/Manifest”)|foreach {
$_.selectnodes(”Identity/Version”) | foreach {
# Check version number MP to import
$versionFile =  $_.get_InnerText()
[int]$countFile = $versionfile.length – ($versionFile.LastIndexOf(”.”) + 1)
[int]$LastDotFile = $versionfile.length – $countFile
[int]$revisionFile = $versionFile.substring($LastDotFile , $countFile)
}
$_.selectnodes(”Name”) | foreach {
$NameFile = $_.get_InnerText()
}
}
write-host “versionFile” $revisionFile
write-host “NameFile” $NameFile
$mps = get-managementpack | where-object {$_.displayname -like $NameFile}
foreach($mp in $mps) {
# Check version number current MP
[string]$VersionMP = $mp.version
[int]$countMP = $versionMP.length – ($versionMP.LastIndexOf(”.”) + 1)
[int]$LastDotMP = $versionMP.length – $countMP
[int]$revisionMP = $versionMP.substring($LastDotMP , $countMP)
write-host “versionMP” $revisionMP
if($revisionFile -gt $revisionMP){
# “File Newer, we can import”
# need to set trap, to determine if import is successful
trap {   ‘Error Category {0}, Error Type {1}, ID: {2}, Message: {3}’ -f
$_.CategoryInfo.Category, $_.Exception.GetType().FullName,
$_.FullyQualifiedErrorID, $_.Exception.Message;   continue }
install-managementpack $File
if ($error) {
$Title = “Something went wrong with import of $NameFile”
$Body = “Name File $NameFile revision $revisionFile The Error message is: $error[0].Exception.Message”
$SmtpClient.Send($from,$to,$title,$Body) �
}
else {  �
$Title = “New updates are imported on $ServerName”
$Body = “Name File ” + $NameFile + ” revision ” + $revisionFile
$SmtpClient.Send($from,$to,$title,$Body)
}
}
if($revisionFile -lt $revisionMP){
$Title = “Update of management pack failed, because of version conflict”
$Body =  “Name File $NameFile  revision saved file $revisionFile Revision current MP $revisionMP”
$SmtpClient.Send($from,$to,$title,$Body)
}
}
}

SCOMSettings.xml

SCOMSERVER
DRIVE:\SVN\SCOM\UnsealedMPs
<DRIVE>\SVN\SCOM\SVNControlledMPs


MAILSERVER
FROM
TO

SCCM: Forcing a Task Sequence to Rerun

There are well known methods to force an advertisement to rerun – including several add-on tools available for the SMS or SCCM console.  To date, however, there are not equivalent methods to force a task sequence to rerun.  Part of this may be because task sequences are typically thought of as focused on Operating System Deployment (OSD) and rerunning these types of distributions are not as common as rerunning advertisements.


While task sequences are the best solution out there for OS Deployments they are much more flexible than just that – including distributing software in very complex scenarios including support of dynamic decisions during execution, handling reboots, enabling specific sequencing of application deployment, etc.  With this kind of power many organizations are using task sequences for software deployment and the ability to force a sequence to rerun on a selective basis and without having to manually logon to individual clients is crucial.  The process to make this happen is very easy.
First, identify your task sequence by ID.  My test sequence is CEN00027.








Note that in my lab this sequence has already run in the past.

The advertisement for the sequence is set with a mandatory execution time – which resulted in the first run.  No other mandatory times have been added.  Further, the advertisement is set to allow rerunning.  If you had an advertisement set to not rerun you should be able to force it to rerun but this would likel require additional WMI and registry edits.  I haven’t tested that specific scenario.

From here, open WMI on the client system of interest and connect to the root\ccm\scheduler namespace.

Click ‘Enum Classes’, select Recursive and then scroll to the bottom and double click on CCM_Scheduler_History() and then click instances.



In the list that shows up, find the entry that corresponds to your task sequence ID and delete it.

With the deletion made, restart the SMS Agent Host service (CCMExec) on the target client.

In a few minutes, the program balloon will pop up indicating the sequence is about to run again.

The process described is manual but could be automated if desired. 

Configuration Manager 2007 Help File Update Wizard

October Update: The Configuration Manager 2007 Help File Update Wizard can be used to update the locally installed help file used by the Configuration Manager console or to install a stand-alone version of the latest available Configuration Manager 2007 Documentation Library. After installing the documentation help update, the latest available help file can also be accessed through the Windows Start menu. If a Configuration Manager console is installed on the computer that the wizard is started on, the existing help documentation will be updated to the latest available version of the Configuration Manager 2007 Documentation Library. If a Configuration Manager 2007 console is not installed on the computer that the wizard is started on, the latest version of the Configuration Manager 2007 Documentation Library help file can be copied locally. In addition, options are available to run the help file update wizard from the command line. See the product read-me document for more details.

Download: Configuration Manager 2007 Help File Update Wizard

OSD App Tree

OSDAppTree2.5.1.pdfOSD App Tree is a configurable way to allow user interaction during a Configuration Manager (ConfigMgr) Operating System Deployment (OSD) Task Sequence (TS). It presents a tree of applications to the interactive user allowing them to choose which ones to install during the TS. Configuration is based on an XML file that contains all possible applications, their settings, and their organization in the tree presented to the end user.
This is a rebranding of OSD App Chooser with a greatly updated look and feel, tooltips, added customization and branding abilities, along with a relatively major new feature: application dependencies.
Any and all feedback is welcome; the application dependencies and enhanced title area are the direct result of feature requests. Please use the contact form on this blog to let me know what you think and for any comments, feature suggestions, or support requests.

Current Version: 2.5.1, release Nov 20, 2009

Documentation: OSDAppTree2.5.1.pdf

Download (includes OSD App Tree, sample config file, and documentation): OSDAppTree2.5.1.zip

ConfigMgr 2007 SP2 Status Message Documentation

!!!! UPDATE !!!!
I've created a table and view containing this data so you can use it in your reports without having to look up what messageIDs like 11708 means.  That article can be found HERE.

!!!! UPDATE2 !!!!
The original Excel spreadsheet said that %1 corresponds to InsStrValue = 1 and %2 corresponds to InsStrValue = 2, etc...however, because that index field is 0-based, %1 actually corresponds to InsStrValue = 0 and %2 corresponds to InsStrValue = 1, etc.  That should be fixed on v2 of the workbook.


I was graciously given access to some raw data for Config Manager involving the status messages and I've put together a spreadsheet similar to the one Microsoft provides for SMS 2003 HERE that shows the status MessageIDs and the corresponding descriptions.
This downloadable excel workbook (at the bottom) lists all status messages that are generated by System Center Configuration Manager (CM) 2007 SP2. You can find specific status messages by using the filters in the spreadsheet. You can filter messages by message ID, by severity (such as error messages, warning messages), by message source (the component that generates the message, like site control manager), by message text or by symbolic name (a short and descriptive meaning for the message which is much easier to understand at a glance than the whole message text).
Here's an example:


Download this Excel workbook and use it for troubleshooting or for in-depth learning of how ConfigManager 2007 SP2 works.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY, and neither do I.

DOWNLOAD THE WORKBOOK HERE:
CM2007_SP2_StatusMessages_v2.xlsx
NOTE:  This was saved using Excel 2007
!!!! UPDATE !!!!
I've created a table and view containing this data so you can use it in your reports without having to look up what messageIDs like 11708 means.  That article can be found HERE.