Tuesday, September 14, 2010

Multi-Threaded FIM MA Export

The performance of the FIM MA on Export can present quite the deployment challenge.  To review, the FIM MA is very fast on import because it reads the FIM Service database directly.  On export the FIM MA acts like any other FIM Service client, whereby it submits Requests to the FIM Service, which are then processed by MPRs.

You’d think the Request processing and all the associated workflows would be the bottleneck here, and of course a system can be configured as such, but the real problem with FIM MA exports is that the MA processes exports one at a time.

So if you have a large number of pending exports on your FIM MA you can suck it up, run the export run profile then take an extended coffee break, or you could hack around a little bit to push the bottleneck back onto the FIM Service’s SQL database.

PowerShell is my favourite tool lately, and makes for pretty simple multi-threading thanks to the Start-Job cmdlet.  At a high level the approach does the following:

  1. Use CSExport.exe to dump the Pending Exports to a giant XML file
  2. Use an XmlReader to rip through the XML file
  3. Every X number of cs-objects, use Start-Job to process the exports
  4. Inside your Start-Job script, use the FIM PowerShell cmdlets to send the updates to FIM

Benefits of this approach are:

  1. The SQL Server that hosts my FIM Service is pegged, which I take to mean the bottleneck is no longer FIM Sync or FIM Service
  2. Deploying in large environments takes a lot less time

The challenges I’ve hit so far include:

  1. The script is a deployment tool, works great in my hands but tends to fail miserably when handed off to somebody without PowerShell and FIM skills
  2. Passing data to PowerShell jobs in Start-Job is nowhere near as cool as simply piping them in, which is where my expectation is set.  I hope improvements are eventually made here so it becomes as easy as pipeline processing in PowerShell functions.
  3. Some attributes are not easy (or possible) to handle with the FIM PowerShell cmdlets, such as DateTime attributes (no matter how well you format them).  An alternative here would be to use the Quest PowerShell cmdlets for FIM, or to just use the FIM Service web service using the CodePlex client or roll your own.
  4. The FIM MA is able to set the CSEntry anchor on export.  Any approach you take here will not be able to do the same.  The workaround is to immediately import the changes from the FIM Service then run a delta sync.  This will fix up the pending exports. Getting this wrong can result in unnecessary exports that will look like duplicates.
  5. The parent script (the one that creates all the child jobs using Start-Job) will take a lot of memory if the jobs are not removed.  by default they stick around in memory until you remove them, so if the job produced a lot of output then you’ll see a lot of memory used by the parent script.

My plan is to demo this script next month at TEC, and hopefully polish it up to share it afterwards.  FIM MA export performance is an issue that the FIM team at Microsoft is well aware of, so maybe they’ll ship an update to address this before I get to post my script ;-)  Yay optimism!


Matt Totty said...

Hey Craig- sounds like the better performance could have saved me tons of effort, embarrasment, and unanswerable questions during deployment.:)

Craig Martin said...

Hey Matt! It can still take a while on large deployments, so you'll notice an uptick in forum activity when I'm running the script ;-)