Friday, January 24, 2020

Adding Structured Data to Event Log Items

Ever want to store data while automating stuff? Often I need to store something but like to avoid writing files or introducing storage systems if I can avoid it, preferring to live off the land.  The scenario I'm working on right now is automating synchronization actions while providing status details.  Another scenario I'm working on is reading objects from one system and uploading them to Azure Log Analytics.  In the upload scenario I need to track the objects I've uploaded to avoid uploading the same object twice.

The solution I've come up with is pretty simple, write an event log entry to the Application event log.  This is pretty simple thanks to the Write-EventLog cmdlet:

     [[-EntryType] <EventLogEntryType>]
     [-Category <Int16>]
     [-RawData <Byte[]>]
     [-ComputerName <String>]

Here is the one line sample:

# Write an entry to the Application log using MyCustomSource
Write-EventLog -LogName Application -Source MyCustomSource -Message foo -EventId 100

That works great (as long as you have already created the custom Source, see New-EventLog).  The fun comes when you want to use code to read the events, and get data from the events.  One approach is to stuff JSON into the event log item Message, similar to what MIM Hybrid Reporting does.

Another approach is to use the Message property for something human-readable (be kind to your event log readers) and use the RawData property for the structured data intended to be read by code.  Here is an example:

# Create a custom event log
New-EventLog -LogName MyCustomLog -Source MyCustomSource

# Create a custom object, convert it to JSON
$json = [PSCustomObject]@{
    fooString = 'the foo'
    fooDate   = Get-Date
    fooArray  = 'foo','bar','baz'
    } | 

# Convert the JSON to a byte array
$jsonBytes = [System.Text.Encoding]::Unicode.GetBytes($json)

# Write an event log item with the raw data
Write-EventLog -LogName MyCustomLog -Source MyCustomSource -Message foo -EventId 100 -RawData $jsonBytes

# Get the event log item we just wrote
$event = Get-EventLog -LogName MyCustomLog -Newest 1

# Convert the RawData back to our object
# Notice how it preserves types (string, date, array)
[System.Text.Encoding]::Unicode.GetString($event.Data) | ConvertFrom-Json 

fooString : the foo
fooDate   : @{value=1/25/2020 2:23:25 AM; DisplayHint=2; DateTime=Friday, January 24, 2020 6:23:25 PM}
fooArray  : {foo, bar, baz}

There you have it, persisting structured data in the event log while maintaining something human readable.  It is more complex than the one-liner for creating a simple event log entry but it persists data without introducing a new storage mechanism, and the storage for the custom event log can be configured like any other event log. 

Friday, January 17, 2020

AAD Connect - ADSync Module Support for Remote PowerShell

Lately I've been having a blast replacing WMI calls with commands from the AAD Connect ADSync PowerShell module. I hit an issue using the ADSync module with PowerShell remoting. To illustrate:

Invoke-Command -ComputerName Sinking -ScriptBlock {Get-ADSyncRunProfileResult}
There was no endpoint listening at net.pipe://localhost/ADSyncManagement that could accept the message. This is often caused by an incorrect address or SOAP 
action. See InnerException, if present, for more details.

It seems some of the commands initiate a connection to an endpoint, which I suspect creates an authentication double-hop issue.

If you are interested in helping prioritize this issue, please vote for the feedback item:
ADSync Cmdlets Fail with Remote PowerShell

The ADSync module has some great commands, help make them better with your votes :-)

Wednesday, January 15, 2020

Azure AD Connect Global Settings

Sync deployments always have some configuration settings hanging around, and usually end up in XML files somewhere on the computer running the synchronization service because there has not been a good way to store configuration settings in the synchronization service itself. That seems to have changed in Azure AD Connect with the Global Settings. Global settings are not well documented but they appear to work pretty well. From what I've observed they are saved to the ADSync database and survive computer restarts. Here's a tour... The command for getting the global settings is Get-ADSyncGlobalSettings.

Get-Help Get-ADSyncGlobalSettings -Full
    Get-ADSyncGlobalSettings [-WhatIf] [-Confirm]  []
        Required?                    false
        Position?                    Named
        Accept pipeline input?       false
        Parameter set name           (All)
        Aliases                      cf
        Dynamic?                     false
        Required?                    false
        Position?                    Named
        Accept pipeline input?       false
        Parameter set name           (All)
        Aliases                      wi
        Dynamic?                     false
        This cmdlet supports the common parameters: Verbose, Debug,
        ErrorAction, ErrorVariable, WarningAction, WarningVariable,
        OutBuffer, PipelineVariable, and OutVariable. For more information, see 
        about_CommonParameters (   

The help content is minimalist but like any good command it can be run without input.

Version          : 396
SqlSchemaVersion : 615
InstanceId       : d112f5f4-4248-4959-8dc8-eac88e531433
Schema           : Microsoft.IdentityManagement.PowerShell.ObjectModel.Schema
Parameters       : {Microsoft.Synchronize.SynchronizationPolicy, Microsoft.SynchronizationOption.JoinCriteria, Microsoft.UserSignIn.DesktopSsoEnabled, Microsoft.Synchronize.MaintenanceEnabled...}

OK, looks like some version detail and a property named Parameters containing the actual settings.  Get-Member sometimes reveals other useful properties and methods, and also tells us the type name:

Get-ADSyncGlobalSettings | Get-Member
   TypeName: Microsoft.IdentityManagement.PowerShell.ObjectModel.GlobalSettings

Name                               MemberType Definition                                                                                                                                                                                                         
----                               ---------- ----------                                                                                                                                                                                                         
AddOrReplaceConfigurationParameter Method     void AddOrReplaceConfigurationParameter(string parameterName, string parameterValue), void AddOrReplaceConfigurationParameter(Microsoft.IdentityManagement.PowerShell.ObjectModel.ConfigurationParameter parameter)
Equals                             Method     bool Equals(System.Object obj)                                                                                                                                                                                     
GetHashCode                        Method     int GetHashCode()                                                                                                                                                                                                  
GetSchema                          Method     System.Xml.Schema.XmlSchema GetSchema(), System.Xml.Schema.XmlSchema IXmlSerializable.GetSchema()                                                                                                                  
GetType                            Method     type GetType()                                                                                                                                                                                                     
IsStagingModeEnabled               Method     bool IsStagingModeEnabled()                                                                                                                                                                                        
ReadXml                            Method     void ReadXml(System.Xml.XmlReader reader), void IXmlSerializable.ReadXml(System.Xml.XmlReader reader)                                                                                                              
ToString                           Method     string ToString()                                                                                                                                                                                                  
WriteXml                           Method     void WriteXml(System.Xml.XmlWriter writer), void WriteXml(System.Xml.XmlWriter writer, bool legacyFormat), void IXmlSerializable.WriteXml(System.Xml.XmlWriter writer)                                             
InstanceId                         Property   guid InstanceId {get;set;}                                                                                                                                                                                         
Parameters                         Property   Microsoft.IdentityManagement.PowerShell.ObjectModel.ParameterKeyedCollection Parameters {get;set;}                                                                                                                 
Schema                             Property   Microsoft.IdentityManagement.PowerShell.ObjectModel.Schema Schema {get;set;}                                                                                                                                       
SqlSchemaVersion                   Property   int SqlSchemaVersion {get;set;}                                                                                                                                                                                    
Version                            Property   int Version {get;set;}                     

Hmm, AddOrReplaceConfigurationParameter looks useful, more on that later in this post..

Looking now at the Parameters property...

Get-ADSyncGlobalSettings | Select-Object -ExpandProperty Parameters
Name                   : Microsoft.OptionalFeature.UserWriteBack
InputType              : String
Scope                  : SynchronizationGlobal
Description            : 
RegexValidationPattern : 
DefaultValue           : 
Value                  : False
Extensible             : False
PageNumber             : 0
Intrinsic              : False
DataType               : String

Name                   : Microsoft.Synchronize.StagingMode
InputType              : String
Scope                  : SynchronizationGlobal
Description            : 
RegexValidationPattern : 
DefaultValue           : 
Value                  : False
Extensible             : False
PageNumber             : 0
Intrinsic              : False
DataType               : String

Each Parameter seems to have a lot of properties, but when viewed with Out-GridView it seems only the Name and Value properties change, the other properties are the same for all Parameters.  Spoiler alert: ADSync does let you specify DataType other than String but it seems to be hard coded to only support String.

Here are the names and values:

Get-ADSyncGlobalSettings | 
Select-Object -ExpandProperty Parameters | 
Format-Table -AutoSize -Property Name, Value, DataType
Name                                                   Value                         DataType
----                                                   -----                         --------
Microsoft.Synchronize.SynchronizationPolicy            Delta                           String
Microsoft.SynchronizationOption.JoinCriteria           AlwaysProvision                 String
Microsoft.UserSignIn.DesktopSsoEnabled                 False                           String
Microsoft.Synchronize.MaintenanceEnabled               True                            String
Microsoft.OptionalFeature.ExportDeletionThresholdValue 10                              String
Microsoft.Version.SynchronizationRuleImmutableTag      V1                              String
Microsoft.SynchronizationOption.AnchorAttribute        mS-DS-ConsistencyGuid           String
Microsoft.OptionalFeature.DirectoryExtensionAttributes                                 String
Microsoft.OptionalFeature.FilterAAD                    False                           String
Microsoft.GroupWriteBack.Forest                                                        String
Microsoft.GroupWriteBack.Container                                                     String
Microsoft.SynchronizationOption.UPNAttribute           userPrincipalName               String
Microsoft.Synchronize.SchedulerSuspended               False                           String
Microsoft.OptionalFeature.DirectoryExtension           False                           String
Microsoft.SynchronizationOption.CustomAttribute                                        String
Microsoft.Synchronize.TimeInterval                     00:30:00                        String
Microsoft.Synchronize.ServerConfigurationVersion                        String
Microsoft.SystemInformation.MachineRole                RolePrimaryDomainController     String
Microsoft.AADFilter.AttributeExclusionList                                             String
Microsoft.OptionalFeature.DeviceWriteBack              False                           String
Microsoft.OptionalFeature.AutoUpgradeState             Enabled                         String
Microsoft.Synchronize.NextStartTime                    Wed, 15 Jan 2020 21:57:01 GMT   String
Microsoft.Synchronize.RunHistoryPurgeInterval          7.00:00:00                      String
Microsoft.OptionalFeature.GroupFiltering               False                           String
Microsoft.ConnectDirectories.WizardDirectoryMode       AD                              String
Microsoft.Synchronize.SynchronizationSchedule          False                           String
Microsoft.OptionalFeature.ExchangeMailPublicFolder     False                           String
Microsoft.OptionalFeature.UserWriteBack                False                           String
Microsoft.Synchronize.StagingMode                      False                           String
Microsoft.OptionalFeature.ExportDeletionThreshold      False                           String
Microsoft.DeviceWriteBack.Forest                                                       String
Microsoft.OptionalFeature.DeviceWriteUp                True                            String
Microsoft.OptionalFeature.HybridExchange               False                           String
Microsoft.AADFilter.ApplicationList                                                    String
Microsoft.DirectoryExtension.SourceTargetAttributesMap                                 String
Microsoft.UserWriteBack.Forest                                                         String
Microsoft.DeviceWriteBack.Container                                                    String
Microsoft.UserWriteBack.Container                                                      String
Microsoft.UserSignIn.SignOnMethod                      PasswordHashSync                String
Microsoft.OptionalFeature.GroupWriteBack               False                           String

On a new installation of AAD Connect there are already a lot of Parameters, neat.  I do not recommend messing Parameters named like Microsoft.*, the same is true for messing with the Windows registry (you're asking for trouble).

Can new Parameters be added?  Well yes they can!  Here's how:

$globalSettings = Get-ADSyncGlobalSettings                           
Set-ADSyncGlobalSettings -GlobalSettings $globalSettings
Version          : 396
SqlSchemaVersion : 615
InstanceId       : d112f5f4-4248-4959-8dc8-eac88e531433
Schema           : Microsoft.IdentityManagement.PowerShell.ObjectModel.Schema
Parameters       : {Microsoft.Synchronize.SynchronizationPolicy, Microsoft.SynchronizationOption.JoinCriteria, Microsoft.UserSignIn.DesktopSsoEnabled, Microsoft.Synchronize.MaintenanceEnabled...}

Note the Version property did not increment.  Calling Get-ADSyncGlobalSettings again shows that the Version actually does increment:

Version          : 397
SqlSchemaVersion : 615
InstanceId       : d112f5f4-4248-4959-8dc8-eac88e531433
Schema           : Microsoft.IdentityManagement.PowerShell.ObjectModel.Schema
Parameters       : {Microsoft.Synchronize.SynchronizationPolicy, Microsoft.SynchronizationOption.JoinCriteria, Microsoft.UserSignIn.DesktopSsoEnabled, Microsoft.Synchronize.MaintenanceEnabled...}

To get the Parameter, just call Get-ADSyncGlobalSettings then use the new Parameter name as the index item:

$globalSettings = Get-ADSyncGlobalSettings
Name                   : fooName
InputType              : String
Scope                  : SynchronizationGlobal
Description            : 
RegexValidationPattern : 
DefaultValue           : 
Value                  : fooValue
Extensible             : False
PageNumber             : 0
Intrinsic              : False
DataType               : String

That's it, a nice place to store configuration values for a synchronization service.  I'll be posting later about experiments with global settings and preventing accidental deletions.

Tuesday, January 14, 2020

Using a Names API to Create Test AD Users

Found this cool little API that provides names, made for a nice and quick little script to throw a bunch of test users into a test Active Directory domain.

### Create a SecureString for the AccountPassword
$newPassword = ConvertTo-SecureString 'N0BoatBrewing!' -AsPlainText -Force

### Get 100 names
$namesContent = Invoke-RestMethod -Uri ''

$namesContent.GetEnumerator() | ForEach-Object {
    ### Split the "FirstName LastName" string into two variables
    $FirstName, $LastName = $PSItem -split ' '   

    ### Construct a couple of strings
    $AccountName = "$FirstName $LastName"
    $DisplayName = "$($FirstName) $($LastName)"

    ### Create the AD User
    New-ADUser -Name $AccountName -GivenName $FirstName -Surname $LastName -SamAccountName $AccountName -DisplayName $DisplayName -Enabled $true -AccountPassword $newPassword -Path 'OU=People,DC=litware,DC=ca'


Azure AD Connect sync: Get-ADSyncConnectorStatistics

WMI gets deprecated in AAD Connect so I am working on updating some AAD Connect scripts to instead use the ADSync PowerShell module.  The existing scripts prevent damage during synchronization by checking thresholds for:
  • Import
    • number of pending import updates
    • number of pending import deletes
    • percentage of pending import updates
    • percentage of pending import deletes
  • Export
    • number of pending export updates
    • number of pending export deletes
    • percentage of pending export updates
    • percentage of pending export deletes
In WMI the MicrosoftIdentityIntegrationServer:MIIS_ManagementAgent class provided the import and export counters, as well as the total number of connector space objects.
In ADSync most of the counters are gone, the rest are moved:

Get-ADSyncConnectorStatistics -ConnectorName
ExportAdds ExportUpdates ExportDeletes TotalConnectors
---------- ------------- ------------- ---------------
         0             0             0              94

Unfortunately not enough detail to get feature parity with the scripts I'm working on because:
  • Import counters are missing (can't prevent damage until export)
  • 'TotalConnectors' is just that - the number of connectors in the connector space as opposed to the total number of objects in the connector space
My plan is to go with what ADSync provides for now, and implement the missing functionality later if it is important enough.  I was a bit discouraged until I found Azure AD Connect sync: Prevent accidental deletes, more on that later.

Thursday, January 09, 2020

AAD Connect - Big Money No WMI!

For years we've enjoyed access to sync functionality via WMI (pronounced 'Whammy'), all the way back to MIIS.  The Windows Management Infrastructure provider for the sync engine provided some useful functionality, including:
  • Start a management agent run or stop an existing management agent run.
  • Get the results of a management agent run.
  • Get statistical information about the connector space or metaverse.
  • Set or change passwords.
  • Get information about FIM Synchronization Service.
  • Search for a connector space object with a specified domain and account name in a global address list, a Windows NT 4.0 domain, or in Active Directory Domain Services (AD DS).
  • Search for a connector space object with a specified domain and user principal name in a global address list, a Windows NT 4.0 domain, or in AD DS.
The WMI provider lives on in MIM but goes away in AAD Connect starting in version
Customers should be informed that the deprecated WMI endpoints for MIIS_Service have now been removed. Any WMI operations should now be done via PS cmdlets.
So heads up!  If you have been programming against the WMI provider then you will need to refactor before upgrading to any version newer than (ask me how I know).  The ADSync PowerShell module does have a ton of cmdlets available, and I'll be sharing experiences with them in the coming posts.

Wednesday, January 08, 2020

MIMDSC - Desired State Configuration Module for Microsoft Identity Manager

Over the years I've automated deployments for MIM using a variety of methods but fell in love with Desired State Configuration enough to apply it to MIM.  DSC was easy to love because at its heart it is state based, much like the synchronization engine has always been but DSC is focused on configuration while the synchronization is focused on identity.

I decided to share some of the DSC resources on GitHub at the MIMDSC repository.  The resources for MIM Sync are largely done.  To see what a synchronization configuration item looks like in DSC, check out the examples in the wiki such as:

Configuration TestMimSyncExportAttributeFlowRule 
    Import-DscResource -ModuleName MimSyncDsc

    Node (hostname) 
        ExportAttributeFlowRule TestMimSyncExportAttributeFlowRule
            ManagementAgentName    = 'TinyHR'
            MVObjectType           = 'SyncObject'
            CDAttribute            = 'JobTitle'
            CDObjectType           = 'person'
            Type                   = 'direct-mapping'
            SrcAttribute           = 'Title'
            SuppressDeletions      = $false
            Ensure                 = 'Present'

I need to start the MIM Service resources but want to see if there is interest first.  If you happen to like both DSC and MIM Service configuration then let me know!

Good To Be Back!

Ah it's good to be back.  In the past few years I've taken a some career adventures (going from consulting back to Microsoft, working as a developer in security incident response and vulnerability management, another stint as a manager).  
Last summer I moved back to an individual contributor role focusing on identity again, this time as a service engineer.  Being excited about DevOps, security and identity makes this job a ton of fun and I find myself figuring things out that seem worth sharing so I'll be posting here again, hopefully often.  The nature of my posts will likely be fun ways to integrate and automate, things that should be simple but require some hacking, ramblings about my career adventures, etc.  Here goes!