Syncing users and groups with Microsoft Active Directory

If your company uses Microsoft Active Directory for user management and you want your users to be able to access Yandex Cloud, you do not need to create Yandex Cloud accounts for your users manually. Instead, you can sync the users and groups created in your Active Directory folder with Yandex Identity Hub.

User and group synchronization is performed by the Identity Hub AD Sync Agent, which can be run on any Linux or Windows server.

How synchronization works:

On the server the synchronization agent is running on, the following TCP ports must be open for incoming and outgoing traffic:

• To access the Yandex Cloud API:

  • 443: For HTTPS.

• To access the Active Directory domain controller:

  • 389: For LDAP.
  • 636: For LDAPS.
  • 135: For MSRPC.
  • 49152:65535: Port range for MSRPC dynamic.

Synchronization objectsSynchronization objects

The Identity Hub AD Sync Agent syncs the following objects with the Active Directory folder:

• Users.

• User attributes.

  Default user attribute mapping table:

  Attribute in Active Directory	Attribute in Identity Hub
  displayName	full_name
  givenName	given_name
  sn	family_name
  mail	email
  telephoneNumber	phone_number
  userPrincipalName	username
  ObjectGUID	external_id

• User groups.

• User group attributes.

  Default user group attribute mapping table:

  Attribute in Active Directory	Attribute in Identity Hub
  name	name
  description	description
  ObjectGUID	external_id

• User memberships in groups.

• User password hashes.

  Active Directory stores user passwords but as hashes, not as plaintext. Yandex Cloud collects the user password hash from the Active Directory folder and uses the modern and secure Argon2 algorithm to generate it own hash based on it.

  Alert

  Yandex Cloud does not store user passwords as plaintext in its databases.

Setting up synchronizationSetting up synchronization

To implement Identity Hub user and group synchronization with Active Directory, you need to do the presetting both on the domain controller side with Active Directory services deployed and on the Yandex Cloud side.

Active Directory domain controller side setupActive Directory domain controller side setup

For the synchronization agent to work correctly on the Active Directory side, do the following:

1. Create a domain user your agent will use to run synchronization.

2. Grant the following permissions to this user:

   • Replicating Directory Changes
   • Replicating Directory Changes All

3. On the domain controller, open the TCP network ports for incoming traffic from the IP address of the server hosting Identity Hub AD Sync Agent:

   • 389: For LDAP.
   • 636: For LDAPS.
   • 135: For MSRPC.
   • 49152:65535: Port range for MSRPC dynamic.

Yandex Cloud side setupYandex Cloud side setup

For the synchronization agent to work correctly on the Yandex Cloud side, do the following:

• Create a service account for synchronization on the Identity Hub side.

• Create and save an authorized key for the service account.

• Assign the following roles to the service account for the organization the user pool is in:

  • organization-manager.userpools.syncAgent
  • organization-manager.groups.externalCreator
  • organization-manager.groups.externalConverter

Identity Hub AD Sync Agent agentIdentity Hub AD Sync Agent agent

Identity Hub AD Sync Agent reads user and user group data in the selected Organization Units (OU) in the Active Directory folder and syncs it with user and user group data in the Identity Hub pool.

On the Active Directory side, the synchronization agent gets user and group data as the user created in the Active Directory domain. To get this data, the agent uses the LDAP and DRSR protocols. The requests go to the Active Directory domain controller address specified in the agent configuration.

On the Yandex Cloud side, the synchronization agent manages users and user groups as a service account with permissions for syncing. Requests to Yandex Cloud go to public endpoint https://organization-manager.api.cloud.yandex.net over HTTPS.

The synchronization agent installation script is available for the following operation systems:

• Linux
• Windows

Synchronization processSynchronization process

During the synchronization process, Identity Hub AD Sync Agent can create, update, or delete users and user groups in Identity Hub. Identity Hub users and groups are synced with Active Directory users and groups in to stages: primary synchronization and incremental synchronization.

Full (primary) synchronizationFull (primary) synchronization

When performing a full synchronization, the agent reads the data of all users, groups, and their attributes in the selected Organization Units in the Active Directory folder and creates the same users and groups with the same attributes in the Identity Hub user pool.

The primary synchronization process for a large number of objects may take a long time. If the full synchronization process is interrupted due to an error, you can restart the agent to resume synchronization from where the previous attempt was interrupted. The agent tracks the progress of full synchronization using process token files in the running agent's directory:

• main_sync_replication_token.json
• password_hash_replication_token.json
• user_control_replication_token.json

After full synchronization is successfully completed, the agent, run as a standalone service or OS service, proceeds to continuously execute a partial (incremental) synchronization.

Partial (incremental) synchronizationPartial (incremental) synchronization

The running agent performs incremental synchronization continuously with the following frequency:

• Syncing user passwords and states: The agent tracks the lock/unlock status of users in the Active Directory domain and user password changes and transfers these updates to Identity Hub at an interval of several seconds. You cannot change the frequency for this synchronization type.
• Syncing other values: The agent tracks other changes in properties, attributes, and parameters of users and groups at an interval set in the agent's configuration file.

During syncing, the user pool may be found to contain a user or user group with names identical to those of the user or user group that need to be synced. In which case, depending on current settings, the agent will either overwrite the data from Active Directory into the existing Identity Hub user or group or return an error message.

Tracked changesTracked changes

During continuous synchronization, the agent tracks the following changes in Active Directory and transfers them to Identity Hub:

• Creating, editing, locking, unlocking, and deleting users.
• Creating, editing, and deleting user groups.
• Changing user and user group attributes.
• Adding users to groups and removing them from groups.
• Changing user passwords.

Synchronization loggingSynchronization logging

Identity Hub AD Sync Agent logs the events taking place during synchronization.

By default, the event and error info is fed into the standard stream named stdout. As an alternative, you can configure saving logs to files in the agent's configuration.

By default, the event info is output in text format, but you can change it to JSON in the agent's configuration.

Additionally, you can set the following logging conditions in the agent's configuration:

• debug
• info
• warn
• error
• dpanic
• panic
• fatal

Agent configurationAgent configuration

The synchronization agent's configuration is set in a YAML file in the following format:

# Default configuration for yc-identityhub-sync-agent
# This is a template - please update with your actual values

userpool_id: "<user_pool_ID>"
cloud_credentials_file_path: "<path_to_file_with_authorized_key>"
replication_tokens_path: "<path_to_directory_with_process_tokens>"
working_directory: "<path_to_agent_working_directory>"

# Active Directory replication API client settings
drsr:
  host: "<domain_controller_IP_address>"
  username: "username"
  password: "password"

# LDAP client settings
ldap:
  host: "ldaps://<domain_controller_IP_address>:636"
  username: "<Active_Directory_username>"
  password: "<Active_Directory_user_password>"
  certificate_path: "<path_to_certificate>"
  insecure_skip_verify: false|true

# Logger configuration
logger:
  level: "<logging_level>"
  format: "plain|json"
  file:
    filename: "<log_file_path>"
    maxsize: 30
    maxbackups: 10

# Sync settings
sync_settings:
  interval: "600s"
  allow_to_capture_users: true|false
  allow_to_capture_groups: true|false
  # Remove the following line if you don't need to replace domain
  replacement_domain: "<user_pool_domain>"
  filter:
    domain: "<Active_Directory_domain_name>"
    organization_units:
      - OU=IdPUsersOU,DC=example,DC=com
      - OU=IdPGroupsOU,DC=example,DC=com
  remove_user_behavior: "remove|block"

Where:

• userpool_id: ID of the user pool in Identity Hub.

• cloud_credentials_file_path: Path to the file containing the authorized key of the service account in Yandex Cloud. For example:

  • /etc/yc-identityhub-sync-agent/authorized_key.json (for Linux)
  • C:\\ProgramData\\YcIdentityHubSyncAgent\\authorized_key.json (for Windows)

  In the cloud_credentials_file_path parameter, you can provide only the file name instead of the full path. In this case, the system will save that file in the working directory specified in working_directory or, if none is specified, in the directory the agent's executable is in.

• replication_tokens_path: Path to the directory storing tokens with info about the current progress of full synchronization processes. This is an optional parameter.

  If this parameter is not set, the system will be saving the tokens in the agent's working directory specified in working_directory or, if none is specified, in the directory the agent's executable is in.

• working_directory: Path to the directory for other files the agent needs to operate. This is an optional parameter.

  If this parameter is not set, the system will use the directory containing the agent's executable as the working directory. By default, the agent's executable resides in the following directories:

  • /etc/yc-identityhub-sync-agent/ (for Linux)
  • C:\\ProgramData\\YcIdentityHubSyncAgent\\ (for Windows)

  Note

  If the cloud_credentials_file_path, replication_tokens_path, and/or logger.level parameters specify paths other than that specified in working_directory, the system use the paths specified in cloud_credentials_file_path, replication_tokens_path, and/or logger.level.

• drsr: DRSR protocol settings for Active Directory authentication of a user with permissions to replicate folder data.

• ldap: LDAPS/LDAP protocol settings for Active Directory authentication:

  Warning

  You can connect to a domain controller over LDAPS or LDAP. LDAPS is the recommended and safe option. Use LDAP only for setup and testing.

  • host: IP address of the Active Directory domain controller. Specify the schema and port number depending on the protocol you use:

    • For LDAPS: ldaps:// is the schema and 636 is the port number.
    • For LDAP: ldap:// is the schema and 389 is the port number.

  • username: Name of the Active Directory domain user with data replication permissions assigned.

  • password: Active Directory domain user password.

  • certificate_path: Path to the public key certificate file required to decrypt traffic from the domain controller. This is a required parameter when using LDAPS.

    If the working_directory parameter specifies the path to the working directory, you can simply specify the certificate file name instead of its full path.

  • insecure_skip_verify: Controls whether to ignore public key certificate validation errors when connecting to a domain controller. This is an optional parameter. The possible values are:

    • false: Certificate validation errors will not be ignored. This is a default value.
    • true: The synchronization agent will ignore certificate validation errors. This may prove effective for synchronization setup and testing. Not recommended for general use.

• logger: Synchronization logging settings:

  • level: Logging level. The possible values are:

    • debug
    • info
    • warn
    • error
    • dpanic
    • panic
    • fatal

  • format: Event info output format. This is an optional parameter. The possible values are:

    • plain: Output the info as plain text. This is a default value.
    • json: Output the info in JSON format.

  • file: Settings for saving logs to files:

    • filename: Path to the file for logging synchronization events.

      In the filename parameter, you can provide only the file name instead of the full path. In this case, the system will save that file in the working directory specified in working_directory or, if none is specified, in the directory the agent's executable is in.

      This is an optional parameter. The default file name is identity_hub.log.

    • maxsize: Maximum size of a single log file, in MB.

    • maxbackups: Maximum number of log files the agent will retain. When this limit is exceeded, the oldest file will be deleted.

    This is an optional parameter. If you do not provide the file settings, logs will be output to the stdout stream and not saved to files.

• sync_settings: Synchronization settings:

  • interval: Incremental synchronization frequency. This is an optional parameter. The default value is 240 seconds.

    Note

    Password and user status synchronization in Active Directory takes place every few seconds at a fixed interval which does not depend on the interval value.

  • allow_to_capture_users: Enables updating an existing user in the Identity Hub user pool if their login matches that of a Active Directory user being synchronized. The possible values are:

    • true: Synchronization agent will update existing Identity Hub users to match their corresponding Active Directory accounts.
    • false: Synchronization agent will not update existing Identity Hub users. If it detects matching logins in the user pool and Active Directory, the synchronization will throw an error.

  • allow_to_capture_groups: Enables updating an existing Identity Hub user group if its name matches that of a Active Directory group being synchronized. The possible values are:

    • true: Synchronization agent will update existing Identity Hub user groups to match their corresponding Active Directory groups.
    • false: Synchronization agent will not update existing Identity Hub groups. If it detects matching group names in the pool and Active Directory, the synchronization will throw an error.

  • replacement_domain: Domain associated with the Identity Hub user pool to which synchronized users and groups belong, e.g., newdomain.idp.yandexcloud.net.

    This is an optional parameter. Specify the replacement_domain value only if the domain name associated with the user pool does not match the domain name on the Active Directory domain controller.

  • filter: Settings for filtering objects to synchronize on the Active Directory side:

    • domain: Domain name in the Active Directory domain controller where the agent will synchronize users and groups.
    • organization_units: List of organization units (OUs) in the Active Directory folder in which the agent will synchronize users and groups.

    If object filtering is not configured, Identity Hub AD Sync Agent will attempt to synchronize all available objects in the Active Directory folder.

  • remove_user_behavior: Controls what action should be applied to users on the Yandex Cloud side if the corresponding ones on the Active Directory side were deleted or ceased to satisfy the conditions specified in sync_settings.filter (e.g., if moved to another organization unit). This is an optional parameter. The possible values are:

    • remove: Users who were deleted ceased to satisfy the filter criteria will be deleted on the Identity Hub side. This is the default action.
    • block: Users who were deleted ceased to satisfy the filter criteria will be deactivated on the Identity Hub side.

  Note

  If synchronization reveals that a Active Directory user group was deleted or ceased to satisfy the filter criteria (e.g., if moved to another organization unit), such a group will be deleted on the Identity Hub side.

See alsoSee also

• Syncing users and groups with Microsoft Active Directory