Unable to sync users that were previously created manually

[tmylonidis]

Hi,

before deploying the idp-scim-sync service, I had created two users manually. After deleting the group these two users were originally members of, the SSO login page was unavailable for these two users. I believe I have removed all possible references to these accounts (in permission sets etc) and created a new group in Google Workspace with only these two users as owners.

However, importing the users to AWS SSO is failing with the following message:
Error: cannot sync groups and their members: error syncing state: error reconciling groups: error creating groups in SCIM Provider: scim: error creating group: statusCode: 400, errCode: 400 Bad Request, errMsg:

Is it possible to address this, or do I need to delete all users and redo the configuration from scratch?

Thanks!

[christiangda]

Hi @tmylonidis sorry to hear that.

Could you provide more evidence:

• Version you are using
• log to understand well the error ( masking your data in case this is confidential: domain, users etc)

I tested this scenario and it is working in the last version, so let me understand better from the log.

You don't need to clean the AWS SSO data before the first sync, but for sure you need to enable the "Automatic provisioning"

NOTE: If the users (email) and groups (name) you have provisioned previously to the first sync doesn't are similar to the news coming from the sync (filter), these users will disappear!

[tmylonidis]

Hi @christiangda, thank you for your response. Please allow me to provide some additional information. Once AWS SSO was enabled using Google Workspace as the IdP, automatic provisioning was NOT enabled. I manually created 2 users in AWS SSO, and on the Google Admin's side I had these two users in a group, and only that group was granted access to the AWS SSO app.

After enabling automatic provisioning and deploying your solution, things were working as expected. However, when I manually deleted the two users I had originally created, alongside with their respective Google Workspace group, even though these two users were part of other Google Workspace groups, they were not created again in AWS SSO. So we are stuck in an intermediate phase where these two users are granted access from a Google Workspace perspective, but they don't exist in AWS SSO. I noticed that the state file stored in the S3 bucket, which the lambda is syncing from, still contains these two users with a proper SCIM id, even after I manually deleted them from AWS SSO.

Another "weird" issue is that renaming groups in Google Workspace can mess things up. For example, we have a group called "AWS Ownership" and the filter configured is "name:AWS*". After renaming this group to "Root AWS Account", the app is still trying to fetch its users for some reason. Can it be because it still exists in the state.json file as well? Shouldn't that be deleted?

The version we are using is 0.0.12. I have attached some logs, only two group names are visible, but this is fine, they are being used just for testing purposes. Do you think that manually editing the state.json file could solve the issue? Ideally we would like to avoid "hacking" solutions, the changes in Google Workspace should be reflected in AWS SSO. Thanks!

[christiangda]

Hi @tmylonidis let me try to help you.

From your previous answer:

I manually created 2 users in AWS SSO, and on the Google Admin's side I had these two users in a group, and only that group was granted access to the AWS SSO app.

I'm confused here, I mean why do you want to create users, groups and members in AWS SSO after the first sync of the idp-scim-sync. The idea is that you only touch the Google Workspace and the magic happens automatically in the AWS SSO side, Right?

When you touch the AWS SSO side manually after the idp-scim-sync was working fine, you are creating an inconsistency between the State File stored in S3 and the AWS SSO side, the program was not built to detect this, the program only detects:

1. The Inconsistencies between the Google Workspace and AWS SSO the first time sync
2. After the the first time sync, the following execution only detects the Inconsistencies between the Google Workspace and the State File and these differences are applied to AWS SSO

Having said that:

• Think in the State File as a cache layer, faster to query than the AWS SSO API
• To fix the inconsistency in this kind of situation, Just delete the State File from the S3 Buckets and the first sync is going to happen again.
• Delete the State File is a SAFE OPERATION, I mean you are deleting a kind of cache layer

Another "weird" issue is that renaming groups in Google Workspace can mess things up. For example, we have a group called "AWS Ownership" and the filter configured is "name:AWS*". After renaming this group to "Root AWS Account", the app is still trying to fetch its users for some reason. Can it be because it still exists in the state.json file as well? Shouldn't that be deleted?

Short answer: No, you shouldn't.

I have never tried this case before because I am confident in the Google Workspace Documentation -> Search for groups, The idp-scim-sync program doesn't filter any, it does the filter in the source, I mean in the Google Workspace using the API. So, any you set as a filter in the idp-scim-sync program will send to the Google API without any change

Also, to be sure about the filter before moving this to the idpscim you could use idpscimcli program, that is why I created this second tool. Google Workspace API Search is "weird", so please try in raw using directly the Google Workspace Documentation -> Search for groups or use idpscimcli before moving this filter to production.

Last Comment About your LOG:

Looks like AWSPMTestGroup existed on the State File and in Google Workspace but not in the AWS SSO, so you were there and created this manually, so why do you get this error?

This could happen when you Delete and Create again the group in the AWS SSO side manually after the first sync.

why?

If the first time the groups were created with the idpscim this store in the state file the ID of this group, so if you go manually and delete and create this again, the ID change and even when the name is the same, the ID in the state file is different, so the idpscim can't do any operation with this group in the AWS SSO because the ID isn't the same, for its this group doesn't exist.

Let me know if this answer helps you!

[tmylonidis]

Hi again @christiangda,

I think after removing all users and groups from AWS SSO, I got it to work again. Also, I think I found what is wrong with the groups that were imported. For some reason, the filter "name:AWS*" does not return the ones prefixed with AWS, but the ones containing the string "AWS" in the group name. After using your tool, I found out that this is an issue with the Google Workspace Directory API. I will try to see if this can be sorted out with them. In any case, thank you for your support and for making this tool!

[christiangda]

Amazing @tmylonidis thanks for keeping me posted. I recommend to you use more than one attribute, you are only using the name use also email or another one allowed in the API.

In our case we are using:

[tmylonidis]

Thank you for your recommendation @christiangda! In case I want to change my filter, do I need to redeploy idp-scim-sync from scratch? I have originally deployed it using the Serverless Application Repository.

[christiangda]

No, you don't. just you have two ways:

1. go to the AWS Lambda and change the filter value directly into the Configuration
2. go to cloudformation and update the template with the new value

[tmylonidis]

Perfect, thank you for all your help!