Azure Cloud Shell integration?

[edyoung]

Hi, I'm the engineering manager for Azure Cloud Shell (https://shell.azure.com)

We've had a couple of queries about including office365-cli in the Cloud Shell image, which would make it available to Cloud Shell users by default. We're looking to deepen our Office 365 coverage in Cloud Shell so I've opened this discussion to see if there is interest.

We have node.js in Cloud Shell and 'npm i @pnp/office365-cli' succeeds. (An end user can't install globally because they are not root.) I've run the CLI as far as 'login' so far.

Roughly speaking, we'd be looking for someone familiar with office365 CLI to try things out in Cloud Shell first just by installing it locally, because you're more familiar with the tool than we are. If that looks promising we can look at including in the image.

It would also be great to support the managed identity support in Cloud Shell. We can use that to avoid the user having to login again if they want to use Office365 using the same identity they used to connect to the Azure portal. This is how az cli avoid prompting in Cloud Shell. See https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-use-vm-token

Let me know what you think!

[waldekmastykarz]

What a great news! We appreciate you reaching out and are thrilled about the idea of having the Office 365 CLI integrated in Azure Cloud Shell. We're all regular users of Azure Cloud Shell and can definitely help out verifying that everything is working as intended.

We have two PRs open that aim to align the Office 365 CLI with the Azure CLI: support for managed identity (#1528) and simplifying storing credentials across the different operating systems (#1529) which we also need to support Azure Functions.

How should we proceed? Do you have a list of scenarios that you'd like us to verify?

[VelinGeorgiev]

@waldekmastykarz , @edyoung If IDENTITY_HEADER and IDENTITY_ENDPOINT are not present the CLI will check for MSI_ENDPOINT and MSI_SECRET and it will try to use them instead.

I have tested the login with identity in Azure Function and Azure VM only, but I can test it in the Cloud Shell on Friday to see if all is good.

Let me know if that suits you.

[waldekmastykarz]

Excellent! In this case we'll proceed with processing #1528 so that we can test it properly shortly 💪

[VelinGeorgiev]

@waldekmastykarz , @edyoung I tested my PR with the Azure Cloud Shell and I am able to get token using MSI and communicate with the Microsoft Graph without issues.

@edyoung is there a way to reliably detect that the CLI is running in the Azure Could Shell? I can see bunch of vars in the env variables. Can I use AZURE_HTTP_USE_AGENT for example or any other of them to detect the Azure Cloud Shell so I can handle conditions within the Office 365 CLI?

[edyoung]

I would check for the presence of ACC_CLOUD. It will have different values for government cloud like US Government cloud, but it will always be set in cloud shell

[VelinGeorgiev]

Thank you very much!

[alexandair]

@edyoung
Just to clarify something about the installation. As a user I can install it locally (or globally, but only with a little bit of hacking), but you are a creator of the container image and can run installation as a root and install it globally.
So, do you plan to install it globally or locally?

[edyoung]

I imagine we'd install it globally

[VelinGeorgiev]

@edyoung, do you know if there is documentation, article or any piece of information on how to assign user-managed identity to the Azure Cloud Shell so I can test with both system managed identity and user-managed identity?

Is it possible at all? :)

[edyoung]

The "managed identity" in cloud shell is the identity of the user - it is neither system-assigned or user-assigned and it can't be configured.

[VelinGeorgiev]

Thank you very much @edyoung! I realized that while testing, but I wanted to confirm it with you 👍

[waldekmastykarz]

We have just released a preview version of the Office 365 CLI with support for managed identity and all the changes we've discussed here. You can test it by installing from npm i -g @pnp/office365-cli@next. Please don't hesitate to let us know if there is anything out of the ordinary and once again, thank you so much for all your help and the opportunity.

[edyoung]

Great! I'd prefer to reference a stable rather than preview release when adding things to Cloud Shell image. Roughly when do you expect to ship a managed-identity-supporting stable build?

[waldekmastykarz]

We ship a new release every month. Next release is planned to go out on May 31. Is there a chance for us to verify the setup that you'd use upfront so that we can decrease the chance of a last-minute issue that would postpone the release with another month?

[edyoung]

It should be possible to install it non-globally and run it from the user directory it gets installed into. I would appreciate if someone familiar with office cli could do that and try some basic manual scenarios. I assume if it works locally it should also work globally. Once a non-preview package is available we can try updating the dockerfile to install the tool as global

[waldekmastykarz]

We'll do it and let you know how the tests went

/cc: @pnp/office-365-cli-maintainers

[VelinGeorgiev]

Hey guys,

Good news!

I did some testing from the shell and here are my results:

User login - Tests while logged in using user name and password

✔️ Login to the CLI - OK
✔️ Commands using AAD Graph APIs - OK
✔️ Commands using Microsoft Graph APIs - OK
✔️ Commands using SharePoint APIs - OK
✔️ Commands using PowerAutomate (Flow) APIs - OK
✔️ Commands using Yammer APIs - OK

Device login - Tests while logged in using the device login flow

✔️ Login to the CLI - OK
✔️ Commands using AAD Graph APIs - OK
✔️ Commands using Microsoft Graph APIs - OK
✔️ Commands using SharePoint APIs - OK
✔️ Commands using PowerAutomate (Flow) APIs - OK
✔️ Commands using Yammer APIs - OK

User managed identity - Tests while logged in using managed identity

✔️ Login to the CLI - OK
✔️ Commands using AAD Graph APIs - OK
✔️ Commands using Microsoft Graph APIs - OK
❌ Commands using SharePoint APIs - NOT OK (throws not supported by MSI audience error)
✔️ Commands using PowerAutomate (Flow) APIs - OK
❌ Commands using Yammer APIs - NOT OK (throws not supported by MSI audience error)

Custom AAD application + Certificate - Tests while logged in using custom AAD application with certificate

✔️ Login to the CLI - OK
✔️ Commands using AAD Graph APIs - OK
✔️ Commands using Microsoft Graph APIs - OK
✔️ Commands using SharePoint APIs - OK
✔️ Commands using PowerAutomate (Flow) APIs - OK
✔️ Commands using Yammer APIs - Not applicable for app (Works with just delegated permissions)

@edyoung, @waldekmastykarz the only issues I experienced are related to the MSI service itself and I do not know if you can influence there. When I am logged with identity and attempt to call SharePoint or Yammer APIs , I get not supported by by MSI audience error.

Even with that, it looks quite positive. Please let me know if you have any questions.

[waldekmastykarz]

Thanks for the extra tests @VelinGeorgiev! @edyoung could it be that you'd need to include additional permission scopes to support communicating with SharePoint and Yammer?

[edyoung]

Thanks for the validation, this does look promising. The cloud shell msi endpoint explicitly lists the audiences it supports. We need to add these, which should be straightforward.

[VelinGeorgiev]

@edyoung just FYI. Bear in mind that the SharePoint audiences are different subdomain per tenant i.e. https//contoso1.sharepoint.com , https'//tenant2.sharepoint.com. I guess this is wildcard when whitelisting the audience in MSI like https://*.sharepoint.com.

[waldekmastykarz]

Hey @edyoung, we have just released a new version of Office 365 CLI (v2.10) that's ready to be included in Azure Cloud Shell. Please let us know if there is anything that we can help with.

[edyoung]

We've been investigating the issue with managed identity audiences. It looks like expanding that to cover arbitrary audiences as needed for sharepoint is more complicated than we anticipated, so will take a bit of time to complete.

We could otherwise start including office 365 cli in the next update. Let me know if you would prefer to have it included with that caveat or wait for the full audience support to come in first. Unfortunately I don't know exactly how long that will take but probably a couple of months

[waldekmastykarz]

Let's include Office 365 CLI in the next update and when the SharePoint scope becomes available, that will be an additional benefit for CLI users.

[Cheneric95]

Hi, @waldekmastykarz @VelinGeorgiev I am Shaolong from cloud shell, working on the integrated the cloud shell with office 365 yammer. We are adding audience yammer into our code base. There's a question need to be explained: Is the yammer should be included in the Fairfax type or the ‘other’ type or both of them included? It looks to me that yammer could be in the ‘other’ type, since all of the Fairfax are usgov related. Or if the yammer has another usgov version?
Thanks for explaining!

[waldekmastykarz]

Awesome! Can't wait to give it a try! 👏

[edyoung]

It'll be a couple weeks until it rolls out. We'll update here once available

[waldekmastykarz]

Awesome! Once again, I appreciate your help 👏

[edyoung]

This is available now.

[waldekmastykarz]

Awesome! Thank you for the update! 👏

[garrytrinder]

We are proud to announce that Office 365 CLI is now part of the Azure Cloud Shell 🎉

Thank you to all who have contributed to the CLI to date and helped us to achieve this major milestone.

[garrytrinder]

Hey @edyoung 👋🏻

We recently released v3 of the CLI under a new name, CLI for Microsoft 365, and therefore we have a new npm package, @pnp/cli-microsoft365 and the old package @pnp/office365-cli has been deprecated, the final version being 2.13.

Are you able to arrange for the Cloud Shell image to be updated to include the new package going forwards?

As always, we really appreciate your help 👍🏻

Announcement: Next version of Office 365 CLI will be v3 #1770

[edyoung]

Sure, but since a release is currently midway it will be the next update (sometime in September) that does this. Our image is now open source so you could also create a PR at GitHub.com/azure/CloudShell if you have time 😃

[garrytrinder]

Absolutely no problem, thank you for the quick response.

Great to see the Shell Image being open sourced ❤️

[Cheneric95]

I just update the tool in the dockerfile at https://github.com/Azure/CloudShell/blob/0d52c2d7113ad12d3dcb2266ce08c50f4fa00dc5/linux/tools.Dockerfile#L50. Thanks From: Edwin Young <[email protected]> Sent: Sunday, August 23, 2020 4:55 PM To: pnp/cli-microsoft365 <[email protected]> Cc: Shaolong Chen <[email protected]>; Mention <[email protected]> Subject: Re: [pnp/cli-microsoft365] Azure Cloud Shell integration? (#1554) Sure, but since a release is currently midway it will be the next update (sometime in September) that does this. Our image is now open source so you could also create a PR at GitHub.com/azure/CloudShell if you have time 😃 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpnp%2Fcli-microsoft365%2Fdiscussions%2F1554%23discussioncomment-55113&data=02%7C01%7CShaolong.Chen%40microsoft.com%7C2c91e100b1bb4f0c016c08d847af2125%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637338164786139275&sdata=2b1wg1XpoPeRTh75zbZTDK2FGxBqGrM5IZGbC94xLRE%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPLV37AZGWP6FVFPKIXHWZDSCGFZZANCNFSM4MY7TF6A&data=02%7C01%7CShaolong.Chen%40microsoft.com%7C2c91e100b1bb4f0c016c08d847af2125%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637338164786139275&sdata=TCX%2FTj9vkPTRhaMkWg8n2G%2Fw7t7iKYgpLCpLtc7B7oo%3D&reserved=0>.

[edyoung]

When I installed just the cli-microsoft365 package, o365 was not available as a command in the shell. Is that expected? Also I noticed that when you run 'm365' the prompt still says 'o365$'.

[garrytrinder]

I've just tested the following on my local machine...

npm i -g @pnp/office365-cli > globally installs v2.13.0
npm i -g @pnp/cli-microsoft365@next > globally installs v3.0.0-beta.d74a198
m365 > displays list of commands in v3.0.0-beta.d74a198
o365 > enters immersive mode o365$ using v2.13.0
npm un -g @pnp/office365-cli > globally removes v2.13.0
o365 > command not found, expected as we have removed immersive mode in @pnp/cli-microsoft365
m365 > displays list of commands in v3.0.0-beta.d74a198

My apologies @edyoung, I mistakenly suggested that o365 alias was still in @pnp/cli-microsoft365 however as @waldekmastykarz has rightly stated, it is not and m365 will be used going forwards.

As you are seeing immersive mode when using m365, then you must have v2.13.0 installed somewhere.

[waldekmastykarz]

If you install both @pnp/office365-cli and @pnp/cli-microsoft365 side by side, I suspect that the m365 executable will keep pointing to @pnp/office365-cli (assuming you installed it first).

[edyoung]

So, it sounds like side-by-side is not the right thing to do. The reason I'm asking is that generally we want to provide some advance notice of breaking changes like removing commands, so it would be nice to provide both for a period before removing the old commands. Is that possible? If not, I guess we will upgrade anyway.

[waldekmastykarz]

What we could try is to install the old package globally and then force-install the new package globally as well:

npm i -g @pnp/office365-cli
npm i -g @pnp/cli-microsoft365 -f

The old package registers o365 and m365 executables. The new package registers only the m365 executable. If you try installing both, you will get an error that npm can't make a symlink to m365 which is already taken. If you use the -f option though, the m365 symlink is overwritten by the new package and that way you can use them side-by-side.

Not sure if there are any side-effects to this, but it seems like a viable option to offer temporary support for both packages.

[Cheneric95]

Thanks for correcting, I made a new branch in pull request branch, which recover the office 365 and add the Microsoft 365. https://github.com/Azure/CloudShell/tree/addMicrosoft365IntoCloudShell Thanks From: Edwin Young <[email protected]> Sent: Monday, August 24, 2020 11:48 PM To: pnp/cli-microsoft365 <[email protected]> Cc: Shaolong Chen <[email protected]>; Mention <[email protected]> Subject: Re: [pnp/cli-microsoft365] Azure Cloud Shell integration? (#1554) Reading the announcement more carefully, seems this is a beta. Do you expect o365 and m365 to work correctly when installed side-by-side? If so I think we should keep o365, add m365, and remove o365 at some point in the future. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpnp%2Fcli-microsoft365%2Fdiscussions%2F1554%23discussioncomment-56137&data=02%7C01%7CShaolong.Chen%40microsoft.com%7C3f82cbe29fbf4289770208d848b1ffae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637339276619611012&sdata=%2Feq%2FRn7ER3Ch2Q7C%2F6M4fP4hmB4aJqTqTQjtbLXWi4A%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPLV37FZHORWE4EB2WOD4W3SCM66ZANCNFSM4MY7TF6A&data=02%7C01%7CShaolong.Chen%40microsoft.com%7C3f82cbe29fbf4289770208d848b1ffae%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637339276619620971&sdata=UmIxIrNaVhlOh4F7cuVJlVmCxz%2Bo7IoSNXRl9wHZDp4%3D&reserved=0>.