Quick Setup for Azure Resource Manager Mode

Azure Resource Template (JSON based Infrastructure as Code) runs with Azure Resource Manager which Azure Python SDK 2.0.0+ supports with new packages and functions. Simple Azure uses the new version of Azure Python SDK to deploy software and infrastructure with Azure Resource Templates.

Previous development is now called ‘legacy’ or ‘classic’ mode of Azure Python SDK with limited features (although it still works to start or terminate Azure Virtual Machines).

This document explains a few changes of using ARM mode and describes how to setup account credentials differently compared to the classic mode. It is also worth to mention that guidelines and instructions from Azure official document or other online articles are insufficient to follow, this is understandable because ARM supports with Azure Python SDK is fairly new (as of September 2016) and some Azure services are in ‘preview’ mode.

Installation of Azure Python SDK

From Pypi:

pip install --pre azure

If you already have the azure package but need to upgrade then add -U option:

pip install --pre azure -U

If you are looking for the latest development, probably downloading code from github.com would be best:

git clone git://github.com/Azure/azure-sdk-for-python.git
cd azure-sdk-for-python
python setup.py install

Additional Packages

From Pypi:

pip install msrest
pip install msrestazure

From github.com repository:

pip install -r requirements.txt

You may encounter some errors like this, if you don’t install additional packages:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/azure/common/credentials.py", line 25, in <module>
raise ImportError("You need to install 'msrest' to use this feature")
ImportError: You need to install 'msrest' to use this feature

Authentication with Service Principal Credentials (ServicePrincipalCredentials)

Similar to AWS IAM service, Azure allows users to have resource access through Active Directory and Service Principal credentials which only require (encrypted) key strings such as clientID, secretKey or tenantID instead of certificate files generated by openssl e.g. .pem. Let’s walk through SDK functions to see how it works.

ServicePrincipalCredentials() from azure.common.credentials requires three arguments: client_id, secret, and tenant to authenticate.

client_id is an unique application id, secret is a encrypted key string registered to the application and tenant is an unique user id.

Getting these values is explained from here: https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/ but app registrations are not described entirely because Admin consent needs to be done additionally. Otherwise, registered apps are not visible in the subscriptions page to add access with Roles.

Note

Remember client_id, secret and tenant values including subscription id because these values are required to authenticate in Simple Azure ARM mode.

Reconsent Step

Follow the steps below:

  • Go to the classic portal
  • Select ‘Active Directory’ and find ‘applications’ tab at the top of the page
  • Search apps by selecting ‘Applications my company owns’ in the search box
  • Select your application and find ‘Users and Groups’ tab at the top of the page
  • Reconsent if the page asks like Admin consent is required prior to assigning users and groups. You can consent via the application by clicking here:

With Azure CLI

It is easier to create a new app and a service principal with access to your subscriptions via Azure CLI. The official documentation is here: https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal-cli/

The two commands complete this step like:

$ azure ad sp create -n <app name> -p <password> --home-page <http or https url> --identifier-uris <http or https url>
$ azure role assignment create --objectId <uuid returned from previous command> -o <Role e.g. Owner or Reader> -c /subscriptions/<subscription ID>/

ServicePrincipalCredentials()

Try to authenticate with the client_id, secret and tenant in Python like :

from azure.common.credentials import ServicePrincipalCredentials as spc
cred = spc(client_id = 'abcdefghi-1234-4555-8173-jklmnopqrstu',secret='abcdEFGHIJ//klmnopqrSTU/',tenant='1234567-abcd-7890-ABCD-1234567890')

If your credentials are invalid, you may see errors like this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 403, in __init__
    self.set_token()
  File "/usr/local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 434, in set_token
    raise_with_traceback(AuthenticationError, "", err)
  File "/usr/local/lib/python2.7/site-packages/msrest/exceptions.py", line 50, in raise_with_traceback
    raise error
msrest.exceptions.AuthenticationError: , InvalidClientIdError: (invalid_request) AADSTS90002: No service namespace named '<wrong id>' was found in the data store.
Trace ID: <UUID>
Correlation ID: <UUID>
Timestamp: 2016-10-04 15:41:24Z

or :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 403, in __init__
    self.set_token()
  File "/usr/local/lib/python2.7/site-packages/msrestazure/azure_active_directory.py", line 434, in set_token
    raise_with_traceback(AuthenticationError, "", err)
  File "/usr/local/lib/python2.7/site-packages/msrest/exceptions.py", line 50, in raise_with_traceback
    raise error
  msrest.exceptions.AuthenticationError: , InvalidClientError: (invalid_client) AADSTS70002: Error validating credentials. AADSTS50012: Invalid client secret is provided.
  Trace ID: <UUID>
  Correlation ID: <UUID>
  Timestamp: 2016-10-04 15:41:33Z

This may occur because your secret is not registered properly or client_id or tenant is not found.

Create a new Resource Group

The first step prior to any deployment would be creating a new resource group and it can be done via ResourceManagmentClient() from azure.mgmt.resource

Let’s try to create a sample group named ‘quickstart-rg-1’ by the following code:

from azure.mgmt.resource import ResourceManagementClient as rmc
client = rmc(cred, 'subscription_id')
client.resource_groups.create_or_update(
          'quickstart-rg-1',
          {
                      'location':'eastus'
          }
      )

Replace the ‘subscription_id’ with a real value.

If you do not have proper permissions, error message looks like:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/azure/mgmt/resource/resources/operations/resource_groups_operations.py", line 223, in create_or_update
    raise exp
msrestazure.azure_exceptions.CloudError: The client '<uuid>' with object id '<uuid>' does not have authorization to perform action 'Microsoft.Resources/subscriptions/resourcegroups/write' over scope '/subscriptions/<subscription_id>/resourcegroups/quickstart-rg-1'.

If your subscription principal is not consent:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/azure/mgmt/resource/resources/operations/resource_groups_operations.py", line 223, in create_or_update
    raise exp
  msrestazure.azure_exceptions.CloudError: The received access token is not valid: at least one of the claims 'puid' or 'altsecid' or 'oid' should be present. If you are accessing as application please make sure service principal is properly created in the tenant.

Authentication in Simple Azure

Simple Azure requires the following information to authenticate:

  • subscription id (identication to your account, e.g. azure account show shows ID)
  • client id (equal to client_id)
  • tenant id (equal to tenant)
  • client secret key (equal to secret)

With Environment Variables

It is recommmend to store the credentials using environment variables instead passing through as Python parameters in code. Use the following environment variable names to store:

  • subscription id: AZURE_SUBSCRIPTION_ID
  • client id: AZURE_CLIENT_ID
  • tenant id: AZURE_TENANT_ID
  • client secret key: AZURE_CLIENT_SECRET

In a simple form, save these in a file and load it before using Simple Azure in a shell. For example:

$ cat <<EOF > ~/.saz/cred
export AZURE_SUBSCRIPTION_ID=5s3ag2s5-2aa1-4828-xxxx-9g8sw72w5w5g
export AZURE_CLIENT_ID=5c5a3ea3-ap34-4pd0-xxxx-2p38ac00aap1
export AZURE_TENANT_ID=5e39a20e-c55a-53de-xxxx-2503a55et6ta
export AZURE_CLIENT_SECRET=xxxx
EOF

Then source it like:

$ source ~/.saz/cred

env command displays environment variables exposed, e.g.:

$ env|grep AZURE
AZURE_SUBSCRIPTION_ID=5s3ag2s5-2aa1-4828-xxxx-9g8sw72w5w5g
AZURE_CLIENT_ID=5c5a3ea3-ap34-4pd0-xxxx-2p38ac00aap1
AZURE_TENANT_ID=5e39a20e-c55a-53de-xxxx-2503a55et6ta
AZURE_CLIENT_SECRET=xxxx

Tips on Getting Credential via Azure CLI

Subscription id and tenant id are found by, for example:

$ azure account show
info:    Executing command account show
data:    Name                        : Simple-Azure
data:    ID                          : 5s3ag2s5-2aa1-4828-xxxx-9g8sw72w5w5g
data:    State                       : Enabled
data:    Tenant ID                   : 5e39a20e-c55a-53de-xxxx-2503a55et6ta
data:    Is Default                  : true
data:    Environment                 : AzureCloud
data:    Has Certificate             : Yes
data:    Has Access Token            : Yes
data:    User name                   : hroe.lee@gmail.com
data:
info:    account show command OK
  • ID represents AZURE_SUBSCRIPTION_ID.
  • Tenant ID represents AZURE_TENANT_ID.

Client id is found by, for example:

$ azure ad app list
info:    Executing command ad app list
+ Listing applications
data:    AppId:                   5c5a3ea3-ap34-4pd0-xxxx-2p38ac00aap1
dqtq:    ObjectId:                dc25d100-1234-4567-bf11-1234e1234dbq
data:    DisplayName:             simpleazure
data:    IdentifierUris:          0=https://simplezure.com/login
data:    ReplyUrls:
data:    AvailableToOtherTenants: False
data:    HomePage:                http://simpleazure.com
data:
info:    ad app list command OK

AppId represents AZURE_CLIENT_ID.

AZURE_CLIENT_SECRET is not visible because it is one-time displayed value from the portal. It is also same as the <password> used in the service principal credential in Azure CLI.