At the current customer I am working for I am creating a lot of VSTS Extensions to deploy Azure Resources.
I want my tasks to be so much user friendly as possible, and one of the things to accomplish this is with Data Source Bindings.
What are Data Source Bindings?
Data source bindings bind a drop-down input field in the UI (e.g. task input) which needs to be dynamically populated with values from ta REST API that needs to be invoked to fetch the list of values.
Where are the Data Sources defined?
The Data sources are defined in the Service Endpoints.
Service endpoints support querying data from external services through REST API.
The data queried can be used to populate task input drop-downs.
In Action
Using the VSTS Rest API we can see all the Service Enpoints Types available:
GET https://{accountName}.visualstudio.com/_apis/serviceendpoint/types?api-version=5.0-preview.1
To make the call working you first need to create a PAT token in you VSTS Account, and add a Basic Authentication Header.
Leave the username empty and the password is the PAT token.
Replace {accountName} in the URL with your VSTS account name.
Like:
GET /_apis/serviceendpoint/types?api-version=5.0-preview.1 HTTP/1.1 Host: olandese.visualstudio.com Authorization: Basic BASE64_EMPTYUSERNAME:PAT Cache-Control: no-cache
The result will be a list of all endpoint types and their data source, here a snippet:
{
"inputDescriptors": [
{
"id": "environment",
"name": "Environment",
"description": "Microsoft Azure Environment for the subscription",
"type": null,
"properties": null,
"inputMode": "combo",
"isConfidential": false,
"useInDefaultDescription": false,
"groupName": null,
"valueHint": null,
"validation": {
"dataType": "string",
"maxLength": 300
},
"values": {
###Code Removed for Readability
],
"dataSources": [
{
"name": "AzureResourceGroups",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/resourcegroups?api-version=2016-02-01",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[*].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureRMResourcesInRGBasedOnType",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/resourceGroups/{{{ResourceGroupName}}}/resources?$filter=resourceType EQ '{{{ResourceType}}}'&api-version=2017-05-10",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[*]",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureStorageAccountRM",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/providers/Microsoft.Storage/storageAccounts?api-version=2015-06-15",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[*].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureStorageAccountRMandClassic",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/resources?api-version=2014-04-01-preview&%24filter=(resourceType%20eq%20'microsoft.storage%2Fstorageaccounts'%20or%20resourceType%20eq%20'microsoft.classicstorage%2Fstorageaccounts')",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[*].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureRMStorageAccountByLocation",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/providers/Microsoft.Storage/storageAccounts?api-version=2015-06-15",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[?(@.location =='{{location}}')].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureRMStorageAccountIdByName",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/providers/Microsoft.Storage/storageAccounts?api-version=2015-06-15",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[?(@.name =='{{storageAccountName}}')].id",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureRMWebAppNames",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/providers/Microsoft.Web/sites?api-version=2015-08-01",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[*].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
},
{
"name": "AzureRMWebAppNamesByType",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/providers/Microsoft.Web/sites?api-version=2015-08-01",
"requestVerb": null,
"requestContent": null,
"resourceUrl": "",
"resultSelector": "jsonpath:$.value[?(@.kind=='{{{ #stringReplace 'linux' ',linux' WebAppKind}}}' || @.kind=='{{{ #stringReplace 'linux' 'app,linux' WebAppKind}}}' || @.kind=='{{{ #stringReplace 'linux' ',linux,container' WebAppKind}}}')].name",
"callbackContextTemplate": null,
"callbackRequiredTemplate": null,
"initialContextTemplate": null,
"headers": [],
"authenticationScheme": null
###Code Removed for Readability
],
"dependencyData": [
{
###Code Removed for Readability
}
],
"name": "azurerm",
"displayName": "Azure Resource Manager",
"description": "Service Endpoint type for Azure Resource Manager connections",
###Code Removed for Readability
}
}
Above you can see the AzureRM endpoint type definition and the “dataSource” array section (from row 21).
You can refer to this data sources in your task if your task use this endpoint type.
Let’s analyse one of the datasources (line 22-34):
{
"name":"AzureResourceGroups",
"endpointUrl":"{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/resourcegroups?api-version=2016-02-01",
"requestVerb":null,
"requestContent":null,
"resourceUrl":"",
"resultSelector":"jsonpath:$.value[*].name",
"callbackContextTemplate":null,
"callbackRequiredTemplate":null,
"initialContextTemplate":null,
"headers":[
],
"authenticationScheme":null
}
We can see the name AzureResourceGroups which we can refer to it in the task, and the REST call.
In the URL there are also some replacement variables like {{{endpoint.url}}} and {{{endopoint.subscriptionId}}}.
This one are filled automatically from the endpoint itself (actually all the variables starting with {{{eindpoint.*}}} will be filled automatically).
We see the “resultSelector”: “jsonpath:$.value[*].name”, where the result of the REST call can be filtered using jsonpath. In this datasource only the names will be returned.
There are also some other variables that you can give from the task, like in the datasource named “AzureRMResourcesInRGBasedOnType” (see lines 36-47), where {{{ResourceGroupName}}} and {{{ResourceType}}} can be provided.
We will use both this datasources in the example below.
Develop
Let’s develop a simple Task to see all this in action.
Here you can get the source.
{
"id":"9656e534-6fa2-4d97-afdb-0ecd2c1f75bd",
"name":"Task-With-DataSources",
"friendlyName":"Task with data sources",
"description":"Example of a task using data source bindings",
"helpMarkDown":"",
"category":"Utility",
"visibility":[
"Build",
"Release"
],
"author":"Marco Mansi",
"version":{
"Major":1,
"Minor":0,
"Patch":4
},
"instanceNameFormat":"",
"groups":[
],
"inputs":[
{
"name":"ConnectedServiceNameSelector",
"aliases":[
"azureConnectionType"
],
"type":"pickList",
"label":"Azure Connection Type",
"required":false,
"helpMarkDown":"",
"defaultValue":"ConnectedServiceNameARM",
"options":{
"ConnectedServiceNameARM":"Azure Resource Manager"
},
"visibleRule":"ConnectedServiceNameSelector = IdontWantToShowThisInput"
},
{
"name":"ConnectedServiceNameARM",
"aliases":[
"azureSubscription"
],
"type":"connectedService:AzureRM",
"label":"Azure Subscription",
"defaultValue":"",
"required":true,
"helpMarkDown":"Azure Resource Manager subscription to configure before running PowerShell"
},
{
"name":"ResourceGroupName",
"type":"pickList",
"label":"Resource Group",
"required":true,
"helpMarkDown":"Provide the name of the Resource Group.",
"properties":{
"EditableOptions":"True"
}
}
],
"dataSourceBindings":[
{
"target":"ResourceGroupName",
"endpointId":"$(ConnectedServiceNameARM)",
"dataSourceName":"AzureResourceGroups"
}
],
"execution":{
"PowerShell3":{
"target":"Main.ps1"
}
}
}
For this demo I have added an input control named “ResourceGroupNameInput” and the type is “pickList”.
The real magic happens at the “dataSourceBindings”, there I tell the task to update the input named “ResourceGroupNameInput” using the REST call from the datasource “AzureResourceGroups” provided by the AzureRM endpoint selected.
I publish my extension and after installing and create an AzureRM endpoint this will be the result:
Now we have a smart dropdown menu which is automatically filled using the credentials of the AzureRM endpoint.
But let’s make it even smarter: let’s say I want another drop-down where I select specific azure resources (like a webapp) from the specified Resource Group.
For this we can use the other DataSource “AzureRMResourcesInRGBasedOnType” which needs a ResourceGroup and a Resource Type as input.
We can chain the inputs so that the value for the Resource Group of the AzureRMResourcesInRGBasedOnType datasource will be based on the Resource Group selected in the first dropdown:
{
"id": "9656e534-6fa2-4d97-afdb-0ecd2c1f75bd",
"name": "Task-With-DataSources",
"friendlyName": "Task with data sources",
"description": "Example of a task using data source bindings",
"helpMarkDown": "",
"category": "Utility",
"visibility": [
"Build",
"Release"
],
"author": "Marco Mansi",
"version": {
"Major": 1,
"Minor": 0,
"Patch": 5
},
"instanceNameFormat": "",
"groups": [],
"inputs": [
{
"name": "ConnectedServiceNameSelector",
"aliases": ["azureConnectionType"],
"type": "pickList",
"label": "Azure Connection Type",
"required": false,
"helpMarkDown": "",
"defaultValue": "ConnectedServiceNameARM",
"options": {
"ConnectedServiceNameARM": "Azure Resource Manager"
},
"visibleRule": "ConnectedServiceNameSelector = IdontWantToShowThisInput"
},
{
"name": "ConnectedServiceNameARM",
"aliases": ["azureSubscription"],
"type": "connectedService:AzureRM",
"label": "Azure Subscription",
"defaultValue": "",
"required": true,
"helpMarkDown": "Azure Resource Manager subscription to configure before running PowerShell"
},
{
"name": "ResourceGroupNameInput",
"type": "pickList",
"label": "Resource Group",
"required": true,
"helpMarkDown": "Provide the name of the Resource Group.",
"properties": {
"EditableOptions": "True"
}
},
{
"name": "WebAppInput",
"type": "pickList",
"label": "Web App",
"required": true,
"helpMarkDown": "Provide the name of the Web App.",
"properties": {
"EditableOptions": "True"
}
}
],
"dataSourceBindings": [
{
"target": "ResourceGroupNameInput",
"endpointId": "$(ConnectedServiceNameARM)",
"dataSourceName": "AzureResourceGroups"
},
{
"target": "WebAppInput",
"endpointId": "$(ConnectedServiceNameARM)",
"dataSourceName": "AzureRMResourcesInRGBasedOnType",
"parameters": {
"ResourceGroupName": "$(ResourceGroupNameInput)",
"ResourceType": "Microsoft.Web/sites"
},
"resultTemplate": "{ "Value" : "{{{name}}}", "DisplayValue" : "{{{name}}}" }"
}
],
"execution": {
"PowerShell3": {
"target": "Main.ps1"
}
}
}
We added a new picklist input named “WebAppInput” and the magic happens again in the “dataSourceBindings”.
I added another data source binding and as parameter for “ResourceGroupName” I provided the value from the input picklist “ResourceGroupNameInput” (in VSTS every field is a variable) .
The other parameter “ResourceType” is hard-coded because I want only to filter on Azure Web Apps (“Microsoft.Web/sites”).
But there’s something more: The datasource “AzureRMResourcesInRGBasedOnType” is giving us back everything from the REST call (“jsonpath:$.value[*]”).
The “resultTemplate” property allow us to format the result and to filter only the name from the REST response and use this for the “Value” and “DisplayValue” for the dropdown.
Here the result:
Even more with custom datasources!
We have seen how to use datasources provided out-of-the-box from the endpoint type.
But let’s say I want to make a specific REST call which is not yet available as datasource in the datasource type definition, I can even specify a “custom” datasource still using the endpoint’s credentials.
Let’s say we want another drop-down which shows all the Azure Maps Accounts in a Resource Group.
The Azure Maps provider is quiet new and not yet available as datasource in the AzureRM datasource type.
We can add a custom datasource with the REST call to be applied:
{
"id": "9656e534-6fa2-4d97-afdb-0ecd2c1f75bd",
"name": "Task-With-DataSources",
"friendlyName": "Task with data sources",
"description": "Example of a task using data source bindings",
"helpMarkDown": "",
"category": "Utility",
"visibility": [
"Build",
"Release"
],
"author": "Marco Mansi",
"version": {
"Major": 1,
"Minor": 0,
"Patch": 13
},
"instanceNameFormat": "",
"groups": [],
"inputs": [
{
"name": "ConnectedServiceNameSelector",
"aliases": ["azureConnectionType"],
"type": "pickList",
"label": "Azure Connection Type",
"required": false,
"helpMarkDown": "",
"defaultValue": "ConnectedServiceNameARM",
"options": {
"ConnectedServiceNameARM": "Azure Resource Manager"
},
"visibleRule": "ConnectedServiceNameSelector = IdontWantToShowThisInput"
},
{
"name": "ConnectedServiceNameARM",
"aliases": ["azureSubscription"],
"type": "connectedService:AzureRM",
"label": "Azure Subscription",
"defaultValue": "",
"required": true,
"helpMarkDown": "Azure Resource Manager subscription to configure before running PowerShell"
},
{
"name": "ResourceGroupNameInput",
"type": "pickList",
"label": "Resource Group",
"required": true,
"helpMarkDown": "Provide the name of the Resource Group.",
"properties": {
"EditableOptions": "True"
}
},
{
"name": "WebAppInput",
"type": "pickList",
"label": "Web App",
"required": true,
"helpMarkDown": "Provide the name of the Web App.",
"properties": {
"EditableOptions": "True"
}
},
{
"name": "MapsAccountInput",
"type": "pickList",
"label": "Azure Maps Account",
"required": true,
"helpMarkDown": "Provide the name Azure Maps Accpunt.",
"properties": {
"EditableOptions": "True"
}
}
],
"dataSourceBindings": [
{
"target": "ResourceGroupNameInput",
"endpointId": "$(ConnectedServiceNameARM)",
"dataSourceName": "AzureResourceGroups"
},
{
"target": "WebAppInput",
"endpointId": "$(ConnectedServiceNameARM)",
"dataSourceName": "AzureRMResourcesInRGBasedOnType",
"parameters": {
"ResourceGroupName": "$(ResourceGroupNameInput)",
"ResourceType": "Microsoft.Web/sites"
},
"resultTemplate": "{ "Value" : "{{{name}}}", "DisplayValue" : "{{{name}}}" }"
},
{
"target": "MapsAccountInput",
"endpointId": "$(ConnectedServiceNameARM)",
"endpointUrl": "{{{endpoint.url}}}/subscriptions/{{{endpoint.subscriptionId}}}/resourceGroups/$(ResourceGroupNameInput)/providers/Microsoft.Maps/accounts?api-version=2018-05-01",
"resultSelector": "jsonpath:$.value[*].name"
}
],
"execution": {
"PowerShell3": {
"target": "Main.ps1"
}
}
}
There is a new pickList input named “MapsAccountInput” and in the “dataSourceBindingSection” a new datasource.
But this one is different, it doesn’t have a “dataSourceName” property, instead we specify a “endpointUrl” and in there is the actual REST call.
We use the {{{endpoint.*}}} variables from the endpoint and $(ResourceGroup) input variable directly in the URL to build the REST call.
In the “resultSelector” I specify I only want the name back.
And here the result:
Conclusion
Using DataSourceBindings in VSTS tasks makes them more “smart” and user-friendly.
And even if there’s no out-of-the-box datasource, you can provide your own one.
Marco Mansi's Blog 
Good stuff Marco
LikeLike