Rasmus Tolstrup Christensen

Azure Search With Powershell

November 21, 2016 | 5 Minute Read

Azure Search is one of the newer kids in the Azure growing family. You can like any other Azure item, interact with it directly from the Azure portal. As you need to do frequent changes in the start of a project or just want to be able to automatically setup your datasource, index and indexes, the portal becomes less usefull. Luckily Azure Search provides a REST API we can easily interact with.

A basic Azure Search setup

The scope for this post will be a setup where data from an Azure hosted SQL Database will be exposed by an Index in Azure Search. To do this we need 3 parts, a datasource, an index and an indexer. The datasource is our SQL database, the index contains the data to expose and the indexer is the “scheduler” to add data to the index. As mentioned Azure Search provides a REST API. To use the API you need to provide an admin key, not a query key, this key will be part of each request to authenticate.

First step, Create A datasource

The first thing we need is a datasource. Here we use Azure Sql Server and one thing you need to think about is how you want to detect changes. Azure Search support a soft-delete approach and “Sql Server Integrated Change Tracking policy”. We’ll use the latter to have items removed from the index, when they are removed from the SQL Server. You need to ensure this is enabled on the SQL Server BEFORE you create the datasource in Azure Search.

$dbBody = @'
{
    "name": 'peopleDb',
    "description": 'people index db',
    "type": 'azuresql',
    "credentials": {"connectionString":'YOUR SQL Connection'},
    "container": {name:'Employees'},
    "dataChangeDetectionPolicy":{ "@odata.type": '#Microsoft.Azure.Search.SqlIntegratedChangeTrackingPolicy'}   
}
'@

$url = "https://company.search.windows.net/datasources?api-version=2015-02-28"

$headers = @{ "api-key" = "YOUR AZURE SEARCH API KEY"}

& Invoke-WebRequest  -Method POST -Uri $url -Body $dbBody -ContentType "application/json" -Headers $headers

Starting from the top, we first define the body of the request. it contains:

  • name the database name.
  • description A textual description of the index.
  • type database type.
  • creadentials the connectionstring to the database.
  • container the table we want to index.
  • dataChangeDetectionPolicy how we want to handle updates.

Next we define the URL, where company is the name of our Azure Search endpoint. The keyword in the url is datasources and also important is the api-version. The value is taken directly from the Azure documentation, it might change in the future.

As mentioned we need an API key for all requests. This must be included as a header in the request, see the $headers in the script above.

Finally we execute the request as POST and define the ContentType as application/json

Second step, Create the Index

The index we create will make use of the container we created above. We need to define which fields in the database should be retrieved, searchable, filterable and facetable by the index and also which field is the Key. We kan make the same set as before this time, we just target the index of the REST API.

$body = @'
{
    "name":'employeeIdx',  
    "fields":
    [
        {"name":'Id',"type":'Edm.String',"key":'true',"searchable":'false',"sortable":'false',"facetable":'false'},
        {"name":'Firstname',"type":'Edm.String'},
        {"name":'Department',"type":'Edm.String',"filterable":'false',"sortable":'false',"facetable":'false'}
        ]          
}
'@

$url = "https://company.search.windows.net/indexes?api-version=2015-02-28"

& Invoke-WebRequest  -Method Post -Uri $url -Body $body -ContentType "application/json" -Headers $headers

You might notice that I omitted the api-key field setup. This is because all of this will be executed in the same script in the real world.

third step, Create the indexer

The third and final step is creating the indexer, a bit name confusing. The indexer is the part of Azure Search which schedules how often our index will be updated from the datasource.

$indexerBody = @'
{
    "name": 'employeeindexer',
    "description": 'indexer for the index employeeIdx. Runs every 5 minute',
    "dataSourceName": 'peopleDb',
    "targetIndexName": 'employeeIdx',
    "schedule": {"interval": 'PT5M', "startTime":'2016-10-11T00:00:00Z'}
    }
'@

$url = "https://company.search.windows.net/indexers?api-version=2015-02-28"

& Invoke-WebRequest  -Method Post -Uri $url -Body $indexerBody -ContentType "application/json" -Headers $headers

This time the target is indexers. Besides specifying the datasource and target index, we also specify the schedule. Here we specify it as interval with a frequency of every 5 min, and a starttime.

build and tear down your indexes

With the script above you are able to fast and consistenly creating indexes and also persisting the script in your version control. One of the reasons I moved into powershell was because I wanted a faster development cycle and this also means I need to be able to tear down my datasource, index and indexer again to make changed and then create it all over and over.

This part is just as easy as the above, a compact edition looks like the following.

#remove indexer
$url = "https://compNY.search.windows.net/indexers/employeeindexer?api-version=2015-02-28"
& Invoke-WebRequest  -Method DELETE -Uri $url -ContentType "application/json" -Headers $headers

#remove index
$url = "https://company.search.windows.net/indexes/employeeIdx?api-version=2015-02-28"
& Invoke-WebRequest  -Method DELETE -Uri $url -ContentType "application/json" -Headers $headers

#remove datasource
$url = "https://company.search.windows.net/datasources/peopleDb?api-version=2015-02-28"
& Invoke-WebRequest  -Method DELETE -Uri $url -ContentType "application/json" -Headers $headers

This time the name of the indexer, index and datasource is specified as part of the URL and the Request method is specified as DELETE.

With these scripts you are able to build and tead down the basic elements in Azure Search easily.

comments powered by Disqus