Integrate Powerful Algolia search engine to Your movie catalog.
Let’s create a movie catalog with huge data set and then use Algolia to implement an unimaginable speed of search engine. Algolia is capable of giving you results from the first keystroke. Speed is not the only thing special about Algolia but its ability tracks the keywords as well. Algolia Allows you to create indices with your datasets. You can search for any word existing in these indices and you will get the desired output for it. Powerful AI system in Algolia will help you even on typos. You can do all this with one interface. You don’t have to specify any specific field or property as you do in the databases. By integrating Algolia to your application, you can get rid of the burden of developing advanced searching systems.
Get your API key
First things first. Algolia provides search engine capabilities through its powerful Restful API. Before using any API, you have to have the API key for it. To get the API key you will need to sign up at Algolia. Click here to go to the sign-up page.
Once you signed up, you will be given a dashboard.
There are many options available to you on your dashboard. You can create indices easily with the Dashboard. Analytics provide you with statistical information about the uses of your Indices and applications. At the moment we will only pay attention to API keys. Go to the API key section, you will see an application ID and two API keys. One API key is search only, while the other one is called admin API key. As the name suggests, search-only API key can be used for searching across indices. You can do everything with admin key including adding data, updating data and deleting data. So, remember to get the Admin API key for this application.
So, let’s get started with the application. Algolia supports almost all the popular programming languages. In this tutorial, we will be using python. Our first task will be to install python client provided by Algolia.
Install python client
Although you can directly access Algolia endpoints from your application, it is recommended the practice to use Algolia API clients. These clients are advanced wrappers around their Rest API. There is open source, performance optimized and provides rich API to interact with Algolia. Installing it is quite simple. Just type the following command in your terminal.
if you don’t have installed pip on your machine, please get it from here.
Our next step will be to instantiate our python client. Open your favorite text editor and add the following code.
Your first step will be to import the algoliasearch client. Then you will create a client object from it. For that, you will need to add your API key. Note that, API key is your identification. Algolia recognizes you by using your API key. Therefore, never give it to anyone else.
Instantiate the Client
Our next step is to instantiate our index. Algolia provides init_index method of the client object for that. Check the following code.
I have used “movie” as my index name. if there is an index with the given name in your account, Algolia will retrieve it and send it to us. Otherwise, Algolia will create a new index. Algolia strongly advises not to give sensitive data as the index name. you should treat it as a publicly available name.
you can see I have call clear_index method on my index. If you already have an index with the given name. Algolia will provide that index with existing data of it. I have run this application several times, so I want to clear the existing data of the index to prevent the duplication of the data.
Fill index with our Data
Next, we come to the most important part of our application. It is adding large sets of data. For this application, I have movie data in 16 files. I have extracted these data and stored them in files. You can use them to try this tutorial when you download this application from Github.
When you are adding data to Algolia, you have to convert them into JSON format. So, I am going to do it next.
Now, we have to add our JSON objects to Algolia index. When you are working with large data sets you have to be careful on several things. You have to be careful not to send a huge number of API calls which will lead to performance issues. On the other hand, you should not send too many data at once which will make the Algolia server overloaded. You have to do it in batches. In this application, I will be using 4 batches. Each one will be having the data of 4 files.
Then we will add those batches to our index using add_objects() method in index object.
Algolia provides Custom Batches out of the box to solve similar problems. There are two reasons why custom batches are useful when working in large datasets.
It will reduce the latency by minimizing the network trips between your application server and Algolia server. Second, it will ensure data integrity as the operations inside batches is done atomically.
Also, Algolia warns us to be cautious of the rate limit. Depending on the objects you send to Algolia, and the operations you perform, there is a limit to a batch. If you exceed that limit, you will be getting errors.
Customize your indexes with powerful configuration options
Algolia has a powerful AI to build very effective indexes. Still, you can give instructions to alogia on how indices should provide results. We have used two configuration options here. those are custom ranking and searchable attributes. ‘customRanking’ attribute sorts result based on the argument given to it. The argument has two parts. Algolia uses a powerful tie-breaking algorithm for that.
I have used sorting based on the ‘title’. I have selected the descending order as well. You can select anything you want based on how you want to rank your data set.
‘searchableAttributes’ are for telling Algolia which attributes in our database is useful for searching. Of course, all the data in our dataset is not useful. With searchable attributes, you can tell Algolia which attributes it should be mainly looking at. When you add searchable attributes, you can use them to search in the index even with typos.
Let’s search with whatever you know about movies
Well, we have done all the important things in our applications. We can search the dataset using our index now. Let’s do it.
There is a movie called “A romance in happy valley”. Let’s search for it. You normally don’t add complete text when searching on google. So, we will do the same here. I will search for “happy valley” And see if I can get the desired movie.
Great! Algolia sends me a JSON Object as the response. It includes many details. If you read the JSON object you will see it includes, title property with full movie name. There was an actor called “Lillian Gish” in my data set. Let’s search for here and see what output we will get.
Algolia says she has acted in “True Heart Sussie” Which is correct. Note that, you are not telling Algolia whether you are looking for a cast or a title. Algolia gets to know it on its own. This behavior is much advanced than normal database queries. That why it is so perfect for a search engine. You can download this application from Github.