Search vs Aggregations

    The search aggregations also know as facets(Field aggregations) are often ignored while using the Search API provided by Loklak. This post aims at clearing up the difference and bring out the clear usage. This extends the previous blog, titled “Know who tweeted the most!” where we discussed about the Search API. Using the same example, the results were viewed using the search aggregations. The following two pictures illustrates the difference.

Selection_074

Selection_058

    The above results are achieved by scraping the aggregation field called mentions, which contains the list of user names who mentioned the search hashtag in their tweets. The result is from the one billion tweets(~1,377,367,341) which Loklak had scraped and indexed. The following screenshots show the JSON object of the search aggregations.

Selection_076

    The above search metadata object shows some important fields which can alter the result. The count is set to zero and the cache is disabled so that the aggregations do not consider the remote search results. The query fields shows the constraints on which the query must be performed. It shows the hashtag for which it must query. Since the search aggregations are going to query the whole Loklak index(over one billion tweets) we can limit the result payload by giving time constraints such as “since” and “until” fields. We can even limit the result display by mentioning a value to the limit field. This helps you to form a search query which gives you a much accurate result. The following is a sample query where you can check out the JSON result. Edit the “q” variable to your search requirement and the time fields too.

http://loklak.org/api/search.json?q=loklak since:2016-01-01 until:2016-03-01&source=cache&count=0&fields=mentions,hashtags

The following shows the aggregations field and the benefits you can get out of it.

Selection_078

    The aggregations field consists of the hashtags and the mentions. The mentions contains all user names denoted by a ‘@’-prefix from the message, which is being listed without the leading ‘@’. The hashtags contains all hashtags appearing in the message, without the leading ‘#’. You have more fields which you can mention in the query according to your need such as limit, source, timezoneOffset, minified etc. For more info on the search aggregations visit the docs page.

You can even visit to the app “knowTheDiff” which guides you through the difference with step by step intro.

Selection_075

The app is still under construction and needs to get more fields into the picture, but can try out the basic difference. Here are some sample queries where you can try out.

Search vs Aggregations

Developer Tools: Build your query using LQL

Writing request queries is definitely a hard job for developers trying to use any API. Sometimes the query strings go wrong and sometimes you’re not getting the output you’ve been expecting. We understood a similar problem in Loklak API for developers and built the Loklak Query Language (LQL).

This tool takes in the fields and the type of query you want to make and dynamically creates the request URL in front of you. You could even test this URL and see the pretty printed JSON responses returned by the server. Here’s the best part, You get to use this with the custom URL of any loklak server that you deploy.

screenshot

The team has put in quite some effort in scripting easy deployment buttons with Heroku, Bluemix, Scalingo, Docker Cloud etc.., and Tools like this help developers build the queries and look for the data they wanted.

Screen Shot 2016-06-01 at 9.09.29 AM

There are a lot of features that are in store and shows the query for a lot of API endpoints. It’s a great tool to play with, In the future, we’d also be integrating the way the queries have to be made for different programming language APIs that we support. So that way you can directly access the required code segments directly and use them with the supporting library.

It’s a lightweight application, every time a change is made in any of the fields in the form, the query gets generated completely on the client side and at the same time the fields change based on the API call that has been chosen.

Have a look at the LQL here or head over to our github and give us feedback or open an issue.

Developer Tools: Build your query using LQL