Live Feeds in loklak Media wall using ‘source=twitter’

Loklak Server provides pagination to provide tweets from Loklak search.json API in divisions so as to improve response time from the server. We will be taking advantage of this pagination using parameter `source=twitter` of the search.json API on loklak media wall. Basically, using parameter ‘source=twitter’ in the API does real time scraping and provides live feeds. To improve response time, it returns feeds as specified in the count (default is 100).

In the blog, I am explaining how implemented real time pagination using ‘source = twitter’ in loklak media wall to get live feeds from twitter.

Working

First API Call on Initialization

The first API call needs to have high count (i.e. maximumRecords = 20) so as to get a higher number of feeds and provide a sufficient amount of feeds to fill up the media wall. ‘source=twitter’ must be specified so that real time feeds are scraped and provided from twitter.

http://api.loklak.org/api/search.json?q=fossasia&callback=__ng_jsonp__.__req0.finished&minified=true&source=twitter&maximumRecords=20&timezoneOffset=-330&startRecord=1

 

If feeds are received from the server, then the next API request must be sent after 10 seconds so that server gets sufficient time to scrap the data and store it in the database. This can be done by an effect which dispatches WallNextPageAction(‘’) keeping debounceTime equal to 10000 so that next request is sent 10 seconds after WallSearchCompleteSuccessAction().

@Effect()
nextWallSearchAction$
= this.actions$
.ofType(apiAction.ActionTypes.WALL_SEARCH_COMPLETE_SUCCESS)
.debounceTime(10000)
.withLatestFrom(this.store$)
.map(([action, state]) => {
return new wallPaginationAction.WallNextPageAction();
});

Consecutive Calls

To implement pagination, next consecutive API call must be made to add new live feeds to the media wall. For the new feeds, count must be kept low so that no heavy pagination takes place and feeds are added one by one to get more focus on new tweets. For this purpose, count must be kept to one.

this.searchServiceConfig.count = queryObject.count;
this.searchServiceConfig.maximumRecords = queryObject.count;return this.apiSearchService.fetchQuery(queryObject.query.queryString, this.searchServiceConfig)
.takeUntil(nextSearch$)
.map(response => {
return new wallPaginationAction.WallPaginationCompleteSuccessAction(response);
})
.catch(() => of(new wallPaginationAction.WallPaginationCompleteFailAction()));
});

 

Here, count and maximumRecords is updated from queryObject.count which varies between 1 to 5 (default being 1). This can be updated by user from the customization menu.

Next API request is as follows:

http://api.loklak.org/api/search.json?q=fossasia&callback=__ng_jsonp__.__req2.finished&minified=true&source=twitter&maximumRecords=1&timezoneOffset=-330&startRecord=1

 

Now, as done above, if some response is received from media wall, next request is sent after 10 seconds after WallPaginationCompleteSuccess() from an effect by keeping debounceTime equal to 10000.

In the similar way, new consecutive calls can be made by keeping ‘source = twitter’ and keeping count low for getting a proper focus on new feed.

Reference

Live Feeds in loklak Media wall using ‘source=twitter’