Adding Tweet Streaming Feature in World Mood Tracker loklak App

The World Mood Tracker was added to loklak apps with the feature to display aggregated data from the emotion classifier of loklak server. The next step in the app was adding the feature to display the stream of Tweets from a country as they are discovered by loklak. With the addition of stream servlet in loklak, it was possible to utilise it in this app.

In this blog post, I will be discussing the steps taken while adding to introduce this feature in World Mood Tracker app.

Props for WorldMap component

The WorldMap component holds the view for the map displayed in the app. This is where API calls to classifier endpoint are made and results are displayed on the map. In order to display tweets on clicking a country, we need to define react props so that methods from higher level components can be called.

In order to enable props, we need to change the constructor for the component –

export default class WorldMap extends React.Component {
    constructor(props) {
        super(props);
        ...
    }
    ...
}

[SOURCE]

We can now pass the method from parent component to enable streaming and other components can close the stream by using props in them –

export default class WorldMoodTracker extends React.Component {
    ...
    showStream(countryName, countryCode) {
        /* Do something to enable streaming component */
        ...
    }
 
    render() {
        return (
             ...
                <WorldMap showStream={this.showStream}/>
             ...
        )
    }
}

[SOURCE]

Defining Actions on Clicking Country Map

As mentioned in an earlier blog post, World Mood Tracker uses Datamaps to visualize data on a map. In order to trigger a piece of code on clicking a country, we can use the “done” method of the Datamaps instance. This is where we use the props passed earlier –

done: function(datamap) {
    datamap.svg.selectAll('.datamaps-subunit').on('click', function (geography) {
        props.showStream(geography.properties.name, reverseCountryCode(geography.id));
    })
}

[SOURCE]

The name and ID for the country will be used to display name and make API call to stream endpoint respectively.

The StreamOverlay Component

The StreamOverlay components hold all the utilities to display the stream of Tweets from loklak. This component is used from its parent components whose state holds info about displaying this component –

export default class WorldMoodTracker extends React.Component {
    ...
    getStreamOverlay() {
        if (this.state.enabled) {
            return (<StreamOverlay
                show={true} channel={this.state.channel}
                country={this.state.country} onClose={this.onOverlayClose}/>);
        }
    }

    render() {
        return (
            ...
                {this.getStreamOverlay()}
            ...
        )
    }
}

[SOURCE]

The corresponding props passed are used to render the component and connect to the stream from loklak server.

Creating Overlay Modal

On clicking the map, an overlay is shown. To display this overlay, react-overlays is used. The Modal component offered by the packages provides a very simple interface to define the design and interface of the component, including style, onclose hook, etc.

import {Modal} from 'react-overlays';

<Modal aria-labelledby='modal-label'
    style={modalStyle}
    backdropStyle={backdropStyle}
    show={true}
    onHide={this.close}>
    <div style={dialogStyle()}>
        ...
    </div>
</Modal>

[SOURCE]

It must be noted that modalStyle and backdropStyle are React style objects.

Dialog Style

The dialog style is defined to provide some space at the top, clicking where, the overlay is closed. To do this, vertical height units are used –

const dialogStyle = function () {
    return {
        position: 'absolute',
        width: '100%',
        top: '5vh',
        height: '95vh',
        padding: 20
        ...
    };
};

[SOURCE]

Connecting to loklak Tweet Stream

loklak sends Server Sent Events to clients connected to it. To utilise this stream, we can use the natively supported EventSource object. Event stream is started with the render method of the StreamOverlay component –

render () {
    this.startEventSource(this.props.channel);
    ...
}

[SOURCE]

This channel is used to connect to twitter/country/<country-ID> channel on the stream and then this can be passed to EventStream constructor. On receiving a message, a list of Tweets is appended and later rendered in the view –

startEventSource(country) {
    let channel = 'twitter%2Fcountry%2F' + country;
    if (this.eventSource) {
        return;
    }
    this.eventSource = new EventSource(host + '/api/stream.json?channel=' + channel);
    this.eventSource.onmessage = (event) => {
        let json = JSON.parse(event.data);
        this.state.tweets.push(json);
        if (this.state.tweets.length > 250) {
            this.state.tweets.shift();
        }
        this.setState(this.state);
    };
}

[SOURCE]

The size of the list is restricted to 250 here, so when a newer Tweet comes in, the oldest one is chopped off. And thanks to fast DOM actions in React, the rendering doesn’t take much time.

Rendering Tweets

The Tweets are displayed as simple cards on which user can click to open it on Twitter in a new tab. It contains basic information about the Tweet – screen name and Tweet text. Images are not rendered as it would make no sense to load them when Tweets are coming at a high rate.

function getTweetHtml(json) {
    return (
        <div style={{padding: '5px', borderRadius: '3px', border: '1px solid black', margin: '10px'}}>
            <a href={json.link} target="_blank">
            <div style={{marginBottom: '5px'}}>
                <b>@{json['screen_name']}</b>
            </div>
            <div style={{overflowX: 'hidden'}}>{json['text']}</div>
            </a>
        </div>
    )
}

[SOURCE]

They are rendered using a simple map in the render method of StreamOverlay component –

<div className={styles.container} style={{'height': '100%', 'overflowY': 'auto',
    'overflowX': 'hidden', maxWidth: '100%'}}>
    {this.state.tweets.reverse().map(getTweetHtml)}
</div>

[SOURCE]

Closing Overlay

With the previous setup in place, we can now see Tweets from the loklak backend as they arrive. But the problem is that we will still be connected to the stream when we click-close the modal. Also, we would need to close the overlay from the parent component in order to stop rendering it.

We can use the onclose method for the Modal here –

close() {
    if (this.eventSource) {
        this.eventSource.close();
        this.eventSource = null;
    }
    this.props.onClose();
}

[SOURCE]

Here, props.onClose() disables rendering of StreamOverlay in the parent component.

Conclusion

In this blog post, I explained how the flow of props are used in the World Mood Tracker app to turn on and off the streaming in the overlay defined using react-overlays. This feature shows a basic setup for using the newly introduced stream API in loklak.

The motivation of such application was taken from emojitracker by mroth as mentioned in fossasia/labs.fossasia.org#136. The changes were proposed in fossasia/apps.loklak.org#315 by @singhpratyush (me).

The app can be accessed live at https://singhpratyush.github.io/world-mood-tracker/index.html.

Resources

Adding Tweet Streaming Feature in World Mood Tracker loklak App

Setting up plugins for test!

I spent most of this summer working on wordpress plugins. So when we were up with a substantial amount, we wanted to test them online. I discussed in a previous blog-post regarding putting an online wordpress implementation through Heroku. Once I was done with internal testing, the plugins were supposed to be released for common testing. Now, since we did not want to risk our service, we couldn’t provide admin rights to new users. So in order to overcome this problem I had the following options at my disposal:

  1. Create a script which would automatically activate and configure all plugins and show a basic plugin interface to users who are not logged in; OR
  2. Create a user with lesser privileges than administrator, but enough to view and modify plugin settings.

The problem with the first approach was its static nature. A user would not be able to test your service if he is not leveraged with all options your program provides. So, in order to ensure rigorous testing, I used the second approach.

WordPress, by default, provides 5 user-types:

  1. Administrator
  2. Editor
  3. Author
  4. Contributor
  5. Subscriber

As none of these user profiles fit the required job specification, I had to create my own user-type. After some brainstorming and searching, I found a pretty useful WordPress plugin (User Role Editor) which creates custom user profiles based on actions already present in WP suite. Once I installed the plugin on our WP installation, I used following steps to create my own user-profile called Loklak Tester. 

  • Click on ‘Users’ menu and then click on ‘User Role Editor’.

Selection_010

  • Here I selected the privileges I wanted for my user. Some of them are shown in the figure.

Selection_009

  • Once I was done, I clicked on Add Role and provided the required user-type name.
  • Below screenshot shows the menu for our new user-profile ‘Loklak Tester’.

Selection_011

Creation of this new user-profile would allow users to login using its credentials. Activate/deactivate, modify, edit plugins and change their settings. This would later act as a demo testing user which could be used by our audience to test our plugins on variety of levels.

Setting up plugins for test!

Loklak gets a makeover !

Things have made tremendous progress since the last time we’ve posted. Loklak has undergone a massive design change and a lot of critical bugs were fixed. The loklak webclient which is the frontend to the loklak server has made tremendous enhancements and progress in this period. We now support an amazing authentication with twitter and the ability to post a tweet, favorite and retweet the tweets you see.

At the same time, the search functionality to the tweets harvested on the loklak server has undergone massive enhancements and it’s now possible to look for tweets which are only images, videos etc.., The tweets can also be visualized based on their location on an interactive map which shows more tweets in an area as you scroll and interact more with the details and markers present on the map.

A profanity filter has also been implemented allowing the users / creators of the wall to create the wall and only showcase the contents that’s suitable for showcasing during a conference where loklak wall is being used.

Screen Shot 2015-07-01 at 5.09.21 pm

Screen Shot 2015-07-01 at 5.19.52 pm

There’s still more exciting developments in progress so that the application reaches the final outcome. This is the 2nd Milestone we’ve accomplished and we’re striving to achieve more and deliver a better experience.

Your feedback is always appreciated, we’d love to know more about what you think. In case you have a feature in mind and want to talk to us, we’re available on the mailing list or you could always file an issue for a feature you’ve always wanted or to report a bug.

Interested in helping us ? Join us in its development on Github, Star us and show your support.

Loklak gets a makeover !