Monetisation of Susi using the Amazon Product API (Part 1)

I’ve worked with the loklak team on Susi for the past month, and we’re in that stage where we are mostly expanding the dataset we have. Susi is still nascent, mind you, but it does show a lot of promise in terms of the idea and the progress we have made.

In the past some posts, I covered OSM Analysis, integrating that into Susi, as well as Bot integration, and I also spoke about the need for Susi to increase its reach (which was the purpose for the Bot integration). For this purpose and the general purpose of making Susi more able to answer different queries, I dug around a lot of APIs, and came across the Amazon Product Advertising API, which answered both the reach and the dataset question.

Through the Amazon API, we can get a wide (really wide) range of information for products in its database. Along with ItemSearch (search Item by Name), ItemLookup (search Item by Amazon ID, known as ASIN), there are a host of other APIs: SimilarityLookup, BrowseNodes, even Virtual Carts (wherein you can add items to a remote virtual cart and get prices etc). And here comes the best thing: since you use the API with your Affiliate / Associate Tag and secret keys, if someone goes into a URL which is marked by your affiliate tag, you get paid for it.

So clearly, we can expand our dataset, as well as get an income by using this API, making it suitable for solving both the reach and the dataset problem. I will explain the usage of this API, as well as its integration into Susi in today’s and two more blog posts. Today, let’s go through the structure, as well as the operations of this API.

The Amazon API is a SOAP based API, which means we get the information as an XML. To access the API, Amazon has an authentication requirement. It gives you a set of API keys (through AWS): an AWS secret key, and an AWS access ID. In addition, you also need to apply for an associate tag at the AWS Affiliates Program. The reason for that is the API gives out URLs marked with our associate tag, and as mentioned above, one earns income after enough number of hits on those URLs. More can be seen here.

Once we have those keys, we decide the Operation that we need to perform: ItemLookup, ItemSearch etc (full list of operations here). And once that is done, we decide the Response Group. The Response Group defines the way that operation returns data, i.e the format, what all data does it return. This makes it very convenient for users to get exactly what they want, which makes it even more ideal for Susi. More about Response Groups here.

So what do we do with all this data? How do we even get the response from the API? That’s the fun part. Let us take an example with the ItemSearch API. We will get the Amazon data, i.e a list of products similar to or matching “the hunger games” for example.

Since Amazon API is SOAP, we need to build up the API request URL. We first decide on a locale. The Amazon locale is to be decided based on your AWS settings. Since I set up my AWS account as a USA locale, I use the USA endpoint of Amazon, namely webservices.amazon.com.

Once this is done we supply the following GET parameters:

1. Service (for most operations we use Service as AWSECommerceService)
2. AWS Access ID
3. AWS Associate Tag
4. Operation (in this case ItemSearch)
5. Keywords (i.e query, in this case “harry potter and the cursed child”)
6. Search Index (this is ONLY used for ItemSearch, in this case we put Search Index as All. It is basically which category to search in, it’s a compulsory param)

7. Response Group (optional, it’s defaulted to Small)
8. Timestamp (time of request)
9. Signature (Amazon uses Hmac algorithm to sign requests, so we need to supply a signature)

My implementation of the Amazon API uses a REST wrapper which uses java.util commands to get the Signature and the Timestamp etc, the rest has to be supplied by us. As of now, let’s see how a sample API request for the USA locale looks like:

http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[AWS Access Key ID]&AssociateTag=[Associate ID]&Operation=ItemSearch&Keywords=the%20hunger%20games&SearchIndex=Books&Timestamp=[YYYY-MM-DDThh:mm:ssZ]&Signature=[Request Signature]

I have then used w3c’s DOM to connect to the API, get the XML, and parse it, but here’s how the sample XML looks like:


<TotalResults>2849</TotalResults>
<TotalPages>285</TotalPages>
<MoreSearchResultsUrl>http://www.amazon.com/gp/redirect.html?linkCode=xm2&SubscriptionId=[AWS Access Key ID]&location=http%3A%2F%2Fwww.amazon.com%2Fgp%2Fsearch%3Fkeywords%3Dthe%2Bhunger%2Bgames%26url%3Dsearch-alias%253Dstripbooks&tag=[Associate ID]&creative=386001&camp=2025</MoreSearchResultsUrl>
<Item>
    <ASIN>0545670314</ASIN>
    <DetailPageURL>http://www.amazon.com/The-Hunger-Games-Trilogy-Mockingjay/dp/0545670314%3FSubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D165953%26creativeASIN%3D0545670314</DetailPageURL>
    <ItemLinks>
        <ItemLink>
            <Description>Technical Details</Description>
            <URL>http://www.amazon.com/The-Hunger-Games-Trilogy-Mockingjay/dp/tech-data/0545670314%3FSubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>Add To Baby Registry</Description>
            <URL>http://www.amazon.com/gp/registry/baby/add-item.html%3Fasin.0%3D0545670314%26SubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>Add To Wedding Registry</Description>
            <URL>http://www.amazon.com/gp/registry/wedding/add-item.html%3Fasin.0%3D0545670314%26SubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>Add To Wishlist</Description>
            <URL>http://www.amazon.com/gp/registry/wishlist/add-item.html%3Fasin.0%3D0545670314%26SubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>Tell A Friend</Description>
            <URL>http://www.amazon.com/gp/pdp/taf/0545670314%3FSubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>All Customer Reviews</Description>
            <URL>http://www.amazon.com/review/product/0545670314%3FSubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
        <ItemLink>
            <Description>All Offers</Description>
            <URL>http://www.amazon.com/gp/offer-listing/0545670314%3FSubscriptionId%3D[AWS Access Key ID]%26tag%3D[Associate ID]%26linkCode%3Dxm2%26camp%3D2025%26creative%3D386001%26creativeASIN%3D0545670314</URL>
        </ItemLink>
    </ItemLinks>
    <ItemAttributes>
        <Author>Suzanne Collins</Author>
        <Manufacturer>Scholastic Press</Manufacturer>
        <ProductGroup>Book</ProductGroup>
        <Title>The Hunger Games Trilogy: The Hunger Games / Catching Fire / Mockingjay</Title>
    </ItemAttributes>
</Item>

One thing to notice is the URLs in the XML result. See how it has the Associate tag and the Access ID mentioned in it? This is how the monetisation happens: when the users buy their products with links having our associate tags on them.

This is obviously just scratching the surface, the API itself has a whole lot of operations. I have given a necessary brief on how the API really works. In my next two blog posts, I will speak on building the Amazon API Service for loklak, and then integrating it into Susi. Feedback is welcome. 🙂

Monetisation of Susi using the Amazon Product API (Part 1)