Frequently asked questions

 

ALGOS :

  1. How does the algorithm work?
    1. The algorithm is loosely based on YouTube’s recommendation algorithm.
    2. It is a neural network with the base layer of collaborative filtering. The collaborative filtering algorithm essentially tries to match a customer’s interaction pattern to an outcome - people who followed this pattern in the past (clicked on products 65,18, 32, purchased products 456 and 43, and added product 365 to the cart) bought product 86. This current customer has a similar pattern, and hence they are is likely to buy product 86.
    3. The next layer adds context – people in Istanbul in June bought more of these products and less of these. The given person is in Istanbul and it is June.
    4. The third layer adds temporal information. Essentially, it starts giving more weightage to the more recent interactions of the customer, and learns what the ideal decay rate should be.
    5. The last layer adds repeatability to the algo – how many times should you show a particular product to the customer before giving up.

 

  1. How does it work for new customers?
    1. For completely new customers (first time visitors ever), the algo utilizes its second layer – i.e., gives contextual popular products. Given the customer is from Istanbul and it is June, the algo starts by showing the most selling products in Istanbul in June.
    2. As the customer keeps clicking, the algo learns in real-time, and keeps improving the subsequent page.

 

  1. How does the algorithm work for non-signed up customers?
    1. The key customer id for the algo is the cookieID. So, even if the customer is not signed-up, we are able to track the customer through the cookieID
    2. BTW, it is our cookieID.

 

  1. We keep changing our products every so often. How does it take care of new products?
    1. Depending on how often you change your products, the algo allocates X% of choices to be coming from new products only – this is essentially to capture data. So, say 80% of your recommendations would be coming from our personalization algo, while 20% of the recommendations would be coming from the new products to capture data.
    2. As new products keep getting old (having enough data that effective personalization algos can work on them), they keep moving to the personalization bucket (80% bucket in the above example).

 

  1. What is the data you use for the algorithm?
    1. We use 3 sets of data for the algo:
      1. User interactions at a cookieID level (for users on website/m-site), customerID level (for all signed-up users), and deviceID level (for users on app) :
        1. Eyeballs
        2. Clicks
        3. Purchases
        4. Add to carts
        5. Add to wishlists
        6. Searches
        7. Filters
      2. Product catalog:  All product info that is displayed on your PDP page
      3. User’s context:
        1. Location from IP   
        2. Device details

 

  1. What data would you be collecting from my website/app?
    1. We collect 3 sets of data. Please see answer to question 5.

 

  1. Would you need our historical data?
    1. It helps but is not worth the trouble. We are happy to start afresh with the data that we collect starting when you sign-me-up.

 

USE-CASES :

  1. How will the algorithm help my business?
    1. The algorithm will deliver the most personalized and relevant products to each visitor, thereby personalizing the experience of each visitor:
      1. Category page re-ordering: Products are re-ordered on the listing page for EACH customer based on their likelihood to click. For any customer, the products that they are likely to buy more are shown on top, while others are shown further down. Leads to more people visiting PDP page, which in-turn leads to higher conversions.
      2. New conversion-optimized boutique pages: New SEO friendly pages are added to your site, with products personalized to each customer. These pages are accessible through the main menu, or through “view more” buttons on the widgets. Leads to a superb customer experience, more people visiting PDP page, which in-turn leads to higher conversions. Also boosts organic traffic.
      3. Recommendation widgets: These are context-aware widgets that show personalized products to each individual on home page, PDP pages and Cart page. Leads to higher conversion rates.
      4. Other widgets: Other optimization widgets are strategically placed within your website to compliment the recommendation widgets. These include “New arrivals”, “Bestsellers”, “Recently viewed” etc.  Leads to higher conversion success of recommendation widgets.
      5. Onsite popups: Popups trigger automatically whenever a particular visitor feels lost or is about to leave. This guides her in the right direction, thereby helping decrease the bounce rate.

 

  1. On my home page/ PDP page, I have some category widgets. Can those be personalized as well?
    1. Yes.

 

  1. How is it different from other recommendation engines in the market?
    1. The are three differences:
      1. We are able to personalize each page (including the PLP which is the most important page).
      2. We are able to create new personalization features for your website. This includes Boutique pages, Popups, etc.
      3. Our integration is, by far, the easiest – 10 minutes of your time with us, vs. 2 months with others.
      4. Our algos are better, giving you better results. This is primarily because of the fact that we are able to read “Eyeball data”, which helps us in differentiating “product seen but not clicked” and “product not seen, and hence not clicked”. You will see this after the integration.

 

INTEGRATION :

  1. How can the algorithm be incorporated to my E-commerce website?
    1. Install our JS onto your website, and leave everything to us.

 

  1. How about app? It does not have JS.
    1. You are right. App integration is slightly more complicated. So, we start with the website, and then, post showing the success, we move to the more complicated app integration in Phase 2.

 

  1. How long would integration with the app take?
    1. Integration on the app would take about 40 hours of tech effort, which is usually split in 1-2 client sprints.
    2. Additionally, in app, because it is an SDK-based integration, going live would also be linked to a new app version release.

 

  1. How do you get my data?
    1. Our JS is able to read it automatically.

 

  1. How do you ensure that the look and feel of my website is maintained?
    1. We copy the CSS files in your website, and replicate the interventions with those CSS files – ensuring that the look and feel of our interventions is EXACTLY the same as yours.

 

  1. How do you get my product feed?
    1. Our JS reads your PLP and PDP pages every 2 hours and gets the product data from there. We might also request you for your product feed, if it is eaily available with you.

 

  1. How do you come to know of my new products and products getting out of stock?
    1. Our JS reads your PLP and PDP pages every 2 hours and gets the product data from there. As you keep adding new products, or your existing products keep getting out of stock, the product gets to know it.

 

  1. Will it make my website slower?
    1. No. All our installations are asynchronous. Hence, they CANNOT make your website slower.
    2. Our API is super-fast – our SLA is a response time of 200 ms for 99% of the hits, which is the fastest in the industry.

 

  1. What if your API fails?
    1. We have a built in full backup – if our API does not respond in 200 ms, the intervention collapses, and your website behaves as if nothing happened – just like it behaves now

 

  1. Where is your server located? What about network latency?
    1. Our server is located in Mumbai on AWS. We use Cloudfront with 1 hour caching, to ensure that we are able to respond within 200 ms anywhere in the world.
    2. If, for some reason, we are not able to meet your latency requirements, we are happy to co-locate the server to shave off another 40-60 ms.

 

  1. We don’t want AWS. Amazon is competition.
    1. We have our backup server on Google. We can make that your default server.

 

  1. How safe is my data? What are the privacy policies?
    1. Your data is as safe or more safe than it is in your website. We use Amazon Web Services to store data. For more information on data security, please visit: https://aws.amazon.com/compliance/data-privacy-faq
    2. We can sign an NDA with you to ensure that your data is safe with us.

 

  1. We use caching on our website. Will it be a hindrance?
    1. We handle it. Essentially, we create a hole in your cached HTML, and show our recommendations in that hole.
    2. The rest of your page remains cached.

 

PROCESS :

  1. How will the integration process work?
    1. Day 0: Install our JS onto your website. If you have access to your product feed, give access to us in any format you have it.
    2. Day 1: We integrate the data ingestion and product ingestion, and start getting the interaction data.
    3. Day 2 and Day 3: We integrate personalization in your website.
    4. Day 4: We make the personalization “limited live”. It becomes available at the following link: www.mywebsite.com/?CrayonDemo=true
    5. Day 4, Day 5, Day 6: We, and you do multiple rounds of testing on the limited live, iteratively solving any issues.
    6. Day 7, Day 8: We make it live and monitor continuously.

 

  1. Will we see the personalization before you make it live?
    1. Absolutely. It will be made “limited live” at the following link: www.mywebsite.com/?CrayonDemo=true
    2. Only once you approve it, will we go live for your customers.

 

  1. Can you make it live for a limited number of customers to start with?
    1. Yes, we can.

 

  1. Can we test the recommendations visually before you make them live?
    1. Yes, you can. However, we strongly suggest that you let the data speak for itself, rather than bringing a human element for the decision making. The algos know what they are doing - let them fail, fall, and learn. With every recommendation that the customer does not click on, the algo improves.

 

  1. How long does it take before the algos are really good?
    1. The algo ideally needs an average of 100 clicks per product. So, if you have 10,000 products, the algo needs 1Mn PDP views. If you have 1 million visits a month, with 3 PDP views per visit, it means that it would take 10 days.
    2. The algo can start with an average of 20 clicks per product. So, if you have 10,000 products, the algo can start converging after 200K PDP views.

 

  1. Can we test this on staging first?
    1. Yes. But it does NOT serve any purpose:
      1. For the UI testing, we will give you access on limited live at this link: www.mywebsite.com/?CrayonDemo=true for you to test anyway.
      2. For the accuracy of recommendations, the staging website does not have any data - so the algo does not get any data to train on, and hence testing on staging does not help.

 

POST LIVE PROCESS:

  1. What happens after we go live?
    1. Week 1, Week 2: The algo tries a large number of models for different customers in different places. Based on the volume you get, 10 to 30 models are tested. By the end of this time, we would deliver the results panel. While we get ourselves accustomed to the panel, it is too early to read into the results.
    2. Week 3, Week 4: The algo learns the optimal mix of models in different scenarios. By end of this, we would be able to start measuring the impact of personalization. Now is a good time to start following the results panel.
    3. Week 5, Week 6: During this time, based on the results, we should be moving to longer-term planning of the engagement.

 

  1. Do you have a panel from where I can manage my engagement?
    1. We will give you access to a basic panel, through which:
      1. You will be able to switch on and switch off the different “interventions”.
      2. You will be able to see the results:
        1. The complete conversion pipeline for each “intervention”.
        2. Incremental impact with personalization in an A-B model.

 

  1. What do you want from us?
    1. Tech :
      1. Putting the JS on the website.
      2. Access to your product feed if you have it easily available.
      3. Creation of a blank page: www.mywebsite.com/boutique/…
    2. Biz:
      1. Once we give you access to www.mywebsite.com/?CrayonDemo=true, we would require you to test it, and give a sign-off to go live.
      2. If you make any big changes to your UI/UX, do let us know beforehand.
    3. Commitment:
      1. Commitment for a longer-term engagement if we show an increase in your key metrics in first 4-6 weeks.

 

  1. How about my m-site?
    1. It works exactly same as website. No problem.

 

  1. How about my app?
    1. App requires a deeper integration. We will do it in Phase 2.

 

  1. So, what exactly are the different phases?
    1. Phase 1:
      1. Personalization on your website and m-site using JS-plugin based quick integration.
    2. Phase 2:
      1. Personalization using deeper API-based integration on your website and m-site.
      2. Personalization on your apps.
      3. Possibly (we are still working on this) - personalization of your digital marketing communication - emails, notifications and advertisements.

 

  1. Is personalization different on App versus web for the same user? Is so how different and why?
    1. Integration:
      1. Integration for website or m-site is done through our JS plugin, and is super easy for the client.
      2. Integration on app needs deeper integration, and hence takes more time for the client.
    2. Go live:
      1. Go-live on website is much easier - we can go live whenever we decide to.
      2. Go-live on app usually needs an app release.
    3. UX and Interventions:
      1. The UX in website and app are somewhat different and may need different interventions. Example, if a person is about to leave on website, then we can show a retention popup. On app, if a person leaves, we can send a “come back” notification.
    4. User behavior and corresponding algo differences:
      1. User behavior on website and app are different. Hence, the mixer algo could converge to different weights in website, m-site and app.
      2. The algo also takes as an input the user device and OS for deciding the choices. Hence, the choices shown may be different in the two.

 

RESULTS :

  1. What can we expect in the results?
    1. Month 1: Increase in CTR’s by 20% or higher (upto 50%)
    2. Month 2: Increase in at least one of the following metrics by 5% or higher (upto 15%):
      1. Page views per visitor
      2. ATC per visitor
      3. Purchases per visitor
    3. Month 3: Increase in all of the following metrics by 5% or higher (upto 15%):
      1. Page views per visitor
      2. ATC per visitor
      3. Purchases per visitor
    4. Month 6: Measurable improvement in the following metrics:
      1. Bounce rate
      2. Repeat visitors
      3. Organic traffic
    5. Month 6 - Month 12 (If we decide to go for personalized digital marketing): Measurable improvement in the following metrics:
      1. Email CTOR’s
      2. Notification Open Rates
      3. Advertisement CTR’s
      4. Advertisement revenue per ad dollar spent