Turning points - My Data Science Internship Experience with Shunya

Sunidhi Brajesh
Intern, Data Science
November 18, 2020
8 Minutes

Turning points, that’s all this post is about. How being at the right place and time helped me to work with one of the highly energetic and motivated people I’ve ever met. The team Shunya has created is no less- a bunch of people willing to make a dent through sheer perseverance. 

Here’s the story of getting on board the EdTech ship and my learnings from building a team at Shunya.

How it all started

I came across a LinkedIn post talking about hiring data science interns. I went through the company’s details and reached out to the founder, Ekta. She replied back in no time with a detailed problem write-up as well! (not too bad, eh? :P)

During our discussion, one thing that connected with me was the vision - building the K12 experience around the offline way of learning. This was something new and challenging I had come across in EdTech, and I accepted the offer immediately.

Scratch scratch scratch!

Building a learning platform from scratch has its own challenges. At Shunya, we use state-of-the-art techniques such as Optical Character Recognition, Text-To-Speech, Speech-To-Text, Taxonomy, Item Response Theory, Ontology and Knowledge Graph to improve online education experience. These topics are complex to implement and require the right mix of people from data science, software engineering, distributed systems and cloud. Take this for example - developing an OCR module involves understanding image processing, OCR model, REST API (to get inference), Postgres, Amazon S3, Redis and AWS. (more details in our Tech blog!)

The product requirement is fulfilled by two data science tracks (track#1 and track#2). These tracks are further split into smaller tasks, each implemented as a micro-service. We start each microservice by building an end-to-end flow for simple use cases and then eventually improve complexity. Track#2 is more research oriented involving IRT, Ontology, Taxonomy and Knowledge graph compared to Track #1 which uses OCR, TTS and Speech-To-Text. We have regular calls with mentors for each track to help us guide in the right direction.

Having previously worked for 4 years in a consumer-facing role at Ola, I was exposed to architecture design but my core work was limited to a particular feature in a module. At Shunya, I could finally use my knowledge of architecture design, distributed systems, and machine learning in designing a scalable backend of machine learning workflows. Previously, I had minimal experience with image, voice and text data with no production level experience in data science. Here, I learnt(from scratch, obviously!) to develop and deploy large scale systems in production as microservices.

Welcome to the team, you’re hired!

With the humongous task ahead of us, it was not possible to accomplish it all with just 4 interns (lol, it never is). We knew we needed more hands on deck. After all, desperate times call for desperate measures (cue- MI theme song).

At this point, we needed more interns for track#2 who could think from a research and application perspective. I have always believed that motivation and skill set are the two most important things for solving any problem. I have taken interviews at my previous role, but building a team was going to be my first experience. One important thing that I learned- taking interviews at a big company (or startup) is mostly limited to evaluating the skillset, but when you’re hiring someone you also need to evaluate for culture fit. 

Hiring is a complex phenomena. You can only learn so much about someone in such a short span. However, we need to workaround this time constraint by building important checkpoints. This was something Ekta provided guidance on. 

Checkpoint#1- Github. I cannot emphasise this enough- keep Github updated. It provides a much needed perspective on how motivated/experienced someone is in their field of interest. 

Checkpoint#2- take home assignments. These serve a two-pronged approach- a) your ability to solve for a problem, and b) your ability to learn and implement. For each candidate, we screen resumes and then immediately go through theirGithub and LinkedIn profiles. This is followed by a take-home assignment and phone interview. After each telephonic round, we discuss individual feedback and take a call.

Both these points also told me how important it is to have depth in my work.  While people fell for a number of repositories, the candidates we selected were the ones , who did a few meaningful things, and knew the details of it. Not just knowing which package they used and the options to solve a problem, but guided by hypothesis. 

In one such problem, I remember we were looking at text not being detected in an image, and how we came up with 5 different hypothesis and ended up scoping for different pre-processing algorithms for those use-cases.  In a normal black box algorithm, you never get to question, reason with yourself and just take that as a limitation to that algorithm.  

The Shunya Life

Though all work is virtual during this pandemic, each day here is a roller-coaster ride with new learnings. Amidst going back-and-forth on potential solutions to the problems we’re solving, I have learnt a lot of things. There will always be more challenges to be solved, what matters is whether the team is motivated enough to power through them. Another important thing is being accessible. It helps a lot to get a fresh set of perspective whenever you’re stuck on any task. We have a great team of mentors, who’re always accessible to help us out.

The entire team syncs up twice a week to discuss the progress, brainstorm on blockers and plan next steps. Everyone is de facto available to help each other irrespective of the track they’re working on. We also have a mid-week chai-pe-charcha session where everyone from tech, product and business comes together to interact and play games (see? we’re fun too).

Join the team!

We’re building the tech infrastructure at Shunya to make learning more interactive. Join us to redefine the ‘tech’ in‘EdTech’.