Creating an Intelligent, Interactive Storybook for Kids

Apparently, 70% of all parents with iPads let their kids use them. I’ve seen kids as young as 2 pick an iPad up and start using it with little or no direction. It seems that with this tablet there is a device that kids can use, regardless or their current educational state or coordination. iPad developers seem to have caught on to this and the kids app market is a steadily growing one, with educational and gaming apps appearing all the time.

I first encountered kids using iPads when I spent time with Ian’s sister and family in Canada earlier this year. I’d taken my iPad with me and downloaded a bunch of kids games for Olivia (3) and Chloe (1) to play with. One of the first thing I noticed, however, was the poor quality of many of the apps, even those with big names like Peppa Pig behind them. Storybooks especially seemed to be nothing more than a digital version of the analogue book with the added disadvantage of not being chewable or usable for hitting your little sister with.

I also looked at how the girls interacted with the educational apps that I downloaded. Often they were no engaging enough to retain interest for more than a minute or two at a time, or did not offer enough incentive to continue when things got hard. Apart from one game, a memory card game, the average time that Olivia engaged with an app was about 2 minutes.  I started to wonder if I could do better and came up with the idea for an interactive storybook that helped kids between the ages of 4 and 7 learn to read.

I saw two problems with trying to create an educational storybook app. One was of entertainment and the other was feedback. While watching Olivia use the educational apps, I noticed that the amount of engagement she had with the app increased if an adult was there with her as she used the app and gave constant feedback and assistance when she needed it. However, the adults were not always available to give her the attention she needed and so she would put the iPad down and do something else instead.

I wondered if we could create the sense of feedback and support that an adult would normally provide by using Speech Recognition to listen to the child as they read the storybook and provide the feedback, encouragement, rewards and assistance at the moment it is required. We could use face detection on the front facing camera on the iPad 2 to determine whether or not the child was reading the book and use that as a guide to whether or not to listen or ignore what the app heard. Combine this with fun interactions, animations, multiple paths through the story and embedded games and I felt we could create an engaging, interactive learning experience.

Working with the Strong Steam team we are creating the Speech Recognition and Face Detection technology to use within the app and are in the process of putting together a prototype app. We’re looking for an early years learning professional with an interest in technology to help us out with how we make the app as engaging as possible while retaining the educational element. If you are one, or you know of one who might be interested, please do get in touch.