I recently wrote a basic overview for the Storybook project that I’m taking to Chile, and I’ve realised having it on here is probably the best place to put it, so here it is.
Project Overview of Interactive Storybook for iPad
from Radical Robot
The Interactive Storybook for iPad project is designed to create an educational storybook for children between the ages of 4 and 7 that will help them learn to read sentences while providing an engaging and entertaining experience. The age range is designed to target children moving from reading single words, to sentences.
Many interactive storybooks are designed to entertain and tell a story, but provide little towards assisting the child to read the story for themselves. In most cases, this assistance is provided by parents or learning professionals, however these people are not always available to assist the child when they want. This project will utilise Speech Recognition technology to listen to the child as they read the story out loud, so as to provide encouragement, feedback, assistance and rewards at the point at which it is needed. Face detection will be used to determine whether or not the child is reading from the app rather than talking to someone off screen and will therefore serve to provide more accuracy to the speech recognition. Speech recognition can be disabled for adult led enjoyment.
As the child reads the story, the app will listen to their progress. When the child reads a word incorrectly, stumbles on pronunciation or takes a long time to read the word (the app only ‘listens’ when the child is looking at the app). It will then step in and prompt the child with assistance using the Phonics learning system. When the child correctly pronounces the word, then audible (‘Well done!’) and visual (animations) feedback is used to provide rewards.
The app will monitor the child’s progress over time (between readings of the story as well as within a reading session) so that feedback can be adjusted to the child’s progress. This means that words that are consistently mispronounced will receive more intervention and greater rewards for success than words that are more often correctly read. The actual form that the feedback/reward system will take is currently in development.
As additional rewards, at the end of every page, the story so far can be animated and interacted with. These animations can be expanded into games that will assist the child with learning the words that they have been struggling with in a fun and interactive way.
The app is designed to be a fun experience whether read together with a parent or alone. When read with a parent then the speech recognition and face detection can be disabled so that mum or dad can provide the learning value and assist their child in playing the games. However, whenever the child wants to play and mum’s busy, then the Speech Recognition can be activated to provide the required assistance and encouragement during self directed play.
This app is still in development, with a working prototype expected around March. Development at this time is focused around creating the speech recognition technology for children.