AR Book experience
In “choose your own adventure” style
by Sze Man Ciu
Immersive and interactive storytelling experience
This project is to create an AR book, to immerse readers in stories by letting them play an active role in the story – the story proceeds according to reader’s choice. It is also an project to experiment with interaction blending the digital and physical worlds.
Project Concept
This project will create a story telling process that combines reading a book and bringing it to life. We are choosing to do a choose your own adventure style book as many other books are being translated into AR for study and recreation, however the ability to choose how the story goes as in the choose your own adventure genre has yet to be done. The goal is to grab younger readers (6 to 14 – ages TBD) attention by bringing the book to life and show animations of the choices made by the reader. We are going to generate an original story based on an old classic poem – The Owl and the Pussycat. We will branch the poem to create some choices, but the goal is to have the reader finish the poem to its conclusion. This will be a proof of concept that we can then take to publishers perhaps to get funding to reboot the choose your own adventure series with visual representation with our app.
Github repository:
github.com/jpverhaeghe/AR_ChooseAdvBook
User journey
Prototype storyline
Our story is loosely based on the classic short poem by Edward Lear ‘The Owl and the Pussycat’, using key moments to advance the adventure.
Creation process
My main role in the project is working with the UX UI and graphic design for print materials, I also work on the AR system and interaction in getting multiple image tracking at the same time, and the collision detection system in the augmented world, as well as creating 3D models for the option cards.
AR interaction
AR image tracking
A system to recognise an image (QR code or drawing pre-determined) on a page and presents the user with the AR scene with an animation of the story for that page.
Collision detection
Users will be able to make their choices by moving the option cards on the page, the system will detect the collision in the AR world to recognise their choices, to show corresponding animation and tell them what page to move to next.
UX UI design
Game instruction at the beginning of the experience, with pop-up UI to guide reader throughout the experience.
To enhance the overall experience, different background music is also added to the scene, to make the experience more immersive.
Graphic design
Graphic design of the book pages, with the page QR code which blends well in the pages. The option cards are also laser-cut, which is easier for readers to move around on the book, and make the experience more game-like.
Reflection
We have successfully created an AR system in a choose your own adventure style, which let reader to make choices, and telling them which page to turn to, for a different storyline. Also, we have implemented a system to track the interaction in the physical world, and give feedback in the digital world.
The most challenging part of the process are to handle the interaction, as we have put extra focus on how to blend the physical and digital world. We have tried with different kind of experience, while not every interaction works well. For example, we are not able to implement finger track moving of objects to player from the virtual page using hands interacting in the real world yet. For our existing system: collision detection when reader move the “option card” in the real worlds for the system detects the collision in the AR world. It sometimes does not work very well, the tracking may be lost when the option card is moved, which affect the detection of collision in the AR world. The lighting and plane also affect the accuracy of the image tracking and collision detection.
For further development, more different types of interaction can be experimented with, to keep experimenting with interaction between the physical and digital world, to blur the boundary of two, making the AR experience and interaction more interesting.
Project by
Sze Man Chiu | Jim Verhaeghe | Philip Kinshuck (2023)