Advances in procedural animation for Virtual Reality [REF2021]

Description of impact

Context
Virtual Reality (VR) technology has matured rapidly over the last seven years into a burgeoning consumer electronics industry. It is estimated that over 171,000,000 people used VR worldwide in 2019 and global sales revenue for 2020 reached approximately USD2,900,000,000 (10-2020), rising to estimated USD5,700,000,000 by 2023. Ocean Rift has played a central role in spearheading this new medium as a popular launch title for most major VR headsets. Indicative of the impact has been a lasting partnership with Samsung, Oculus (Facebook) and Google, with Ocean Rift being developed in tandem with their prototype and commercial technology between 2013 and 2020.

The first version of Ocean Rift was created in 2013, when ap Cenydd developed a series of free demo applications for the Oculus Rift development kit. The demo was very popular and used by enthusiasts worldwide to demonstrate and evangelize VR. In the then fledgling industry, these original demos pioneered several techniques which later became ubiquitous, including comfortable movement systems to reduce simulator sickness and the use of motion controls to effectively navigate 3D space.

Ocean Rift is the world’s first VR safari park, where users are transported into a unique underwater world containing over 50 animals, spread across 14 habitats including coral reefs, lagoons, the arctic and prehistoric seas. The app features state-of-the-art procedural animation techniques developed specifically for the project, which aims to bring animals like dolphins, whales, sharks, sea lions and prehistoric reptiles to life. This algorithmic approach to virtual creatures is unique to Ocean Rift and has helped push the boundaries of VR.

Industry and Consumer impact
Based on the success of the original demo, ap Cenydd was invited by Samsung and Oculus in late 2013 to develop Ocean Rift as a launch title for the first modern consumer VR product, the Samsung Gear VR. This involved working confidentially on cutting edge VR hardware and software, under close collaboration, support and feedback from Samsung and Oculus.

Ocean Rift was initially launched in September 2014 alongside the Gear VR Innovator Edition and Samsung Galaxy Note 4 smartphone [5.1]. A year later it was a launch title for the consumer edition of the Gear VR, alongside Samsung’s flagship Galaxy S7. The app was also bundled with the phone in North America and Europe to hundreds of thousands of consumers (actual sales figures are unavailable due to a Non-Disclosure Agreement). It has since been available at launch alongside all of Samsung’s flagship phones and has often been demoed by both Samsung and Oculus in their stores and at international trade shows. In 2019, SK Telecom, South Korea’s largest telecoms company, bundled Ocean Rift with the flagship Samsung S10 5G phone during the launch of the world’s first 5G network.

The significance of Ocean Rift to the VR industry is demonstrated by the inclusion of the app as a launch title for most major VR platforms since the Gear VR, including the HTC Vive (2016), Google Daydream (2016), Oculus Rift (2016), Oculus Touch (2018), Xiaomi Mi VR (China, 2018), Oculus Go by Facebook (2018) and Oculus Quest by Facebook (2019). Being a launch title is by invitation only and Ocean Rift’s continual presence demonstrates its enduring impact on an industry worth an estimated USD2,900,000,000 (10-2020). For the Oculus Go by Facebook, the app’s logo was featured on the commercial packaging alongside giants like Hulu and Netflix.

It is estimated that across premium, free and demo versions, Ocean Rift has been downloaded over 1,800,000 times by the general public. The app has a glowing reception on VR app stores, based on over 7000 reviews [5.2]. It has charted internationally at #1 on several occasions between 2014 and 2018 and is a rare example of a leading non-violent VR application suitable for all ages. It is commonly used as the first app to introduce someone to VR, and has been used extensively for entertainment, education and increasingly to improve health.

Healthcare benefits
There is growing evidence that VR can benefit mental health and quality of life in general. In ‘Tim’s New World’, a man with a rare form of Cerebral Palsy is seen using Ocean Rift as a form of escapism [5.3]. The app has been used in retirement communities for similar reasons and is often used as a form of therapeutic distraction for chronic pain conditions and treatments. VR and Ocean Rift in particular have also been shown to be effective at reducing fear and anxiety levels in children during blood drawing [5.4], reducing pain and anxiety in breast cancer patients [5.5], reducing pain during unmedicated childbirth [5.6], and as an analgesic for acute and chronic pain in adults [5.7].Ocean Rift has also been used to improve the quality of life for MS patients during infusions [5.8], and to safely immerse patients for brain mapping during awake craniotomy surgery [5.9].

Supporting innovation in education
Ocean Rift has also been spearheading VR as a powerful education tool in partnership with a USD10,000,000 (10-2017) Oculus initiative [5.10]. This feature allows users to learn about life in Earth’s oceans while being immersed in it, including learning about each creature’s habitat, diet and lifecycle. This represents a completely new way of learning about the natural world, including the grave threats posed by pollution and climate change. The app has been showcased during special events at the American Natural History Museum in New York and has been translated into French, German, Latin American Spanish, Japanese, Korean, Simplified Chinese and Welsh.

This work has also led to many opportunities and research directions for Bangor University and the wider community. Beneficial activities include media appearances, open day demos, public seminars, festivals and school visits; which all in turn benefit admissions. The app also places Bangor research at the very forefront of VR, through ongoing collaboration with leading tech companies. This environment enhances the student experience, through research-led teaching in modules such as Imperative PAI and Games Design, Applications of AI, and dissertation and PhD projects, where students have access to the latest VR hardware and world-leading expertise.

Impact Summary for the General Public

Bangor University’s development of Virtual Reality (VR) application Ocean Rift, one of the first and most popular consumer VR apps in the world, has helped spearhead the now burgeoning global VR industry, in partnership with Samsung, Facebook and Google. Ocean Rift has been an invite-only launch title for most major VR headsets between 2014 and 2019, including the Samsung Gear VR, Oculus Rift, HTC Vive, Google Daydream, Xiaomi Mi, Oculus Go and Oculus Quest. With an estimated 1,800,000 downloads it receives consistent positive reviews by users and developers. The app was based on underpinning research in VR best practices and procedural animation, including new computer graphics and artificial intelligence techniques for producing life-like animation and behaviour. Recent global benefits have been generated from its use in educational and healthcare contexts.

Description of the underpinning research

This case study is based on the work of Dr Llyr ap Cenydd and Dr Bill Teahan, Bangor University, including foundational research in novel computer animation techniques, the development of an industry-leading VR application called Ocean Rift, and subsequent research undertaken between 2013 and 2019.

Over the past seven years, VR has grown rapidly from a hobbyist emerging technology to a multibillion-dollar industry. Consumer headsets are now capable of generating viscerally immersive experiences. This rapid progress has required a rethink on how we design, model and interact with virtual worlds, and best practices are constantly evolving. This is particularly important for virtual characters, as the intimacy of VR requires a much greater attention to animation detail and behaviour. Research undertaken at Bangor University has been at the forefront of this process.

Several Bangor University innovations have contributed to advances in the VR industry, most notably the use of procedural animation techniques for bringing virtual animals like sharks, dolphins and sea lions to life. Procedurally animated characters can be described as ‘puppets that can pull their own strings’, where artificial intelligence is used to synthesise animation with the range of behaviour comparable to real-life equivalents. There is great synergy between the strengths of procedural animation and the challenges of VR.

Foundational research was undertaken by Dr Llyr ap Cenydd and Dr Bill Teahan from 2008 - 2013, including the development of novel procedural animation techniques for synthesising the locomotion of virtual arthropods [3.1]. The research investigated how the dexterous motions of creatures like spiders can be simulated in real-time using approaches derived from biology and robotics. The resulting virtual creatures could move autonomously within an arbitrary environment. Crucially the work also demonstrated how the approach could be applied to a wide range of animals, such as insects, crustaceans and reptiles. This research informed the development of similar techniques for animating aquatic creatures in Ocean Rift [3.2, 3.3].

The first version of Ocean Rift was developed in 2013, as a free tech demo exploring ways of simulating an underwater environment in VR. Its global popularity led to a partnership with Samsung and Oculus (Facebook), and the app was developed further as a launch title for the first consumer VR device, the Samsung Gear VR.

Bangor researchers’ subsequent access to prototypical hardware through collaboration with Oculus, Samsung, Microsoft, Valve and Google facilitated innovation at the cutting-edge of the VR industry, with research undertaken in parallel with the app’s evolution. Several case studies were published on how techniques developed for Ocean Rift helped define VR best practices, including comfortable movement in VR, motion controller-based swimming, immersive-level design, user-creature interaction and educational features [3.4].

Procedural animation systems can be challenging to optimize due to the sheer number of parameters that control behaviour. Bangor research went on to investigate how crowd-sourcing and genetic algorithms could be used to fine-tune the behaviour of virtual dolphins towards greater realism [3.2, 3.3]. This led to research that indicated that people perceive animation and behaviour differently in VR compared to normal screens [3.5] and these insights further informed app development. In addition, well-optimised VR applications are critical in ensuring that simulator sickness is not triggered due to any sustained drop in performance. Optimisation can also facilitate more complex scenes, or conversely conserve battery life. Novel techniques for optimising underwater environments for mobile VR were published [3.6], based on the pioneering techniques developed at Bangor for the Gear VR version of Ocean Rift.

Beneficiaries and reach of impact

Virtual Reality (VR) technology has matured rapidly over the last seven years into a burgeoning consumer electronics industry. Ocean Rift has played a central role in spearheading this new medium as a popular launch title for most major VR headsets. Indicative of the impact has been a lasting partnership with Samsung, Oculus (Facebook) and Google, with Ocean Rift being developed in tandem with their prototype and commercial technology from 2013-2020.
Another key beneficiary are consumers. Ocean Rift has been downloaded an estimated 1.8 million times, with the average VR headset used by several people. The app has a glowing reception on VR app stores, based on over 7000 reviews [7]. It has charted internationally at #1 on several occasions from 2014-2018 and is a rare example of a leading non-violent VR application suitable for all ages. It is commonly used as the first app to introduce someone to VR, and has been used extensively for entertainment, education and increasingly to improve mental health and quality of life.

In Ocean Rift, users are transported into a unique underwater world. The app is the world’s first VR safari park, containing over 50 animals, spread across 14 habitats including coral reefs, lagoons, the arctic and prehistoric seas. The app features state of the art procedural animation techniques developed specifically for the project, which aims to bring animals like dolphins, whales, sharks, sea lions and prehistoric reptiles to life. This algorithmic approach to virtual creatures is unique to Ocean Rift and has helped push the boundaries of VR.
The first version of Ocean Rift was created in 2013, when Llyr was developing a series of free demo applications for the Oculus Rift development kit. The demo was very popular and used by enthusiasts worldwide to demonstrate and evangelize VR. In the then fledgling industry, these original demos pioneered several techniques which later became ubiquitous, including comfortable movement systems to reduce simulator sickness and the use of motion controls to effectively navigate 3D space [4][6].

Based on the success of the original demo, Llyr was invited by Samsung and Oculus in late 2013 to develop Ocean Rift as a launch title the first modern consumer VR product, the Samsung Gear VR. This involved working in secret on cutting edge VR hardware and software, under close collaboration, support and feedback from Samsung and Oculus.
Ocean Rift was initially launched in September 2014 alongside the Gear VR Innovator Edition and Samsung Galaxy Note 4 smartphone [8]. A year later it was a launch title for the consumer edition of the Gear VR, alongside Samsung’s flagship Galaxy S7. The app was also bundled with the phone in North America and Europe to hundreds of thousands of consumers. It has since been available at launch alongside all of Samsung’s flagship phones and has often been demoed by both Samsung and Oculus in their stores and at international trade shows. In 2019, SK Telecom, South Korea’s largest telecoms company, bundled Ocean Rift with the flagship Samsung S10 5G phone during the launch of the world’s first 5G network.

Ocean Rift has been a launch title for most major VR platforms since the Gear VR, including the HTC Vive (2016), Google Daydream (2016), Oculus Rift (2016), Oculus Touch (2018), Xiaomi Mi VR (China, 2018), Oculus Go by Facebook (2018) and Oculus Quest by Facebook (2019). Being a launch title is by invite only and Ocean Rift’s continual presence demonstrates its lasting impact on the industry.

Ocean Rift has also been spearheading VR as a powerful education tool in partnership with Oculus [9]. This feature allows users to learn about life in Earth’s oceans while being immersed in it, including learning about each creature’s habitat, diet and lifecycle [10]. This represents a completely new way of learning about the natural world, including the grave threats posed by pollution and climate change. The narrator of the app, Katie Flamman, has been nominated for several awards based on her performance [11]. The app has also been showcased during special events at the American Natural History Museum in New York and has been translated into both Simplified Chinese and Welsh.
There is growing evidence that VR can benefit mental health and quality of life in general. In “Tim’s New World”, a man with a rare form of Cerebral Palsy is seen using Ocean Rift as a form of escapism [12]. The app has been used in retirement communities for similar reasons and is often used as a form of therapeutic distraction for chronic pain conditions and treatments. VR and Ocean Rift in particular has also been shown to be effective at reducing fear and anxiety levels in children during blood drawing [13], reducing pain and anxiety in breast cancer patients [14], and as an analgesic for acute and chronic pain in adults [15]. It is also being used to improve the quality of life for MS patients during infusions [16].
This work has also led to many opportunities and research directions for Bangor University and the wider community. Beneficial activities include media appearances, open day demos, public seminars, festivals and school visits; which all in turn benefit admissions. The app also places Bangor research at the very forefront of VR, through ongoing collaboration with leading tech companies. This environment also enhances the student experience, through research-led teaching in modules such as Imperative Programming, AI and Games Design, Applications of AI, and dissertation and PhD projects, where students have access to the latest VR hardware and world-leading expertise.

Impact status	Ongoing
Impact date	1 Aug 2013 → 31 Jul 2020
Category of impact	Technological, Health/Quality of life