Our products are based on proprietary in-house research. We run a cutting-edge, multi-disciplinary research team in artificial intelligence (AI) and virtual reality (VR) that not only develops the technologies underlying our products, but also performs fundamental research in these areas. Our research has been published in the top AI and VR venues.
It is our unique multidisciplinary approach, drawing on our backgrounds in physics, computer graphics and vision, applied mathematics, and machine learning, that allows us to bring new insights to solve open problems across these fields.
Efficient Generalized Spherical CNNs
International Conference on Learning Representations (ICLR) (2021)
Abstract: Many problems across computer vision and the natural sciences require the analysis of spherical data, for which representations may be learned efficiently by encoding equivariance to rotational symmetries. We present a generalized spherical CNN framework that encompasses various existing approaches and allows them to be leveraged alongside each other. The only existing non-linear spherical CNN layer that is strictly equivariant has complexity O(C2L5), where C is a measure of representational capacity and L the spherical harmonic bandlimit. Such a high computational cost often prohibits the use of strictly equivariant spherical CNNs. We develop two new strictly equivariant layers with reduced complexity O(CL4) and O(CL3 log L), making larger, more expressive models computationally feasible. Moreover, we adopt efficient sampling theory to achieve further computational savings. We show that these developments allow the construction of more expressive hybrid models that achieve state-of-the-art accuracy and parameter efficiency on spherical benchmark problems.
Scattering Networks on the Sphere for Scalable and Rotationally Equivariant Spherical CNNs
International Conference on Learning Representations (ICLR) (2022)
Abstract: Convolutional neural networks (CNNs) constructed natively on the sphere have been developed recently and shown to be highly effective for the analysis of spherical data. While an efficient framework has been formulated, spherical CNNs are nevertheless highly computationally demanding; typically they cannot scale beyond spherical signals of thousands of pixels. We develop scattering networks constructed natively on the sphere that provide a powerful representational space for spherical data. Spherical scattering networks are computationally scalable and exhibit rotational equivariance, while their representational space is invariant to isometries and provides efficient and stable signal representations. By integrating scattering networks as an additional type of layer in the generalized spherical CNN framework, we show how they can be leveraged to scale spherical CNNs to the high-resolution data typical of many practical applications, with spherical signals of many tens of megapixels and beyond.
Reducing Cybersickness in 360-Degree Virtual Reality
Multisensory Research, Volume 35, Issue 2 (2021)
Abstract: Despite the technological advancements in Virtual Reality (VR), users are constantly combating feelings of nausea and disorientation, the so called cybersickness. Triggered by a sensory conflict between the visual and vestibular systems, cybersickness symptoms cause discomfort and hinder the immersive VR experience. Here we investigated cybersickness in 360-degree VR. In 360-degrees VR experiences, movement in the real world is not reflected in the virtual world, and therefore self-motion information is not corroborated by matching visual and vestibular cues, which may potentially induce cybersickness. We have evaluated whether an Artificial Intelligence (AI) software designed to supplement the VR experience with artificial 6-degree-of-freedom motion may reduce sensory conflict, and therefore cybersickness. Explicit (questionnaires) and implicit (physiological responses) measurements were used to measure cybersickness symptoms during and after VR exposure. Our results confirmed a reduction in feelings of nausea during the AI supplemented 6-degree-of-freedom motion VR. Through improving the congruency between visual and vestibular cues, users can experience more engaging, immersive and safe virtual reality, which is critical for the application of VR in educational, medical, cultural and entertainment settings.
Technical Blog Posts
For a more accessible introduction to our in-house research, have a look at our related blog posts:
What Einstein Can Teach Us About Machine Learning
Harnessing symmetry in machine learning
In many ways physics and machine learning share a common goal: to formulate models of observed phenomena. In achieving this goal physicists have long understood the importance of symmetry. In this post we look at how the ideas of symmetry from physics may be leveraged as guiding principles in machine learning. Read more on Towards Data Science.
Solving Cybersickness with AI
How geometric AI techniques can be used to alleviate cybersickness
While 360° VR can transport you anywhere in the world to enjoy, for example, virtual entertainment, tourism, cultural or educational experiences, today’s technology supports only 3 DOF motion, where you cannot move around in the virtual world. This limits realism and can induce cybersickness. Fortunately, recently developed geometric AI techniques tailored specifically to 360° photos and videos, realised in Kagenova’s copernic360 technology, can synthesise 6 DOF to allow you to move about in the virtual world, alleviating cybersickness and substantially enhancing realism. Read more on Towards Data Science.
Geometric Deep Learning for Spherical Data
By encoding an understanding of the translational symmetry of the physical world, convolutional neural networks (CNNs) have revolutionised computer vision. In this post we investigate how the principles underlying the success of CNNs may be transferred to the range of problems for which the data exhibits complex geometry, such as the sphere. Read more on Towards Data Science.
Efficient Generalized Spherical CNNs
Hybrid rotationally equivariant spherical CNNs
Notions of spherical convolution offer a promising route to unlocking the potential of deep learning for the variety of problems in which spherical data are prevalent. However, the introduction of non-linearity is a challenge. In this post we explore how ideas originating in quantum physics may be applied to overcome this barrier. We introduce new approaches for implementing these ideas efficiently in practice. Read more on Towards Data Science.
We make use of and contribute to a number of open source codes developed initially by Kagenova team members.
S2LET: Fast wavelets on the sphere
Functionality to compute fast wavelet transforms of signals on the sphere based on the approach outlined in McEwen et al. (2018) and references therein. Axisymmetric, directional and spin wavelet transforms are supported.
The technology behind our products is patented in a number of juristictions, with further jurisdictions and patents pending.
Method and system for providing at least a portion of content having six degrees of freedom motion
Configuring an immersive experience
Rendering an immersive experience