How Will VR Change Image Analysis and Visualization?
People are visual learners, and VR could be the next step in making sense of increasingly larger data sets. Image analysis will offer new insights.
A few decades ago, most data was neatly organized in tables. Visual representations included bar graphs, pie-charts or line graphs. The volume, velocity, and variety of data have increased exponentially, and has become a phenomenon called Big Data.
Have visualization techniques kept up with the changes? Most importantly, how can we analyze and extract information from non-standard data such as pictures, camera recordings or medical imagery? What are the right tools to comprehend the knowledge trapped in an image? Virtual reality (VR) can be a natural extension of the 2D visualization techniques of the last century, perfectly adapted to the new challenges.
Reasons to use VR for data visualization
VR is fascinating and still regarded as a craze for the gaming community and is slowly gaining mainstream traction. In fact, this tool could offer a quick and efficient way to grasp relevant information from billions of data points simultaneously in a multi-dimensional environment. It also gives the user the ability to manipulate data in a very intuitive way—almost child-like—by touching and pointing.
Boost Speed
Help users understand data as fast as possible by letting them interact with it. Working with a large data set could become time-consuming and complicated. By using VR, the user can become immersed in the problem. For example, a medical student could learn about the body by looking at each layer of the body and zooming in on the organs up to the cellular level.
The great thing about our brains is that they are entirely adapted for processing visual content, even if it is rich in information. This way of learning or studying is more natural compared to reading a book or taking notes. In fact, it resembles our primitive way of accumulating information by experience.
Multi-dimensional Interaction
VR helps to put data into perspective. 3D rendering creates a new universe which the user will explore at a macro or micro level. Image analysis can be combined with sounds to gain more insights. Also, through selective filtering, the user can look at only those areas or items that are interesting to them at a particular moment.
This approach works great for any set of data which has more than two dimensions and therefore would be difficult to visualize using standard graphs.
A 360-degree view
Entering a virtual world offers a whole new understanding of the depicted phenomenon. Just like on a line chart one can see patterns like cycles, in a VR representation the data analyst can find new insights more efficiently than just looking at data. Through visualization, the user gets access to the essence of the phenomenon promptly.
Technical Requirements of VR
Creating a compelling VR representation can be demanding, both from a computational perspective and obtaining the actual image at a high frame rate, consistently. Each VR frame should be rendered twice, one for each eye. The guidelines of the device, such as this one from Oculus Rift give the necessary constraints related to the granulation of the image, the textures and even the number of vertices.
Keeping in mind these considerations, next come the requirements dictated by the human body. These are necessary to create a flawless experience and maintain the illusion of another world.
Challenges
The minimum requirement is 60 fps, but industry experts push this limit towards 75 fps or even 90 fps to ensure fluidity. Also, the system should react to head movements in less than 20 ms to prevent VR sickness. Add to this the sheer size of the headset device, and you already have a significant challenge. Another limitation is that we can only process around 1KB of information on a flat screen.
One of the most significant challenges of data visualization is the concentration of many data points in the same areas, creating clusters. When this happens, it would be useful to be able to dig deeper into the structure and zoom in.
Solutions
A simple solution to motion sickness is to help the user feel more grounded. We are used to 3D landmarks in the surrounding environment. An excellent VR representation preserves some of these landmarks by placing the user in a virtual room. Simply adding walls to the situation offers the illusion of a familiar and restrained space. The user feels more comfortable in a “room” setting than a completely borderless one.
Another solution offered by VR is that you can use it to represent multidimensional data. For example, by creating more complex objects, each of the dimensions and even color of an object could act as codes for specific characteristics. Also, by being able to enter inside the representation, you can have a very different perspective of the analyzed phenomenon.
Not only does VR create a visual representation of large datasets, but it can offer context and drill down into the structure. The final goal should be that the user has enough freedom to use the visualization as they need and consider fit for their purposes. Through the use of image analysis software, the initial image can be split into smaller parts or the data points can be used to create real-time dense point cloud images.
Last but not least, the problem of optimization can be fixed by a careful analysis of possible bottlenecks in rendering.
Conclusions
VR can help data analysts to make the transition from plots and charts to 3D representations of complex states. It will have a game-changer role in the transformation of data into accessible insights.
The main reasons to pursue this technique are related to the necessity of finding new ways to make sense of the increasingly more massive data sets. Secondly, it is necessary to evaluate the non-standard data from a multidimensional perspective and to give each user the freedom to interact with data as they please and need.
VR comes with multiple technical challenges, and the current state of development is still in its infancy, but there is heightened interest in the advantages offered by this technique.