Google has always believed in the power of immersive virtual reality, but to create the most realistic sense of presence, the content displayed in VR needs to be as close as possible to what it sees.
The light field is an advanced capture, stitching and rendering algorithm. By generating motion parallax and very realistic textures and illumination for statically captured images, the light field can provide you with a very high quality presence. In order to demonstrate the potential of this technology, Google recently officially released the free application "Welcome to Light Fields" on Steam, supporting HTC Vive, Oculus Rift and WMR heads.
Google's light field technology
Capture and process light fields
For the light field, nearby objects seem to be quite close to you. When you move your head, they will look very different. There are fewer changes in distant objects, and different objects emit different light, so you will get a strong hint that you are in 3D space. When browsing through VR heads that support location tracking, the light field can achieve a truly stunning VR experience based on clips captured in the real world.
The reason for this is that the light field can record all the different rays entering the space. To record them, Google adapted a GoPro Odyssey Jump camera platform that was bent into a vertical arc of 16 lenses and mounted on a rotating platform.
Left: Time-lapse video of a spherical light field taken on the deck of the Space Shuttle Discovery.
Right: Light field rendering allows us to sample and interpolate the light recorded by the lens and synthesize a new scene view anywhere in the spherical space.
It takes approximately 1 minute for the camera platform to rotate and record approximately 1000 outward viewing angles on a 70 cm sphere. This provides a two foot wide amount of light, which determines the size of the headspace, which is the space the user turns to explore the scene. To render the view to the head, Google samples the light from the camera position on the surface of the sphere, constructing a new view from inside the sphere and matching the way the user moves the head. They are aligned and compressed in a custom dataset file. According to Xiaobian, Google reads this dataset file through special rendering software as a Unity game engine plugin.
Record the world with light field
Google has chosen several special venues to test their light field camera platform. Google said that they love the lacquered teak and mahogany interiors of the Gamble House, the glossy ceramics and sparkles of the Mosaic Tile House, and the sunshine of the St. Stephen's Church in Granada. Stained stained glass windows. Most importantly, the Smithsonian Institution's Aerospace Museum and 3D Digital Office allowed Google to enter NASA's Discovery spacecraft, providing us with an astronaut's perspective on this unopened flight deck. Finally, Google recorded a series of light fields and experimented with how eye contact can be applied to a 6-degree-of-freedom experience.
VR video is a bright technology for exploring the world. Although it's just an experiment, the light field shows us the level of realism that the VR experience can achieve.
The difference between VR video and panoramic video
What is the difference between VR video and panoramic video?
We use a picture to illustrate the difference between 360-degree panoramic video and VR video:
The panoramic video is well understood, which is different from the traditional viewing angle of traditional video, allowing people to watch 360 degrees freely. On the basis of this, VR video also allows people to freely move and watch in the video (providing a 360-degree free view of any position in the scene).
The panoramic video can be 3D or not, and can be viewed through the screen or with glasses. The VR video must be viewed with the head and must be 3D.
The panoramic video is played linearly, that is, played back according to the timeline. And VR video allows users to watch at different locations at the same time (somewhat like a sci-fi movie paused in time, then the viewer moves freely in the time and space of the freeze, watching the surrounding characters and scenery).
What are the similarities between panoramic video and VR video?
1. Both can be 3D (the panoramic video also has 2D version);
2. Can be viewed 360 degrees up and down, left and right;
3. Have a sense of visual immersion.
What is the difference?
The biggest difference is that VR video allows viewers to walk around in the scene, but panoramic video is not available, where the camera is located, where the viewer must be, at most 360 degrees can be viewed in situ.
If it is a VR movie, the traditional "Lens Movement, Scene Switching, Zoom In or Zoom Out" may be replaced, and the audience determines the lens position. You can think of VR video as "3D panoramic video + free movement".
Therefore, what everyone is seeing now is "pseudo-VR video", and the VR video on the major video websites is basically 360-degree panoramic video. In fact, most of the "VR camera" slogan marketing is a 360-degree panoramic camera. So, we can't help but ask, what is the real VR video?
Real VR video is currently only available via CG
How can I shoot real VR video? Unfortunately, it is still difficult to achieve with current technical means and equipment capabilities. So far, we haven't seen a real VR video actually taken. Because a camera can only capture light in one position, and to achieve free movement in the scene, it is necessary to capture the light anywhere in the scene. Obviously we can't place so many cameras in the scene, which is contradictory. Can Google's light field technology solve this problem?
Is there any possibility in the future? The answer is yes.
To take a true VR video, we need a light field camera.
What is a light field camera? This is definitely a black technology and it is difficult to express it in a few words. Probably means to capture and record the light signals from different angles through a matrix camera (very many micro cameras), and then use the computer to synthesize images at any position.
Lytro's 360-degree light field camera released last year
Immerge, with hundreds of lenses and image sensors inside, is divided into five “layers†(you can imagine 20 layers of GoPro cameras combined on each floor), in addition to dedicated server and editing tools.
Unlike traditional cameras, light field cameras record the direction of light in addition to recording color and light intensity information – this is the origin of the “light field†technology.
With the direction information of the light incident from all directions, the algorithm can be used to analyze and reverse the environment to restore a three-dimensional environment model. This is similar to the results of Computer Modeling (CG), except that CG is a subjective "fictional" model and the light field camera is a reverse-oriented "objective" reduction model.
With the 3D model, the computer can calculate the visual image at any position in real time, and there is no problem with 3D+360 degree panorama. That is to say, the scene captured by the position of the light field camera is real, and the scene seen after freely moving the position is synthesized by the computer later.
Lytro admits that at present they can only make the angle of view move around a certain range and back and forth, and it is not enough to "see" the back of the object (perhaps to calculate and model the reflected light). So to achieve true VR video, or computer modeling and then real-time rendering output is the easiest.
All VR games today can be seen as VR video. Not only can 3D display +360 degree panorama, but also free to move, even control the timeline is no problem. However, VR videos are modeled with computers, and what are they shooting? Should we continue to discuss the shooting process of VR video?
Of course, although the panoramic video is a "pseudo VR video", the panoramic video can still be considered as the initial stage of VR video. Because the panoramic video plus 3D effect can also bring immersion, compared with ordinary video has a qualitative leap, the application field is also very broad, it is worth trying and exploring.
In the future, the presentation of VR will tend to be diversified, and the development of the first display itself will move from the current single screen to the realization of the light field. At present, the domestic and foreign research institutions represented by Magic Leap explore the light field head-mounted display. This kind of head display can make people focus on different depths of virtual objects instead of a display screen. Finally, in terms of application, Google Welcome to Light Field provides us with a good example, better light field. The application method needs further research and exploration. For example, the virtual fusion of the virtual light field and the real world and the natural interaction will become the hotspots of the academic and industrial circles. "
There are some draft products here, which will not be displayed on the website. These products can be modified later to enrich the content of the website. If you want to know more about our company, you can go to our website to check, there will be what you want, or you can consult us. Nice to meet you and looking forward to our cooperation.
Draft-waiting For Releasing,Draft-waiting For Releasing,Draft-waiting For Releasing
ETOP WIREHARNESS LIMITED , https://www.etopwireharness.com