WO2018176927A1

WO2018176927A1 - Binocular rendering method and system for virtual active parallax computation compensation

Info

Publication number: WO2018176927A1
Application number: PCT/CN2017/117130
Authority: WO
Inventors: 赵凤萍; 邓金富
Original assignee: 上海讯陌通讯技术有限公司
Priority date: 2017-04-01
Filing date: 2017-12-19
Publication date: 2018-10-04
Also published as: CN107071384B; CN107071384A

Abstract

The present invention provides a binocular rendering method and system for virtual active parallax computation compensation. The method comprises: active parallax computation: performing binocular rendering adaptation on an initial user to obtain a dedicated adaptive parallax synchronization matrix corresponding to the user; image parallax compensation and cropping: using the dedicated adaptive parallax synchronization matrix of the user as a cropping factor for virtual active parallax compensation, cropping a complete image into left and right eye groups; and binocular rendering: pre-rendering dichoptic images of the same monocular camera on a waiting overlay canvas of a next scene, and rendering and projecting, according to a fixed time sequence period, a composite image of the prepared overlay canvas on a target display hardware layer in one pass. In the present invention, a single camera is used as basic image acquisition hardware, and ghosting causing dizziness is reduced since, after software-based parallax computation compensation, the effect approaches one in which a wearer directly observes natural objects with their own eyes. Optimal adjustment of parallax is actively achieved, and a cropped single view source is reversely compensated, achieving a more immersive visual effect.

Description

Binary rendering method and system for virtual active disparity calculation compensation

Technical field

The present invention relates to the field of virtual reality image processing technologies, and in particular, to a binocular rendering method and system for virtual active disparity calculation compensation.

Background technique

The distance between the eyes of a person is a certain distance, so when observing a three-dimensional object, since the two eyes are horizontally separated at two different positions, the observed image of the object is different, and there is an aberration between them. The existence of aberrations, through the human brain, can feel stereoscopic effects. How to compensate the rendering with the virtual active parallax method to enhance the look and feel of the virtual reality is the focus of the present invention.

The imaging foundation relied on in virtual reality is based on the simulation of binoculars in the natural environment of two different images with aberrations, while the monocular camera acquisition device is still the mainstream (smartphone, tablet, notebook, headset, Cameras, etc., binoculars are not suitable for popularization due to the specialization of the use scene and the limitations of physical hardware. So how to use the monocular camera to capture the simulated dual-purpose framing effect and compensate the rendering through virtual active parallax is also difficult and different.

At present, the prior art deals with the above difficulties, and still has the following drawbacks:

1) The single image is directly divided by the split screen device, and the difference image seen by the binocular can be stabilized according to the actual physiological parallax of the viewer (because of wearing the device, due to the distance of the pupil) and the like;

2) The image captured by the single camera is ultimately from an observer origin, and the images of the two observer origins obtained by the dual cameras respectively introduce a ghost of the parallax due to the target object, and the elimination of the ghosting depends on the external device;

3) After the binocular acquisition, the synchronization between the two images is directly related to whether the user causes vertigo, and the image color, saturation and exposure difference of the two channels will also reduce the virtual reality perception;

4) The traditional dual-purpose view content is fixed parallax that is immutable.

Based on the above limitations and defects, the present invention proposes a binocular rendering method for virtual active disparity calculation compensation for a single-purpose imaging device. The method strives to bring the parallax fitness of the subject closer to the human eyes and directly observe the parallax brought in by nature, and image the virtual active difference part of the left and right eyes. After the rendering preprocessing is completed, the binocular is rendered once to ensure synchronization and obtain enhancement. Virtual reality experience.

After searching, the application number is: 201610666409.6, and the name is: “A virtual reality mobile dynamic time frame compensation rendering system and method”, the method is: applying frame rendering to generate an application frame buffer sequence, and extracting in the application frame buffer sequence The latest or most recent application frame performs secondary rendering to obtain a time frame, and the time frame is sent to the shared buffer, and is refreshed by the screen reading time frame rendering result under the timing control of the vertical synchronization management module. Through the design of the shared buffer, the GPU renders the result directly to the screen refresh buffer, reducing the delay of the multi-level cache exchange. Through vertical synchronization time management, the GPU's rendering time is controlled, and the conflict between GPU rendering and screen refresh reading is avoided, so that the picture can be displayed normally at low delay without tearing.

The method in the above document solves the problem that the split screen image is not synchronized due to the split screen rendering when the number of cache frames is relatively large, which is related to the parallax compensation and the binocular rendering problem close to the real presence experience to be solved by the present invention. Differently, in addition, the present invention performs pre-processing before final rendering to the screen to achieve the same effect of the comparative invention.

Application No.: 201511001007.6, entitled "A Method and Apparatus for Adjusting Parallax of Virtual Reality", wherein the method of the present invention mainly comprises: acquiring a parallax range preset by a virtual reality model, the parallax range being a parallax upper limit value And a range of a parallax lower limit value; obtaining a stereoscopic slice source and a disparity value corresponding to the stereoscopic slice source, the stereoscopic slice source being a slice source displayed in a virtual reality scene; adjusting the parallax range according to the parallax range The disparity value of the stereoscopic sheet source such that the disparity value of the stereoscopic sheet source does not exceed the parallax range. Compared with the prior art, the present invention can improve the display effect of displaying a new stereoscopic scene in a stereoscopic scene.

The methods in the above documents mainly focus on and improve the display effect of displaying a new stereoscopic scene in a stereoscopic scene, which is different from the parallax compensation problem to be solved by the present invention and the binocular rendering problem close to the real on-the-spot experience.

Application No.: 201410302221.4, entitled "Binocular three-dimensional graphics rendering method and related system", including a projection transformation step, the projection transformation step includes: adding a middle plane as a projection plane between the near plane and the far plane, and the near plane and the far side The primitives between the planes are projected onto the midplane. The invention projects the primitive between the near plane and the far plane onto the middle plane through the added midplane, and the primitive between the near plane and the middle plane has a stereoscopic effect of the screen, between the middle plane and the far plane The primitives have a stereo effect of entering the screen; thus, the existing "outline" and "on screen" effects can be rendered without using special hardware when using the existing rendering pipeline in the 3D display device.

The method in the above document focuses on the rendering of three-dimensional graphics by utilizing the hardware characteristics of the 3D display device, which is solved by the present invention based entirely on software without relying on the parallax compensation binocular rendering method of the 3D display device in the virtual reality device. Not the same problem.

Summary of the invention

In view of the deficiencies in the prior art, an object of the present invention is to provide a binocular rendering method and system for virtual active disparity calculation compensation.

The binocular rendering method for virtual active disparity calculation compensation provided by the present invention includes the following steps:

Active parallax calculation step: performing binocular rendering adaptation on the initial user, and obtaining a dedicated adaptive parallax synchronization matrix of the corresponding user;

Image disparity compensation and clipping step: the user's exclusive adaptive parallax synchronization matrix is used as the clipping factor of the virtual active disparity compensation, and a complete image is cut into left and right eye combinations, which are respectively displayed in the left and right eye framing windows;

Binocular rendering step: pre-rendering the same monocular camera-viewed image on the waiting overlay canvas of the next field, and projecting the prepared overlay canvas composite image rendering target virtual reality header at a time according to a fixed time period. Wear the device's display hardware layer.

Preferably, the active disparity calculation step comprises:

Step A1: selecting a plurality of original images of different resolutions from the image library, and performing binocular rendering on the user wearing the virtual reality wearing device for the first time through the original image, based on the M points at the fixed position in the original image. The positions are adapted one by one; wherein M is a positive integer, and the positions of the M points are selected based on the form of the adaptive parallax synchronization matrix;

Step A2: obtaining the optimal parallax correction parameter and the visual range of each region in the original image by using the user's adaptive feedback, and generating the adapted parallax synchronization matrix of the user; wherein the original image has a visual range that is greater than the left and right eyes respectively. Range of vision;

Step A3: automatically generate an active parallax compensation standard suitable for the user's own eyes according to the wearer's sensory comfort response, and use the active parallax compensation standard as the best fitting adjustment standard for each subsequent view.

Preferably, the image disparity compensation and clipping step comprises:

Step B1: The left and right eyes are respectively based on the parallax tailoring factors of different regions as the clipping basis of the current wearer to view the virtual reality, and the complete image is cut into the optimal left and right eye combinations, and the optimal left and right eye combinations are:

In the initial adjustment, the wearer determines the best fit result according to his own wearing habits. The best fit result is the best adjustment target, saves the compensation matrix generated by the optimal adjustment target, and makes the compensation matrix The dynamic adjustment is used as a reference, and the compensation matrix is a calibration matrix, which is also called an optimal left and right eye combination;

Step B2: The two-way divided image obtained by the step B1 is used as two subsets of the original image pixel ensemble, and the two subsets have an intersection; the clipping is performed by the virtual active parallax compensation clipping factor, so that the left and right eyes see the image. Consistent with the principle of maximizing the optimal parallax compensation, the principle of maximizing the optimal parallax compensation is: recording data based on single or multiple active corrections as a set of candidate compensation, in a period of wearing, recording and screening The preferred compensation matrix for different light, scene, and wearing angle, and the maximum binocular synthesis field of view as the auxiliary target;

Step B3: performing pre-processing on the image obtained by the subsequent monocular camera, and forming a separate view image of the two-way coordinate system, wherein the sub-view compensation parameters include: according to different observation angles, observation distances, Light, color, image conversion, and image cropping.

Preferably, the period of the fixed timing in the binocular rendering step is to render each image by a specified frequency timing.

A binocular rendering system for virtual active disparity calculation compensation according to the present invention includes the following modules:

Active disparity calculation module: used for performing binocular rendering adaptation on the initial user, and obtaining a dedicated adaptive disparity synchronization matrix of the corresponding user;

Image disparity compensation and trimming module: used as a tailoring disparity synchronization matrix of the user as a clipping factor of virtual active disparity compensation, and cuts a complete image into left and right eye combinations, which are respectively displayed in the left and right eye framing windows;

Binocular rendering module: used to pre-render the same monocular camera image on the waiting overlay canvas of the next field, and project the prepared overlay canvas composite image rendering to the target virtual at a time according to the fixed timing period. Realistic display device hardware layer.

Preferably, the active disparity calculation module includes:

Parallax synchronization matrix selection sub-module: selecting a plurality of original images with different resolutions from the image library, and performing binocular rendering on the user wearing the virtual reality wearing device for the first time through the original image, based on the fixed position in the original image The positions of the M points are adapted one by one; wherein M is a positive integer, and the positions of the M points are selected based on the form of the adaptive parallax synchronization matrix;

Adapting the parallax synchronization matrix generation sub-module: obtaining the optimal parallax correction parameters and the visual range of each region in the original image through the user's adaptive feedback, and generating the adaptive parallax synchronization matrix of the user; wherein the original image has a range of visual forces Greater than the left and right eyesight alone;

The best fit calibration standard generation sub-module: automatically generates an active parallax compensation standard suitable for the user's own eyes according to the wearer's sensory comfort response, and based on the active parallax compensation standard as the best fit adjustment for each subsequent view standard.

Preferably, the image disparity compensation and cropping module comprises:

The best left and right eye combination generation sub-module: the left and right eyes are respectively based on the parallax clipping factors of different regions as the clipping basis of the current wearer to view the virtual reality, and a complete image is cut into the best left and right eye combination, the best left and right eye Eye combination means:

The cropping sub-module: the two-way divided image obtained by the cropping is used as two subsets of the original image pixel complete set, the two subsets have an intersection; the cropping factor of the virtual active parallax compensation is used to compensate the cropping, so that the left and right eyes see the image conforming The principle of maximizing the optimal parallax compensation, the principle of maximizing the optimal parallax compensation is: the data based on single or multiple active corrections as a set of candidate compensation, recorded and selected in a period of wearing Including: different compensation modes for different light, scene, and wearing angle, and the maximum binocular synthesis field of view as the auxiliary target;

The sub-view image generation sub-module: pre-processes the image obtained by the subsequent monocular camera by the pre-view compensation parameter to form a separate bi-directional coordinate system, and the sub-view compensation parameter refers to the observation environment: Observe the angle, the observation distance, the brightness, and the color when the image is transformed and the image is cropped.

Compared with the prior art, the present invention has the following beneficial effects:

1. The binocular rendering method for virtual active parallax calculation compensation provided by the present invention uses a single camera as the hardware foundation of the image acquisition device, and compensates for the wearer's own eyes to directly observe the natural and reduce the ghosting caused by vertigo by software parallax calculation. According to the individual difference of the wearer, the optimal adjustment of the parallax can be actively obtained, and the single view source of the cropping can be compensated in reverse to achieve a more realistic immersive look.

2. The binocular rendering method of the virtual active parallax calculation compensation provided by the invention ensures the timing synchronization in the binocular rendering process, the color and the luminosity are consistent, and the two-way imaging has other effects, which helps the wearing body to dynamically modify the rendering content to Meet the virtual effects of binoculars close to reality.

3. The binocular rendering method for virtual active parallax calculation compensation provided by the invention can provide targeted and precise adjustment for the parallax of the view content, which not only satisfies the synchronization requirement, but also satisfies the difference of parallax and reduces dizziness.

DRAWINGS

Other features, objects, and advantages of the present invention will become apparent from the Detailed Description of Description

1 is a flowchart of a binocular rendering method for virtual active disparity calculation compensation provided by the present invention;

2 is a schematic diagram of pre-calibration of left and right eyes for virtual active disparity calculation compensation provided by the present invention;

3 is a standard diagram of a best fit for a binocular generated by a user;

Figure 4 is a schematic diagram showing the result of the cropping compensation;

Figure 5 is a new view of the render composition.

detailed description

The invention will now be described in detail in connection with specific embodiments. The following examples are intended to further aid the invention to those skilled in the art, but are not intended to limit the invention in any way. It should be noted that a number of changes and modifications may be made by those skilled in the art without departing from the inventive concept. These are all within the scope of protection of the present invention.

Active parallax calculation step: performing binocular rendering on the initial user to obtain a dedicated adaptive parallax synchronization matrix of the corresponding user. Specifically, since the single camera input source is selected, the active parallax calculation mentioned in the present invention is virtual active parallax calculation, that is, when the user first uses, firstly, several dual-purpose images are rendered to the wearer, respectively. The nine point positions shown in Fig. 2 are adapted one by one, and the parameters of the optimal parallax correction and the range of visual acuity of each region are obtained by the user's adaptive feedback (minimum of 3 points of 1/3/5/7/9) And, in turn, generate a wearer-specific adaptive parallax synchronization matrix. The original image's visual range is always greater than the left and right eyesight's separate visual range. Finally, the active parallax compensation standard suitable for the viewer's own eyes is achieved, and the compensation based on this will be the best fit adjustment for each subsequent view, as shown in FIG.

Image Disparity Compensation and Trimming Step: The user's exclusive adaptive parallax synchronization matrix is used as the clipping factor of the virtual active parallax compensation, and a complete image is cut into left and right eye combinations, which are respectively displayed in the left and right eye view windows. Specifically, the adaptive parallax synchronization matrix that best fits the wearer is used as the clipping factor of the virtual active disparity compensation, and the original image is a complete set of pixels, and the left and right eyes respectively use the disparity clipping factor of different regions as the current wearer to view the virtual Based on the realistic tailoring, a complete image is cut into the best left and right eye combination, and the field of view is displayed in a single way in the left and right eye viewfinder windows. The two-way split image thus cropped is a subset of the complete set, but the two subsets overlap and have non-overlapping portions. The split compensation trim is shown in Figure 4. By dividing the compensation of the sub-view, it is ensured that the left and right eyes are in accordance with the maximum of the optimal parallax compensation, so that each subsequent image is acquired by the monocular camera, and then pre-processed into two coordinates according to the above-mentioned sub-view compensation parameters. There are other separate images.

Binocular rendering step: pre-rendering the image of the same monocular camera to the waiting overlay on the next field, and projecting the prepared overlay composite image at the target display hardware map at a time according to the period of the fixed timing. Floor. Specifically, the image rendering is designed to synthesize the image of the same monocular camera, and the pre-rendering is synthesized on the waiting superimposed canvas of the next field. According to the period of the fixed timing, the prepared superimposed canvas is rendered and projected at one time. The hardware layer is displayed in the target, because in the pre-processing binocular rendering has ensured the optimal combination of color, chromatic aberration and parallax, the image has been synthesized in the fixed timing cycle to ensure optimal combination of binocular timing, as shown in Figure 5. In order to obtain a similar wearable adjuster, I directly observe the binocular experience of nature.

The technical solution in the present invention will be described in more detail below with reference to specific embodiments.

As shown in FIG. 1, the binocular rendering method for virtual active disparity calculation compensation provided by the present invention includes the following steps:

Step S1: randomly selecting several original images of different resolutions from the template library;

Step S2: pre-marking 5 to 9 parallax standard crosses on the selected original image;

Step S3: attempting to separately project the synthesized area in the binocular display;

Step S4: the calibration cross is adjusted by the wearer according to his own perception, if the non-vertigo binocular fitting that meets the maximum viewing angle is performed, step S5 is performed, otherwise, step S3 is performed;

Step S5: storing the final compensation adjustment parameter of the calibration cross according to the self-perception of the wearer, and attempting to adjust the compensation parameters of the plurality of sets according to different resolutions;

Step S6: start frame processing of the actual single-view video data stream, perform reverse cutting based on the pre-prepared parallax compensation adjustment value, and cut out the left-eye frame and the right-eye frame from the original image respectively;

Step S7: pre-synthesizing the binoculars with different cuts, reconstructing a new composite image with a larger overall size than the original view, waiting for rendering in the binocular viewfinder;

Step S8: Rendering into a new picture at one time.

Example 1 (Virtual Reality Immersion Entertainment)

In the virtual reality immersive entertainment such as roller coaster, deep sea tour and game, if the collection device is a single camera or a composite image into a single image data source, the virtual active device can perform virtual active parallax compensation rendering on the virtual device side to the user. Provides the best immersive entertainment experience with low stun ghosting and high image quality.

Embodiment 2 (360 degree panoramic broadcast)

When 360-degree panoramic collection and live broadcast of sports events or concerts, multi-channel image arrays usually perform image fusion in real time, thus losing the unique perspective of multiple cameras. Through the present invention, virtual active devices can be virtualized on the virtual reality device side. Parallax compensated rendering provides the user with the best panoramic live broadcast experience with low stun ghosting and high image quality.

Those skilled in the art will appreciate that the system provided by the present invention and its various devices can be logically gated, except that the system provided by the present invention and its various devices are implemented in purely computer readable program code. Switches, ASICs, programmable logic controllers, and embedded microcontrollers are used to achieve the same functionality. Therefore, the system and its various devices provided by the present invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within hardware components; A device that implements various functions is considered to be either a software module that implements a method or a structure within a hardware component.

The specific embodiments of the present invention have been described above. It is to be understood that the invention is not limited to the specific embodiments described above, and various changes or modifications may be made by those skilled in the art without departing from the scope of the invention. The features of the embodiments and the embodiments of the present application may be combined with each other arbitrarily without conflict.

Claims

A binocular rendering method for virtual active disparity calculation compensation, comprising the following steps:

Active parallax calculation step: performing binocular rendering adaptation on the initial user, and obtaining a dedicated adaptive parallax synchronization matrix of the corresponding user;

Image disparity compensation and clipping step: the user's exclusive adaptive parallax synchronization matrix is used as the clipping factor of the virtual active disparity compensation, and a complete image is cut into left and right eye combinations, which are respectively displayed in the left and right eye framing windows;

Binocular rendering step: pre-rendering the same monocular camera-viewed image on the waiting overlay canvas of the next field, and projecting the prepared overlay canvas composite image rendering target virtual reality header at a time according to a fixed time period. Wear the device's display hardware layer.
The binocular rendering method for virtual active disparity calculation compensation according to claim 1, wherein the active disparity calculation step comprises:

Step A1: selecting a plurality of original images of different resolutions from the image library, and performing binocular rendering on the user wearing the virtual reality wearing device for the first time through the original image, based on the M points at the fixed position in the original image. The positions are adapted one by one; wherein M is a positive integer, and the positions of the M points are selected based on the form of the adaptive parallax synchronization matrix;

Step A2: obtaining the optimal parallax correction parameter and the visual range of each region in the original image by using the user's adaptive feedback, and generating the adapted parallax synchronization matrix of the user; wherein the original image has a visual range that is greater than the left and right eyes respectively. Range of vision;

Step A3: automatically generate an active parallax compensation standard suitable for the user's own eyes according to the wearer's sensory comfort response, and use the active parallax compensation standard as the best fitting adjustment standard for each subsequent view.
The binocular rendering method for virtual active disparity calculation compensation according to claim 1, wherein the image disparity compensation and clipping step comprises:

Step B1: The left and right eyes are respectively based on the parallax tailoring factors of different regions as the clipping basis of the current wearer to view the virtual reality, and the complete image is cut into the optimal left and right eye combinations, and the optimal left and right eye combinations are:

In the initial adjustment, the wearer determines the best fit result according to his own wearing habits. The best fit result is the best adjustment target, saves the compensation matrix generated by the optimal adjustment target, and makes the compensation matrix The dynamic adjustment is used as a reference, and the compensation matrix is a calibration matrix, which is also called an optimal left and right eye combination;

Step B2: The two-way divided image obtained by the step B1 is used as two subsets of the original image pixel ensemble, and the two subsets have an intersection; the clipping is performed by the virtual active parallax compensation clipping factor, so that the left and right eyes see the image. Consistent with the principle of maximizing the optimal parallax compensation, the principle of maximizing the optimal parallax compensation is: recording data based on single or multiple active corrections as a set of candidate compensation, in a period of wearing, recording and screening The preferred compensation matrix for different light, scene, and wearing angle, and the maximum binocular synthesis field of view as the auxiliary target;

Step B3: performing pre-processing on the image obtained by the subsequent monocular camera, and forming a separate view image of the two-way coordinate system, wherein the sub-view compensation parameters include: according to different observation angles, observation distances, Light, color, image conversion, and image cropping.
The binocular rendering method for virtual active disparity calculation compensation according to claim 1, wherein the period of the fixed timing in the binocular rendering step is to render the respective images by the specified frequency timing.
A binocular rendering system for virtual active disparity calculation compensation, comprising the following modules:

Active disparity calculation module: used for performing binocular rendering adaptation on the initial user, and obtaining a dedicated adaptive disparity synchronization matrix of the corresponding user;

Image disparity compensation and trimming module: used as a tailoring disparity synchronization matrix of the user as a clipping factor of virtual active disparity compensation, and cuts a complete image into left and right eye combinations, which are respectively displayed in the left and right eye framing windows;

Binocular rendering module: used to pre-render the same monocular camera image on the waiting overlay canvas of the next field, and project the prepared overlay canvas composite image rendering to the target virtual at a time according to the fixed timing period. Realistic display device hardware layer.
The virtual active disparity calculation compensation binocular rendering system according to claim 5, wherein the active disparity calculation module comprises:

Parallax synchronization matrix selection sub-module: selecting a plurality of original images with different resolutions from the image library, and performing binocular rendering on the user wearing the virtual reality wearing device for the first time through the original image, based on the fixed position in the original image The positions of the M points are adapted one by one; wherein M is a positive integer, and the positions of the M points are selected based on the form of the adaptive parallax synchronization matrix;

Adapting the parallax synchronization matrix generation sub-module: obtaining the optimal parallax correction parameters and the visual range of each region in the original image through the user's adaptive feedback, and generating the adaptive parallax synchronization matrix of the user; wherein the original image has a range of visual forces Greater than the left and right eyesight alone;

The best fit calibration standard generation sub-module: automatically generates an active parallax compensation standard suitable for the user's own eyes according to the wearer's sensory comfort response, and based on the active parallax compensation standard as the best fit adjustment for each subsequent view standard.
The virtual active disparity calculation compensation binocular rendering system according to claim 5, wherein the image disparity compensation and cropping module comprises:

The best left and right eye combination generation sub-module: the left and right eyes are respectively based on the parallax clipping factors of different regions as the clipping basis of the current wearer to view the virtual reality, and a complete image is cut into the best left and right eye combination, the best left and right eye Eye combination means:

In the initial adjustment, the wearer determines the best fit result according to his own wearing habits. The best fit result is the best adjustment target, saves the compensation matrix generated by the optimal adjustment target, and makes the compensation matrix The dynamic adjustment is used as a reference, and the compensation matrix is a calibration matrix, which is also called an optimal left and right eye combination;

The cropping sub-module: the two-way divided image obtained by the cropping is used as two subsets of the original image pixel complete set, the two subsets have an intersection; the cropping factor of the virtual active parallax compensation is used to compensate the cropping, so that the left and right eyes see the image conforming The principle of maximizing the optimal parallax compensation, the principle of maximizing the optimal parallax compensation is: the data based on single or multiple active corrections as a set of candidate compensation, recorded and selected in a period of wearing Including: different compensation modes for different light, scene, and wearing angle, and the maximum binocular synthesis field of view as the auxiliary target;

The sub-view image generation sub-module: pre-processes the image obtained by the subsequent monocular camera by the pre-view compensation parameter to form a separate bi-directional coordinate system, and the sub-view compensation parameter refers to the observation environment: Observe the angle, the observation distance, the brightness, and the color when the image is transformed and the image is cropped.