CN111556304B - Panoramic image processing method, device and system - Google Patents
Panoramic image processing method, device and system Download PDFInfo
- Publication number
- CN111556304B CN111556304B CN202010322704.6A CN202010322704A CN111556304B CN 111556304 B CN111556304 B CN 111556304B CN 202010322704 A CN202010322704 A CN 202010322704A CN 111556304 B CN111556304 B CN 111556304B
- Authority
- CN
- China
- Prior art keywords
- panoramic image
- image
- panoramic
- mask
- rectangular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 88
- 230000009466 transformation Effects 0.000 claims abstract description 65
- 238000004364 calculation method Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000004458 analytical method Methods 0.000 claims abstract description 28
- 238000010191 image analysis Methods 0.000 claims description 12
- 230000001131 transforming effect Effects 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 208000012886 Vertigo Diseases 0.000 abstract description 6
- 231100000889 vertigo Toxicity 0.000 abstract description 6
- 230000008569 process Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 7
- 239000000654 additive Substances 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0088—Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
Abstract
The embodiment of the invention discloses a panoramic image processing method, a panoramic image processing device and a panoramic image processing system, wherein the method comprises the following steps: carrying out projection transformation on the original panoramic image, and splitting the original panoramic image into left and right rectangular panoramic images; performing parallax analysis on the left and right eye rectangular panoramic images; performing mask region calculation processing on the left and right eye rectangular panoramic images according to the parallax analysis result, and performing mask addition processing on the left and right eye rectangular panoramic images according to the calculation processing result; combining the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and outputting the target panoramic image. By adopting the method, the stereoscopic parallax of the top view angle image and the bottom view angle image in the original image can be adjusted by utilizing the image mask, and the stereoscopic vision conflict problem of the existing panoramic image is solved, so that the vertigo feeling of a user when using VR equipment is effectively improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a panoramic image processing method and a panoramic image processing device. In addition, an electronic device and a storage medium are also related.
Background
In recent years, panoramic images have been widely used in social life. When the user uses equipment such as Virtual Reality (VR) glasses, helmet, can experience the immersive sense of immersing that panoramic image brought. When binocular stereoscopic parallax is added to a panoramic image, the immersion and the fidelity are further enhanced, but incorrect binocular stereoscopic parallax causes strong visual conflict and vertigo, and the viewing comfort is seriously affected.
Currently, it is a common panoramic image content generation method to shoot a real scene by using a panoramic camera or to produce a virtual scene based on three-dimensional software such as 3DS MAX and MAYA. When adding binocular stereo parallax to a panoramic image, an additional stereo camera needs to be added to a conventional 2D panoramic camera (a camera in a real scene or a virtual camera in three-dimensional software) system. In the human stereoscopic vision, two-viewpoint images having a slight difference are fused by horizontally arranging left and right eyes, and according to this principle, the two-viewpoint images are also generally horizontally arranged in a real stereoscopic camera system or a virtual camera system. When a panoramic image shot by a horizontally arranged camera system is viewed in VR equipment, binocular stereoscopic parallax is correct when a left eye horizontal viewing angle and a right eye horizontal viewing angle or a small-angle inclined viewing angle are observed. However, when the user raises his head (looks at the sky) or lowers his head (looks at the ground) at a large angle, the left and right eye angles and the stereo shooting angle are difficult to coincide, which causes the binocular stereo parallax to be incorrect, resulting in a strong sense of discomfort in the user's vision. Therefore, it is necessary to provide an improvement solution to the deficiency of the binocular stereoscopic parallax in the existing panoramic image.
Disclosure of Invention
Therefore, the embodiment of the invention provides a panoramic image processing method to solve the problem that the stereoscopic vision conflict of a panoramic image in the prior art causes incorrect binocular stereoscopic parallax and causes strong discomfort in the vision of a user.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
the panoramic image processing method provided by the embodiment of the invention comprises the following steps: carrying out projection transformation on the original panoramic image, and splitting the original panoramic image into left and right rectangular panoramic images; performing parallax analysis on the left and right eye rectangular panoramic images; performing mask region calculation processing on the left and right eye rectangular panoramic images according to the parallax analysis result, and performing mask addition processing on the left and right eye rectangular panoramic images according to the calculation processing result; combining the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and outputting the target panoramic image.
Further, performing parallax analysis on the left-eye and right-eye rectangular panoramic images specifically includes: and calculating the stereo parallax of the left-eye and right-eye rectangular panoramic images to obtain a parallax image.
Further, the performing mask region calculation processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the parallax analysis result, and performing mask addition processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the calculation processing result specifically includes: based on the disparity map, calculating the areas of the left and right eye rectangular panoramic images needing to be added with the top view angle masks and the bottom view angle masks, and adding the top view angle masks and the bottom view angle masks to the corresponding areas of the left and right eye rectangular panoramic images respectively.
Further, the projection transformation is performed on the original panoramic image, and the original panoramic image is split into left and right eye rectangular panoramic images, specifically: according to a preset panoramic image projection transformation rule, transforming an original panoramic image format into a rectangular binocular stereoscopic panoramic image in an area projection format, and splitting a rectangular panoramic image in the rectangular binocular stereoscopic panoramic image into left and right eye rectangular panoramic images with the same size.
Further, based on the disparity map, calculating an area of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image, where a top view mask and a bottom view mask need to be added, and adding the top view mask and the bottom view mask to corresponding areas of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image respectively, specifically including: obtaining an initial value of an image area, a parallax processing threshold range, a mask color value or a brightness value of a preset top view angle and a preset bottom view angle needing to be added with a mask; and according to the image region initial value and the parallax processing threshold range, calculating the range of adding a top view angle mask and a bottom view angle mask in the parallax image, and respectively adding the top view angle mask and the bottom view angle mask in the corresponding ranges of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the mask color values or the brightness values.
Further, merging the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, and restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, specifically: and combining the left and right eye rectangular panoramic images added with the top view angle shade and the bottom view angle shade into a rectangular binocular stereoscopic panoramic image, and performing inverse transformation on the rectangular binocular stereoscopic panoramic image in the area projection format into a target panoramic image consistent with the original panoramic image format according to a preset panoramic image projection transformation rule.
An embodiment of the present application further provides a panoramic image processing apparatus, including: the transformation and splitting unit is used for carrying out projection transformation on the original panoramic image and splitting the original panoramic image into left and right rectangular panoramic images; a parallax analysis unit for performing parallax analysis on the left and right eye rectangular panoramic images; the mask processing unit is used for performing mask region calculation processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the parallax analysis result and performing mask addition processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the calculation processing result; and the panoramic image restoration unit is used for merging the left and right eye rectangular panoramic images added with the masks into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and outputting the target panoramic image.
Further, the parallax analysis unit is specifically configured to calculate a stereoscopic parallax of the left-eye and right-eye rectangular panoramic images to obtain a parallax map.
Further, the mask processing unit is specifically configured to: based on the disparity map, calculating the areas of the left and right eye rectangular panoramic images needing to be added with the top view angle masks and the bottom view angle masks, and adding the top view angle masks and the bottom view angle masks to the corresponding areas of the left and right eye rectangular panoramic images respectively.
Further, the transforming and splitting unit is specifically configured to: according to a preset panoramic image projection transformation rule, transforming an original panoramic image format into a rectangular binocular stereoscopic panoramic image in an area projection format, and splitting a rectangular panoramic image in the rectangular binocular stereoscopic panoramic image into left and right eye rectangular panoramic images with the same size.
Further, the mask processing unit is specifically configured to: obtaining an initial value of an image area, a parallax processing threshold range, a mask color value or a brightness value of a preset top view angle and a preset bottom view angle needing to be added with a mask; and according to the image region initial value and the parallax processing threshold range, calculating the range of adding a top view angle mask and a bottom view angle mask in the parallax image, and respectively adding the top view angle mask and the bottom view angle mask in the corresponding ranges of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the mask color values or the brightness values.
Further, the panoramic image restoration unit is specifically configured to: and combining the left and right eye rectangular panoramic images added with the top view angle shade and the bottom view angle shade into a rectangular binocular stereoscopic panoramic image, and performing inverse transformation on the rectangular binocular stereoscopic panoramic image in the area projection format into a target panoramic image consistent with the original panoramic image format according to a preset panoramic image projection transformation rule.
An embodiment of the present application further provides a panoramic image processing system, including: the device comprises an image transformation module, an image analysis module, a mask processing module and an image restoration module; the image transformation module is used for performing projection transformation on the original panoramic image and splitting the original panoramic image into left and right rectangular panoramic images; the image analysis module is used for connecting the image transformation module, receiving the left and right eye rectangular panoramic images processed by the image transformation module, performing parallax analysis on the left and right eye rectangular panoramic images and outputting a parallax map; the mask processing module is used for connecting the image transformation module and the image analysis module, and performing top view mask and bottom view mask region calculation processing on the parallax image received from the image analysis module to obtain a region range needing to be added with masks; respectively carrying out top view mask and bottom view mask adding processing on the left and right eye rectangular panoramic images received from the image conversion module; the image restoration module is used for connecting the mask processing module, receiving the left and right eye rectangular panoramic images added with the masks, combining the left and right eye rectangular panoramic images added with the masks into a binocular stereoscopic panoramic image, and restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule.
Correspondingly, an embodiment of the present application further provides an electronic device, including: a processor and a memory; the memory is used for storing a program of a panoramic image processing method, and the electronic device executes any one of the panoramic image processing methods after being powered on and running the program of the panoramic image processing method through the processor.
Accordingly, an embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium contains one or more program instructions, and the one or more program instructions are used for a server to execute the panoramic image processing method according to any one of the above embodiments.
By adopting the panoramic image processing method, the stereoscopic parallax of the top view angle image and the bottom view angle image in the original image can be adjusted by utilizing the image mask, and the stereoscopic vision conflict problem of the conventional panoramic image is solved, so that the vertigo sense of a user when the user uses VR equipment is effectively improved; meanwhile, the mask range can be automatically adjusted according to the size of the absolute value of the stereo parallax, the influence of the minimized added mask on the original image is reduced, and therefore the immersion and the fidelity of the original image are more kept on the premise of maximizing the comfort degree.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
Fig. 1 is a flowchart of a panoramic image processing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a panoramic image processing apparatus according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an electronic device according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a panoramic image processing system according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an implementation process of adding a mask area according to an embodiment of the present invention;
fig. 6 is a schematic diagram of transformation and inverse transformation of a projection format of a panoramic image according to an embodiment of the present invention;
FIG. 7 is a block diagram of a panoramic image file processing system according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a left-eye and right-eye rectangular panoramic image according to an embodiment of the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes an embodiment of the panoramic image processing method according to the present invention in detail. As shown in fig. 1, which is a flowchart of a panoramic image processing method according to an embodiment of the present invention, the specific implementation process includes the following steps:
step S101: and carrying out projection transformation on the original panoramic image, and splitting the original panoramic image into left and right eye rectangular panoramic images.
In the embodiment of the present invention, the implementation process of performing projection transformation on the original panoramic image and splitting the original panoramic image into the left and right rectangular panoramic images may include: according to a preset panoramic image projection transformation rule, transforming an original panoramic image format into a rectangular binocular stereoscopic panoramic image in an area projection format, and splitting a rectangular panoramic image in the rectangular binocular stereoscopic panoramic image into left and right eye rectangular panoramic images with the same size. Specifically, according to a preset panoramic image projection transformation rule, original panoramic image formats such as spherical surface, cylindrical projection (ERP), pseudo-cylindrical projection and cubic projection (CMP) can be uniformly transformed into a rectangular binocular stereoscopic panoramic image in a cylindrical equal-area projection (EAP) format, and a rectangular panoramic image in the rectangular binocular stereoscopic panoramic image is further split into left and right rectangular panoramic images P with the same sizelAnd PrIn which P islAnd PrThe image width and height can be noted as W and H, respectively. At this time, Pl(w, h) and PrAnd (W, H) respectively represents the color value (or brightness value) of the pixel with the coordinate (W, H) in the rectangular panoramic images of the left eye and the right eye, W is more than or equal to 1 and less than or equal to W, and H is more than or equal to 1 and less than or equal to H.
Step S102: and performing parallax analysis on the left and right eye rectangular panoramic images.
After the original panoramic image is transformed and split into the left and right rectangular panoramic images in step S101, a disparity map may be obtained by performing disparity analysis on the left and right rectangular panoramic images in this step.
In the embodiment of the present invention, performing parallax analysis on the left-eye and right-eye rectangular panoramic images specifically includes: and calculating the stereo parallax of the left-eye and right-eye rectangular panoramic images to obtain a parallax image. Specifically, the right-eye rectangular panoramic image P may be displayedrFor reference, a left-eye rectangular panoramic image P is calculated and obtained by using a disparity map algorithm of preset binocular stereo views such as a left-right consistency constraint principle, a dynamic programming method, a fixed window area matching method and an image segmentation methodlThe parallax value of each pixel is obtained to obtain a parallax image D of the left-eye rectangular panoramic imagelr. In the implementation process, the disparity map D can be generally usedlrStored as 8-bit integer gray scale image, i.e. pixel D in parallax imagelr(w, h) grayscale representation of the left-eye rectangular panoramic image PlThe magnitude of the parallax for a pixel with a medium coordinate (w, h), typically-127 ≦ Dlr(W, H) is less than or equal to 127, wherein W is less than or equal to 1 and less than or equal to W, and H is less than or equal to 1 and less than or equal to H.
Step S103: and performing mask region calculation processing on the left and right eye rectangular panoramic images according to the parallax analysis result, and performing mask addition processing on the left and right eye rectangular panoramic images according to the calculation processing result.
After the disparity map is obtained in step S102, a mask region calculation process and a mask addition process may be performed on the left-and-right-eye rectangular panoramic images in this step according to the disparity map.
In an embodiment of the present invention, the performing mask region calculation processing on the left-eye and right-eye rectangular panoramic images according to the parallax analysis result, and performing mask addition processing on the left-eye and right-eye rectangular panoramic images according to the calculation processing result specifically includes: based on the disparity map, calculating the areas of the left and right eye rectangular panoramic images needing to be added with the top view angle masks and the bottom view angle masks, and adding the top view angle masks and the bottom view angle masks to the corresponding areas of the left and right eye rectangular panoramic images respectively. Specifically, firstly, obtaining an initial value of an image area, a parallax processing threshold range, a mask color value or a brightness value of a preset top view angle and a preset bottom view angle to which a mask needs to be added; then, according to the image region initial value and the parallax processing threshold range, the range of adding a top view angle mask and a bottom view angle mask is calculated in the parallax image, and according to the mask color value or the brightness value, the top view angle mask and the bottom view angle mask are added in the corresponding range of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image respectively.
In the specific implementation process, the preset initial values of the image areas of which the top view angle and the bottom view angle need to be added with the masks can be obtained in advance, and the initial values of the image areas of which the top view angle and the bottom view angle need to be added with the masks are respectively marked as RtAnd RbWherein, 0<Rt≤H/2≤Rb<H (when R ist，RbWhen H/2, representing the mask region to search from the horizontal center line of the rectangular panoramic image); obtaining a set parallax processing threshold value, wherein the parallax processing threshold value is recorded as d0Wherein d is0The value range is usually an integer between 0 and 127 and is consistent with the parallax gray scale map; and obtaining a set mask color value or brightness value, which is recorded as c0Generally c0Is an integer value between 0 and 255, and is not particularly limited herein; further, adding an initial value R of an image area of a mask according to the top view angle and the bottom view angletAnd RbAnd a parallax processing threshold d0In the disparity map DlrCalculating the range of adding top view mask and bottom view mask, and according to the mask color value or brightness value c0And respectively adding a top view angle mask and a bottom view angle mask in corresponding ranges in the left and right eye rectangular panoramic images.
Wherein, as shown in FIG. 5, the above-mentioned parallax map DlrThe process of calculating the additive top view mask range may comprise: (1) if R istIf not, the top view angle does not need to be added with a mask, and the step of adding the top view angle mask range calculation is quit; otherwise, executing the following step (2); (2) if in the disparity map DlrMiddle ordinate is equal to RtAny one of the pixels Dlr(w，Rt) The vision ofAbsolute value of difference | Dlr(w，Rt)|≥d0Then, the range of the rectangular panoramic image of the left eye and the right eye to which the top view angle mask needs to be added is the vertical coordinate interval [1, R ]t](i.e., line 1 to line R in the left-right eye rectangular panorama imagetAll pixels of the row) to complete the calculation step of adding the top view angle mask range; if the ordinate is equal to RtAll pixel parallax absolute values of (1) are less than d0If yes, executing the step (3); (3) r is to betDecrease in value by 1, i.e. Rt＝Rt-1, repeating step (1).
The implementation process of adding the top view angle mask in the left and right eye rectangular panoramic images may include: (1) setting all pixel color values of the vertical coordinates in the left-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pl(w，h)＝c0，W is more than or equal to 1 and less than or equal to W, h is more than or equal to 1 and less than or equal to Rt(ii) a (2) Setting all pixel color values of the vertical coordinates in the right-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pr(w，h)＝c0，W is more than or equal to 1 and less than or equal to W, h is more than or equal to 1 and less than or equal to Rt。
The in-parallax image DlrThe implementation of the calculating the added base view mask range may include: (1) if R isbH, the bottom view does not need to add a mask, and the step of adding the bottom view mask range calculation is exited; otherwise, executing the following step (2); (2) if it is in the disparity map DlrMiddle ordinate is equal to RbAny one of the pixels Dlr(w，Rb) Absolute value of parallax | Dlr(w，Rb)|≥d0Then, the range of the rectangular panoramic image of the left and right eyes to which the bottom view angle mask needs to be added is the vertical coordinate interval [ R ]b，H](i.e., the R-th panoramic image in the left-and right-eye rectangular panoramic image)bAll pixels from row to row H) to complete the calculation step of adding the bottom view angle shade range; if the ordinate is equal to RbAll pixel parallax absolute values of (1) are less than d0Then, the following step (3) is executed; (3) r is to bebIncrease in value by 1, i.e. Rb＝Rb+1, the above step (1) is repeatedly executed.
The implementation process of adding the bottom view angle mask in the left-eye and right-eye rectangular panoramic images may include: (1) setting all pixel color values of the vertical coordinates in the rectangular panoramic image of the left eye in the calculated bottom view angle shade range as preset values c0I.e. Pl(w，h)＝c0W is more than or equal to 1 and less than or equal to W, RbH is less than or equal to H; (2) setting all pixel color values of the vertical coordinates in the right-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pr(w，h)＝c0W is more than or equal to 1 and less than or equal to W, Rb≤h≤H。
Step S104: combining the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and outputting the target panoramic image.
After the mask adding process in step S103, the left and right eye rectangular panoramic images with the masks added thereto may be merged into a binocular stereoscopic panoramic image, and subjected to inverse transformation to be restored into a target panoramic image.
In the embodiment of the present invention, the combining the left and right eye rectangular panoramic images with the masks added thereto into a binocular stereoscopic panoramic image, and restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule specifically includes: and combining the left and right eye rectangular panoramic images added with the top view angle shade and the bottom view angle shade into a rectangular binocular stereoscopic panoramic image, and performing inverse transformation on the rectangular binocular stereoscopic panoramic image in the area projection format into a target panoramic image consistent with the original panoramic image format according to a preset panoramic image projection transformation rule.
In a specific implementation process, the left and right eye rectangular panoramic images added with the top view angle shade and the bottom view angle shade can be combined into a conventional rectangular binocular stereoscopic panoramic image; and then according to a preset panoramic image projection transformation rule, inversely transforming the rectangular panoramic image in the format of cylindrical equal-area projection (EAP) into the format (as shown in figure 6) corresponding to the original panoramic image, such as spherical projection, cylindrical projection (ERP), pseudo-cylindrical projection, cubic projection (CMP), and the like, and finally outputting the processed new panoramic image (namely, the target panoramic image in accordance with the format of the original panoramic image).
In a complete implementation, the following may be included:
firstly, a 4K ultrahigh-definition binocular stereoscopic panoramic image is manufactured through live-action shooting or CG.
Secondly, transforming the original binocular stereoscopic panoramic image into a binocular stereoscopic rectangular image through an equal-area cylindrical projection (EAP) transformation formula, wherein the aspect ratio of the image is 1:1, and the resolution is 3840 multiplied by 3840; wherein the left eye image and the right eye image have the same size and are arranged from top to bottom; the left-eye rectangular panoramic image PlRight-eye rectangular panoramic image PrIs split out of PlAnd PrThe image height and width may be H1920 and W3840, respectively; pl(w, h) and Pr(w, h) respectively represents pixel color values (or luminance values) of coordinates (w, h) in the rectangular panoramic images of the left and right eyes; wherein h is more than or equal to 1 and less than or equal to 1920, and w is more than or equal to 1 and less than or equal to 3840. The 2K video maximum may be 2048 and the 4K video maximum may be 4096.
Third, to PlAnd PrCalculating binocular stereo parallax of the images to obtain a parallax image Dlr(ii) a Parallax map DlrThe pixel value range of each pixel can be-127 to 127, and the pixel value range represents the stereo parallax size of the left-eye and right-eye rectangular panoramic images at the corresponding positions, and the larger the absolute value of the parallax is, the larger the stereo parallax is, the stronger the stereo perception is.
The fourth step is that in the parallax map DlrCalculating top view mask and bottom view mask regions: firstly, setting initial values of a top view angle area and a bottom view angle area of the additive mask as R respectivelyt＝320、Rb1600 (i.e. 1/6 for the top and bottom view initial regions of the left and right eye images), disparity threshold d020 pixels in size (about 1.5 ° viewing angle range, stereoscopic parallax angle is noticeable to the human eye); then, the following steps are carried out on the disparity map DlrRespectively calculating the areas of the top view angle and the bottom view angle needing to be added with the masks: (1) if R ist0, no mask is added to the top view angleContinuing to execute the following step (4); otherwise, executing the step (2); (2) if in the disparity map DlrMiddle ordinate is equal to RtAny one of the pixels Dlr(w，Rt) Absolute value of parallax | Dlr(w，Rt)|≥d0Then, the range of the rectangular panoramic image of the left eye and the right eye to which the top view angle mask needs to be added is the vertical coordinate interval [1, R ]t](i.e., line 1 to line R in the left-right eye rectangular panorama imagetAll pixels of the row), finishing the calculation of the top view angle mask range, and continuing to execute the step (4); if the ordinate is equal to RtAll pixel parallax absolute values of (1) are less than d0If yes, executing the step (3); (3) r is to betDecrease in value by 1, i.e. Rt＝Rt-1, repeatedly performing step (1); (4) if R isbH, the bottom view angle does not need to be added with a mask, and the mask range calculation step is completed; otherwise, executing the step (5); (5) if in the disparity map DlrMiddle ordinate is equal to RbAny one of the pixels Dlr(w，Rb) Absolute value of parallax | Dlr(w，Rb)|≥d0Then, the range of the rectangular panoramic image of the left and right eyes to which the bottom view angle mask needs to be added is the vertical coordinate interval [ R ]b，H](i.e., the R-th panoramic image in the left-and right-eye rectangular panoramic image)bAll pixels from row to row H) to complete the calculation of the mask range; if the ordinate is equal to RbAll pixel parallax absolute values of (1) are less than d0If yes, executing step (6); (6) r is to bebIncrease in value by 1, i.e. Rb＝Rb+1, repeat step (4).
And a fifth step of setting a mask color (brightness) value to 255 (pure white), and adding top view and bottom view masks in the left and right eye rectangular panoramic images respectively according to the following steps: (1) setting all pixel color values of the vertical coordinates in the left-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pl(w，h)＝c0W is more than or equal to 1 and less than or equal to W, h is more than or equal to 1 and less than or equal to Rt(ii) a (2) Setting all pixel color values of the vertical coordinates in the right-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pr(w，h)＝c0，W is more than or equal to 1 and less than or equal to W, h is more than or equal to 1 and less than or equal to Rt(ii) a (3) Setting all pixel color values of the vertical coordinates in the rectangular panoramic image of the left eye in the calculated bottom view angle shade range as preset values c0I.e. Pl(w，h)＝c0W is more than or equal to 1 and less than or equal to W, RbH is less than or equal to H; (4) setting all pixel color values of the vertical coordinates in the right-eye rectangular panoramic image in the calculated top view angle shade range as preset values c0I.e. Pr(w，h)＝c0，W is more than or equal to 1 and less than or equal to W, Rb≤h≤H。
Sixthly, recombining the left-eye and right-eye rectangular panoramic images with the masks added into a 4K binocular stereoscopic panoramic image format which is arranged side by side up and down, wherein the resolution ratio of the panoramic image format is 3840 multiplied by 3840 (the length-width ratio is 1: 1);
and seventhly, restoring the processed 4K binocular stereoscopic panoramic image into the image format in the first step through an inverse transformation formula of cylindrical equal-area projection (EAP), and obtaining the target panoramic image.
And finally, outputting the target panoramic image.
By adopting the panoramic image processing method, the stereoscopic parallax of the top view angle image and the bottom view angle image in the original image can be adjusted by utilizing the image mask, and the stereoscopic vision conflict problem of the conventional panoramic image is solved, so that the vertigo feeling of a user when using VR equipment is effectively improved.
Corresponding to the panoramic image processing method, the invention also provides a panoramic image processing device. Since the embodiment of the apparatus is similar to the above method embodiment, the description is simple, and please refer to the description of the above method embodiment, and the following description of an embodiment of a panoramic image processing apparatus is only illustrative. Fig. 2 is a schematic view of a panoramic image processing apparatus according to an embodiment of the present invention.
The invention relates to a panoramic image processing device, which comprises the following parts:
and the transforming and splitting unit 201 is configured to perform projection transformation on the original panoramic image and split the original panoramic image into left and right rectangular panoramic images.
A disparity analyzing unit 202, configured to perform disparity analysis on the left-eye and right-eye rectangular panoramic images.
And the mask processing unit 203 is configured to perform mask region calculation processing on the left-eye and right-eye rectangular panoramic images according to the parallax analysis result, and perform mask addition processing on the left-eye and right-eye rectangular panoramic images according to the calculation processing result.
And the panoramic image restoration unit 204 is configured to merge the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, restore the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and output the target panoramic image.
By adopting the panoramic image processing device, the stereoscopic parallax of the top view angle image and the bottom view angle image in the original image can be adjusted by utilizing the image mask, and the stereoscopic vision conflict problem of the conventional panoramic image is solved, so that the vertigo feeling of a user when using VR equipment is effectively improved.
Corresponding to the panoramic image processing method, the invention also provides a panoramic image processing system. Since the embodiment of the system is similar to the above method embodiment, the description is simple, and for the relevant points, reference may be made to the description of the above method embodiment, and the following description of an embodiment of a panoramic image processing system is only illustrative. Please refer to fig. 4 and 7.
The panoramic image processing system comprises the following parts: the device comprises an image transformation module, an image analysis module, a mask processing module and an image restoration module.
The image transformation module is used for inputting an original panoramic image, performing projection transformation on the original panoramic image, and splitting the original panoramic image into left and right rectangular panoramic images. Specifically, the module may convert the original panoramic image into a rectangular panoramic image in an Equal Area Projection (EAP) format, further split the rectangular panoramic image into rectangular panoramic images of left and right eyes with the same size (as shown in fig. 8), and output a rectangular panoramic image P of the left and right eyeslAnd Pr；PlAnd PrThe image width and height are W and H, respectively; pl(w, h) and PrAnd (W, H) respectively represents the color value (or brightness value) of the pixel with the coordinate (W, H) in the rectangular panoramic images of the left eye and the right eye, wherein W is more than or equal to 1 and less than or equal to W, and H is more than or equal to 1 and less than or equal to H.
The image transformation module performs cylindrical equal-area projection on the original panoramic image, which means that a plurality of common panoramic image formats such as spherical projection, cylindrical projection, pseudo-cylindrical projection, cubic projection and the like are transformed into a rectangular panoramic image format in a unified manner through a cylindrical equal-area projection transformation formula, so that the processing of a subsequent module is facilitated.
The image analysis module is used for connecting the image transformation module, receiving the left and right eye rectangular panoramic images processed by the image transformation module, performing parallax analysis on the left and right eye rectangular panoramic images and outputting a parallax map. In the specific implementation process, the disparity map D can be obtainedlrStored as 8-bit integer gray scale image, i.e. pixel D in parallax imagelr(w, h) grayscale representation left-eye image PlThe magnitude of the parallax for a pixel with a medium coordinate (w, h), typically-127 ≦ Dlr(W, H) is less than or equal to 127, wherein W is less than or equal to 1 and less than or equal to W, and H is less than or equal to 1 and less than or equal to H.
The mask processing module is used for connecting the image transformation module and the image analysis module, and performing top view mask and bottom view mask region calculation processing on the parallax image received from the image analysis module to obtain a region range needing to be added with masks; and the top view mask and the bottom view mask adding processing are respectively carried out on the left and right eye rectangular panoramic images received from the image transformation module.
When the mask processing module calculates the mask range, the initial values of the image areas of which the top view angle and the bottom view angle need to be added with masks are respectively set as RtAnd Rb，0<Rt≤H/2≤Rb<H (when R ist，RbWhen H/2, representing the mask region to search from the horizontal center line of the rectangular panoramic image); manually setting parallax processing threshold value as d0，d0The value range is usually an integer between 0 and 127 and is consistent with the parallax gray scale map; according to Rt、RbAnd d0Value in the disparity map DlrTop of the middle computerThe extent of the view mask and the bottom view mask.
The image restoration module is used for connecting the mask processing module, receiving the left and right eye rectangular panoramic images added with the masks, combining the left and right eye rectangular panoramic images added with the masks into a binocular stereoscopic panoramic image, and restoring the binocular stereoscopic panoramic image into a target panoramic image consistent with an input format according to a preset inverse transformation rule.
By adopting the panoramic image processing system, the stereoscopic parallax of the top view angle image and the bottom view angle image in the original image can be adjusted by utilizing the image mask, and the stereoscopic vision conflict problem of the conventional panoramic image is solved, so that the vertigo feeling of a user when using VR equipment is effectively improved.
Corresponding to the panoramic image processing method, the invention also provides electronic equipment. Since the embodiment of the electronic device is similar to the above method embodiment, the description is relatively simple, and please refer to the description of the above method embodiment, and the electronic device described below is only schematic. Fig. 3 is a schematic view of an electronic device according to an embodiment of the present invention.
The electronic device specifically includes: a processor 301 and a memory 302; the memory 302 is used for executing one or more program instructions, and is used for storing a program of a panoramic image processing method, and after the server is powered on and runs the program of the panoramic image processing method through the processor 301, the server executes any one of the panoramic image processing methods. The electronic device of the present invention may be a server.
Corresponding to the panoramic image processing method, the invention also provides a computer storage medium. Since the embodiment of the computer storage medium is similar to the above method embodiment, the description is simple, and please refer to the description of the above method embodiment, and the computer storage medium described below is only schematic.
The computer storage medium contains one or more program instructions for executing the panoramic image processing method by a server.
In an embodiment of the invention, the processor or processor module may be an integrated circuit chip having signal processing capabilities. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (ddr Data Rate SDRAM), Enhanced SDRAM (ESDRAM), synclink DRAM (SLDRAM), and Direct memory bus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory. Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above-mentioned embodiments, objects, technical solutions and advantages of the present invention are further described in detail, it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (9)
1. A panoramic image processing method is characterized by comprising the following steps: carrying out projection transformation on the original panoramic image, and splitting the original panoramic image into left and right rectangular panoramic images; performing parallax analysis on the left and right eye rectangular panoramic images; performing mask region calculation processing on the left and right eye rectangular panoramic images according to the parallax analysis result, and performing mask addition processing on the left and right eye rectangular panoramic images according to the calculation processing result; combining the left and right eye rectangular panoramic images with the masks added into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule, and outputting the target panoramic image;
the performing mask region calculation processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the parallax analysis result, and performing mask addition processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the calculation processing result specifically include: based on the disparity map, calculating the areas of the left and right eye rectangular panoramic images needing to be added with the top view angle masks and the bottom view angle masks, and adding the top view angle masks and the bottom view angle masks to the corresponding areas of the left and right eye rectangular panoramic images respectively.
2. The method of processing a panoramic image according to claim 1, wherein performing disparity analysis on the left-eye and right-eye rectangular panoramic images specifically includes: and calculating the stereo parallax of the left-eye and right-eye rectangular panoramic images to obtain a parallax image.
3. The panoramic image processing method according to claim 1, wherein the original panoramic image is subjected to projective transformation and split into left and right eye rectangular panoramic images, specifically: according to a preset panoramic image projection transformation rule, transforming an original panoramic image format into a rectangular binocular stereoscopic panoramic image in an area projection format, and splitting a rectangular panoramic image in the rectangular binocular stereoscopic panoramic image into left and right eye rectangular panoramic images with the same size.
4. The method of claim 1, wherein the computing, based on the disparity map, the regions of the left and right rectangular panoramic images to which the top view mask and the bottom view mask are added, and the adding the top view mask and the bottom view mask to the corresponding regions of the left and right rectangular panoramic images respectively comprises: obtaining an initial value of an image area, a parallax processing threshold range, a mask color value or a brightness value of a preset top view angle and a preset bottom view angle needing to be added with a mask; and according to the image region initial value and the parallax processing threshold range, calculating the range of adding a top view angle mask and a bottom view angle mask in the parallax image, and respectively adding the top view angle mask and the bottom view angle mask in the corresponding ranges of the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the mask color values or the brightness values.
5. The panoramic image processing method according to claim 1, wherein the left and right eye rectangular panoramic images with the masks added thereto are merged into a binocular stereoscopic panoramic image, and the binocular stereoscopic panoramic image is restored to a target panoramic image according to a preset inverse transformation rule, specifically: and combining the left and right eye rectangular panoramic images added with the top view angle shade and the bottom view angle shade into a rectangular binocular stereoscopic panoramic image, and performing inverse transformation on the rectangular binocular stereoscopic panoramic image in the area projection format into a target panoramic image consistent with the original panoramic image format according to a preset panoramic image projection transformation rule.
6. A panoramic image processing apparatus, comprising: the transformation and splitting unit is used for carrying out projection transformation on the original panoramic image and splitting the original panoramic image into left and right rectangular panoramic images; a parallax analysis unit for performing parallax analysis on the left and right eye rectangular panoramic images; the mask processing unit is used for performing mask region calculation processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the parallax analysis result and performing mask addition processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the calculation processing result; the panoramic image restoration unit is used for merging the left and right eye rectangular panoramic images added with the masks into a binocular stereoscopic panoramic image, restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule and outputting the target panoramic image;
the performing mask region calculation processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the parallax analysis result, and performing mask addition processing on the left-eye rectangular panoramic image and the right-eye rectangular panoramic image according to the calculation processing result specifically include: based on the disparity map, calculating the areas of the left and right eye rectangular panoramic images needing to be added with the top view angle masks and the bottom view angle masks, and adding the top view angle masks and the bottom view angle masks to the corresponding areas of the left and right eye rectangular panoramic images respectively.
7. A panoramic image processing system, comprising: the device comprises an image transformation module, an image analysis module, a mask processing module and an image restoration module; the image transformation module is used for performing projection transformation on the original panoramic image and splitting the original panoramic image into left and right rectangular panoramic images; the image analysis module is used for connecting the image transformation module, receiving the left and right eye rectangular panoramic images processed by the image transformation module, performing parallax analysis on the left and right eye rectangular panoramic images and outputting a parallax map; the mask processing module is used for connecting the image transformation module and the image analysis module, and performing top view mask and bottom view mask region calculation processing on the parallax image received from the image analysis module to obtain a region range needing to be added with masks; respectively carrying out top view mask and bottom view mask adding processing on the left and right eye rectangular panoramic images received from the image conversion module; the image restoration module is used for connecting the mask processing module, receiving the left and right eye rectangular panoramic images added with the masks, combining the left and right eye rectangular panoramic images added with the masks into a binocular stereoscopic panoramic image, and restoring the binocular stereoscopic panoramic image into a target panoramic image according to a preset inverse transformation rule.
8. An electronic device, comprising: a processor; and a memory for storing a program of the panoramic image processing method, wherein the electronic device executes the panoramic image processing method according to any one of claims 1 to 5 after being powered on and the program of the panoramic image processing method is executed by the processor.
9. A computer-readable storage medium containing one or more program instructions for execution by a server of the panoramic image processing method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010322704.6A CN111556304B (en) | 2020-04-22 | 2020-04-22 | Panoramic image processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010322704.6A CN111556304B (en) | 2020-04-22 | 2020-04-22 | Panoramic image processing method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111556304A CN111556304A (en) | 2020-08-18 |
| CN111556304B true CN111556304B (en) | 2021-12-31 |
Family
ID=72003888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010322704.6A Active CN111556304B (en) | 2020-04-22 | 2020-04-22 | Panoramic image processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111556304B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767412A (en) * | 2017-09-11 | 2018-03-06 | 西安中兴新软件有限责任公司 | A kind of image processing method and device |
| CN108521568A (en) * | 2018-03-26 | 2018-09-11 | 天津大学 | The assay method of stereo-picture comfort level disparity range based on salient region |
| CN108810512A (en) * | 2018-04-24 | 2018-11-13 | 宁波大学 | A kind of object-based stereo-picture depth method of adjustment |
| CN109564705A (en) * | 2016-07-29 | 2019-04-02 | 索尼公司 | Image processing apparatus and image processing method |
| US10616568B1 (en) * | 2019-01-03 | 2020-04-07 | Acer Incorporated | Video see-through head mounted display and control method thereof |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6380881B2 (en) * | 2012-07-31 | 2018-08-29 | Ｔｉａｎｍａ Ｊａｐａｎ株式会社 | Stereoscopic image display apparatus, image processing apparatus, and stereoscopic image processing method |
-
2020
- 2020-04-22 CN CN202010322704.6A patent/CN111556304B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109564705A (en) * | 2016-07-29 | 2019-04-02 | 索尼公司 | Image processing apparatus and image processing method |
| CN107767412A (en) * | 2017-09-11 | 2018-03-06 | 西安中兴新软件有限责任公司 | A kind of image processing method and device |
| CN108521568A (en) * | 2018-03-26 | 2018-09-11 | 天津大学 | The assay method of stereo-picture comfort level disparity range based on salient region |
| CN108810512A (en) * | 2018-04-24 | 2018-11-13 | 宁波大学 | A kind of object-based stereo-picture depth method of adjustment |
| US10616568B1 (en) * | 2019-01-03 | 2020-04-07 | Acer Incorporated | Video see-through head mounted display and control method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111556304A (en) | 2020-08-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5654138B2 (en) | Hybrid reality for 3D human machine interface | |
| Rahaman et al. | Virtual view synthesis for free viewpoint video and multiview video compression using Gaussian mixture modelling | |
| KR20170017700A (en) | Electronic Apparatus generating 360 Degrees 3D Stereoscopic Panorama Images and Method thereof | |
| CN111047709B (en) | Binocular vision naked eye 3D image generation method | |
| US11812066B2 (en) | Methods, devices and stream to encode global rotation motion compensated images | |
| CN112017222A (en) | Video panorama stitching and three-dimensional fusion method and device | |
| CN106228530B (en) | A kind of stereography method, device and stereo equipment | |
| CN110769323B (en) | Video communication method, system, device and terminal equipment | |
| CN111612878B (en) | Method and device for making static photo into three-dimensional effect video | |
| WO2018225518A1 (en) | Image processing device, image processing method, program, and telecommunication system | |
| US20190266802A1 (en) | Display of Visual Data with a Virtual Reality Headset | |
| KR20180108106A (en) | Apparatus for reproducing 360 degrees video images for virtual reality | |
| CN112927271A (en) | Image processing method, image processing apparatus, storage medium, and electronic device | |
| WO2023169283A1 (en) | Method and apparatus for generating binocular stereoscopic panoramic image, device, storage medium, and product | |
| CN111556304B (en) | Panoramic image processing method, device and system | |
| CN115314658A (en) | Video communication method and system based on three-dimensional display | |
| CN111629194B (en) | Method and system for converting panoramic video into 6DOF video based on neural network | |
| CN114513646A (en) | Method and device for generating panoramic video in three-dimensional virtual scene | |
| WO2019026183A1 (en) | Image generation device and image generation method | |
| KR20210066825A (en) | Coding and decoding of omnidirectional video | |
| WO2024055379A1 (en) | Video processing method and system based on character avatar model, and related device | |
| US10257488B2 (en) | View synthesis using low resolution depth maps | |
| CN113436304B (en) | Image rendering method and device and head-mounted display equipment | |
| CN117061720B (en) | Stereo image pair generation method based on monocular image and depth image rendering | |
| CN104767986A (en) | Depth of Field (DOF) image correction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240311 Address after: 314050 9F, No. 705, Asia Pacific Road, Nanhu District, Jiaxing City, Zhejiang Province Patentee after: ZHEJIANG YANGTZE DELTA REGION INSTITUTE OF TSINGHUA University Guo jiahuodiqu after: Zhong Guo Address before: No.152 Huixin Road, Nanhu District, Jiaxing City, Zhejiang Province 314000 Patentee before: ZHEJIANG FUTURE TECHNOLOGY INSTITUTE (JIAXING) Guo jiahuodiqu before: Zhong Guo |