WO2020098721A1 - 虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质 - Google Patents

虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质 Download PDF

Info

Publication number
WO2020098721A1
WO2020098721A1 PCT/CN2019/118202 CN2019118202W WO2020098721A1 WO 2020098721 A1 WO2020098721 A1 WO 2020098721A1 CN 2019118202 W CN2019118202 W CN 2019118202W WO 2020098721 A1 WO2020098721 A1 WO 2020098721A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
image data
polar coordinate
pixels
coordinate system
Prior art date
Application number
PCT/CN2019/118202
Other languages
English (en)
French (fr)
Inventor
孔德辉
徐科
杨维
游晶
刘欣
艾吉松
Original Assignee
深圳市中兴微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中兴微电子技术有限公司 filed Critical 深圳市中兴微电子技术有限公司
Priority to US17/293,749 priority Critical patent/US11908107B2/en
Publication of WO2020098721A1 publication Critical patent/WO2020098721A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/10—Selection of transformation methods according to the characteristics of the input images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G06T3/604—Rotation of whole images or parts thereof using coordinate rotation digital computer [CORDIC] devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Definitions

  • Embodiments of the present disclosure relate to the field of virtual reality (Virtual Reality, VR), for example, to an image rendering method, device, device and computer storage medium of a VR device, which can be displayed for a virtual reality head-mounted display device (HMD)
  • the system performs geometric distortion correction (Graphic Distortion Correction, GDC).
  • VR technology uses the near-field display system to build realistic scenes, allowing users to experience large spaces in a small area. To achieve this goal, it is particularly important to build a realistic and efficient near-field display system.
  • VR technology has been widely used, for example, in telemedicine, education and training, and industrial simulation.
  • a VR system is composed of a source generation end, a data processing end, and a data presentation end; the source generation end is realized by a motion camera or a model composition based on Graphics (Processing Unit, GPU), and the data processing end is set to realize data splicing and transmission And extraction, the data presentation end is set to present the extracted content to the user.
  • the data presentation end is the most compact module connecting the user and the content, which directly affects the user experience, and the user experience is obviously affected by the data presentation end. How to reduce the distortion at the data presentation end (such as the distortion introduced by HMD) is an urgent problem to be solved.
  • Embodiments of the present disclosure provide an image rendering method, apparatus, device, and computer storage medium of a VR device, which can reduce distortion at the data rendering end of the VR device.
  • An embodiment of the present disclosure provides an image presentation method of a VR device.
  • the method includes:
  • the pole of the polar coordinate system is the intersection of the center of the field of view of the VR device and the display plane;
  • the image data to be presented is presented based on the convex lens.
  • An embodiment of the present disclosure also provides an image presentation device for a VR device.
  • the device includes: an acquisition module, a processing module, and a presentation module, where
  • Acquisition module set to acquire input image data
  • the processing module is set to form a corresponding relationship between input image data and output image data in a polar coordinate system; obtain image data to be presented according to the output image data; the pole of the polar coordinate system is the center of the field of view of the VR device and Show the intersection of planes;
  • the rendering module is configured to render the image data to be rendered based on the convex lens.
  • An embodiment of the present disclosure also provides an image rendering device of a VR device, the device including: a processor and a memory for storing a computer program that can run on the processor; wherein,
  • An embodiment of the present disclosure also provides a VR device.
  • the VR device includes any one of the foregoing devices or any one of the foregoing devices, and further includes: a source generating terminal and a data processing terminal.
  • Embodiments of the present disclosure also provide a computer storage medium that stores a computer program on the computer storage medium.
  • the computer program is executed by a processor to implement the above method.
  • FIG. 1 is a flowchart of an image presentation method of a VR device provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a geometric distortion model of a convex lens according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of differences between a polar coordinate system and a Cartesian coordinate system according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a method for mapping input space to output space in an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of a method for mapping output space to input space in an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of the composition structure of an image presentation device of a VR device according to an embodiment of the present disclosure
  • FIG. 7 is a schematic structural diagram of an image presentation device of a VR device according to an embodiment of the present disclosure.
  • the HMD in the VR device can realize the image presentation through the convex lens group, that is, the image data to be presented can be presented based on the device of the convex lens at the data presentation end; the convex lens group in the HMD is set to increase the depth of field of the content to be presented, which can be reduced The user's discomfort in near-field viewing, but at the same time caused a series of geometric distortions (Graphic Distortion, GD).
  • the convex lens in HMD has the center symmetry characteristic with the focus as the core. In actual implementation, you can use the data rendering end of the VR device to receive the input image data.
  • the input image data is the data obtained after processing by the source generating end and the data processing end of the VR device.
  • the input image data can be a rectangular image Data, for example, the input image data is image data with a resolution of A * B, A represents the number of horizontal pixels of the input image data, and B represents the number of vertical pixels of the input image data; in practical applications, the data of the VR device is presented The end can process the input image data and present it.
  • the data presentation end of the VR device must solve two problems: 1) how to reduce the distortion introduced by the HMD device, and 2) how to make the model flexibly adapt to multiple devices; the first problem is reflected in Improve the authenticity of user viewing content.
  • the purpose of the second aspect is to reduce the dependence of distortion correction on resolution and equipment through modeling.
  • An embodiment of the present disclosure proposes an image presentation method of a VR device, which can be applied to the data presentation end of the VR device.
  • FIG. 1 is a flowchart of an image presentation method of a VR device according to an embodiment of the present disclosure. As shown in FIG. 1, the process may include steps 1010 to 1040.
  • Step 1010 Acquire input image data.
  • the input image data includes the coordinates and pixel values of a plurality of pixels; wherein the pixel values are used to represent the luminance component or the chrominance components of the pixels, for example, the pixel values may include Red Green Blue (Red Green Blue) , RGB) R component, G component or B component of space, may also include Y component, Cb component or Cr component of YCbCr space.
  • Red Green Blue Red Green Blue
  • RGB Red Green Blue
  • R component Red Green Blue
  • G component or B component of space may also include Y component, Cb component or Cr component of YCbCr space.
  • Step 1020 Under the polar coordinate system, the corresponding relationship between the input image data and the output image data is formed; the pole of the polar coordinate system is the intersection of the center of the field of view of the VR device and the display plane.
  • the output image data can be obtained according to the input image data and the correspondence between the input image data and the output image data.
  • the solution for implementing the GDC may be to perform pre-distortion processing on the input image data.
  • the principle of the pre-distortion processing will be described below.
  • HMD represents the mapping method of the above convex lens
  • f (x, y) represents the output result of the data processing end, that is, the pixel value of the pixel with Cartesian coordinates (x, y) output from the data processing end
  • f (x, y) is consistent with the content in the actual scene in size, position, color and shape
  • f '(x, y) represents the content presented after the above convex lens.
  • the further away from the center of the convex lens the more magnified the area.
  • This non-uniform distortion changes the relative size and position of the object, and due to different color distortions, obvious halos will appear on the edges of the object On the other hand, it increases the difficulty of numerical quantization.
  • the non-uniform distortion caused by the convex lens can be suppressed by the distortion suppression method represented by equation (2).
  • HMD represents the above-mentioned mapping method of input image data to output image data (used to represent the mapping relationship between multiple pixels of input image data to multiple pixels of output image data).
  • the coordinates of multiple pixels of the input image data can be mapped according to a specific mapping method in a polar coordinate system to obtain output image data; the specific mapping method is used to represent the convex lens Inverse mapping of image enlargement mapping.
  • the image magnification mapping method of the convex lens needs to be determined.
  • the parameters (including distortion parameters) of the convex lens can be determined and stored in advance, and the parameters of the convex lens can be obtained by reading, and then based on The parameters of the convex lens determine the image enlargement mapping method of the convex lens, and further determine the inverse mapping of the image enlargement mapping method of the convex lens, that is, determine the specific mapping method.
  • the distortion suppression scheme under the Cartesian coordinate system is mostly used, and the Cartesian coordinate system uses two or more intersecting at the origin
  • the number axis constitutes a plane radial coordinate system, including a rectangular coordinate system and an oblique coordinate system.
  • the commonly used Cartesian coordinate system is a rectangular coordinate system.
  • the position of any point can be calibrated using the coordinate of the corresponding point on the number axis.
  • the direction of the number axis is used as the unit direction.
  • the unit length indicates the distance from the origin in this direction. . Calculate the distance of the point from the origin by the square and the square.
  • a distortion suppression scheme in a polar coordinate system may be adopted.
  • the polar coordinate system is the distance from the reference point to the origin (called the polar diameter ⁇ ) and the angle between the reference line and the origin and the horizontal line (called the Polar angle ⁇ ) to describe the position of the reference point.
  • the limit ⁇ is greater than or equal to zero, and ⁇ is between 0 and 2 ⁇ , each point on the plane except the poles has a unique polar coordinate description ( ⁇ , ⁇ ).
  • the advantage of the Cartesian coordinate system is that it is intuitive and easy to understand, but in actual calculations, the polar coordinate system has some advantages in modeling and has a wide range of applications in calculus.
  • the GD for the lens system has the characteristic of radial homogeneity, that is, the deformation generated at the same point as the focal distance is the same; for the convex lens in the HMD device, the predistortion is equivalent to the image shrinking to the focus. For rectangular images, the formation Pincushion-like distortion.
  • the coordinates of multiple pixels in the input image data are respectively converted from the Cartesian coordinate system to the polar coordinate system;
  • Example sexually the coordinates of multiple pixels in the input image data can be converted from the Cartesian coordinate system to the polar coordinate system according to the display resolution of the VR device, respectively.
  • the display resolution of the VR device may be the display resolution of the HMD in the VR device.
  • the display resolution of the HMD in the VR device indicates the resolution of the data to be displayed, which may be video source data or a graphics processor (Graphics Processing Unit, GPU) The resolution of the data drawn.
  • the display resolution of the above HMD may be the display resolution of the HMD corresponding to the monocular, for example, the output resolution of the HMD is 2560 * 1440, then the display resolution of the HMD corresponding to the monocular is 1280 * 1440;
  • the display resolution of the HMD may be determined according to the aspect ratio of the input monocular resolution. For example, if the input monocular resolution is 1200 * 1200, the display resolution of the HMD is 1280 * 1208.
  • the display resolution of the VR device can be predetermined and stored in the VR device. In this way, the display resolution of the VR device can be obtained by reading.
  • the input image data are coordinate points in the plurality of pixels in the Cartesian coordinate system of the coordinate (x, y), according to a Cartesian coordinate system to a polar coordinate system conversion Formula, calculate the corresponding polar coordinate data ( ⁇ , ⁇ ), and then get the image grid in the polar coordinate system.
  • the polar coordinate data ( ⁇ , ⁇ ) it is necessary to obtain the coordinates (x 0 , y 0 ) of the intersection of the center of the field of view of the VR device and the display plane in the Cartesian coordinate system; in practical applications, The coordinates of the intersection of the center of the field of view of the VR device and the display plane can be predetermined and stored. In this way, the coordinates of the intersection of the center of the field of view of the VR device and the display plane can be obtained by reading.
  • the coordinate conversion formula of the Cartesian coordinate system to the polar coordinate system is the same, that is, for the display resolution of the same VR device, a unified Cartesian Coordinate conversion formula from coordinate system to polar coordinate system.
  • the polar diameter of the polar coordinate data after coordinate conversion can be scaled to (0,1).
  • the display resolution of the VR device can be expressed as m * n, where m and n are both greater than 1; the coordinate conversion formula of the Cartesian coordinate system to the polar coordinate system described above is described by formula (5) below.
  • a table can be used to express the coordinate conversion formula of the Cartesian coordinate system to the polar coordinate system; the coordinate system conversion corresponding to the multiple display resolutions of the VR device can be predetermined Table, the above coordinate system conversion table is used to represent the coordinate mapping relationship of multiple pixel points in the image from Cartesian coordinate system to polar coordinate system.
  • a coordinate system conversion table (Look-Up-Table (LUT)) corresponding to a plurality of predetermined display resolutions may be obtained, according to the display resolution of the VR device, and the multiple A coordinate system conversion table corresponding to each display resolution converts the coordinates of multiple pixel points in the input image data from the Cartesian coordinate system to the polar coordinate system, respectively.
  • LUT Look-Up-Table
  • the coordinate system conversion table corresponding to the display resolution of the VR device may be selected from the coordinate system conversion tables corresponding to the multiple display resolutions according to the display resolution of the VR device, and the selected coordinates
  • the system conversion table converts the coordinates of multiple pixels in the input image data from the Cartesian coordinate system to the polar coordinate system, respectively.
  • a unified coordinate system conversion table can be used for the display resolution of the same VR device, when different VR devices (such as different HMD devices) have the same display resolution,
  • the reuse of the unified coordinate system conversion table is easy to implement in hardware design, which can reduce the dependence on the VR device model or HMD device model when performing coordinate conversion, and overcomes the parameter setting of the VR device or HMD device in the related art Sensitive defects facilitate the adaptation to different VR devices or different HMD devices, and improve the ease of expansion of the embodiments of the present disclosure.
  • Step 1030 Obtain image data to be presented according to the output image data.
  • the image data to be presented is realized by the convex lens group of the VR device, and the image data to be presented includes the coordinates and pixel values of a plurality of pixels.
  • the image data to be presented includes image data obtained by determining the pixel values of multiple pixel points of the output image data; the pixel values of multiple pixel points in the output image data can be determined according to the output image The coordinates and pixel values of multiple pixels in the data are used to obtain the image data to be presented.
  • mapping method For the implementation of determining the pixel values of multiple pixels in the output image data, for example, it can be implemented based on two mapping methods, where the first mapping method is denoted as the mapping method from the input space to the output space, and the second The mapping method is referred to as the mapping method from the output space to the input space; the two mapping methods are described below.
  • the first mapping method the mapping method from input space to output space.
  • a polar coordinate pixel grid of the input space is generated; each grid in the polar coordinate pixel grid of the input space corresponds to a pixel of the input image data.
  • the polar of the input space The pixel points corresponding to the coordinate pixel grid may not include all pixel points in the image corresponding to the input image data.
  • the pixel points corresponding to the multiple grids in the polar coordinate pixel grid of the input space are respectively mapped to obtain the mapped multiple discrete pixel points;
  • the mapped multiple discrete pixel points are all in the polar coordinate pixel grid in the output space.
  • the zoom-in mapping method belongs to the non-uniform zoom-in mapping method.
  • the specific mapping method is the non-uniform zoom-out mapping method. Therefore, when the polar coordinate pixel grid of the input space is a uniformly distributed grid, the polar coordinate pixel grid of the output space is output. Is non-uniform.
  • the pixel values of multiple discrete pixels after mapping can be determined.
  • the pixel value of each discrete pixel after mapping is the input image data in the polar coordinate pixel grid of the input space before mapping
  • the pixel values of the corresponding pixel points in the input space that is, the pixel points of the polar coordinate pixel grid of the input space form a one-to-one correspondence with the mapped discrete pixel points, and the pixel value of each discrete pixel point after the mapping is the corresponding input space
  • the pixel values of the multiple pixel points of the polar coordinate pixel grid in the output space are determined.
  • the pixel value of the corresponding pixel can be determined by interpolation operation; in one embodiment, it can be based on The polar coordinate pixel grid of the output space to determine the neighborhood of the pixel points that do not belong to the polar coordinate pixel grid of the output space; perform interpolation operations according to the pixel values of multiple pixel points in the neighborhood of the pixel point To get the pixel value of the pixel; for example, when i takes a positive integer, if the ith pixel of the output image data does not belong to a pixel in the polar coordinate pixel grid of the output space, you can output In the spatial polar coordinate pixel grid, the neighborhood of the i-th pixel of the output image data is determined, and interpolation operations are performed according to the pixel values of multiple pixels in the neighborhood of the i-th pixel of the output image data to obtain The pixel value of the ith
  • pixel interpolation based on the polar coordinate system when performing interpolation operations based on the pixel values of multiple pixels in the neighborhood of the corresponding pixel, pixel interpolation based on the polar coordinate system may be used.
  • Multi-pixel-based interpolation is affected by two factors. One is the queue collection class library Tape number, that is, the number of pixels participating in the interpolation, and the second is the weight, which determines the influence of the pixels participating in the interpolation on the interpolation operation result. In one embodiment, the weight of each pixel is adjusted according to the number of tapes.
  • the selection scheme of the Tape number here is the same as the scheme in the Cartesian coordinate system, that is, the p ⁇ q neighborhood of the pixel position after the mapping is selected, p and q are integers greater than 1; in selecting the pixel weights, the polar coordinate system
  • the interpolation scheme needs to comprehensively consider the difference between the polar diameter and the polar angle, and the pixel interpolation method in the polar coordinate system is exemplarily described below through equation (6).
  • ( ⁇ 1 , ⁇ 0 ) represents the coordinates of the pixel point that needs to be interpolated to obtain the pixel value
  • ( ⁇ i , ⁇ j ) represents the neighborhood of the pixel point that needs to be interpolated to obtain the pixel value
  • PixelOutput ( ⁇ 1 , ⁇ 0 ) represents the pixel value of the pixel obtained by interpolation
  • PixelInput ( ⁇ i , ⁇ j ) represents the pixel of the pixel whose coordinate is ( ⁇ i , ⁇ j )
  • the value, weight i, j represents the normalized weight of the pixel with coordinates ( ⁇ i , ⁇ j ).
  • the weight sum can be quantized to the power of 2, which is realized by multiply-add and shift operations.
  • the mapping weight scheme can be different. Therefore, the weight can be determined according to the difference between polar diameter and polar angle.
  • different polar-based The interpolation method in the coordinate system is similar to the bilinear interpolation and trilinear interpolation method; in actual implementation, an interpolation scheme similar to Lanczos can be used, or a bilinear interpolation method based on polar diameter and polar angle can be used.
  • a bilinear interpolation method may be used, and interpolation operations are performed according to pixel values of multiple pixels in the neighborhood of the pixels.
  • the pixel value of any pixel of the output image data can be directly determined as the polar coordinate pixel grid of the output space
  • the pixel value of the corresponding pixel for example, when i takes a positive integer, if the ith pixel of the output image data belongs to a pixel in the polar coordinate pixel grid of the output space, the ith pixel of the image data is output
  • the pixel value of the pixel is: the pixel value of the corresponding pixel in the polar coordinate pixel grid of the output space.
  • the advantage of using the first mapping method described above is that it can output pixel information more efficiently.
  • the disadvantage is that in one embodiment, a large amount of data needs to be cached to ensure that all pixels required for interpolation have been mapped, which increases hardware.
  • Designed storage resource consumption for example, static random access memory (Static Random-Access Memory, SRAM) consumption
  • SRAM static random access memory
  • the second mapping method the mapping method from the output space to the input space.
  • a polar coordinate pixel grid of the input space is generated; in an embodiment, each grid in the polar coordinate pixel grid of the input space corresponds to a pixel point of the input image data, in an embodiment
  • the pixels corresponding to the polar coordinate pixel grid of the input space may not include all pixels in the image corresponding to the input image data.
  • mapping method of the output image data to the input image data for example, the inverse mapping of the above specific mapping method
  • multiple pixels in the image corresponding to the output image data are respectively mapped to the polar coordinate pixel network of the input space Grid to obtain multiple discrete pixels after mapping.
  • the polar diameter can be remapped according to the parameters of the HMD, that is, each pixel of the output image data is mapped according to the mapping method of the output image data to the input image data to obtain the output image data
  • the pixel position ( ⁇ 0 , ⁇ 0 ) is mapped to the polar coordinate pixel grid of the input space ( ⁇ 1 , ⁇ 0 ); according to the characteristics of the convex lens mapping method, the distortion caused by the convex lens mapping method is symmetrical about the focal point of the convex lens, Therefore, when the pole of the polar coordinate system is the intersection of the center of the field of view of the VR device and the display plane, the distortion caused by the convex lens mapping method is only the radial distortion in the polar coordinate system. Therefore, during remapping (the specific mapping method Before and after the inverse mapping), the coordinates of the pixel point will only change the polar diameter, and the polar angle will not change.
  • the remapping can be explained by equation (7).
  • h () represents a function corresponding to the mapping method of output image data to input image data.
  • the polar coordinate pixel network that does not belong to the input space can be determined by interpolation
  • the pixel value of the pixel point of the grid; in an embodiment, the neighborhood of the pixel point that does not belong to the polar coordinate pixel grid of the input space can be determined according to the polar coordinate pixel grid of the input space;
  • the pixel values of multiple pixels in the neighborhood of the pixel are interpolated to obtain the pixel value of the pixel; for example, when i takes a positive integer, if the i If the pixel does not belong to a pixel in the polar coordinate pixel grid of the input space, the neighbor of the i-th pixel of the plurality of discrete pixel points after the mapping can be determined in the polar coordinate pixel grid of the input space Field, interpolating according to the pixel values of multiple pixels in the neighborhood of the i-th pixel of the multiple discrete pixels after the mapping
  • the interpolation operation method is the same as the interpolation operation method in the first mapping method.
  • the pixel value of any one of the mapped multiple discrete pixels can be directly determined Is the pixel value of the corresponding pixel in the polar coordinate pixel grid of the input space; for example, when i takes a positive integer, if the i-th pixel of the plurality of discrete pixels after mapping belongs to the polar coordinate pixel of the input space
  • the pixel value of the ith pixel of the multiple discrete pixels after the mapping is: the pixel value of the corresponding pixel in the polar coordinate pixel grid of the input space.
  • the pixel values of the plurality of discrete pixel points after the mapping can be obtained according to the pixel values of the plurality of mapped discrete pixel points; that is, By mapping the output image data to the input image data, each pixel before and after the mapping forms a one-to-one correspondence.
  • the pixel value of each pixel of the output image data can be determined as: The pixel value of the corresponding pixel in the multiple discrete pixels.
  • the advantage of using the above second mapping method is that it can effectively reduce the input line buffer and make full use of all the information of the input pixels; in one embodiment, according to the above second mapping method, multiple pixels of the output image data The polar coordinates of the points are mapped to the input space one by one. Because the polar coordinate pixel grid of the input space is uniform, each time a pixel is mapped, the pixels involved in the interpolation can be uniquely determined according to the mapping results and the neighborhood parameters; In an embodiment, compared with the first mapping method, there is no need to cache the mapping results of multiple pixels, which can effectively reduce the input line cache.
  • One potential drawback of the above second mapping method is that if pixels are mapped to the polar coordinate pixel grid in the input space, the interval between the mapped multiple discrete pixels is uneven, and for the mapped multiple discrete The neighbor range selected by the pixels at the same time cannot contain all the pixels of the polar coordinate pixel grid of the input space. It is possible that some pixels in the polar coordinate pixel grid of the input space do not participate in the interpolation process, resulting in the input image data information.
  • this defect can be weakened by increasing the number of interpolated Tapes, but the increase in the number of Tapes will also cause the interpolation complexity to become larger; this kind of defect will be obvious when the distortion difference between adjacent output pixels is large (such as output resolution The effective edge area with a lower rate), but with the continuous improvement of the display resolution, the defect is no longer obvious (when the display resolution of the input and output is guaranteed to be above 2K, the defect is difficult to distinguish with the naked eye). Therefore, the embodiment of the present disclosure generally adopts the second mapping method for mapping.
  • mapping method from the input image data to the output image data is the specific mapping method described above
  • the convex lens will enlarge the content of the image
  • the reverse mapping of the convex lens mapping method will reduce the content of the image.
  • the image area before mapping by the above-mentioned specific mapping method for example, a rectangular area
  • black borders such as black borders caused by pincushion distortion
  • the pixels of the image area mapped by the mapping method involve the input signal (that is, whether they are derived from the pixel mapping of the input image data), and are processed according to the judgment result. In one embodiment, if the judgment result is yes, you can Zero the pixel value of the pixel.
  • the area where multiple pixels in the image corresponding to the input image data are located can be removed from the area where multiple pixels in the image corresponding to the output image data are located to obtain an edge pixel area;
  • the pixel values of multiple pixels are set to zero, and the to-be-presented is generated according to the coordinates and pixel values of multiple pixels of the output image data and the coordinates and pixel values of multiple pixels of the edge pixel area Image data.
  • Step 1040 Present the image data to be presented based on a convex lens.
  • Embodiments of the present disclosure provide an image rendering method, apparatus, device, and computer storage medium of a VR device to obtain input image data; under a polar coordinate system, a corresponding relationship between input image data and output image data is formed; the polar coordinate system The pole of is the intersection of the center of the field of view of the virtual reality VR device and the display plane; the image data to be presented is obtained according to the output image data; and the image data to be presented is presented based on the convex lens.
  • the convex lens mapping method has a center symmetry characteristic about the focal point
  • the convex lens mapping method has a center symmetry characteristic about the pole.
  • the embodiments of the disclosure only The mapping result can be obtained by polar diameter mapping in polar coordinate system, without polar angle mapping, and the calculation complexity is lower than the mapping method in Cartesian coordinate system, which is convenient for hardware implementation.
  • steps 1010 to 1030 can be implemented by the processor of the VR device, and the above processor can be located at the data rendering end of the VR device.
  • Step 1040 can be implemented by using the processor of the VR device in combination with the hardware device of the data rendering end. It can be an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), digital signal processor (Digital Signal Processor, DSP), digital signal processing device (Digital Signal Processing Device, DSPD), programmable logic device (Programmable Logic Device, PLD) ), At least one of a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, and a microprocessor.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD programmable logic device
  • FPGA field programmable gate array
  • CPU Central Processing Unit
  • controller central processing unit
  • microcontroller and a microprocessor.
  • the coordinate mapping scheme is shifted from the Cartesian coordinate system to the polar coordinate system, which can improve the effect of GDC; in the embodiment of the present disclosure, each pixel in the input image data is obtained
  • the coordinate data ( ⁇ , ⁇ ) in the polar coordinate system maps multiple pixel points in the input image data according to a specific mapping method (polar diameter mapping) to obtain output image data, and the coordinate data of the pixel points of the output image data is written as ( ⁇ ', ⁇ )
  • the mapping process of the above specific mapping method does not correct the polar angle.
  • the pixel value of the corresponding pixel is set to zero, that is, zero padding; if the pixels of the image area mapped by the specific mapping method are in accordance with After being mapped by the above specific mapping method, the pixel value of the corresponding pixel can be obtained through interpolation operation, and the brightness or chromaticity result f ( ⁇ ′, ⁇ ) of the pixel to be presented can be obtained.
  • the method of the embodiment of the present disclosure does not solve the distance from the pixel point to the focal point
  • the distance of pixels in the coordinate system is not redistributed according to the following formula:
  • r ' represents the polar diameter mapped according to the mapping method of the convex lens.
  • the above two sets of operations are also not suitable for hardware implementation.
  • the embodiments of the present disclosure do not perform two sets of operations, which can reduce the computational complexity of the VR device when rendering images.
  • the interpolation in the Cartesian coordinate system uses the relative position of the reference point and the center, and the interpolation in the polar coordinate system uses the polar diameter of the reference point and the point to be interpolated.
  • the polar angle information has a better fit with the distortion model, which in turn can improve the GDC effect.
  • a GDC scheme based on a polar coordinate system is provided, which is completed pixel by pixel in the polar coordinate system Distance mapping is different from the scheme based on block mapping in the related art.
  • all pixels of the input image data are mapped through the position coordinates of the input space and the output space, that is, the accuracy of each pixel can be very high, and
  • block-based mapping schemes in addition to the high accuracy of boundary points, the accuracy of other points continuously decreases as the distance from the boundary increases.
  • the embodiments of the present disclosure only involve polar-diameter mapping, and the computational complexity is lower than that in the Cartesian coordinate system.
  • the polar diameter and polar angle parameters are used to realize the interpolation operation in the polar coordinate system to obtain the output result, which reduces the sensitivity of the distance map to the parameters and can improve the accuracy of the interpolation.
  • FIG. 2 is a schematic diagram of a geometric distortion model of a convex lens according to an embodiment of the present disclosure.
  • a small circle represents pixels of input image data
  • a large circle represents distortion caused by a convex lens mapping method corresponding to each pixel.
  • the center position represents the focal position of the convex lens. It can be seen that the degree of distortion depends on the distance of the pixel from the focal point, which corresponds to the polar diameter in the polar coordinate system.
  • FIG. 3 is a schematic diagram of the difference between the polar coordinate system and the Cartesian coordinate system involved in an embodiment of the present disclosure.
  • a small circle represents a pixel point.
  • the polar diameters of two pixel points A and B The same, but the polar angles are different;
  • the weight of the interpolation operation depends on the distance between the two pixels in the x direction and y direction, as shown by line segment AB in the figure;
  • the The weight of the interpolation operation depends on the difference between the polar diameter and polar angle of the pixel.
  • the distance of the pixel point in the polar angle is not directly equal to its Euclidean distance; similarly, in the polar coordinate system and the Cartesian coordinate system, the degree of distortion of the Euclidean distance from the reference point is also inconsistent. Therefore, the GDC scheme in the polar coordinate system is more in line with the distortion principle of the HMD device. At the same time, using the polar coordinate grid calculated only once can reduce the sensitivity to the HMD device parameters.
  • the pixel points corresponding to a plurality of grids in the polar coordinate pixel grid of the input space may be mapped according to the first mapping method described above, to obtain a plurality of mapped discrete pixel points;
  • the position of multiple discrete pixel points in the polar coordinate pixel grid of the output space is assigned weights according to the interpolation method based on the polar coordinate system (using the bilinear-based polar coordinate interpolation weight distribution method).
  • the interpolation calculation result ie, pixel value
  • FIG. 4 is a schematic diagram of a mapping method from an input space to an output space in an embodiment of the present disclosure.
  • a circle represents a grid of polar coordinate pixels in the input space
  • a five-pointed star represents a plurality of discrete pixel points after the mapping.
  • the middle numbers 1 to 5 represent the 5 different discrete pixel points after the mapping; in one embodiment, the mapped multiple discrete pixel points are not uniform with respect to the polar coordinate pixel grid of the input space.
  • the relative relationship with the polar angle determines that one pixel of the input space (represented by a solid circle) can have a weighted influence on the surrounding 5 different discrete pixels (represented by the curve with arrows in Figure 4); in Figure 4, with arrows
  • the curve of indicates that pixels in the input space are assigned weights to the mapped multiple discrete pixels.
  • A1 Generation of polar coordinate pixel grid.
  • the focus information is used to characterize the center point of the field of view, usually corresponding to a pixel of the display resolution, and the default setting is the center of the monocular area; here, the depth of field of the 3D video can be modified by adjusting the focus to adapt The interpupillary distance of the user.
  • B1 GDC mapping pixel by pixel.
  • the first mapping method described above can be used for GDC mapping; the GDC parameters are used for polar coordinate mapping from input space to output space, and each polar coordinate mapping from input space to output space is reserved. The polar angle information of the pixel points and the polar diameter information are corrected.
  • x N represents the Nth pixel participating in the interpolation
  • x j represents the jth pixel participating in the interpolation
  • j takes 1 to N
  • N is an integer greater than 1
  • weight N represents the Nth participating
  • weight j represents the weight of the jth pixel participating in the interpolation
  • ⁇ weight q represents the sum of the weights of all pixels participating in the interpolation
  • x output represents the result of the interpolation operation.
  • the interpolation is stopped and the calculation result is output.
  • C1 Pixel interpolation in polar coordinate system.
  • the interpolation operation is performed on the pixels.
  • the interpolation method based on the polar coordinate system is characterized in that its weight design depends on the relative relationship of the polar coordinate space, and needs to meet the close distance The point weight is not lower than the point far away.
  • dis abs ( ⁇ 0 - ⁇ 1 ) ⁇ max ( ⁇ 0 , ⁇ 1 ) + abs ( ⁇ 0 - ⁇ 1 ), the calculation participates The distance between the interpolated multiple pixels and the pixels that need to be interpolated to obtain pixel values
  • dis i represents the distance between the i-th plurality of pixels participating in interpolation and the pixels that need to be interpolated to obtain pixel values.
  • the weight of the pixels participating in interpolation can be expressed by equation (9).
  • steps B1 and C1 may be performed pixel by pixel for each color channel, thereby the output result may simultaneously suppress color distortion.
  • the coordinate grid of the input space and the coordinate grid of the output space for the display allocation rate of the VR device.
  • the coordinate grid of the input space and the coordinate grid of the output space use the polar diameter at [0, 1], the polar angle Polar coordinate representation in the range of [0, 2 ⁇ ]; for HMD, the display resolution (ie, output resolution) is usually fixed. Therefore, two LUT tables can be set to establish the coordinate network of the input space. Grid and coordinate grid of output space.
  • the coordinates of each LUT table represent the Cartesian coordinate system x, y coordinates, and the value represents its polar diameter or polar angle.
  • each pixel of the output image data is mapped to a polar coordinate pixel of the input space according to the inverse mapping of the specific mapping method (that is, the image magnification mapping method of the convex lens) Grid to get the mapped multiple discrete pixels; according to the position of the mapped multiple discrete pixels in the polar coordinate pixel grid of the input space, select the neighboring pixels in the polar coordinate pixel grid of the input space Interpolation operation is performed, and the interpolation weights are in accordance with the bilinear interpolation weight allocation scheme, which can be referred to equation (8).
  • the specific mapping method that is, the image magnification mapping method of the convex lens
  • FIG. 5 is a schematic diagram of a mapping method from output space to input space in an embodiment of the present disclosure.
  • a circle represents a polar coordinate pixel grid of the input space
  • a five-pointed star represents a plurality of pixel points of the output image data after mapping Discrete pixels, with numbers 1 to 3 in Figure 5 respectively representing 3 different pixels in the output image data;
  • the polar coordinate pixel grid of the input space is not uniform with respect to the pixels of the output image data .
  • the relative coordinates of the polar diameter and polar angle determine that the pixels of the polar coordinate pixel grid of the surrounding 5 input spaces have a heavy influence on one pixel of the output image data (represented by a solid five-pointed star); in Figure 5, with The curve of the arrow indicates that pixels in the input space assign weights to pixels in the output image data.
  • the mapping method from the output space to the input space is adopted, the corresponding polar coordinate positions of the luminance or chrominance components are shifted, and interpolation is performed according to the polar diameter and polar angle difference after the shift, which can be explained by the following three steps .
  • A2 Generation of polar coordinate pixel grid.
  • step A1 The implementation of this step is the same as the implementation of step A1.
  • the second mapping method described above can be used for GDC mapping; in one embodiment, the polar coordinate mapping of the output space to the input space is performed by using GDC parameters, and each polar coordinate mapping of the output space to the input space is reserved. The polar angle information of each pixel is corrected for the polar diameter information.
  • x j represents the jth pixel participating in the interpolation
  • j takes 1 to N
  • N is an integer greater than 1
  • weight j represents the weight of the jth pixel participating in the interpolation
  • x output represents the result of the interpolation operation.
  • formula (10) embodies the polar coordinate mapping from the output space to the input space, which is convenient for pipeline design in hardware design and Reduce cache requirements.
  • C2 Pixel interpolation in polar coordinate system.
  • the interpolation operation is performed on the pixels.
  • the interpolation method based on the polar coordinate system is characterized in that its weight design depends on the relative relationship of the polar coordinate space, and needs to meet the close distance The point weight is not lower than the point far away.
  • steps B2 and C2 can be performed pixel by pixel for each color channel, thereby the output results can simultaneously suppress color distortion.
  • the pixel points corresponding to a plurality of grids in the polar coordinate pixel grid of the input space may be mapped according to the first mapping method described above, to obtain a plurality of mapped discrete pixel points;
  • the position of multiple discrete pixels in the polar coordinate pixel grid of the output space is assigned weights according to the interpolation method based on the polar coordinate system (the polar coordinate interpolation weight distribution method based on Lanczos interpolation). After the distribution of the weights of the discrete pixels is completed, the result of the interpolation operation (that is, the pixel value) of the multiple discrete pixels after the mapping is output by weighted sum.
  • the presentation method of the VR device can be described by the following three steps.
  • A3 Generation of polar coordinate pixel grid.
  • step A1 The implementation of this step is the same as the implementation of step A1.
  • step B1 The implementation of this step is the same as the implementation of step B1.
  • C3 Pixel interpolation in polar coordinate system.
  • the interpolation operation is performed on the pixels.
  • the interpolation method based on the polar coordinate system is characterized in that its weight design depends on the relative relationship of the polar coordinate space, and needs to meet the close distance The point weight is not lower than the point far away.
  • weight_origin sinc (dis2) / sinc (dis2 ⁇ N) (11)
  • weight_origin represents the initial weight of pixels
  • N represents the number of pixels participating in the interpolation, that is, the number of Tapes.
  • ( ⁇ 1 , ⁇ 0 ) represents the coordinates of the pixels that need to be interpolated to obtain pixel values
  • PixelOutput ( ⁇ 1 , ⁇ 0 ) represents the coordinates of the pixels with coordinates ( ⁇ 1 , ⁇ 0 )
  • Pixel value, ( ⁇ i , ⁇ j ) represents the coordinates of the pixels in the neighborhood of the pixel with coordinates ( ⁇ 1 , ⁇ 0 )
  • weight_origini, j represents the initial value of the pixels with coordinates ( ⁇ i, ⁇ j ) Weight
  • PixelInput ( ⁇ i, ⁇ j ) represents the pixel value of the pixel with coordinates ( ⁇ i, ⁇ j ).
  • steps B3 and C3 may be performed pixel by pixel for each color channel, thereby the output result may simultaneously suppress color distortion.
  • each LUT table represent the Cartesian coordinate system x, y coordinates, and the value represents its polar diameter or polar angle.
  • mapping method from the output space to the input space may refer to FIG. 4, and Embodiment 3 of the present disclosure has been described with reference to FIG. 5.
  • the presentation method of the VR device can be described by the following steps.
  • A4 Generation of polar coordinate pixel grid.
  • step A1 The implementation of this step is the same as the implementation of step A1.
  • C4 Pixel interpolation in polar coordinate system.
  • the embodiment of the present disclosure proposes an image rendering device of the VR device, which can be applied to the VR device.
  • FIG. 6 is a schematic diagram of the composition structure of an image rendering device of a VR device according to an embodiment of the present disclosure. As shown in FIG.
  • the obtaining module 601 is set to obtain input image data
  • the processing module 602 is set to form a corresponding relationship between input image data and output image data in a polar coordinate system; obtain image data to be presented according to the output image data; the poles of the polar coordinate system are the views of the virtual reality VR device The intersection of the field center and the display plane;
  • the rendering module 603 is configured to render the image data to be rendered based on the convex lens.
  • the processing module 602 is further configured to, before forming the correspondence between the input image data and the output image data, coordinate the coordinates of multiple pixels in the input image data according to the display resolution of the VR device Convert from Cartesian coordinate system to polar coordinate system respectively.
  • the processing module 602 is configured to obtain a coordinate system conversion table corresponding to a plurality of predetermined display resolutions respectively; according to the display resolution of the VR device and the plurality of display resolutions
  • Corresponding coordinate system conversion tables respectively convert the coordinates of multiple pixel points in the input image data from Cartesian coordinate system to polar coordinate system; the coordinate system conversion table is set to indicate that multiple pixel points in the image are respectively from Cartesian Coordinate mapping relationship from coordinate system to polar coordinate system.
  • the processing module 602 is configured to determine the pixel values of multiple pixels in the output image data, and generate the coordinates according to the coordinates and pixel values of the multiple pixels in the output image data. Image data to be presented.
  • processing module 602 is set to:
  • a polar coordinate pixel grid of the input space is generated, and each grid in the polar coordinate pixel grid of the input space corresponds to a pixel point of the input image data;
  • the pixel points corresponding to the multiple grids in the polar coordinate pixel grid of the input space are respectively mapped to obtain the mapped multiple discrete pixel points;
  • any pixel in the image corresponding to the output image data does not belong to a plurality of pixels in the polar coordinate pixel grid of the output space, it is determined according to the polar coordinate pixel grid of the output space The neighborhood of the pixel point that does not belong to the polar coordinate pixel grid of the output space; performing interpolation operations according to the pixel values of a plurality of pixel points in the neighborhood of the pixel point to obtain the pixel value of the pixel point.
  • processing module 602 is set to:
  • each grid in the polar coordinate pixel grid of the input space corresponds to a pixel point of the input image data
  • mapping method of the output image data to the input image data multiple pixels in the image corresponding to the output image data are respectively mapped to polar coordinate pixel grids in the input space to obtain the mapped multiple discrete pixels ;
  • any one of the plurality of discrete pixels after the mapping does not belong to a pixel in the polar coordinate pixel grid of the input space, determine according to the polar coordinate pixel grid of the input space A neighborhood of the pixel point that does not belong to the polar coordinate pixel grid of the input space; performing interpolation operations according to pixel values of a plurality of pixel points in the neighborhood of the pixel point to obtain the pixel value of the pixel point;
  • the pixel values of the plurality of discrete pixels after the mapping are obtained.
  • the processing module 602 is configured to use a bilinear interpolation method, a trilinear interpolation method or a Lanczos interpolation method, and according to pixels of a plurality of pixels in the neighborhood of the pixels Values are interpolated.
  • the pixel value is used to represent a luminance component or a chrominance component of a pixel.
  • the processing module 602 is further configured to remove the area where multiple pixels in the image corresponding to the input image data are located in the image corresponding to the output image data, Obtain the edge pixel area; set the pixel values of multiple pixels in the edge pixel area to zero, according to the coordinates and pixel values of the multiple pixel points of the output image data, and the number of edge pixel areas The coordinates and pixel values of each pixel point generate the image data to be presented.
  • the acquisition module 601 and the processing module 602 can be composed of a central processing unit (CPU), a microprocessor (Micro Processing Unit, MPU), and a digital signal processor (Digital Signal Processor) located in a VR device. DSP), or Field Programmable Gate Array (FPGA), etc.
  • the above presentation module 603 can be implemented by a hardware device in the VR device such as CPU, MPU, DSP, or FPGA combined with the data presentation end of the VR device .
  • multiple functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of this embodiment essentially or does related technology
  • the contributed part or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes multiple instructions to make a computer device (which may be a personal computer, server , Or a network device, etc.) or a processor (processor) executes all or part of the steps of the method described in this embodiment.
  • the aforementioned storage media include: Universal Serial Bus flash disk (Universal Serial Bus flash disk, U disk), mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), Various media that can store program codes, such as magnetic disks or optical disks.
  • the computer program instructions corresponding to the method in this embodiment may be stored on a storage medium such as an optical disk, a hard disk, or a USB flash drive.
  • a storage medium such as an optical disk, a hard disk, or a USB flash drive.
  • FIG. 7 it shows an image rendering device 70 of a VR device provided by an embodiment of the present disclosure.
  • the device may include: a memory 71 and a processor 72; wherein,
  • the memory 71 is configured to store computer programs and data
  • the processor 72 is configured to execute a computer program stored in the memory to implement any method in the foregoing embodiment.
  • the memory 71 may be a volatile memory (volatile memory), such as RAM; or a non-volatile memory (non-volatile memory), such as ROM, flash memory (flash memory), and hard disk Drive (HDD) or Solid-State Drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor 72.
  • volatile memory volatile memory
  • non-volatile memory non-volatile memory
  • ROM read-only memory
  • flash memory flash memory
  • HDD hard disk Drive
  • SSD Solid-State Drive
  • the processor 72 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.
  • the electronic device used to implement the above processor function may also be other, and the embodiment of the present disclosure is not specifically limited.
  • An embodiment of the present disclosure proposes a VR device, which includes the image rendering apparatus of any VR device or any type of VR-based image rendering device in the embodiment of the present disclosure, and further includes a source generating terminal and a data processing terminal.
  • the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer usable storage media (including disk storage and optical storage, etc.) containing computer usable program code.
  • Each flow and / or block in the flowchart and / or block diagram and a combination of the flow and / or block in the flowchart and / or block diagram may be implemented by computer program instructions.
  • These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processing machine, or other programmable data processing device to produce a machine that enables the generation of instructions executed by the processor of the computer or other programmable data processing device
  • These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device
  • the instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Geometry (AREA)

Abstract

本公开实施例提供了一种虚拟现实VR设备的图像呈现方法、装置、设备和计算机存储介质，所述方法包括：获取输入图像数据；在极坐标系下，形成输入图像数据和输出图像数据的对应关系；所述极坐标系的极点为VR设备的视场中心与显示平面的交点；根据所述输出图像数据得到待呈现图像数据；将所述待呈现图像数据基于凸透镜进行呈现。

Description

虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质
本申请要求在2018年11月16日提交中国专利局、申请号为201811368846.5的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开实施例涉及虚拟现实(Virtual Reality，VR)领域，例如涉及一种VR设备的图像呈现方法、装置、设备和计算机存储介质，可以针对虚拟现实头戴显示设备(Head Mount Device，HMD)显示系统，进行几何畸变校正(Graphic Distortion Correction，GDC)。
背景技术
VR技术借助近场显示系统构建逼真的现实场景，使用户可以在小范围内体验大空间。为了达到这一目的，搭建逼真高效的近场显示系统尤为重要。为了摆脱时间空间的限制，VR技术得到了广泛应用，例如在远程医疗、教育培训和工业仿真等方面。一个VR系统，由源生成端、数据处理端、数据呈现端构成；源生成端通过运动相机或者基于图形处理器(Graphics Processing Unit，GPU)的模型构图实现，数据处理端设置为实现数据拼接传输和提取，数据呈现端设置为将提取的内容呈现给用户。
数据呈现端是连接用户与内容的最紧密的模块，直接影响用户的体验，用户体验受数据呈现端的影响非常明显。而如何在数据呈现端降低失真(如HMD引入的失真)，是亟待解决的问题。
发明内容
本公开实施例提供了一种VR设备的图像呈现方法、装置、设备和计算机存储介质，能够在VR设备的数据呈现端降低失真。
本公开实施例提供了一种VR设备的图像呈现方法，所述方法包括：
获取输入图像数据；
在极坐标系下，形成输入图像数据和输出图像数据的对应关系；所述极坐标系的极点为VR设备的视场中心与显示平面的交点；
根据所述输出图像数据得到待呈现图像数据；
将所述待呈现图像数据基于凸透镜进行呈现。
本公开实施例还提供了一种VR设备的图像呈现装置，所述装置包括：获取模块、处理模块和呈现模块，其中
获取模块，设置为获取输入图像数据；
处理模块，设置为在极坐标系下，形成输入图像数据和输出图像数据的对应关系；根据所述输出图像数据得到待呈现图像数据；所述极坐标系的极点为VR设备的视场中心与显示平面的交点；
呈现模块，设置为将所述待呈现图像数据基于凸透镜进行呈现。
本公开实施例还提供了一种VR设备的图像呈现设备，所述设备包括：处理器和用于存储能够在处理器上运行的计算机程序的存储器；其中，
所述处理器设置为运行所述计算机程序时，执行上述方法。
本公开实施例还提供了一种VR设备，所述VR设备包括上述任意一种装置或上述任意一种设备，还包括：源生成端和数据处理端。
本公开实施例还提供了一种计算机存储介质，该计算机存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述方法。
附图说明
图1为本公开实施例提供的VR设备的图像呈现方法的流程图；
图2为本公开实施例涉及的凸透镜的几何失真的模型示意图；
图3为本公开实施例涉及的极坐标系与笛卡尔坐标系的差异点的示意图；
图4为本公开实施例中输入空间到输出空间的映射方法的示意图；
图5为本公开实施例中输出空间到输入空间的映射方法的示意图；
图6为本公开实施例的VR设备的图像呈现装置的组成结构示意图；
图7为本公开实施例的VR设备的图像呈现设备的结构示意图。
具体实施方式
以下结合附图及实施例，对本公开进行说明。此处所描述的实施例仅仅用以解释本公开，并不用于限定本公开。
VR设备中的HMD可以通过凸透镜组来实现图像呈现，即，可以将待呈现图像数据基于位于数据呈现端的凸透镜的设备来呈现；HMD中的凸透镜组，设置为增加待呈现内容的景深，可以降低用户近场观看的不适，但同时导致了一 系列的几何失真(Graphic Distortion，GD)。HMD中的凸透镜具有以焦点为核心的中心对称特性。实际实施时，可以利用VR设备的数据呈现端接收输入图像数据，输入图像数据为经VR设备的源生成端和数据处理端处理后得出的数据，可选地，输入图像数据可以是矩形图像数据，例如输入图像数据是分辨率为A*B的图像数据，A表示输入图像数据的横向像素点个数，B表示输入图像数据的纵向像素点个数；实际应用中，VR设备的数据呈现端可以对输入图像数据进行处理后进行呈现。
相关技术中，VR设备的数据呈现端要解决两个方面的问题：1)如何降低通过HMD设备引入的失真，2)如何使模型可以灵活地适配多种设备；第一方面的问题体现在提升用户观看内容的真实性。第二方面的问题目的在于通过建模降低失真矫正对分辨率和设备的依赖。
基于上述记载的内容，提出以下实施例。
实施例一
本公开实施例提出了一种VR设备的图像呈现方法，可以应用于VR设备的数据呈现端。
图1为本公开实施例的VR设备的图像呈现方法的流程图，如图1所示，该流程可以包括：步骤1010至步骤1040。
步骤1010：获取输入图像数据。
在一实施例中，输入图像数据包括多个像素点的坐标和像素值；其中，像素值用于表示像素点的亮度分量或色度分量，例如，像素值可以包括红绿蓝(Red Green Blue，RGB)空间的R分量、G分量或B分量，也可以包括YCbCr空间的Y分量、Cb分量或Cr分量。
步骤1020：在极坐标系下，形成输入图像数据和输出图像数据的对应关系；所述极坐标系的极点为VR设备的视场中心与显示平面的交点。
实际应用中，可以根据输入图像数据、以及输入图像数据和输出图像数据的对应关系，得出输出图像数据。
针对VR设备的数据呈现端带来的图像失真，实现GDC的方案可以是针对输入图像数据进行预失真处理，下面对预失真处理的原理进行说明。
数据呈现端的凸透镜带来的失真可以通过式(1)进行说明。
Figure PCTCN2019118202-appb-000001
式(1)中，HMD表示上述凸透镜的映射方式，f(x,y)表示数据处理端输出结果，即，数据处理端输出的笛卡尔坐标为(x,y)的像素点的像素值；f(x,y)与实际场景中的内容在大小、位置、颜色和形状上保持一致；f'(x,y)表示经过上述凸透镜后呈现的内容。
在一实施例中，距离凸透镜中心越远的区域放大越明显，这种非均匀失真一方面改变了物体的相对大小和位置，并且由于不同色彩的失真不同，物体的边缘会出现明显的光晕，另一方面，加大了数值量化的难度。
针对上述凸透镜带来的非均匀失真，可以采用式(2)表示的失真抑制方式进行抑制。
Figure PCTCN2019118202-appb-000002
式(2)中，HMD'表示预失真处理；可以看出，数据处理端输出结果f(x,y)并不直接进行显示(即不直接通过上述凸透镜进行呈现)，而是对数据处理端输出结果f(x,y)进行预失真处理；将预失真处理的结果g(x,y)进行显示(即通过上述凸透镜进行呈现)；在一实施例中，HMD'表示HMD的逆映射，示例性地，HMD表示桶形失真对应的映射方式时，HMD'表示枕形失真对应的映射方式，HMD表示枕形失真对应的映射方式时，表示桶形失真对应的映射方式。
本公开实施例中，HMD'表示上述输入图像数据至输出图像数据的映射方式(用于表示输入图像数据多个像素点至输出图像数据的多个像素点的映射关系)。
作为本步骤的一种实现方式，可以在极坐标系下，对输入图像数据多个像素点的坐标按照特定映射方式进行映射，得到输出图像数据；所述特定映射方式用于表示所述凸透镜的图像放大映射方式的逆映射。
为了确定上述特定映射方式，需要确定上述凸透镜的图像放大映射方式，实际实施时，可以预先确定并存储上述凸透镜的参数(包括畸变参数)，可以通过读取的方式获取上述凸透镜的参数，进而根据上述凸透镜的参数确定上述凸透镜的图像放大映射方式，进而确定出上述凸透镜的图像放大映射方式的逆映射，即，确定出上述特定映射方式。
参照式(1)和式(2)，相关技术中，针对上述凸透镜带来的非均匀失真，多采用笛卡尔坐标系下的失真抑制方案，笛卡尔坐标系采用相交于原点的两条或者多条数轴，构成平面放射坐标系，包括直角坐标系和斜角坐标系。常用的笛卡尔坐标系为直角坐标系，在平面内，任何一点的位置可以采用数轴上对应的点的坐标来标定，利用数轴的方向作为单位方向，用单位长度表示该方向上距离原点的远近。通过平方和开方计算该点距离原点的距离。
本公开实施例中可以采用极坐标系下的失真抑制方案，极坐标系是通过参考点距离原点的距离(称为极径ρ)以及参考点与原点的连线与水平线的夹角(称为极角θ)来描述参考点的位置。当限制ρ大于或等于零，θ介于0到2π时，平面上除了极点之外，每个点都有唯一的极坐标描述(ρ，θ)。
下面通过式(3)和式(4)说明极坐标与笛卡尔直角坐标的转换关系。
Figure PCTCN2019118202-appb-000003
x＝ρcosθ,y＝ρsinθ    (4)
笛卡尔坐标系的优势在于直观便于理解，但是实际计算中极坐标系具有建模上的一些优势，在微积分中具有广泛的应用。针对透镜系统的GD具有径向同性的特性，即和焦点距离相同的点产生的形变相同；对于HMD设备中的凸透镜，预失真则等价于图像向焦点产生收缩，对于矩形图像而言，形成类似枕形的失真。
基于上述记载的内容，在一实施例中，在形成输入图像数据和输出图像数据的对应关系前，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系；示例性地，可以根据所述VR设备的显示分辨率，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
VR设备的显示分辨率可以是VR设备中HMD的显示分辨率，在一实施例中，VR设备中HMD的显示分辨率表示待显示的数据分辨率，可以是视频源数 据或者图形处理器(Graphics Processing Unit，GPU)绘制的数据的分辨率。在一个示例中，上述HMD的显示分辨率可以是HMD对应到单眼的显示分辨率，例如，HMD的输出分辨率为2560*1440，则HMD对应到单眼的显示分辨率为1280*1440；在另一个示例中，可以根据输入单眼分辨率的纵横比确定HMD的显示分辨率，例如，输入单眼分辨率为1200*1200，则HMD的显示分辨率为1280*1208。
VR设备的显示分辨率可以预先确定并存储在VR设备中，如此，可以通过读取的方式获取VR设备的显示分辨率。
本公开实施例中，可以根据VR设备的显示分辨率，分别对输入图像数据中多个像素点在笛卡尔坐标系的坐标(x,y) ，按照笛卡尔坐标系到极坐标系的坐标转换公式，计算得出相应的极坐标数据(ρ,θ)，进而得出极坐标系下的图像网格。
在一实施例中，计算极坐标数据(ρ,θ)，需要获取笛卡尔坐标系下VR设备的的视场中心与显示平面的交点的坐标(x 0,y 0)；在实际应用中，VR设备的的视场中心与显示平面的交点的坐标可以预先确定并进行存储，如此，可以通过读取的方式获取VR设备的视场中心与显示平面的交点的坐标。
在一实施例中，VR设备的显示分辨率相同时，上述笛卡尔坐标系到极坐标系的坐标转换公式是相同的，即，针对相同的VR设备的显示分辨率，可以采用统一的笛卡尔坐标系到极坐标系的坐标转换公式。
实际应用中，为了保证坐标转换前后的数据内容的一致，不丢失输入数据，可以将坐标转换后的极坐标数据的极径缩放到(0,1)范围内。
VR设备的显示分辨率可以表示为m*n，这里，m和n均大于1；下面通过式(5)说明上述笛卡尔坐标系到极坐标系的坐标转换公式。
Figure PCTCN2019118202-appb-000004
在实际应用中，对于VR设备的任意一种显示分辨率，可以利用表格来表示 笛卡尔坐标系到极坐标系的坐标转换公式；可以预先确定VR设备的多种显示分辨率对应的坐标系转换表，上述坐标系转换表用于表示图像中多个像素点从笛卡尔坐标系到极坐标系的坐标映射关系。
在实际实施时，可以获取预先确定的多种显示分辨率分别对应的坐标系转换表(查找表(Look-Up-Table，LUT))，根据所述VR设备的显示分辨率、以及所述多种显示分辨率分别对应的坐标系转换表，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
在一实施例中，可以根据VR设备的显示分辨率，在多种显示分辨率分别对应的坐标系转换表中，选取出VR设备的显示分辨率对应的坐标系转换表，根据选取出的坐标系转换表，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
在一实施例中，由于针对相同的VR设备的显示分辨率，可以采用统一的坐标系转换表，因而，在不同的VR设备(例如不同的HMD设备)具有相同的显示分辨率时，通过对统一的坐标系转换表的复用，便于在硬件设计中实现，可以在进行坐标转换时减少对VR设备型号或HMD设备型号的依赖，克服了相关技术中对VR设备或HMD设备的参数设定敏感的缺陷，便于实现对不同的VR设备或不同的HMD设备的适配，提升了本公开实施例的易拓展性。
步骤1030：根据所述输出图像数据得到待呈现图像数据。
在一实施例中，待呈现图像数据通过VR设备的凸透镜组来实现图像呈现的数据，待呈现图像数据包括多个像素点的坐标和像素值。
本公开实施例中，待呈现图像数据包括输出图像数据在确定多个像素点的像素值后得到的图像数据；可以确定所述输出图像数据中多个像素点的像素值，根据所述输出图像数据中多个像素点的坐标和像素值，得到所述待呈现图像数据。
对于确定输出图像数据中多个像素点的像素值的实现方式，示例性地，可以基于两种映射方法实现，其中，第一种映射方法记为输入空间到输出空间的映射方法，第二种映射方法记为输出空间到输入空间的映射方法；下面分别对这两种映射方法进行说明。
第一种映射方法：输入空间到输出空间的映射方法。
针对输入图像数据，生成输入空间的极坐标像素网格；输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点，在一实施例中，输入空间的极坐标像素网格对应的像素点可以不包括输入图像数据对应的图像中所有像素点。
按照输入图像数据至输出图像数据的映射方式，将所述输入空间的极坐标像素网格中的多个网格分别对应的像素点进行映射，得到映射后的多个离散像素点；根据所述映射后的多个离散像素点，生成输出空间的极坐标像素网格，所述输出空间的极坐标像素网格中的每个网格对应所述映射后的多个离散像素点中的一个像素点；在一实施例中，映射后的多个离散像素点均处于输出空间的极坐标像素网格中，在输入图像数据至输出图像数据的映射方式为上述特定映射方式时，由于凸透镜的图像放大映射方式属于非均匀放大的映射方式，则特定映射方式为非均匀缩小的映射方式，因而，在输入空间的极坐标像素网格为均匀分布的网格时，输出空间的极坐标像素网格是非均匀的。
实际应用中，可以确定映射后的多个离散像素点的像素值，在一实施例中，映射后的每个离散像素点的像素值为映射前输入空间的极坐标像素网格中输入图像数据中对应像素点的像素值，即，输入空间的极坐标像素网格的像素点与映射后的离散像素点形成一一对应关系，映射后的每个离散像素点的像素值为对应的输入空间的极坐标像素网格中的像素点的像素值。
在确定出映射后的多个离散像素点的像素值后，便确定了输出空间的极坐标像素网格的多个像素点的像素值。
当输出图像数据对应的图像中任一个像素点不属于所述输出空间的极坐标像素网格中像素点时，可以通过插值运算来确定对应像素点的像素值；在一实施例中，可以根据输出空间的极坐标像素网格，确定不属于所述输出空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值；例如，当i取正整数时，如果输出图像数据的第i个像素点不属于所述输出空间的极坐标像素网格中像素点时，则可以在输出空间的极坐标像素网格中，确定出输出图像数据的第i个像素点的邻域，根据输出图像数据的第i个像素点的邻域中多个像素点的像素值进行插值运算，得到输出图像数据的第i个像素点的像素值。
本公开实施例中，在根据对应像素点的邻域中多个像素点的像素值进行插值运算时，可以采用基于极坐标系下的像素插值，基于多像素的插值受到两个因素的影响，一个是队列集合类库Tape数，即参与插值的像素点数，二是权重，确定参与插值的像素点对插值运算结果的影响。在一实施例中，会根据Tape数调整每个像素点的权重。此处Tape数的选择方案和笛卡尔坐标系下的方案一致，即选择映射后像素位置的p×q邻域，p和q均为大于1的整数；在选择像素权重上，极坐标系下的插值方案需要综合考虑极径与极角的差异，下面通过式(6)示例性地说明极坐标系下的像素插值方法。
Figure PCTCN2019118202-appb-000005
式(6)中，(ρ 1,θ 0)表示需要通过插值运算得出像素值的像素点的坐标，(ρ i,θ j)表示需要通过插值运算得出像素值的像素点的邻域中像素点的坐标，PixelOutput(ρ 1,θ 0)表示通过插值运算得出的像素点的像素值，PixelInput(ρ i,θ j)表示坐标为(ρ i,θ j)的像素点的像素值，weight i,j表示坐标为(ρ i,θ j)的像素点的归一化权重。
在利用硬件实现式(6)时，可以将权重和量化到2的幂次，通过乘加与移位操作实现。该过程中根据极径和极角的距离，映射权重的方案可以不同，由此，可以根据极径和极角的差异确定权重，借鉴笛卡尔坐标系下的插值方法，可以引入不同的基于极坐标系下的插值方法，类似于双线性插值和三线性插值方法；实际实施时，可以采用类似于Lanczos的插值方案，也可以采用基于极径和极角的双线性插值方法，
可选地，可以采用双线性插值方法、三线性插值方法或兰索斯(Lanczos)插值方法，并根据所述像素点的邻域中多个像素点的像素值进行插值运算。
当输出图像数据的任一像素点属于所述输出空间的极坐标像素网格中像素点时，可以直接将输出图像数据的任一像素点的像素值确定为输出空间的极坐标像素网格中对应像素点的像素值；例如，当i取正整数时，如果输出图像数据的第i个像素点属于所述输出空间的极坐标像素网格中像素点时，则输出图像数据的第i个像素点的像素值为：输出空间的极坐标像素网格中对应像素点的像素值。
采用上述第一种映射方法的优点在于：可以更高效地输出像素信息，缺点 在于：在一实施例中，需要缓存大量的数据才可以确保插值需要的像素点已经全部映射完成，这增加了硬件设计的存储资源消耗(例如静态随机存取存储器(Static Random-Access Memory，SRAM)消耗)；在一实施例中，，由于映射之后的极坐标像素网格不均匀，所以存在相同邻域内映射的点数并不完全相同，插值权重的设计更加复杂的问题。
第二种映射方法：输出空间到输入空间的映射方法。
针对输入图像数据，生成输入空间的极坐标像素网格；在一实施例中，输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点，在一实施例中，输入空间的极坐标像素网格对应的像素点可以不包括输入图像数据对应的图像中所有像素点。
按照所述输出图像数据至输入图像数据的映射方式(例如为上述特定映射方式的逆映射)，将所述输出图像数据对应的图像中的多个像素点分别映射至输入空间的极坐标像素网格，得到映射后的多个离散像素点。
示例性地，可以根据HMD的参数对极径进行重映射，即，按照输出图像数据至输入图像数据的映射方式，对所述输出图像数据的每个像素点进行映射，得出输出图像数据的像素点(ρ 0,θ 0)在输入空间的极坐标像素网格的映射后位置(ρ 1,θ 0)；根据凸透镜映射方式的特点，凸透镜映射方式带来的失真关于凸透镜的焦点对称，因此，在极坐标系的极点为VR设备的视场中心与显示平面的交点时，凸透镜映射方式带来的失真只是极坐标系中径向的失真，因而，在重映射(所述特定映射方式的逆映射)前后，像素点的坐标只会发生极径的变化，极角不发生变化，重映射可以通过式(7)进行说明。
ρ 1＝h(ρ 0)      (7)
式(7)中，h()表示输出图像数据至输入图像数据的映射方式对应的函数。
当映射后的多个离散像素点中的任一个像素点不属于所述输入空间的极坐标像素网格中的像素点时，可以通过插值运算来确定不属于所述输入空间的极坐标像素网格的所述像素点的像素值；在一实施例中，可以根据输入空间的极坐标像素网格，确定不属于所述输入空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值；例如，当i取正整数时，如果映射后的多个离散像素点的第i个 像素点不属于所述输入空间的极坐标像素网格中像素点时，则可以在输入空间的极坐标像素网格中，确定出映射后的多个离散像素点的第i个像素点的邻域，根据映射后的多个离散像素点的第i个像素点的邻域中多个像素点的像素值进行插值运算，得到映射后的多个离散像素点的第i个像素点的像素值。
在一实施例中，插值运算的方法与第一种映射方法中插值运算的方法相同。
当映射后的多个离散像素点的任一个像素点属于所述输入空间的极坐标像素网格中像素点时，可以直接将映射后的多个离散像素点的任一个像素点的像素值确定为输入空间的极坐标像素网格中对应像素点的像素值；例如，当i取正整数时，如果映射后的多个离散像素点的第i个像素点属于所述输入空间的极坐标像素网格中像素点时，则映射后的多个离散像素点的第i个像素点的像素值为：输入空间的极坐标像素网格中对应像素点的像素值。
在确定出映射后的多个离散像素点的像素值后，可以根据所述映射后的多个离散像素点的像素值，得出所述输出图像数据的多个像素点的像素值；即，通过输出图像数据至输入图像数据的映射方式，映射前和映射后的每个像素点形成一一对应关系，此时，可以将输出图像数据的每个像素点的像素值确定为：映射后的多个离散像素点中对应像素点的像素值。
采用上述第二种映射方法的优点在于：可以有效地降低输入行缓存，并充分利用输入像素的全部信息；在一实施例中，根据上述第二种映射方法，将输出图像数据的多个像素点的极坐标分别逐一映射至输入空间，由于输入空间的极坐标像素网格是均匀的，所以，每实现一个像素点的映射，可以根据映射结果和邻域参数唯一确定参与插值的像素点；在一实施例中，与第一种映射方法相比，不需要缓存多个像素点的映射结果，进而可以有效地降低输入行缓存。
上述第二种映射方法的一个潜在缺陷在于，如果像素点映射至输入空间的极坐标像素网格时，映射后的多个离散像素点的间隔是不均匀的，而针对映射后的多个离散像素点同时选择的邻域范围不能包含输入空间的极坐标像素网格的所有的像素点，可能输入空间的极坐标像素网格中有些像素点并不参与插值过程，从而导致输入图像数据信息的遗漏；该缺陷可以通过增加插值的Tape数进行削弱，但是Tape数的增加也会导致插值复杂度变大；这种缺陷在相邻输出像素间畸变差异较大的情况下会明显(比如输出分辨率较低的有效边缘区域)， 但是随着显示分辨率的不断提升，该缺陷已经不再明显(输入输出的显示分辨率均保证在2K以上时，该缺陷已经肉眼难以分辨)。因此，本公开实施例通常采用第二种映射方法进行映射。
无论采用上述哪种映射方法，在输入图像数据至输出图像数据的映射方式为上述特定映射方式时，由于凸透镜会放大图像的内容，凸透镜映射方式的逆映射会缩小图像的内容，因此，与按照上述特定映射方式映射前的图像区域(例如为矩形区域)相比，按照上述特定映射方式映射后的图像区域会存在一部分黑边(例如枕形失真导致的黑边)，可以判断，按照上述特定映射方式映射后的图像区域的像素点是否涉及输入信号(即是否由输入图像数据的像素点映射得出的)，根据判断结果进行处理，在一实施例中，如果判断结果为是，则可以将像素点的像素值置零。
在一实施例中，可以将输入图像数据对应的图像中的多个像素点所在区域除去输出图像数据对应的图像中的多个像素点所在区域，得到边缘像素点区域；将边缘像素点区域的多个像素点的像素值置零，根据所述输出图像数据的多个像素点的坐标和像素值、以及所述边缘像素点区域的多个像素点的坐标和像素值，生成所述待呈现图像数据。
步骤1040：将所述待呈现图像数据基于凸透镜进行呈现。
本公开实施例提供一种VR设备的图像呈现方法、装置、设备和计算机存储介质，获取输入图像数据；在极坐标系下，形成输入图像数据和输出图像数据的对应关系；所述极坐标系的极点为虚拟现实VR设备的视场中心与显示平面的交点；根据所述输出图像数据得到待呈现图像数据；将所述待呈现图像数据基于凸透镜进行呈现。如此，由于视场中心与凸透镜的焦点均处于凸透镜的主光轴上，因而，凸透镜映射方式关于焦点具有中心对称特性时，凸透镜映射方式关于极点具有中心对称特性，进而，本公开实施例只需要在极坐标系下通过极径映射便可得出映射结果，无需进行极角映射，在计算复杂度上低于笛卡尔坐标系下的映射方式，便于硬件实现。
实际应用中，步骤1010至步骤1030可以由VR设备的处理器实现，上述处理器可以位于VR设备的数据呈现端，步骤1040可以利用VR设备的处理器结合数据呈现端的硬件设备实现，上述处理器可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、 可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)、中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。
本公开实施例针对VR设备的数据呈现端引入的GD，将坐标映射方案从笛卡尔坐标系迁移至极坐标系，可以提升GDC的效果；本公开实施例中，获取输入图像数据中每个像素点在极坐标系下的坐标数据(ρ,θ)，针对输入图像数据中多个像素点按照特定映射方式进行映射(极径映射)，得到输出图像数据，输出图像数据像素点的坐标数据记为(ρ',θ)，根据HMD中凸透镜映射方式的特性，上述特定映射方式的映射过程并不进行极角的修正。如果上述特定映射方式映射后的图像区域的像素点不涉及输入信号时，则将对应像素点的像素值置零，即，进行零填充；如果特定映射方式映射后的图像区域的像素点是按照上述特定映射方式映射得出的，则可以通过插值运算得出对应像素点的像素值，可以得出待呈现的像素点的亮度或色度结果f(ρ',θ)。
与相关技术中采用笛卡尔坐标系下的GDC方案相比，本公开实施例的方法并不求解像素点到焦点的距离
Figure PCTCN2019118202-appb-000006
也不按照以下公式进行像素点在坐标系下的距离重新分配：
Figure PCTCN2019118202-appb-000007
其中，r'表示按照凸透镜的映射方式映射后的极径。上述这两组运算也不适于硬件实现，相对地，本公开实施例不进行两组运算，可以降低VR设备的图像呈现时的计算复杂度。
另外，由于这种凸透镜带来的径向失真以焦点为中心，笛卡尔坐标系下的插值利用参考点与中心的相对位置，极坐标系下的插值采用参考点与待插值点的极径与极角信息，与失真模型的拟合度更高，进而，可以提升GDC的效果。
本公开实施例中，克服相关技术中GDC方法中计算复杂度较高的问题和对HMD设备参数设定敏感的缺陷，提供了基于极坐标系的GDC方案，通过在极坐标系下逐像素完成距离映射，与相关技术中基于块映射的方案不同，本公开中输入图像数据的所有像素点均通过输入空间与输出空间的位置坐标映射，即每个像素点的精度均可以达到很高，而相关技术中基于块的映射方案，除了边 界点的精度很高之外，其他点的精度随着与边界的距离增加而不断降低。而且由于本公开实施涉及的凸透镜的失真为非均匀性的，所以基于块的处理在一些直线特征明显的场景中，失真明显。同时，本公开实施例只涉及极径映射，计算复杂度上低于笛卡尔坐标系下的方案。另外，通过建立极坐标系与笛卡尔坐标系之间的LUT表变换关系，在硬件设计中非常便于实现。借助极径与极角参数实现极坐标系下的插值运算获取输出结果，降低距离映射对参数的敏感性，可以提升插值的准确性。
下面通过附图对本公开实施例带来的效果进行进行说明。
图2为本公开实施例涉及的凸透镜的几何失真的模型示意图，图2中，小圆圈表示输入图像数据的像素点，大圆圈代表每个像素点对应的凸透镜映射方式带来的失真，参照图2，中心位置表示凸透镜的焦点位置，可以看出，失真的程度取决于像素点距离焦点的距离，这一距离正好对应了极坐标系下的极径。
图3为本公开实施例涉及的极坐标系与笛卡尔坐标系的差异点的示意图，如图3所示，小圆圈表示像素点，在极坐标系下，A和B两个像素点极径相同，但是极角不同；而在笛卡尔坐标系下，进行插值运算时的权重取决于两个像素点在x方向和y方向的距离，图中线段AB所示；在极坐标系下，进行插值运算时的权重取决于像素点在极径和极角的差值。当极径相同时，像素点在极角上的距离并不直接等于其欧式距离；同样地，在极坐标系与笛卡尔坐标系下，距离参考点相同的欧氏距离失真程度也不一致。所以极坐标系下的GDC方案更符合HMD设备的失真原理，同时利用只计算一次的极坐标网格可以降低对HMD设备参数的敏感性。
实施例二
在本公开实施例的基础上，进行举例说明。
建立针对VR设备的显示分配率的输入空间的坐标网格和输出空间的坐标网格，对于输入空间的坐标网格和输出空间的坐标网格，用极径在[0，1]、极角在[0，2π]范围内的极坐标表示；对于HMD，其显示分辨率(即输出分辨率)是固定的，因此，可以设定两个LUT表，分别用于建立输入空间的坐标网格和输出空间的坐标网格。通过在极坐标系下进行坐标映射，可以将每帧图像的坐 标计算复杂度降低为O(1)。
可以按照上述记载的第一种映射方法，将所述输入空间的极坐标像素网格中的多个网格分别对应的像素点进行映射，得到映射后的多个离散像素点；根据映射后的多个离散像素点在输出空间的极坐标像素网格中的位置，按照基于极坐标系下的插值方法进行权重分配(采用基于双线性的极坐标插值权重分配方法)，在针对映射后的多个离散像素点的权重分配完毕后，通过加权和的方式输出映射后的多个离散像素点的插值运算结果(即像素值)。
图4为本公开实施例中输入空间到输出空间的映射方法的示意图，如图4所示，圆圈表示输入空间的极坐标像素网格，五角星表示映射后的多个离散像素点，图4中数字1至5分别表示映射后的5个不同的离散像素点；在一实施例中，映射后的多个离散像素点相对于输入空间的极坐标像素网格是不均匀的，通过极径与极角的相对关系确定输入空间的一个像素点(用实心圆圈表示)可以对周围5个不同的离散像素点有权重影响(图4中用带箭头的曲线表示)；图4中，带箭头的曲线表示输入空间的像素点向映射后的多个离散像素点分配权重。在采用输入空间到输出空间的映射方法时，亮度或色度分量进行对应的极坐标位置偏移，根据偏移之后的极径和极角差异进行插值即可，可以通过以下步骤进行说明。
A1：极坐标像素网格的生成。
读取HMD设备的显示分辨率、凸透镜畸变参数、焦点信息以及待显示源的分辨率。
在一实施例中，焦点信息用于表征视场中心点，通常对应到显示分辨率的一个像素点，缺省设置为单眼区域的中心；此处可以通过调节焦点修正3D视频的景深，适配用户的瞳距。
将输入图像数据的坐标按照式(5)进行坐标转换，得到极坐标系下输入图像数据中多个像素点的坐标；此处，极坐标系下最大极径为1，保证输入空间的信息可以全部投影到输出空间。
B1：逐像素点进行GDC映射。
在一实施例中，可以采用上述记载的第一种映射方法进行GDC映射；利用GDC参数进行输入空间到输出空间的极坐标映射，在进行输入空间到输出空间 的极坐标映射时，保留每个像素点的极角信息，修正极径信息。
当所有参与分配插值运算的权重的像素点的像素值均统计完成后，根据式(8)输出运算结果。
Figure PCTCN2019118202-appb-000008
式(8)中，x N表示第N个参与插值的像素点，x j表示第j个参与插值的像素点，j取1至N，N为大于1的整数；weight N表示第N个参与插值的像素点的权重，weight j表示第j个参与插值的像素点的权重，∑weight q表示所有参与插值的像素点的权重和，x output表示插值运算结果。
本公开实施例中，满足N个像素点参与插值之后即停止插值并输出运算结果。
C1：极坐标系下的像素插值。
根据本公开实施例记载的内容，无论采用哪种映射方法，均针对像素点进行插值运算，基于极坐标系的插值方法特点在于其权重设计依赖于极坐标空间的相对关系，需要满足距离近的点权重不低于距离远的点。参考笛卡尔坐标系下的权重获取方式，此处给出如下实现方案：
基于水平垂直均为2Tape的线性关系的权重设计，对于极坐标系下的两个像素点(ρ 0,θ 0)和(ρ 1,θ 1)，如果其极径相同(ρ 0＝ρ 1)，那么这两个像素点的距离为dis＝abs(θ 0-θ 1)×ρ 0。如果极角相同(θ 0＝θ 1)，那么这两个像素点的距离为dis＝abs(ρ 0-ρ 1)。如果极径与极角均不相等，那么这两个像素点的距离为dis＝abs(θ 0-θ 1)×max(ρ 0,ρ 1)+abs(ρ 0-ρ 1)，计算参与插值的多个像素点与需要通过插值运算得出像素值的像素点的距离和
Figure PCTCN2019118202-appb-000009
其中，dis i表示第i个参与插值的多个像素点与需要通过插值运算得出像素值的像素点的距离，参与插值的像素点的权重可以用式(9)表示。
Figure PCTCN2019118202-appb-000010
在一实施例中，根据不同颜色通道的GDC参数差异，可以针对每个颜色通道分别逐像素进行步骤B1和C1，由此输出的结果可以同时抑制色彩失真。
实施例三
在本公开实施例的基础上，进行举例说明。
建立针对VR设备的显示分配率的输入空间的坐标网格和输出空间的坐标网格，对于输入空间的坐标网格和输出空间的坐标网格，用极径在[0，1]、极角在[0，2π]范围内的极坐标表示；对于HMD，其显示分辨率(即输出分辨率)通常是固定的，因此，可以设定两个LUT表，分别用于建立输入空间的坐标网格和输出空间的坐标网格，示例性地，每个LUT表的坐标表示笛卡尔坐标系x，y坐标，值代表其极径或者极角。
可以按照上述记载的第二种映射方法，按照所述特定映射方式的逆映射(即上述凸透镜的图像放大映射方式)，将所述输出图像数据的每个像素点映射至输入空间的极坐标像素网格，得到映射后的多个离散像素点；根据映射后的多个离散像素点在输入空间的极坐标像素网格的位置，在输入空间的极坐标像素网格中，选择临近的像素点进行插值运算，插值权重按照双线性插值的权重分配方案，可以参考式(8)。
图5为本公开实施例中输出空间到输入空间的映射方法的示意图，如图5所示，圆圈表示输入空间的极坐标像素网格，五角星表示输出图像数据的像素点映射后的多个离散像素点，图5中用数字1至3分别表示输出图像数据中3个不同的像素点；在一实施例中，输入空间的极坐标像素网格相对于输出图像数据的像素点是不均匀的，通过极径与极角的相对关系确定周围5个输入空间的极坐标像素网格的像素点对输出图像数据的一个像素点(用实心五角星表示)有权重影响；图5中，带箭头的曲线表示输入空间的像素点向输出图像数据的像素点分配权重。在采用输出空间到输入空间的映射方法时，亮度或色度分量 进行对应的极坐标位置偏移，根据偏移之后的极径和极角差异进行插值即可，可以通过以下三个步骤进行说明。
A2：极坐标像素网格的生成。
本步骤的实现方式与步骤A1的实现方式相同。
B2：逐像素点进行GDC映射。
这里可以采用上述记载的第二种映射方法进行GDC映射；在一实施例中，利用GDC参数进行输出空间到输入空间的极坐标映射，在进行输出空间到输入空间的极坐标映射时，保留每个像素点的极角信息，修正极径信息。
当所有参与分配插值运算的权重的像素点的像素值均统计完成后，根据式(10)输出运算结果。
Figure PCTCN2019118202-appb-000011
式(10)中，x j表示第j个参与插值的像素点，j取1至N，N为大于1的整数，weight j表示第j个参与插值的像素点的权重，
Figure PCTCN2019118202-appb-000012
表示所有参与插值的像素点的权重和，x output表示插值运算结果。
根据式(10)得出的运算结果与通过式(8)得出的运算结果虽然相同，但是式(10)体现了从输出空间到输入空间的极坐标映射，在硬件设计中便于流水线设计并降低缓存需求。
C2：极坐标系下的像素插值。
根据本公开实施例记载的内容，无论采用哪种映射方法，均针对像素点进行插值运算，基于极坐标系的插值方法特点在于其权重设计依赖于极坐标空间的相对关系，需要满足距离近的点权重不低于距离远的点。参考笛卡尔坐标系下的权重获取方式，此处给出如下实现方案：
基于水平垂直均为2Tape的线性关系的权重设计，对于极坐标系下的两个像素点(ρ 0,θ 0)和(ρ 1,θ 1)，如果其极径相同(ρ 0＝ρ 1)，那么这两个像素点的距离 为dis＝abs(θ 0-θ 1)×ρ 0。如果极角相同(θ 0＝θ 1)，那么这两个像素点的距离为dis＝abs(ρ 0-ρ 1)。如果极径与极角均不相等，那么这两个像素点的距离为dis＝abs(θ 0-θ 1)×max(ρ 0,ρ 1)+abs(ρ 0-ρ 1)，计算参与插值的多个像素点与需要通过插值运算得出像素值的像素点的距离和DIS＝dis i，其中，dis i表示第i个参与插值的多个像素点与需要通过插值运算得出像素值的像素点的距离，参与插值的像素点的权重可以用式(9)表示。
在一实施例中，根据不同颜色通道的GDC参数差异，可以针对每个颜色通道分别逐像素进行步骤B2和C2，由此输出的结果可以同时抑制色彩失真。
实施例四
在本公开实施例的基础上，进行举例说明。
建立针对VR设备的显示分配率的输入空间的坐标网格和输出空间的坐标网格；这里，建立输入空间的坐标网格和输出空间的坐标网格与本公开实施例二相同。
可以按照上述记载的第一种映射方法，将所述输入空间的极坐标像素网格中的多个网格分别对应的像素点进行映射，得到映射后的多个离散像素点；根据映射后的多个离散像素点在输出空间的极坐标像素网格中的位置，按照基于极坐标系下的插值方法进行权重分配(基于Lanczos插值的极坐标插值权重分配方法)，在针对映射后的多个离散像素点的权重分配完毕后，通过加权和的方式输出映射后的多个离散像素点的插值运算结果(即像素值)。
本公开实施例中，输入空间到输出空间的映射方法可以参照图4所示，本公开实施例二已经针对图4作出说明。
本公开实施例中，VR设备的呈现方法可以通过以下三个步骤进行说明。
A3：极坐标像素网格的生成。
本步骤的实现方式与步骤A1的实现方式相同。
B3：逐像素点进行GDC映射。
本步骤的实现方式与步骤B1的实现方式相同。
C3：极坐标系下的像素插值。
根据本公开实施例记载的内容，无论采用哪种映射方法，均针对像素点进行插值运算，基于极坐标系的插值方法特点在于其权重设计依赖于极坐标空间的相对关系，需要满足距离近的点权重不低于距离远的点。参考笛卡尔坐标系下的权重获取方式，此处给出如下实现方案：
基于多Tape的非线性插值权重设计，对于多Tape的情况，可以参考Lanczos插值，利用步骤C1或C2中的像素点距离计算方法，但是采用Lanczos插值的权重算法，即将笛卡尔坐标系下相邻像素间距设置为1，按照以下公式对像素点间的距离dis进行等比例放大：dis2＝dis×Q，其中Q为放大因子，dis2表示放大结果。按照式(11)计算每个像素点的初始权重。
weight_origin＝sinc(dis2)/sinc(dis2×N)     (11)
式(11)中，weight_origin表示像素点的初始权重，N表示参与插值的像素点的个数，即Tape数。
在计算出每个像素点的初始权重后，按照式(12)计算每个参与插值的像素点后重新进行归一化。
Figure PCTCN2019118202-appb-000013
式(12)中，(ρ 1,θ 0)表示需要通过插值运算得出像素值的像素点的坐标，PixelOutput(ρ 1,θ 0)表示坐标为(ρ 1,θ 0)的像素点的像素值，(ρ i,θ j)表示坐标为(ρ 1,θ 0)的像素点的邻域中的像素点的坐标，weight_origini, j表示坐标为(ρi,θ j)的像素点的初始权重，PixelInput(ρi,θ j)表示坐标为(ρi,θ j)的像素点的像素值。
在一实施例中，根据不同颜色通道的GDC参数差异，可以针对每个颜色通道分别逐像素进行步骤B3和C3，由此输出的结果可以同时抑制色彩失真。
实施例五
在本公开实施例的基础上，进行举例说明。
建立针对VR设备的显示分配率的输入空间的坐标网格和输出空间的坐标 网格，对于输入空间的坐标网格和输出空间的坐标网格，用极径在[0，1]、极角在[0，2π]范围内的极坐标表示；对于HMD，其显示分辨率(即输出分辨率)通常是固定的，因此，可以设定两个LUT表，分别用于建立输入空间的坐标网格和输出空间的坐标网格，示例性地，每个LUT表的坐标表示笛卡尔坐标系x，y坐标，值代表其极径或者极角。
可以按照上述记载的第二种映射方法，按照所述特定映射方式的逆映射(即上述凸透镜的图像放大映射方式)，将所述输出图像数据对应的图像中的多个像素点分别映射至输入空间的极坐标像素网格，得到映射后的多个离散像素点；根据映射后的多个离散像素点在输入空间的极坐标像素网格的位置，在输入空间的极坐标像素网格中，选择临近的像素点进行插值运算，插值权重按照Lanczos插值的权重分配方案，可以参考式(11)和式(12)。
本公开实施例中，输出空间到输入空间的映射方法可以参照图4所示，本公开实施例三已经针对图5作出说明。
本公开实施例中，VR设备的呈现方法可以通过以下步骤进行说明。
A4：极坐标像素网格的生成。
本步骤的实现方式与步骤A1的实现方式相同。
B4：逐像素点进行GDC映射。
本步骤的实现方式与步骤B2的实现方式相同。
C4：极坐标系下的像素插值。
本步骤的实现方式与步骤C3的实现方式相同。
实施例六
在本公开前述实施例提出的VR设备的图像呈现方法的基础上，本公开实施例提出了一种VR设备的图像呈现装置，可以应用于VR设备中。
图6为本公开实施例的VR设备的图像呈现装置的组成结构示意图，如图6所示，所述装置包括获取模块601、处理模块602和呈现模块603，其中
获取模块601，设置为获取输入图像数据；
处理模块602，设置为在极坐标系下，形成输入图像数据和输出图像数据的 对应关系；根据所述输出图像数据得到待呈现图像数据；所述极坐标系的极点为虚拟现实VR设备的视场中心与显示平面的交点；
呈现模块603，设置为将所述待呈现图像数据基于凸透镜进行呈现。
在一实施方式中，所述处理模块602，还设置为在形成输入图像数据和输出图像数据的对应关系之前，根据所述VR设备的显示分辨率，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
在一实施方式中，所述处理模块602，是设置为获取预先确定的多种显示分辨率分别对应的坐标系转换表；根据所述VR设备的显示分辨率、以及所述多种显示分辨率分别对应的坐标系转换表，将输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系；所述坐标系转换表设置为表示图像中多个像素点分别从笛卡尔坐标系到极坐标系的坐标映射关系。
在一实施方式中，所述处理模块602，是设置为确定所述输出图像数据中多个像素点的像素值，根据所述输出图像数据中多个像素点的坐标和像素值，生成所述待呈现图像数据。
在一实施方式中，所述处理模块602，是设置为：
针对输入图像数据，生成输入空间的极坐标像素网格，所述输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点；
按照输入图像数据至输出图像数据的映射方式，将所述输入空间的极坐标像素网格中的多个网格分别对应的像素点进行映射，得到映射后的多个离散像素点；根据所述映射后的多个离散像素点，生成输出空间的极坐标像素网格，所述输出空间的极坐标像素网格中的每个网格对应所述映射后的多个离散像素点中的一个像素点；
在所述输出图像数据对应的图像中的任一个像素点不属于所述输出空间的极坐标像素网格中的多个像素点的情况下，根据所述输出空间的极坐标像素网格，确定不属于所述输出空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值。
在一实施方式中，所述处理模块602，是设置为：
针对输入图像数据，生成输入空间的极坐标像素网格；所述输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点；
按照所述输出图像数据至输入图像数据的映射方式，将所述输出图像数据对应的图像中的多个像素点分别映射至输入空间的极坐标像素网格，得到映射后的多个离散像素点；
在所述映射后的多个离散像素点中的任一个像素点不属于所述输入空间的极坐标像素网格中的像素点的情况下，根据所述输入空间的极坐标像素网格，确定不属于所述输入空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值；
根据所述映射后的多个离散像素点的像素值，得出所述输出图像数据的多个像素点的像素值。
在一实施方式中，所述处理模块602，是设置为采用双线性插值方法、三线性插值方法或兰索斯Lanczos插值方法，并根据所述像素点的邻域中多个像素点的像素值进行插值运算。
在一实施方式中，所述像素值用于表示像素点的亮度分量或色度分量。
在一实施方式中，所述处理模块602，还设置为将所述输入图像数据对应的图像中的多个像素点所在区域除去所述输出图像数据对应的图像中的多个像素点所在区域，得到边缘像素点区域；将所述边缘像素点区域的多个像素点的像素值置零，根据所述输出图像数据的多个像素点的坐标和像素值、以及所述边缘像素点区域的多个像素点的坐标和像素值，生成所述待呈现图像数据。
实际应用中，上述获取模块601和处理模块602均可由位于VR设备中的中央处理器(Central Processing Unit，CPU)、微处理器(Micro Processor Unit，MPU)、数字信号处理器(Digital Signal Processor，DSP)、或现场可编程门阵列(Field Programmable Gate Array，FPGA)等实现，上述呈现模块603可以由位于VR设备中的CPU、MPU、DSP、或FPGA等结合VR设备的数据呈现端的硬件设备实现。
另外，在本实施例中的多个功能模块可以集成在一个处理单元中，也可以是每个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实 现。
所述集成的单元如果以软件功能模块的形式实现并非作为独立的产品进行销售或使用时，可以存储在一个计算机可读取存储介质中，本实施例的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括多个指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或processor(处理器)执行本实施例所述方法的全部或部分步骤。而前述的存储介质包括：通用串行总线闪存盘(Universal Serial Bus flash disk，U盘)、移动硬盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等多种可以存储程序代码的介质。
本实施例中的方法对应的计算机程序指令可以被存储在光盘，硬盘，U盘等存储介质上，当存储介质中的方法对应的计算机程序指令被一电子设备读取或被执行时，实现前述实施例的任意一种方法。
参见图7，其示出了本公开实施例提供的一种VR设备的图像呈现设备70，该设备可以包括：存储器71和处理器72；其中，
所述存储器71，设置为存储计算机程序和数据；
所述处理器72，设置为执行所述存储器中存储的计算机程序，以实现前述实施例的任意一种方法。
在实际应用中，上述存储器71可以是易失性存储器(volatile memory)，例如RAM；或者非易失性存储器(non-volatile memory)，例如ROM，快闪存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合，并向处理器72提供指令和数据。
上述处理器72可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。在一实施例中，对于不同的设备，用于实现上述处理器功能的电子器件还可以为其它，本公开实施例不作具体限定。
实施例七
在本公开实施例提出了一种VR设备，包括本公开实施例中的任意一种VR 设备的图像呈现装置或任意一种VR生的图像呈现设备，还包括：源生成端和数据处理端。
本公开的实施例可提供为方法、系统、或计算机程序产品。因此，本公开可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。
本公开是参照根据本公开实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。

Claims (13)

  1. 一种虚拟现实VR设备的图像呈现方法，包括：
    获取输入图像数据；
    在极坐标系下，形成所述输入图像数据和输出图像数据的对应关系；所述极坐标系的极点为VR设备的视场中心与显示平面的交点；
    根据所述输出图像数据得到待呈现图像数据；
    将所述待呈现图像数据基于凸透镜进行呈现。
  2. 根据权利要求1所述的方法，在所述形成所述输入图像数据和输出图像数据的对应关系之前，还包括：
    根据所述VR设备的显示分辨率，将所述输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
  3. 根据权利要求2所述的方法，其中，所述根据所述VR设备的显示分辨率，将所述输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系，包括：
    获取预先确定的多种显示分辨率分别对应的坐标系转换表，所述坐标系转换表用于表示图像中多个像素点分别从笛卡尔坐标系到极坐标系的坐标映射关系；
    根据所述VR设备的显示分辨率、以及所述多种显示分辨率分别对应的坐标系转换表，将所述输入图像数据中多个像素点的坐标分别从笛卡尔坐标系转换到极坐标系。
  4. 根据权利要求1所述的方法，其中，所述根据所述输出图像数据得到待呈现图像数据，包括：
    确定所述输出图像数据中多个像素点的像素值，根据所述输出图像数据中多个像素点的坐标和像素值，生成所述待呈现图像数据。
  5. 根据权利要求4所述的方法，其中，所述确定所述输出图像数据中多个像素点的像素值，包括：
    针对所述输入图像数据，生成输入空间的极坐标像素网格，所述输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点；
    按照所述输入图像数据至所述输出图像数据的映射方式，将所述输入空间的极坐标像素网格中的多个网格分别对应的像素点进行映射，得到映射后的多个离散像素点；根据所述映射后的多个离散像素点，生成输出空间的极坐标像素网格，所述输出空间的极坐标像素网格中的每个网格对应所述映射后的多个 离散像素点中的一个像素点；
    在所述输出图像数据对应的图像中的任一个像素点不属于所述输出空间的极坐标像素网格中的像素点的情况下，根据所述输出空间的极坐标像素网格，确定不属于所述输出空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值。
  6. 根据权利要求4所述的方法，其中，所述确定所述输出图像数据中多个像素点的像素值，包括：
    针对所述输入图像数据，生成输入空间的极坐标像素网格；所述输入空间的极坐标像素网格中的每个网格对应所述输入图像数据的一个像素点；
    按照所述输出图像数据至所述输入图像数据的映射方式，将所述输出图像数据对应的图像中的多个像素点分别映射至所述输入空间的极坐标像素网格，得到映射后的多个离散像素点；
    在所述映射后的多个离散像素点中的任一个像素点不属于所述输入空间的极坐标像素网格中的像素点的情况下，根据所述输入空间的极坐标像素网格，确定不属于所述输入空间的极坐标像素网格的所述像素点的邻域；根据所述像素点的邻域中多个像素点的像素值进行插值运算，得到所述像素点的像素值；
    根据所述映射后的多个离散像素点的像素值，得出所述输出图像数据的多个像素点的像素值。
  7. 根据权利要求5或6所述的方法，其中，所述根据所述像素点的邻域中多个像素点的像素值进行插值运算，包括：
    采用双线性插值方法、三线性插值方法或兰索斯Lanczos插值方法，并根据所述像素点的邻域中多个像素点的像素值进行插值运算。
  8. 根据权利要求4所述的方法，其中，所述像素值用于表示像素点的亮度分量或色度分量。
  9. 根据权利要求4所述的方法，其中，所述根据所述输出图像数据中多个像素点的坐标和像素值，生成所述待呈现图像数据，包括：
    将所述输入图像数据对应的图像中的多个像素点所在区域除去所述输出图像数据对应的图像中的多个像素点所在区域，得到边缘像素点区域；
    将所述边缘像素点区域的多个像素点的像素值置零，根据所述输出图像数据的多个像素点的坐标和像素值、以及所述边缘像素点区域的多个像素点的坐标和像素值，生成所述待呈现图像数据。
  10. 一种虚拟现实VR设备的图像呈现装置，包括：获取模块、处理模块和 呈现模块，其中：
    所述获取模块，设置为获取输入图像数据；
    所述处理模块，设置为在极坐标系下，形成所述输入图像数据和输出图像数据的对应关系；根据所述输出图像数据得到待呈现图像数据；所述极坐标系的极点为VR设备的视场中心与显示平面的交点；
    所述呈现模块，设置为将所述待呈现图像数据基于凸透镜进行呈现。
  11. 一种虚拟现实VR设备的图像呈现设备，包括：处理器和用于存储能够在所述处理器上运行的计算机程序的存储器；其中，
    所述处理器设置为运行所述计算机程序的情况下，执行权利要求1至9任一项所述的方法。
  12. 一种虚拟现实VR设备，所述VR设备包括权利要求10所述的装置或权利要求11所述的设备，还包括：源生成端和数据处理端。
  13. 一种计算机存储介质，所述计算机存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1至9中任一项所述的方法。
PCT/CN2019/118202 2018-11-16 2019-11-13 虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质 WO2020098721A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/293,749 US11908107B2 (en) 2018-11-16 2019-11-13 Method and apparatus for presenting image for virtual reality device, device and non-transitory computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811368846.5A CN111199518B (zh) 2018-11-16 2018-11-16 Vr设备的图像呈现方法、装置、设备和计算机存储介质
CN201811368846.5 2018-11-16

Publications (1)

Publication Number Publication Date
WO2020098721A1 true WO2020098721A1 (zh) 2020-05-22

Family

ID=70730391

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/118202 WO2020098721A1 (zh) 2018-11-16 2019-11-13 虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质

Country Status (3)

Country Link
US (1) US11908107B2 (zh)
CN (1) CN111199518B (zh)
WO (1) WO2020098721A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10768442B1 (en) * 2019-03-18 2020-09-08 Brelyon Inc. Display system providing concentric light field and monocular-to-binocular hybridization
CN113763229A (zh) * 2020-06-01 2021-12-07 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质
CN113436291B (zh) * 2021-06-21 2024-03-19 北京达佳互联信息技术有限公司 图像的处理方法和装置
CN113558551A (zh) * 2021-09-26 2021-10-29 深圳市资福医疗技术有限公司 一种图像获取方法及应用其的胶囊内窥镜
CN113726981B (zh) * 2021-09-30 2022-05-17 北京耐德佳显示技术有限公司 图像数据处理方法、电子设备及存储介质
WO2023123357A1 (zh) * 2021-12-31 2023-07-06 Oppo广东移动通信有限公司 图像处理的方法和装置
CN115511831B (zh) * 2022-09-27 2023-04-04 佳木斯大学 一种组织胚胎病理切片的数据分析处理系统及方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130236067A1 (en) * 2008-04-17 2013-09-12 Biometricore, Inc. Computationally Efficient Feature Extraction and Matching Iris Recognition
CN104567818A (zh) * 2014-12-31 2015-04-29 浙江工业大学 一种便携式全天候主动全景视觉传感器
CN107206692A (zh) * 2015-04-24 2017-09-26 惠普发展公司有限责任合伙企业 三维对象表示
CN107369132A (zh) * 2017-07-21 2017-11-21 网易（杭州）网络有限公司 获取虚拟现实设备图像畸变规律的方法、系统及终端设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6317127B1 (en) * 1996-10-16 2001-11-13 Hughes Electronics Corporation Multi-user real-time augmented reality system and method
US7813585B2 (en) * 2003-07-28 2010-10-12 Olympus Corporation Image processing apparatus, image processing method, and distortion correcting method
US9628766B2 (en) 2012-02-22 2017-04-18 Sony Corporation Display device, image processing device and image processing method, and computer program
CN104735464A (zh) * 2015-03-31 2015-06-24 华为技术有限公司 一种全景视频交互传输方法、服务器和客户端
WO2017079162A1 (en) * 2015-11-02 2017-05-11 Castar, Inc Method of immersive rendering for wide field of view
KR101785027B1 (ko) * 2016-01-14 2017-11-06 주식회사 라온텍 화면 왜곡 보정이 가능한 디스플레이 장치 및 이를 이용한 화면 왜곡 보정 방법
CN107220925B (zh) * 2017-05-05 2018-10-30 珠海全志科技股份有限公司 一种实时虚拟现实加速方法及装置
KR102473840B1 (ko) * 2017-11-21 2022-12-05 삼성전자주식회사 디스플레이 드라이버 및 모바일 전자 기기
CN108427595B (zh) * 2018-03-13 2021-07-27 网易（杭州）网络有限公司 虚拟现实中用户界面控件显示位置的确定方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130236067A1 (en) * 2008-04-17 2013-09-12 Biometricore, Inc. Computationally Efficient Feature Extraction and Matching Iris Recognition
CN104567818A (zh) * 2014-12-31 2015-04-29 浙江工业大学 一种便携式全天候主动全景视觉传感器
CN107206692A (zh) * 2015-04-24 2017-09-26 惠普发展公司有限责任合伙企业 三维对象表示
CN107369132A (zh) * 2017-07-21 2017-11-21 网易（杭州）网络有限公司 获取虚拟现实设备图像畸变规律的方法、系统及终端设备

Also Published As

Publication number Publication date
US11908107B2 (en) 2024-02-20
US20220122233A1 (en) 2022-04-21
CN111199518B (zh) 2024-03-26
CN111199518A (zh) 2020-05-26

Similar Documents

Publication Publication Date Title
WO2020098721A1 (zh) 虚拟现实设备的图像呈现方法、装置、设备和计算机存储介质
US10152814B2 (en) Image distortion compensation display device and image distortion compensation method using the same
WO2018201652A1 (zh) 一种实时虚拟现实加速方法及装置
TWI423659B (zh) 影像校正方法與相關影像校正系統
CN108090880B (zh) 一种图像的反畸变处理方法以及装置
CN106204712B (zh) 分段线性不规则光栅化
CN101605270B (zh) 生成深度图的方法和装置
TWI503791B (zh) 針對各向異性過濾運算細節位準之技術
WO2016155382A1 (zh) 生成马赛克图像的方法和装置
JP2008515353A (ja) 二次元画像の拡大およびピンチング
WO2019101005A1 (zh) 像素补偿方法、装置和终端设备
RU2697928C1 (ru) Способ сверхразрешения изображения, имитирующего повышение детализации на основе оптической системы, выполняемый на мобильном устройстве, обладающем ограниченными ресурсами, и мобильное устройство, его реализующее
KR20220047284A (ko) 포비티드 렌더링을 위한 시스템들 및 방법들
CN104902139A (zh) 一种头戴显示器和头戴显示器的视频数据处理方法
CN113302658A (zh) 并行纹理采样
CN113643414A (zh) 一种三维图像生成方法、装置、电子设备及存储介质
CN103096102B (zh) 影像形变方法
US20160189350A1 (en) System and method for remapping of image to correct optical distortions
CN116363290A (zh) 一种用于大规模场景三维重建的纹理贴图生成方法
WO2016004667A1 (zh) 增强视频图像光滑度及清晰度的超分辨率重构方法
TW201917698A (zh) 多維影像投射裝置及其多維影像校正方法
KR20050046775A (ko) 풋프린트의 분석 및 수정 방법
JP2017016511A (ja) 歪み補正画像処理装置及びプログラム
KR100684558B1 (ko) 텍스쳐 밉매핑 장치 및 방법
JP2002260003A (ja) 映像表示装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19885093

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19885093

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15.09.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19885093

Country of ref document: EP

Kind code of ref document: A1