CN112132740A - Video image display method, device and system - Google Patents
Video image display method, device and system Download PDFInfo
- Publication number
- CN112132740A CN112132740A CN201910553681.7A CN201910553681A CN112132740A CN 112132740 A CN112132740 A CN 112132740A CN 201910553681 A CN201910553681 A CN 201910553681A CN 112132740 A CN112132740 A CN 112132740A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- video image
- correction
- map
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012937 correction Methods 0.000 claims abstract description 131
- 230000008569 process Effects 0.000 claims description 22
- 238000009434 installation Methods 0.000 claims description 17
- 238000013507 mapping Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 230000007704 transition Effects 0.000 abstract description 10
- 230000000007 visual effect Effects 0.000 abstract description 6
- 230000000694 effects Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 230000004927 fusion Effects 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The application provides a video image display method, a device and a system, wherein the method comprises the following steps: determining a two-dimensional correction map and a three-dimensional perspective corresponding to a video image acquired by a camera; and fusing the two-dimensional correlogram and the three-dimensional perspective view by using the interpolation variable, and displaying the fused image. The two-dimensional correction graph and the three-dimensional perspective graph are fused and displayed by utilizing the interpolation variable, so that not only can all image information be displayed, but also 3D-to-2D or 2D-to-3D animation transition can be realized, and stronger visual impact force is brought to a user.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, and a system for displaying a video image.
Background
The fisheye video is a video image acquired by adopting an ultra-wide-angle lens, the visual angle range of the fisheye video is 220-230 degrees, and the picture distortion effect is strong. When two-dimensional (2D) display is performed, distortion of a video image needs to be corrected according to the principle of camera imaging, and the whole picture can be displayed on a screen by adjusting an image area. When three-dimensional (3D) display is performed, it is necessary to map image coordinates of a video image and three-dimensional coordinates of a three-dimensional model, and a user views the video image by setting a three-dimensional viewing angle.
However, the two-dimensional display method is poor in three-dimensional effect although it can display all image information, and the three-dimensional display method is poor in three-dimensional effect but can display only partial image information.
Disclosure of Invention
In view of the above, the present application provides a method, an apparatus and a system for displaying video images to solve the technical defects of the current display method.
According to a first aspect of embodiments of the present application, there is provided a video image display method, the method including:
determining a two-dimensional correction map and a three-dimensional perspective corresponding to a video image acquired by a camera;
and fusing the two-dimensional correction map and the three-dimensional perspective map by using the interpolation variable, and displaying the fused image.
According to a second aspect of embodiments of the present application, there is provided a video image display apparatus, the apparatus comprising:
the determining module is used for determining a two-dimensional correction map and a three-dimensional perspective corresponding to a video image acquired by the camera;
and the display module is used for fusing the two-dimensional correction map and the three-dimensional perspective map by using an interpolation variable and displaying the fused image.
According to a third aspect of embodiments herein, there is provided a video image display system, the system comprising;
the camera is used for acquiring a video image and sending the video image to the electronic equipment;
the electronic equipment is used for determining a two-dimensional correction map and a three-dimensional perspective corresponding to the video image; fusing the two-dimensional correction map and the three-dimensional perspective map by using an interpolation variable to obtain a fused image;
and the display is used for displaying the fused image.
By applying the embodiment of the application, the two-dimensional correction graph and the three-dimensional perspective graph corresponding to the video image acquired by the camera are determined, then the two-dimensional correction graph and the three-dimensional perspective graph are fused by utilizing the interpolation variable, and the fused image is displayed.
Based on the above description, the two-dimensional correction graph and the three-dimensional perspective graph are fused by using the interpolation variable and then displayed, so that not only can all image information be displayed, but also the animation transition from 3D to 2D or from 2D to 3D can be realized, and a stronger visual impact force is brought to a user.
Drawings
FIG. 1A is a flow chart illustrating an embodiment of a method for displaying video images according to an exemplary embodiment of the present application;
FIG. 1B is an original fisheye image according to the embodiment of FIG. 1A;
FIG. 1C is a schematic diagram of an image coordinate system and a screen coordinate system shown in accordance with the embodiment of FIG. 1A;
FIG. 1D is a two-dimensional rectification diagram corresponding to a fisheye image shown in the embodiment of FIG. 1A;
FIG. 1E is a schematic diagram of a two-dimensional correction map shown in the embodiment of FIG. 1A displayed on a screen according to the present application;
FIG. 1F is a schematic diagram of a three-dimensional model coordinate system and a screen coordinate system according to the embodiment of FIG. 1A;
FIG. 1G is a schematic view of an observation point shown in the embodiment of FIG. 1A;
FIG. 1H is a three-dimensional perspective view of the present application from a different perspective view point, as shown in the embodiment of FIG. 1A;
FIG. 1I is a schematic view of a transition effect of a fusion change according to the embodiment of FIG. 1A;
FIG. 2 is a flow chart illustrating an embodiment of another video image display method according to an exemplary embodiment of the present application;
FIG. 3 is a block diagram of a video image display system according to an exemplary embodiment of the present application;
fig. 4 is a block diagram of an embodiment of a video image display apparatus according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
At present, the two display modes adopted for the fisheye video are either two-dimensional display modes capable of seeing all image information or three-dimensional display modes capable of seeing only partial image information, and both display modes have the defect of self display, so that the visual impact force brought to a user is weak, and the two display modes are not high in adaptability.
In order to solve the above problems, the present application provides a video image display method, which includes determining a two-dimensional rectification map and a three-dimensional perspective map corresponding to a video image acquired by a camera, and then fusing the two-dimensional rectification map and the three-dimensional perspective map by using an interpolation variable, and displaying the fused image.
Based on the above description, the two-dimensional rectification graph and the three-dimensional perspective graph are fused by using the interpolation variable and then displayed, so that not only can all image information be displayed, but also 3D-to-2D or 2D-to-3D animation transition can be realized, and a stronger visual impact force is brought to a user.
Fig. 1A is a flowchart illustrating an embodiment of a video image display method according to an exemplary embodiment of the present application, where the video image display method can be applied to an electronic device (e.g., a mobile terminal, a PC), and the video image is taken as a fisheye image for example.
As shown in fig. 1A, the video display method includes the following steps:
step 101: and determining a two-dimensional correction map and a three-dimensional perspective corresponding to the video image acquired by the camera.
Before two-dimensional correction, a correction function needs to be determined, different correction function forms exist under different camera installation modes (such as top installation, bottom installation, side installation and the like), and the calculation amount of the correction function is large, and when correction is performed, correction calculation is performed on each pixel point in a video image by using the correction function, so that the correction calculation amount of one frame of the video image is large.
Illustratively, assume that the pixel coordinates (u, v) in the two-dimensional correlogram correspond to the pixel coordinates (u, v) in the video image1，v1) The following formula is involved in the correction calculation:
u’＝(u-cu)/fu
v’＝(v-cv)/fv
R＝u’2+v’2
dR＝1+R2k1+R2k2+R3k3
dTX＝2p1u’v’+p2(R+2u’2)
dTY＝2p2u’v’+p1(R+2v’2)
u1＝(u’dR+dTX)fu+cu
v1＝(v’dR+dTY)fv+cv
wherein, cu、cv、fu、fvIs the camera internal reference, k1、k2、k3、p1、p2Is a distortion parameter.
Based on the above, in order to reduce the amount of calculation on one hand and unify various correction function forms on the other hand, a frame of two-dimensional correction graph with the same size as the video image is established, then the correction function is utilized to perform correction pre-calculation on each pixel coordinate in the two-dimensional correction graph to obtain the pixel coordinate corresponding to the video image, and then the pixel coordinate corresponding to each pixel coordinate in the two-dimensional correction graph in the video image is stored in the two-dimensional correction lookup table for subsequent direct search correction.
Wherein, different camera installation modes all can correspond there is a two-dimentional correction look-up table to under the different installation scenes through switching over two-dimentional correction look-up table alright realize the conversion of two-dimentional correction picture fast. It will be appreciated by those skilled in the art that the form of the corrective function used for different installation modes in establishing the two-dimensional corrective look-up table can be implemented by correlation techniques.
In one example, for the process of determining the two-dimensional correction map corresponding to the video image captured by the camera, the two-dimensional correction lookup table corresponding to the installation mode of the camera is selected from the pre-established two-dimensional correction lookup tables, and then the two-dimensional correction map corresponding to the video image is obtained according to the two-dimensional correction lookup table, and the obtained two-dimensional correction map is converted into a screen coordinate system of the display.
For example, as shown in fig. 1B, the original fisheye image shows a severely distorted picture, and when fig. 1B is displayed, the adopted display relation is Color (w, h) ═ GetPixel (u, v); where (w, h) represents screen coordinates in the screen coordinate system, (u, v) represents pixel coordinates in the image coordinate system, and GetPixel represents padding of color values at (u, v) to (w, h) of the display.
As shown in fig. 1C, (a) is an image coordinate system of the image, the vertex at the lower left corner of the image is taken as an origin, the horizontal direction is taken as a horizontal axis, and the vertical direction is taken as a vertical axis, and (b) is a screen coordinate system of the display, the screen center of the display is taken as an origin, the horizontal direction is taken as a horizontal axis, and the vertical direction is taken as a vertical axis, after normalization, the value range of the pixel coordinate (u, v) is 0 to 1, the value range of the screen coordinate (w, h) is-1 to 1, and the relationship between the two is:
w＝2u-1
h＝2v-1
as shown in fig. 1D, the two-dimensional rectification graph corresponding to the video image is shown, and the displayed picture has no distortion, and when the display is shown in fig. 1D, the two-dimensional rectification graph needs to be converted into the screen coordinate system of the display to be displayed, as shown in fig. 1E, (a) is the screen of the display, (b) is the two-dimensional rectification graph, and (c) is the original fisheye image, and the pixel coordinate F point (u, v) in the two-dimensional rectification graph and the pixel coordinate Q point (u, v) in the video image are obtained1，v1) The relationship between them is: (u)1，v1) The pixel coordinate Q point (u, v) in the video image is obtained by looking up the corresponding two-dimensional rectification look-up table (lookop 2D (u, v))1，v1) From the above-described image-to-screen conversion relation (w, h) ═ 2(u, v) -1 shown in fig. 1C, the rectification map can be converted into the screen coordinate system of the display, and displayed using the Color (w, h) ═ GetPixel (u, v) display relation.
It should be noted that, because the user selects different correction modes, such as no loss in correction, large span, and the like, and the correction function forms are also different, the influence of the correction mode on the correction function needs to be considered when the two-dimensional correction lookup table is established, and the pre-established two-dimensional correction lookup table corresponds to the installation mode and the correction mode, so that the conversion of the two-dimensional correction map can be quickly realized by switching the two-dimensional correction lookup table in different installation modes and correction modes.
In another example, for the process of determining the two-dimensional correction map corresponding to the video image captured by the camera, the information of the correction mode input from the outside is received, the installation mode of the camera and the two-dimensional correction lookup table corresponding to the correction mode are selected from the pre-established two-dimensional correction lookup table, then the two-dimensional correction map corresponding to the video image is obtained according to the two-dimensional correction lookup table, and the obtained two-dimensional correction map is converted into the screen coordinate system of the display.
In an embodiment, for the process of determining the three-dimensional perspective view corresponding to the video image captured by the camera, the three-dimensional model map corresponding to the video image may be obtained according to a pre-established three-dimensional mapping lookup table, and the obtained three-dimensional model map may be converted into the three-dimensional perspective view according to the externally input observation viewpoint information and the perspective conversion relationship between the screen coordinate system of the display and the three-dimensional model coordinate system.
The three-dimensional model related to the pre-established three-dimensional mapping lookup table can be a spherical model, a cylindrical model, a conical model and the like. Different three-dimensional models can be pre-calculated in different conversion function forms, the establishment principle of the three-dimensional mapping lookup table is consistent with that of the two-dimensional correction lookup table, and pixel coordinates of each pixel in the three-dimensional model image, which correspond to the pixel coordinates in the video image, are recorded in the pre-established three-dimensional mapping lookup table for subsequent direct searching and conversion.
The process of converting the three-dimensional rendering from the three-dimensional model map is described in detail below:
taking the three-dimensional model as the cone model as an example, as shown in fig. 1F, (a) is a three-dimensional model coordinate system of the cone model, (b) is a screen coordinate system of the display, fov represents a view range angle in the three-dimensional model in the vertical direction, n represents a value of a near clipping plane of the three-dimensional model, F represents a value of a far clipping plane of the three-dimensional model, and a perspective transformation relationship between P (x, y, z) in the three-dimensional model coordinate system and Q (w, h) in the screen coordinate system is as follows:
the aspect represents the ratio of the length to the width of the screen of the display, the aspects, n, f and fov are known quantities, and the depth represents the distance between the display and the lens of the camera.
After perspective transformation, even in a three-dimensional display mode, only local image information can be seen by observing the display, so that a three-dimensional perspective view capable of being displayed can be obtained only by changing viewpoints. In the process of changing the viewpoint, as shown in fig. 1G, taking the observation viewpoint information at least including the pitch angle pitch, the heading angle yaw, and the distance Scale between the viewpoint V and the observation target as an example, the calculation process of the viewpoint matrix View is as follows:
right＝up×forward
head＝forward×right
therefore, the conversion relation between the three-dimensional model coordinate system and the screen coordinate system of the display is as follows:
(w,h)＝PerspectiveView(x,y,z)
where Perspective represents a Perspective transformation matrix, and View represents a View transformation matrix.
The three-dimensional model graph can be converted into a three-dimensional perspective view under a screen coordinate system for displaying through the conversion relation between the three-dimensional model coordinate system and the screen coordinate system of the display, and the display relation of the three-dimensional perspective view is as follows:
Color(w,h)＝GetPixel(LookUp3D(PerspectiveView-1(w,h)))
wherein, LookUp3D represents the search of three-dimensional mapping LookUp table, PerspectView-1And the GetPixel represents the color value of a certain pixel coordinate in the acquired video image.
As shown in fig. 1H, fig. 1 (1), 2 (2), (3), (4), (5), and (6) are three-dimensional perspective views displayed by a display when a user inputs different viewing point information, and an original fisheye image is shown in fig. 1B.
It should be further noted that, the order of obtaining the two-dimensional correction drawing and the three-dimensional perspective drawing is not limited in the present application, and may be sequential, or may be obtained simultaneously.
Step 102: and fusing the two-dimensional correlogram and the three-dimensional perspective view by using the interpolation variable and displaying the fused image through a display.
Wherein, the value range of the interpolation variable is [0, 1 ].
In an embodiment, as can be seen from the above description of step 101, since the determined two-dimensional correction chart and the three-dimensional rendering map are both located in the screen coordinate system of the display, the two-dimensional correction chart and the three-dimensional rendering map can be directly fused, and the fused map is displayed, where the fusion display process may be: and traversing each interpolation variable according to a specified sequence, fusing the pixel value corresponding to each screen coordinate in the two-dimensional correction graph with the pixel value corresponding to each screen coordinate in the three-dimensional perspective view according to the currently traversed interpolation variable, and displaying the fused pixel values through a display.
During the traversal, an interpolation variable can be taken at preset intervals (the smaller the interval is, the smoother the animation transition is), the traversal order can be from 0 to 1, or from 1 to 0, the traversal order is different, the animation transition effect is different, the traversal order of the interpolation variable is assumed to be from 0 to 1, and the display effect of the video image can be the animation transition effect from 3D to 2D.
In an embodiment, for a process of fusing a pixel value corresponding to each screen coordinate in the two-dimensional correctional map with a pixel value corresponding in the three-dimensional perspective view according to the currently traversed interpolation variable and displaying the fused pixel value through the display, a first interpolation coefficient of the two-dimensional correctional map and a second interpolation coefficient of the three-dimensional perspective view may be determined according to the currently traversed interpolation variable, a sum of the first interpolation coefficient and the second interpolation coefficient is 1, and the pixel value corresponding in the two-dimensional correctional map and the pixel value corresponding in the three-dimensional perspective view are fused using the first interpolation coefficient and the second interpolation coefficient.
Before determining a first interpolation coefficient of the two-dimensional correction map and a second interpolation coefficient of the three-dimensional perspective map, determining an interpolation function form, wherein when the interpolation function needs to meet the conditions, an interpolation coefficient obtained by the interpolation function is 0 when an interpolation variable is 0; the interpolation coefficient obtained by the interpolation function is 1 when the interpolation variable is 1.
Based on the condition that the interpolation function needs to satisfy, the form of the interpolation function may be l (x) or l (x) xnAnd n is a positive integer greater than or equal to 1.
Illustratively, assume that the fused relationship between the two-dimensional correctional map and the three-dimensional rendering is:
Color(w,h)＝L(x)Color2D(w,h)+(1-L(x))Color3D(w,h)
wherein, Color2D (w, h) represents a Color value of a screen coordinate (w, h) in the two-dimensional correlogram, Color3D (w, h) represents a Color value of the screen coordinate (w, h) in the three-dimensional perspective view, l (x) represents a first interpolation coefficient of the two-dimensional correlogram, i.e., an interpolation coefficient obtained by an interpolation function, x represents an interpolation variable, and 1-l (x) represents a second interpolation coefficient of the three-dimensional perspective view. From this, the sum of the first interpolation coefficient and the second interpolation coefficient is 1.
Based on the above fusion relation, when the traversal order of the interpolation variables is 0 to 1, the transition effect of displaying the video image is from 3D to 2D, and when the traversal order of the interpolation variables is 1 to 0, the transition efficiency of displaying the video image is from 2D to 3D.
As shown in fig. 1I, the fusion change effect of the two-dimensional correlogram and the three-dimensional perspective view has not only the animation effect of dynamic switching, but also the 3D effect with all the image information of the original fisheye image retained.
In the embodiment of the application, the two-dimensional correction map and the three-dimensional perspective map corresponding to the video image acquired by the camera are determined, and then the two-dimensional correction map and the three-dimensional perspective map are fused by using the interpolation variable, and the fused image is displayed by the display.
Based on the above description, the two-dimensional rectification graph and the three-dimensional perspective graph are fused by using the interpolation variable and then displayed, so that not only can all image information be displayed, but also 3D-to-2D or 2D-to-3D animation transition can be realized, and a stronger visual impact force is brought to a user.
Fig. 2 is a flowchart illustrating another video image display method according to an exemplary embodiment of the present application, and based on the embodiment illustrated in fig. 1A, the video image display method includes the following steps:
step 201: and establishing a two-dimensional correction lookup table, a three-dimensional mapping lookup table and an interpolation function form.
Step 202: and receiving the video image, and respectively determining a two-dimensional correction image and a three-dimensional perspective image corresponding to the video image by using the two-dimensional correction lookup table and the three-dimensional mapping lookup table.
Step 203: and fusing the two-dimensional correction graph and the three-dimensional perspective graph by using an interpolation function form, and outputting and displaying.
For the process from step 201 to step 203, the detailed implementation thereof can refer to the related description from step 101 to step 102, and is not described again.
Fig. 3 is a block diagram of a video image display system according to an exemplary embodiment of the present application, and as shown in fig. 3, the video image display system includes:
the camera 310 is used for acquiring a video image and sending the video image to the electronic equipment;
an electronic device 320 for determining a two-dimensional rectification map and a three-dimensional perspective map corresponding to the video image; fusing the two-dimensional correction map and the three-dimensional perspective map by using an interpolation variable to obtain a fused image;
and a display 330 for displaying the fused image.
Fig. 4 is a block diagram of an embodiment of a video image display apparatus according to an exemplary embodiment of the present application, which can be applied to an electronic device, as shown in fig. 4, and the video image display apparatus includes:
a determining module 410, configured to determine a two-dimensional rectification map and a three-dimensional perspective map corresponding to a video image acquired by a camera;
and a display module 420 for fusing the two-dimensional correlogram and the three-dimensional perspective view by using the interpolation variable and displaying the fused image.
In an optional implementation manner, the determining module 410 is specifically configured to select, from a pre-established two-dimensional correction lookup table, a two-dimensional correction lookup table corresponding to an installation mode of a camera in a process of determining a two-dimensional correction map corresponding to a video image acquired by the camera, where a pixel coordinate of each pixel in the two-dimensional correction map corresponding to a pixel coordinate in the video image is recorded in the two-dimensional correction lookup table; and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
In an optional implementation manner, the determining module 410 is specifically configured to receive externally input information of a rectification mode in a process of determining a two-dimensional rectification map corresponding to a video image acquired by a camera; selecting an installation mode of the camera and a two-dimensional correction lookup table corresponding to the correction mode from a pre-established two-dimensional correction lookup table, wherein a pixel coordinate of each pixel in a two-dimensional correction image, which corresponds to a pixel coordinate in a video image, is recorded in the two-dimensional correction lookup table; and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
In an optional implementation manner, the determining module 410 is specifically configured to, in a process of determining a three-dimensional perspective view corresponding to a video image acquired by a camera, obtain a three-dimensional model map corresponding to the video image according to a pre-established three-dimensional mapping lookup table, where a pixel coordinate of each pixel in the three-dimensional model map corresponding to a pixel coordinate in the video image is recorded in the three-dimensional mapping lookup table; and converting the obtained three-dimensional model image into a three-dimensional perspective view according to the perspective conversion relation between the screen coordinate system and the three-dimensional model coordinate system of the display and the externally input observation viewpoint information.
In an alternative implementation, the two-dimensional correctional map and the three-dimensional perspective map are both located in a screen coordinate system of a display; the display module 420 is specifically configured to traverse each interpolation variable according to a specified order, fuse a pixel value corresponding to each screen coordinate in the two-dimensional correction map and a pixel value corresponding to each screen coordinate in the three-dimensional perspective view according to the currently traversed interpolation variable, and display the fused pixel values through the display; wherein, the value range of the interpolation variable is more than or equal to 0 and less than or equal to 1.
In an optional implementation manner, the display module 420 is specifically configured to determine, according to the currently traversed interpolation variable, a first interpolation coefficient of the two-dimensional correctional graph and a second interpolation coefficient of the three-dimensional perspective graph in the process of fusing the corresponding pixel value of each screen coordinate in the two-dimensional correctional graph and the corresponding pixel value in the three-dimensional perspective graph according to the currently traversed interpolation variable, where a sum of the first interpolation coefficient and the second interpolation coefficient is 1; and fusing the corresponding pixel value in the two-dimensional correction map and the corresponding pixel value in the three-dimensional perspective view by using the first interpolation coefficient and the second interpolation coefficient.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (13)
1. A method for displaying video images, the method comprising:
determining a two-dimensional correction map and a three-dimensional perspective corresponding to a video image acquired by a camera;
and fusing the two-dimensional correlogram and the three-dimensional perspective view by using the interpolation variable, and displaying the fused image.
2. The method of claim 1, wherein determining a two-dimensional correlogram corresponding to a video image captured by a camera comprises:
selecting a two-dimensional correction lookup table corresponding to the installation mode of the camera from a pre-established two-dimensional correction lookup table, wherein a pixel coordinate of each pixel in a two-dimensional correction image, which corresponds to a video image, is recorded in the two-dimensional correction lookup table;
and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
3. The method of claim 1, wherein determining a two-dimensional correlogram corresponding to a video image captured by a camera comprises:
receiving externally input information of a correction mode;
selecting an installation mode of the camera and a two-dimensional correction lookup table corresponding to the correction mode from a pre-established two-dimensional correction lookup table, wherein a pixel coordinate of each pixel in a two-dimensional correction image, which corresponds to a pixel coordinate in a video image, is recorded in the two-dimensional correction lookup table;
and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
4. The method of claim 1, wherein determining a three-dimensional perspective view corresponding to a video image captured by a camera comprises:
obtaining a three-dimensional model map corresponding to the video image according to a pre-established three-dimensional mapping lookup table, wherein the three-dimensional mapping lookup table records pixel coordinates of each pixel in the three-dimensional model map, which correspond to the pixel coordinates in the video image;
and converting the obtained three-dimensional model image into a three-dimensional perspective view according to the perspective conversion relation between the screen coordinate system and the three-dimensional model coordinate system of the display and the externally input observation viewpoint information.
5. The method of claim 1, wherein the two-dimensional correctional map and the three-dimensional perspective view are both located in a screen coordinate system of a display, and fusing the two-dimensional correctional map and the three-dimensional perspective view and displaying a fused image using an interpolated variable, comprises:
traversing each interpolation variable according to a specified sequence, fusing a pixel value corresponding to each screen coordinate in the two-dimensional correction graph with a pixel value corresponding to each screen coordinate in the three-dimensional perspective view according to the currently traversed interpolation variable, and displaying the fused pixel values through the display;
wherein, the value range of the interpolation variable is more than or equal to 0 and less than or equal to 1.
6. The method of claim 5, wherein fusing the corresponding pixel value in the two-dimensional correctional map for each screen coordinate with the corresponding pixel value in the three-dimensional perspective view according to the currently traversed interpolation variable comprises:
determining a first interpolation coefficient of the two-dimensional correction map and a second interpolation coefficient of the three-dimensional perspective map according to the currently traversed interpolation variable, wherein the sum of the first interpolation coefficient and the second interpolation coefficient is 1;
and fusing the corresponding pixel value in the two-dimensional correction map and the corresponding pixel value in the three-dimensional perspective view by using the first interpolation coefficient and the second interpolation coefficient.
7. A video image display apparatus, characterized in that the apparatus comprises:
the determining module is used for determining a two-dimensional correction map and a three-dimensional perspective corresponding to a video image acquired by the camera;
and the display module is used for fusing the two-dimensional correction map and the three-dimensional perspective map by using an interpolation variable and displaying the fused image.
8. The apparatus according to claim 7, wherein the determining module is specifically configured to, in a process of determining a two-dimensional correction map corresponding to a video image captured by a camera, select a two-dimensional correction lookup table corresponding to an installation mode of the camera from pre-established two-dimensional correction lookup tables, where a pixel coordinate of each pixel in the two-dimensional correction map corresponds to a pixel coordinate in the video image is recorded in the two-dimensional correction lookup table; and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
9. The device according to claim 7, wherein the determining module is specifically configured to receive externally input information of the rectification mode in the process of determining the two-dimensional rectification graph corresponding to the video image captured by the camera; selecting an installation mode of the camera and a two-dimensional correction lookup table corresponding to the correction mode from a pre-established two-dimensional correction lookup table, wherein a pixel coordinate of each pixel in a two-dimensional correction image, which corresponds to a pixel coordinate in a video image, is recorded in the two-dimensional correction lookup table; and obtaining a two-dimensional correction image corresponding to the video image according to the two-dimensional correction lookup table, and converting the obtained two-dimensional correction image into a screen coordinate system of the display.
10. The apparatus according to claim 7, wherein the determining module is specifically configured to, in the process of determining the three-dimensional perspective view corresponding to the video image captured by the camera, obtain a three-dimensional model map corresponding to the video image according to a pre-established three-dimensional mapping lookup table, where a pixel coordinate of each pixel in the three-dimensional model map corresponds to a pixel coordinate in the video image is recorded in the three-dimensional mapping lookup table; and converting the obtained three-dimensional model image into a three-dimensional perspective view according to the perspective conversion relation between the screen coordinate system and the three-dimensional model coordinate system of the display and the externally input observation viewpoint information.
11. The apparatus of claim 7, wherein the two-dimensional correctional graph and the three-dimensional perspective view are both located in a screen coordinate system of a display;
the display module is specifically used for traversing each interpolation variable according to a specified sequence, fusing a pixel value corresponding to each screen coordinate in the two-dimensional correction graph with a pixel value corresponding to each screen coordinate in the three-dimensional perspective view according to the currently traversed interpolation variable, and displaying the fused pixel values through the display; wherein, the value range of the interpolation variable is more than or equal to 0 and less than or equal to 1.
12. The apparatus according to claim 11, wherein the display module is configured to determine a first interpolation coefficient of the two-dimensional correlogram and a second interpolation coefficient of the three-dimensional perspective view according to the currently traversed interpolation variable, in the process of fusing the corresponding pixel value of each screen coordinate in the two-dimensional correlogram with the corresponding pixel value in the three-dimensional perspective view according to the currently traversed interpolation variable, and a sum of the first interpolation coefficient and the second interpolation coefficient is 1; and fusing the corresponding pixel value in the two-dimensional correction map and the corresponding pixel value in the three-dimensional perspective view by using the first interpolation coefficient and the second interpolation coefficient.
13. A video image display system, characterized in that the system comprises;
the camera is used for acquiring a video image and sending the video image to the electronic equipment;
the electronic equipment is used for determining a two-dimensional correction map and a three-dimensional perspective corresponding to the video image; fusing the two-dimensional correction map and the three-dimensional perspective map by using an interpolation variable to obtain a fused image;
and the display is used for displaying the fused image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910553681.7A CN112132740B (en) | 2019-06-25 | 2019-06-25 | Video image display method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910553681.7A CN112132740B (en) | 2019-06-25 | 2019-06-25 | Video image display method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112132740A true CN112132740A (en) | 2020-12-25 |
| CN112132740B CN112132740B (en) | 2023-08-25 |
Family
ID=73849377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910553681.7A Active CN112132740B (en) | 2019-06-25 | 2019-06-25 | Video image display method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112132740B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040247173A1 (en) * | 2001-10-29 | 2004-12-09 | Frank Nielsen | Non-flat image processing apparatus, image processing method, recording medium, and computer program |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
| CN106846410A (en) * | 2016-12-20 | 2017-06-13 | 北京鑫洋泉电子科技有限公司 | Based on three-dimensional environment imaging method and device |
| CN109308686A (en) * | 2018-08-16 | 2019-02-05 | 北京市商汤科技开发有限公司 | A kind of fish eye images processing method and processing device, equipment and storage medium |
| WO2019096323A1 (en) * | 2017-11-20 | 2019-05-23 | 杭州海康威视数字技术股份有限公司 | Fisheye image processing method and apparatus, and electronic device |
-
2019
- 2019-06-25 CN CN201910553681.7A patent/CN112132740B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040247173A1 (en) * | 2001-10-29 | 2004-12-09 | Frank Nielsen | Non-flat image processing apparatus, image processing method, recording medium, and computer program |
| CN105894549A (en) * | 2015-10-21 | 2016-08-24 | 乐卡汽车智能科技（北京）有限公司 | Panorama assisted parking system and device and panorama image display method |
| CN106846410A (en) * | 2016-12-20 | 2017-06-13 | 北京鑫洋泉电子科技有限公司 | Based on three-dimensional environment imaging method and device |
| WO2019096323A1 (en) * | 2017-11-20 | 2019-05-23 | 杭州海康威视数字技术股份有限公司 | Fisheye image processing method and apparatus, and electronic device |
| CN109308686A (en) * | 2018-08-16 | 2019-02-05 | 北京市商汤科技开发有限公司 | A kind of fish eye images processing method and processing device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112132740B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109547766B (en) | Panoramic image generation method and device | |
| JP6561216B2 (en) | Generating intermediate views using optical flow | |
| CN109348119B (en) | Panoramic monitoring system | |
| JP7002056B2 (en) | 3D model generator and 3D model generation method | |
| US9299152B2 (en) | Systems and methods for image depth map generation | |
| EP2862356B1 (en) | Method and apparatus for fusion of images | |
| TWI524734B (en) | Method and device for generating a depth map | |
| US20170111582A1 (en) | Wide-Area Image Acquiring Method and Apparatus | |
| CN105243637B (en) | One kind carrying out full-view image joining method based on three-dimensional laser point cloud | |
| CN104756489B (en) | A kind of virtual visual point synthesizing method and system | |
| CN103971352A (en) | Rapid image splicing method based on wide-angle lenses | |
| JPH08331607A (en) | Three-dimensional display image generating method | |
| US20100302234A1 (en) | Method of establishing dof data of 3d image and system thereof | |
| CN108200360A (en) | A kind of real-time video joining method of more fish eye lens panoramic cameras | |
| JP2012019399A (en) | Stereoscopic image correction device, stereoscopic image correction method, and stereoscopic image correction system | |
| CN111866523A (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| Park et al. | Efficient viewer-centric depth adjustment based on virtual fronto-parallel planar projection in stereo 3D images | |
| CN115170637A (en) | Virtual visual angle image construction method and device, control equipment and readable storage medium | |
| CN108564654B (en) | Picture entering mode of three-dimensional large scene | |
| CN114449303A (en) | Live broadcast picture generation method and device, storage medium and electronic device | |
| US11043019B2 (en) | Method of displaying a wide-format augmented reality object | |
| CN116860112B (en) | Combined scene experience generation method, system and medium based on XR technology | |
| JP4908350B2 (en) | Image processing method and imaging apparatus using the image processing method | |
| CN107743222A (en) | A kind of image processing method and three-dimensional panorama VR collectors based on collector | |
| JPH0981746A (en) | Two-dimensional display image generating method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |