WO2019192358A1 - Procédé et appareil permettant de synthétiser une vidéo panoramique, et dispositif électronique - Google Patents

Procédé et appareil permettant de synthétiser une vidéo panoramique, et dispositif électronique Download PDF

Info

Publication number
WO2019192358A1
WO2019192358A1 PCT/CN2019/079726 CN2019079726W WO2019192358A1 WO 2019192358 A1 WO2019192358 A1 WO 2019192358A1 CN 2019079726 W CN2019079726 W CN 2019079726W WO 2019192358 A1 WO2019192358 A1 WO 2019192358A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
images
panoramic
fisheye
external reference
Prior art date
Application number
PCT/CN2019/079726
Other languages
English (en)
Chinese (zh)
Inventor
王泽文
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2019192358A1 publication Critical patent/WO2019192358A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present application relates to the field of image processing technologies, and in particular, to a panoramic video synthesis method, apparatus, and electronic device.
  • the fisheye camera is an ultra-wide-angle camera that simulates the fish's upward looking water surface.
  • the fisheye camera has an angle of view greater than or equal to 180 degrees.
  • the fisheye camera has the advantages of large field of view angle, can accommodate many scenes, and can be adapted to small space shooting. Therefore, it is widely used in the fields of virtual reality technology, robot navigation, visual monitoring, intelligent assisted driving and the like.
  • the two fisheye cameras are set back to back to form a double fisheye device, so that by splicing the fisheye images taken by the two fisheye cameras, 360 can be obtained.
  • Panoramic image of degree
  • the fisheye image taken with the fisheye camera has a very wide viewing angle, and the distortion of the fisheye image taken is very large. Therefore, when splicing the fisheye images taken by the two fisheye cameras, distortion correction processing is required.
  • the internal reference and the external reference of each fisheye camera are calibrated by means of offline calibration, and the fisheye image of the collected fisheye image is spherically projected by the calibration internal reference of each fisheye camera to obtain a spherical projection view, and then The spherical projection image is spliced according to the external reference of each fisheye camera to obtain a panoramic image.
  • the hardware cost of the hardware device for realizing panoramic image synthesis is too high, and the process of panoramic image synthesis is complicated and inefficient.
  • the purpose of the embodiments of the present application is to provide a panoramic video synthesis method, apparatus, and electronic device to reduce the hardware cost of panoramic video synthesis and improve the efficiency of panoramic video synthesis.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides a method for synthesizing a panoramic video, where the method includes:
  • the video to be spliced is spliced to synthesize a panoramic video.
  • an embodiment of the present application provides a panoramic video synthesizing apparatus, where the apparatus includes:
  • the obtaining module is configured to obtain each original video captured by two fisheye cameras in the fisheye device;
  • a search module configured to search for a pre-established panoramic synthesis mapping table, and respectively generate a splicing relationship between each video to be spliced corresponding to each original video and the video to be spliced, and the panoramic synthesis mapping table is based on each fisheye camera Extracting the external parameters of the fisheye camera calibrated in each of the collected external reference calibration images, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively processing the calibration images of the external indexes;
  • the splicing module is configured to splicing the video to be spliced according to the splicing relationship to synthesize a panoramic video.
  • an embodiment of the present application provides an electronic device, including a processor and a memory;
  • the memory is configured to store a computer program
  • the processor when used to execute a computer program stored on the memory, implements the method steps as described in the first aspect.
  • the embodiment of the present application provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, implementing the method according to the first aspect step.
  • the original video captured by the two fisheye cameras in the fisheye device is obtained, and the pre-established panoramic synthesis mapping table is searched, and each to-be-spliced video corresponding to each original video is separately generated.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters. Processed.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • FIG. 1 is a schematic flowchart of a method for synthesizing a panoramic video according to an embodiment of the present application
  • Figure 2a is a corresponding fisheye image without clipping and offset
  • Figure 2b is a corresponding fisheye image with no cropping and offset
  • Figure 2c is a corresponding fisheye image with cropping and no offset
  • Figure 2d is a corresponding fisheye image with cropped offset
  • FIG. 3 is a schematic flowchart of a process for generating a panoramic synthesis mapping table according to an embodiment of the present application
  • 4a is a schematic diagram of an effective area of an external reference calibration image collected by a first fisheye camera in a fisheye device according to an embodiment of the present application;
  • 4b is a schematic diagram of an effective area of an external reference calibration image collected by a second fisheye camera in a fisheye device according to an embodiment of the present application;
  • FIG. 5 is a schematic structural diagram of a panoramic synthesis mapping table according to an embodiment of the present application.
  • FIG. 6 is a schematic flow chart of a spherical projection according to an embodiment of the present application.
  • FIG. 7 is a schematic flow chart of spherical expansion according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a panoramic video generation process according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a panoramic video synthesizing apparatus according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the embodiment of the present application provides a panoramic video synthesis method, device, and electronic device.
  • the panoramic video synthesis method provided by the embodiment of the present application is first introduced.
  • the execution method of the panoramic video synthesis method provided by the embodiment of the present application may be a fisheye device including a core processing chip, or may be an electronic device having a video processing and synthesizing function, and implement the panoramic video synthesis method provided by the embodiment of the present application.
  • the manner may be at least one of software, hardware circuits, and logic circuits disposed in the execution body.
  • a panoramic video synthesis method provided by an embodiment of the present application may include the following steps.
  • the fisheye device includes two fisheye cameras, each having an angle of view greater than or equal to 180 degrees, so the total field of view of the fisheye device is greater than or equal to 360 degrees.
  • the fisheye device may or may not include a core processing chip. If the fisheye device includes a core processing chip, the original video captured by each fisheye camera can be obtained by the core processing chip in the fisheye device; if the fisheye device does not include the core processing chip, it needs to have a video processing synthesis function.
  • the electronic device acquires each original video captured by each fisheye camera in the fisheye device.
  • the two fisheye cameras may not have consistent key parameters and installation parameters, including but not limited to camera internal parameters, field of view, distortion, focal length, etc.; installation parameters include, but are not limited to, installation location, installation angle, and the like.
  • the total field of view of the fisheye device is greater than or equal to 360 degrees.
  • the panoramic composition map is pre-established and stored in the execution body, and the panorama synthesis map is used to synthesize the panoramic video.
  • the fisheye image collected by the fisheye device has a circular effective area, as shown in Fig. 2a, the half of the maximum length of the effective area in the horizontal direction and the vertical direction is the level of the fisheye image.
  • the focal length in the direction and vertical direction as shown in equation (1):
  • R xi is the horizontal focal length of the fisheye image acquired by the i-th fisheye camera
  • R yi is the vertical focal length of the fisheye image acquired by the i-th fisheye camera
  • W i is collected by the i-th fisheye camera
  • H i is the maximum length of the effective area of the fisheye image acquired by the i-th fisheye camera in the vertical direction.
  • R i is the projection radius of the fisheye image acquired by the i-th fisheye camera.
  • the center of the effective area of the fisheye image is the projection center
  • the angle between the pixel point (x i , y i ) on the fisheye image collected by the i-th fisheye camera and the radius of the fisheye is ⁇ i is:
  • the distance r i between the pixel point (x i , y i ) on the fisheye image collected by the i-th fisheye camera and the center of the fisheye image effective area collected by the i-th fisheye camera is:
  • the pitch angle ⁇ i of the spherical coordinate system corresponding to the fisheye image coordinate system acquired by the i-th fisheye camera is:
  • the spherical coordinates corresponding to the fisheye image coordinates acquired by the i-th fisheye camera are:
  • the fisheye image is cropped and the main point is offset.
  • FIG. 2b when the main point of the fisheye image is shifted, the main point of the fisheye image is no longer the center of the effective area of the fisheye image, and the focal length of the fisheye image cannot be calculated by the above conventional method;
  • FIG. 2c when the fisheye image is cropped, the effective area of the fisheye image is no longer a complete circle.
  • the circle detection by the conventional method will fail, and the accurate radius of the effective area of the fisheye image cannot be obtained.
  • Fig. 2d there is also a case where the fisheye image is both cropped and shifted, and in this case, it is less likely to obtain an accurate effective area radius and focal length.
  • the internal reference of the fisheye camera is used for spherical projection, which has higher precision and better adaptability.
  • the pre-calibrated external reference calibration image collected by the fisheye camera is processed to establish a panoramic synthesis mapping table, so that when the original video is collected, the original video can be directly spliced by using the panoramic synthesis mapping table, thereby improving the video splicing. effectiveness.
  • the manner in which the panoramic synthesis mapping table is established may include the following steps.
  • the calibration images of the external parameters collected by the two fisheye cameras and the external parameters of each fisheye camera calibrated in the calibration images of the external reference are obtained.
  • the outer edge of the effective area of each external reference calibration image is detected.
  • the outer edge of the effective area of each external reference calibration image is fitted by curve fitting, and the outer contour of the effective area of each external reference calibration image is obtained.
  • the internal parameters of each fisheye camera that collects the calibration images of the external parameters are determined.
  • each of the external reference calibration images is spherically projected, and each spherical projection image corresponding to each external reference calibration image is generated.
  • each spherical projection image is spherically expanded to obtain spherical expansion images corresponding to the spherical projection images.
  • the relative positional relationship between the spherical expansion images is obtained.
  • the spherical expansion images are processed according to the relative positional relationship to obtain a panoramic synthesis mapping table.
  • a process of generating a panoramic composition map is provided.
  • Each spherical image captured by the two fisheye cameras is projected and expanded by spherical projection to generate corresponding spherical expansion images, and the spherical expansion image is an undistorted image, and two spherical surfaces are obtained according to the external parameters of each fisheye camera.
  • the relative positional relationship between the images is expanded, and the two spherical expansion images are processed according to the relative positional relationship to obtain a panoramic composite mapping table.
  • the processing of the spherical expansion image may include transforming and cutting the two spherical expansion images;
  • the transformation of the two spherical expansion images may be: using one of the spherical expansion images as a reference, and transforming another spherical expansion image to achieve the purpose of seamless mosaic.
  • the process of establishing the panoramic synthesis mapping table may also be established according to an inverse process, establishing an empty panoramic mapping table, and establishing a correspondence between the mapping table and the calibration image by inverse transformation and pseudo-projection expansion.
  • the manner in which the panoramic synthesis mapping table is established may include the following steps.
  • the calibration images of the external parameters collected by the two fisheye cameras and the external parameters of each fisheye camera calibrated in the calibration images of the external reference are obtained.
  • the outer edge of the effective area of each external reference calibration image is detected.
  • the outer edge of the effective area of each external reference calibration image is fitted by curve fitting, and the outer contour of the effective area of each external reference calibration image is obtained.
  • the internal parameters of each fisheye camera that collects the calibration images of the external parameters are determined.
  • a first panoramic mapping table is established, wherein the content of the first panoramic mapping table is empty.
  • the relative positional relationship between each preset area in the first panoramic mapping table is obtained.
  • inverse transformation is performed on each preset area according to the relative positional relationship, and an inverse transformation area of each preset area is obtained.
  • each of the calibration images is spherically projected according to the internal reference of each fisheye camera, and each spherical projection image corresponding to each external reference calibration image is generated.
  • the spherical projection images are spherically developed to obtain spherical projection images corresponding to the respective spherical projection images.
  • the panoramic composition mapping table is determined according to the correspondence between each spherical expansion image and the inverse transformation region of each preset region.
  • the panoramic synthesis mapping table can be established by the above positive process or inverse process, and the process of establishing the panoramic synthesis mapping table in the positive process is shorter, and the accuracy of establishing the panoramic synthesis mapping table in the reverse process is higher.
  • the external reference calibration image is an image pre-acquired by each fisheye camera in the fisheye device based on the respective external reference when the panoramic composite mapping table is established, and the external reference calibration image may be a pre-photographed image, or may be based on each fisheye in advance. Any video frame, or multiple video frames, of the video samples collected by the respective external parameters of the camera; the external reference calibration image may also be extracted from the original video before the panoramic video is generated for the original video collected by each fisheye camera. Any video frame or multiple video frames.
  • the external reference of the fisheye camera is pre-calibrated in the external reference calibration image, for example, the installation position and installation height of the fisheye camera. Therefore, the external parameters of each fisheye camera can be obtained directly by the calibration images of the external parameters. Through the detection of the outer edge of the effective area, the outer contour of the effective area of the external reference calibration image can be obtained, and the internal reference of the fisheye camera can be determined according to the graphic formed by the outer contour.
  • the method for detecting the outer edge of the effective region of each external reference calibration image may be: detecting the outer edge of the effective region by recognizing the transition edge of the black and white region of the image; or searching for the outermost edge of the effective region for the straight line search manner At a distance, the outer edge of the effective area is obtained by connecting the four farthest points.
  • the manner of detecting the outer edge of the effective area belongs to the protection scope of the embodiment, and details are not described herein again.
  • the effective area of each external reference calibration image may be an elliptical area
  • the step of fitting the outer edge of the effective region of each external reference calibration image by curve fitting to obtain the outer contour of the effective region of each external reference calibration image may be specifically: Using the curve fitting, the outer edge of the effective area of each external reference calibration image is fitted, and an ellipse formed by the outer contour of the effective area of each external reference calibration image is obtained.
  • the step of collecting the internal parameters of each fisheye camera of each external reference calibration image is determined based on the outer contour of the effective region of the external reference calibration image, and specifically: the image is calibrated for each external reference, and the collection is determined based on the formed ellipse.
  • the internal reference of each fisheye camera of the external reference calibration image includes: horizontal focal length, vertical focal length and principal point coordinates.
  • the horizontal focal length is the long axis of the ellipse along the horizontal axis
  • the vertical focal length is the ellipse along the vertical axis.
  • the short axis of the direction, the coordinates of the principal point are the center coordinates of the ellipse.
  • the external reference calibrates the ellipse in the image and the ellipse in the external reference calibration image acquired by the second fisheye device shown in Figure 4b.
  • the ellipse area is the effective area of the external reference calibration image. Therefore, the long axis fx of the ellipse along the x direction (horizontal axis direction) is the horizontal focal length of the fisheye camera, and the short axis fy along the y direction (vertical axis direction) is the fisheye.
  • the focal length of the camera and the coordinates of the center of the ellipse are the coordinates of the point (xc, yc).
  • the panoramic composition mapping table includes a mapping table of two fisheye cameras, and there is an overlapping area with the same content between the mapping tables of the two fisheye cameras.
  • the panoramic composition mapping table structure is divided into two parts, a first fisheye camera mapping table 501 and a second fisheye camera mapping table 502, respectively, which are collected by two fisheye cameras.
  • the reference maps the mapping relationship of the image in the plane panorama.
  • the first fisheye camera mapping table 501 and the second fisheye camera mapping table 502 each have an overlapping area, representing an overlapping area of the two fisheye cameras on the field of view, and two fisheye cameras in the overlapping area on the field of view.
  • the content is the same.
  • the splicing relationship between the video to be spliced and the spliced video to be spliced corresponding to the original video may be generated for each original video by using a pre-established panoramic compositing mapping table.
  • each of the external reference calibration images is spherically projected according to an internal parameter of each fisheye camera, and corresponding to each external reference calibration image is generated.
  • the steps of projecting images on the spherical surface may be specifically: according to the horizontal focal length and the vertical focal length of each fisheye camera, using a preset projection radius calculation formula to determine the projection radius of each external reference calibration image; respectively, each fisheye camera
  • the coordinates of the principal point are the coordinates of the projection center.
  • the preset projection conversion formula is used to perform spherical projection on the calibration images of the external parameters collected by each fisheye camera to obtain the spherical projection expressions.
  • the spherical projection expression is used to generate the external parameters.
  • Each spherical projection image corresponding to the image is calibrated.
  • the spherical projection process of the embodiment of the present application is provided, and the spherical projection is performed by using the internal reference of the fisheye camera.
  • the preset projection radius calculation formula is as shown in the formula (7):
  • f xi is the horizontal focal length of the i-th fisheye camera
  • f yi is the vertical focal length of the i-th fisheye camera
  • R xi is the horizontal focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R Yi is the vertical focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera.
  • ⁇ i is the angle between the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera and the projection radius
  • r i is the external reference calibration image acquired by the i-th fisheye camera.
  • the distance between the pixel point (x i , y i ) and the principal point coordinate (C xi , C yi ) of the i-th fisheye camera, and ⁇ i is the spherical surface corresponding to the external reference calibration image coordinate system acquired by the i-th fisheye camera
  • the pitch angle of the coordinate system
  • the step of performing spherical expansion on each spherical projection image to obtain each spherical expansion image corresponding to each spherical projection image may be specifically:
  • Each of the spherical projection images is spherically expanded by using a preset spherical expansion formula, and each spherical expansion image corresponding to each spherical projection image is generated.
  • spherical projection and spherical expansion are to simulate a fisheye imaging model, and correct the distorted calibration image to an undistorted spherical expansion image for subsequent image mosaic.
  • (x zi , y zi ) is the spherical coordinate (u i , v i , w i ) corresponding to the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera in the spherical expansion image
  • the coordinate corresponding to ⁇ zi is the angle between the coordinate point (x zi , y zi ) and the vertical direction of the coordinate system
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera
  • R xi is The horizontal focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R yi is the vertical focal length of the external reference calibration image acquired by the i-th fisheye camera
  • ⁇ zi is collected by the i-th fisheye camera
  • the internal parameter is the only parameter that can truly reflect the main point offset and focal length change, and the internal parameters of the fisheye camera are used to establish
  • the spherical projection link in the process of panoramic synthesis of the mapping table will significantly improve the spherical projection effect and ensure the correction effect of the image after the spherical surface is expanded.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal reference of the fisheye camera calculated according to the external reference calibration image, for one two fisheye camera
  • the fisheye device with fixed relative position only needs to store the panoramic synthesis map.
  • the fisheye device collects the video the original video captured by the two fisheye cameras is searched using the panoramic synthesis map to obtain the table.
  • the video is spliced, and the spliced video is spliced to generate a panoramic video.
  • the splicing relationship is given in the splicing relationship, for example, the first column of pixels of the second video to be spliced overlaps with the twentieth column of the first video to be spliced, and the second video to be spliced is flipped 45 After the degree overlaps with the first video to be stitched and so on.
  • the video frames may be spliced, and the spliced video frames are video-combined to obtain a panoramic video.
  • the S102 may include: searching a pre-established panoramic synthesis mapping table, and generating a video frame to be spliced corresponding to each video frame in each original video and a video frame to be spliced Splicing relationship; for each video frame to be spliced generated by the same original video, a video to be spliced is generated by a preset video generation technology.
  • S103 may include: splicing corresponding video frames to be spliced in each video to be spliced according to a splicing relationship between the video frames to be spliced, and obtaining a plurality of spliced images; using a preset three-dimensional projection strategy, respectively Three-dimensional projection of each of the plurality of mosaic images is performed, and each three-dimensional panoramic image corresponding to each mosaic image is obtained; and based on each three-dimensional panoramic image, a panoramic video is generated by using a preset video generation technology.
  • the splicing relationship between the video frames to be spliced and the video frames to be spliced corresponding to each video frame in the original video can be obtained based on the panoramic compositing mapping table. Since the video is generated by the video frame, the preset video generation technology can be used. The video to be spliced is generated, and the preset video generation technology specifies the order of the video frames in the video and the playback speed when the video is generated.
  • the corresponding video frames to be spliced may be spliced to obtain a plurality of spliced images, and the spliced image is three-dimensionally projected by using a three-dimensional projection strategy, and the three-dimensional projection is performed by using a preset video generation technology.
  • the resulting 3D panorama is video converted to generate a panoramic video.
  • the generation of the panoramic video may also be a two-dimensional video generated by the splicing technique, and the two-dimensional video is three-dimensionally projected to obtain a panoramic video.
  • S103 may be specifically: splicing each video to be spliced according to a splicing relationship to generate a two-dimensional video; and performing three-dimensional projection of the two-dimensional video according to a preset three-dimensional projection strategy, Panoramic video.
  • the original video captured by the two fisheye cameras is used to look up the table by using the panoramic synthesis mapping table to obtain two videos to be stitched, and according to the stitching relationship, two videos to be stitched are stitched, and two are generated.
  • Dimensional video through spherical rendering of 2D video, can generate 3D spherical panoramic video.
  • the method of projecting onto the corresponding three-dimensional model can support a variety of three-dimensional panoramic display forms, and expand the display mode of the fisheye device.
  • the video frames captured by the two fisheye cameras can also be extracted, and the panoramic synthesis map is used to perform the table lookup to obtain two subgraphs to be stitched, and two to be The spliced subgraph is image spliced to obtain a two-dimensional panoramic image.
  • a two-dimensional panoramic video is generated, and the two-dimensional panoramic video is spherically rendered to generate a three-dimensional spherical panoramic video.
  • the 2D panoramic video can generate corresponding 3D spherical panoramic video through cylindrical projection, cube projection and other projection methods.
  • the two-dimensional panoramic image may be spherically projected to obtain a three-dimensional panoramic image, and a three-dimensional spherical panoramic video is generated by a video generation technique, which is not specifically limited herein.
  • the original video captured by the two fisheye cameras in the fisheye device is obtained, and the pre-established panoramic synthesis mapping table is searched, and each of the to-be-spliced videos corresponding to the original video and the stitching between the to-be-spliced videos are respectively generated. Relationship, according to the splicing relationship, splicing each video to be spliced to synthesize a panoramic video.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters. Processed.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • the embodiment of the present application further provides a panoramic video synthesizing device.
  • the panoramic video synthesizing device includes the following modules.
  • the obtaining module 910 is configured to acquire each original video captured by two fisheye cameras in the fisheye device.
  • the search module 920 is configured to search for a pre-established panoramic synthesis mapping table, and respectively generate a splicing relationship between each video to be spliced corresponding to each original video and the video to be spliced, and the panoramic synthesis mapping table is based on each fisheye
  • the external parameters of the fisheye camera calibrated in the calibration images of the external parameters collected by the camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters are respectively processed to obtain the calibration images of the external indexes.
  • the splicing module 930 is configured to splicing the video to be spliced according to the splicing relationship to synthesize a panoramic video.
  • the acquiring module 910 may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the external reference of the fisheye camera may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the external reference of the fisheye camera may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the external reference of the fisheye camera may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the external reference of the fisheye camera may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the external reference of the fisheye camera may be further configured to: acquire, for each of the external reference calibration images respectively collected
  • the device may further include: a detecting module, configured to detect an outer edge of an effective region of the external reference calibration image; and a curve fitting module, configured to fit an effective region of the external reference calibration image by using a curve fitting An outer edge, an outer contour of an effective area of the external reference calibration image is obtained; an internal parameter determining module, configured to determine, according to the outer contour of the effective area of the external reference, the respective images of the external reference calibration images are collected An internal reference of the fisheye camera; and a projection module, configured to perform spherical projection on each of the external reference calibration images according to the internal parameters of the fisheye cameras, to generate respective spherical projection images corresponding to the external reference calibration images; And a module for performing spherical expansion on the spherical projection images to obtain a spherical expansion image corresponding to each spherical projection image; and obtaining a module, configured to obtain the spherical expansion according to the external parameters of each of the fisheye cameras a relative positional relationship between the images
  • the acquiring module 910 may be further configured to: acquire, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration targets in the external reference calibration images.
  • the device may further include: a detecting module, configured to detect an outer edge of an effective region of the external reference calibration image; and a curve fitting module, configured to fit an effective region of the external reference calibration image by using a curve fitting An outer edge, an outer contour of an effective area of the external reference calibration image is obtained; an internal parameter determining module, configured to determine, according to the outer contour of the effective area of the external reference, the respective images of the external reference calibration images are collected An internal parameter of the fisheye camera; an establishing module, configured to establish a first panoramic mapping table, wherein the content of the first panoramic mapping table is empty; and an obtaining module, configured to obtain the first according to an external parameter of each fisheye camera An inverse positional relationship between each preset area in a panoramic mapping table; an inverse transform module, configured to perform inverse transform on each preset area according to the relative positional relationship, to obtain an inverse transformation of each preset area a projection unit, configured to perform spherical projection on the external reference calibration images according to the internal parameters of the fisheye cameras, and
  • the effective area may be an elliptical area.
  • the curve fitting module may be specifically configured to: fit the outer edge of the effective region of the external reference calibration image by curve fitting, and obtain an ellipse formed by the outer contour of the effective region of the external reference calibration image.
  • the internal parameter determining module may be configured to: calibrate an image for each of the external parameters, and determine, according to the formed ellipse, an internal parameter of each fisheye camera that collects the calibration images of the external parameters, where the internal reference includes: a horizontal direction a focal length, a vertical direction focal length, and a principal point coordinate, wherein the horizontal direction focal length is a long axis of the ellipse along a horizontal axis direction, and the vertical direction focal length is a short axis of the ellipse along a longitudinal axis direction, the main point The coordinates are the center coordinates of the ellipse.
  • the projection module may be specifically configured to: determine, according to a horizontal focal length and a vertical focal length of each fisheye camera, using a preset projection radius calculation formula a projection radius of each external reference calibration image; respectively, using the coordinates of the main point of each fisheye camera as a projection center coordinate, and performing a spherical projection on each of the external reference calibration images collected by each fisheye camera by using a preset projection conversion formula Obtaining each spherical projection expression; using each of the spherical projection expressions, generating each spherical projection image corresponding to each of the external reference calibration images.
  • the preset projection radius calculation formula may be:
  • f xi is the horizontal focal length of the i-th fisheye camera
  • f yi is the vertical focal length of the i-th fisheye camera
  • R xi is the horizontal focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R yi is the vertical direction focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera.
  • the preset projection conversion formula may be:
  • ⁇ i is the angle between the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera and the projection radius
  • r i is the external reference calibration image acquired by the i-th fisheye camera.
  • the distance between the pixel point (x i , y i ) and the principal point coordinate (C xi , C yi ) of the i-th fisheye camera, and ⁇ i is the spherical surface corresponding to the external reference calibration image coordinate system acquired by the i-th fisheye camera The elevation angle of the coordinate system.
  • the spherical projection expression can be:
  • the expansion module may be specifically configured to: perform spherical expansion on the spherical projection images by using a preset spherical expansion formula, and generate each corresponding to the spherical projection images.
  • the spherical expansion image, the preset spherical expansion formula may be:
  • (x zi , y zi ) is the spherical coordinate (u i , v i , w i ) corresponding to the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera in the spherical expansion image
  • the coordinate corresponding to ⁇ zi is the angle between the coordinate point (x zi , y zi ) and the vertical direction of the coordinate system
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera
  • R xi is The horizontal focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R yi is the vertical focal length of the external reference calibration image acquired by the i-th fisheye camera
  • ⁇ zi is collected by the i-th fisheye camera
  • the panoramic composition mapping table may include a mapping table of each of the two fisheye cameras, and an overlapping area with the same content exists between the mapping tables of the two fisheye cameras. .
  • the searching module 920 may be specifically configured to: search a pre-established panoramic composite mapping table, and generate each video frame to be stitched corresponding to each video frame in each original video.
  • the splicing relationship between the video frames to be spliced is described; for each video frame to be spliced generated by the same original video, the video to be spliced is generated by using a preset video generation technology.
  • the splicing module 930 may be configured to: splicing corresponding video frames to be spliced in the video to be spliced according to the splicing relationship between the video frames to be spliced, to obtain multiple spliced images; Using a preset three-dimensional projection strategy, three-dimensionally projecting each of the plurality of mosaic images to obtain a three-dimensional panoramic image corresponding to each of the mosaic images; and using the preset video generation technology based on the three-dimensional panoramic images , generate a panoramic video.
  • the splicing module 930 may be specifically configured to: splicing the video to be spliced according to the splicing relationship to generate a two-dimensional video;
  • the three-dimensional video is three-dimensionally projected to obtain a panoramic video.
  • the original video captured by the two fisheye cameras in the fisheye device is obtained, and the pre-established panoramic synthesis mapping table is searched, and each of the to-be-spliced videos corresponding to the original video and the stitching between the to-be-spliced videos are respectively generated. Relationship, according to the splicing relationship, splicing each video to be spliced to synthesize a panoramic video.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters. Processed.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • the embodiment of the present application provides an electronic device, as shown in FIG. 10, including a processor 1001 and a memory 1002.
  • the memory 1002 is configured to store a computer program.
  • the processor 1001 is configured to execute a computer program stored on the memory 1002. , to achieve the following steps:
  • the panoramic composition mapping table is based on the external parameters of the fisheye camera calibrated in each of the external reference calibration images collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively
  • the external reference calibration image is processed, and the video to be spliced is spliced according to the splicing relationship to synthesize the panoramic video.
  • the processor 1001 may further implement the following steps: acquiring, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration of each of the external reference calibration images.
  • An outer reference of the fisheye camera detecting an outer edge of the effective region of the external reference calibration image; fitting the outer edge of the effective region of the external reference calibration image by curve fitting to obtain the calibration image of the external reference An outer contour of the effective area; determining an internal reference of each fisheye camera that collects the external reference calibration image based on an outer contour of the effective area of the external reference calibration image; according to the internal reference of each fisheye camera
  • Each of the external reference calibration images is spherically projected to generate respective spherical projection images corresponding to the external reference calibration images; and the spherical projection images are spherically expanded to obtain spherical projection images corresponding to the spherical projection images.
  • the processor 1001 may further implement the following steps: acquiring, for each of the external reference calibration images respectively collected by the two fisheye cameras, and the calibration of each of the external reference calibration images.
  • An outer reference of the fisheye camera detecting an outer edge of the effective region of the external reference calibration image; fitting the outer edge of the effective region of the external reference calibration image by curve fitting to obtain the calibration image of the external reference An outer contour of the effective area; determining an internal parameter of each fisheye camera that collects the external reference calibration image based on an outer contour of the effective area of the external reference calibration image; establishing a first panoramic mapping table, the first panoramic view The content of the mapping table is empty; according to the external parameters of the fisheye cameras, the relative positional relationship between each preset area in the first panoramic mapping table is obtained; according to the relative positional relationship, the pre-preparing Performing an inverse transformation on the region to obtain an inverse transformation region of each of the preset regions; and performing spherical projection on the calibration images of the external parameters according to the
  • the effective area is an elliptical area.
  • the processor 1001 can implement the step of using the curve fitting to fit the outer edge of the effective region of the external reference calibration image to obtain the outer contour of the effective region of the external reference calibration image.
  • the following steps are: fitting the outer edge of the effective region of the external reference calibration image by curve fitting to obtain an ellipse formed by the outer contour of the effective region of each external reference calibration image.
  • the processor 1001 when implementing the step of determining the internal parameters of each fisheye camera of each external reference calibration image, by implementing the outer contour of the effective region of the external reference calibration image, may specifically implement the following steps: And determining, according to the formed ellipse, an internal parameter of each fisheye camera that collects the calibration images of the external parameters, where the internal parameters include: a horizontal focal length, a vertical focal length, and a principal point coordinate,
  • the horizontal direction focal length is a long axis of the ellipse along a horizontal axis direction
  • the vertical direction focal length is a short axis of the ellipse along a longitudinal axis direction
  • the principal point coordinate is a center coordinate of the ellipse.
  • the processor 1001 performs spherical projection on the external reference calibration images according to the internal parameters of the fisheye cameras to generate the external parameters.
  • the following steps may be specifically implemented: determining, according to the horizontal focal length and the vertical focal length of each fisheye camera, using a preset projection radius calculation formula to determine the external reference calibration a projection radius of the image; respectively, using the coordinates of the main point of each fisheye camera as a projection center coordinate, and performing a spherical projection on each of the external reference calibration images collected by the fisheye cameras by using a preset projection conversion formula to obtain each spherical surface a projection expression; each of the spherical projection images corresponding to each of the external reference calibration images is generated by using the respective spherical projection expressions.
  • the preset projection radius calculation formula may be:
  • f xi is the horizontal focal length of the i-th fisheye camera
  • f yi is the vertical focal length of the i-th fisheye camera
  • R xi is the horizontal focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R yi is the vertical direction focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera.
  • the preset projection conversion formula may be:
  • ⁇ i is the angle between the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera and the projection radius
  • r i is the external reference calibration image acquired by the i-th fisheye camera.
  • the distance between the pixel point (x i , y i ) and the principal point coordinate (C xi , C yi ) of the i-th fisheye camera, and ⁇ i is the spherical surface corresponding to the external reference calibration image coordinate system acquired by the i-th fisheye camera The elevation angle of the coordinate system.
  • the spherical projection expression can be:
  • the processor 1001 when the processor 1001 performs the spherical expansion of the spherical projection images to obtain the spherical expansion images corresponding to the spherical projection images, The following steps may be implemented: performing a spherical expansion on the spherical projection images by using a preset spherical expansion formula to generate respective spherical expansion images corresponding to the spherical projection images, and the preset spherical expansion formula is:
  • (x zi , y z i) is the spherical coordinate (u i , v i , w i ) corresponding to the pixel point (x i , y i ) on the external reference calibration image acquired by the i-th fisheye camera.
  • ⁇ zi is the angle between the coordinate point (x zi , y zi ) and the vertical direction of the coordinate system
  • R i is the projection radius of the external reference calibration image acquired by the i-th fisheye camera
  • R xi The horizontal direction focal length of the external reference calibration image acquired by the i-th fisheye camera
  • R yi is the vertical focal length of the external reference calibration image acquired by the i-th fisheye camera
  • ⁇ zi is collected in the i-th fisheye camera
  • the panoramic composition mapping table includes a mapping table of each of the two fisheye cameras, and an overlapping area with the same content exists between the mapping tables of the two fisheye cameras.
  • the processor 1001 is configured to implement the search for a pre-established panoramic synthesis mapping table, and respectively generate a video to be stitched corresponding to each original video and a video between the to-be-spliced videos.
  • the step of splicing the relationship the following steps may be implemented: searching for a pre-established panoramic synthesis mapping table, and generating a splicing relationship between each video frame to be spliced corresponding to each video frame in each original video and the video frames to be spliced
  • a video to be spliced is generated by a preset video generation technology.
  • the processor 1001 when implementing the step of splicing the video to be spliced according to the splicing relationship and synthesizing the panoramic video, may specifically implement the following steps: according to the splicing between the video frames to be spliced And splicing the corresponding video frames to be spliced in the video to be spliced to obtain a plurality of spliced images; respectively, performing three-dimensional projection on each spliced image in the plurality of spliced images by using a preset three-dimensional projection strategy Obtaining each three-dimensional panoramic image corresponding to each of the mosaic images; and generating a panoramic video by using a preset video generation technology based on the three-dimensional panoramic images.
  • the processor 1001 when the processor 1001 implements the step of splicing the video to be spliced according to the splicing relationship and synthesizing the panoramic video, the following steps may be specifically implemented: According to the splicing relationship, the video to be spliced is spliced to generate a two-dimensional video; and the two-dimensional video is three-dimensionally projected according to a preset three-dimensional projection strategy to obtain a panoramic video.
  • Data transmission between the memory 1002 and the processor 1001 may be performed by means of a wired connection or a wireless connection, and the computer device may communicate with other devices through a wired communication interface or a wireless communication interface. It should be noted that only an example of transmitting data between the processor 1001 and the memory 1002 via the bus is shown in FIG. 10, and is not limited to a specific transmission mode.
  • the above memory may include a RAM (Random Access Memory), and may also include an NVM (Non-volatile Memory), such as at least one disk storage.
  • the memory may also be at least one storage device located away from the processor.
  • the processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processor) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processor
  • ASIC Application) Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • other programmable logic device discrete gate or transistor logic device, discrete hardware components.
  • the processor of the electronic device reads and runs the computer program stored in the memory, and the computer program executes the panoramic video synthesis method provided by the embodiment of the present application at runtime, thereby enabling: obtaining the fisheye
  • the original video captured by the two fisheye cameras in the device is searched for a pre-established panoramic composite mapping table, and the stitching relationship between each video to be stitched and the video to be stitched corresponding to each original video is generated respectively, and each stitching relationship is determined according to the stitching relationship. Splicing the video for splicing and synthesizing the panoramic video.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters. Processed.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • the embodiment of the present application provides a computer readable storage medium for storing a computer program, which is used to execute at runtime: the embodiment of the present application The panoramic video synthesis method provided.
  • the computer readable storage medium stores a computer program that executes the panoramic video synthesis method provided by the embodiment of the present application at runtime, and thus can realize: obtaining the original respectively captured by two fisheye cameras in the fisheye device.
  • the video is searched for a pre-established panoramic composite mapping table, and each of the to-be-spliced videos corresponding to the original video and the splicing relationship between the to-be-spliced videos are respectively generated, and the video to be spliced is spliced according to the splicing relationship to synthesize the panoramic video.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters. Processed.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • the embodiment of the present application provides an application program for performing the panoramic video synthesis method provided by the embodiment of the present application.
  • the application performs the panoramic video synthesis method provided by the embodiment of the present application at runtime, so that the original video captured by the two fisheye cameras in the fisheye device can be obtained, and the pre-established panoramic synthesis can be found.
  • the mapping table respectively generates a splicing relationship between each video to be spliced corresponding to each original video and each video to be spliced, and splicing the spliced videos according to the splicing relationship to synthesize the panoramic video.
  • the panoramic synthesis mapping table is based on the external parameters of the fisheye camera calibrated in the external reference calibration image collected by each fisheye camera, and the internal parameters of the fisheye camera calculated according to the calibration images of the external parameters, respectively, and respectively performing calibration images of the external parameters.
  • the video splicing can be completed according to the panoramic synthesis mapping table, and the external parameters of each fisheye camera are calibrated in the collected external reference calibration image, and the internal parameters of each fisheye camera can be calculated according to the external reference calibration image. Only one external calibration is required, and the internal and external parameters of each fisheye camera can be obtained. It is not necessary to separately configure the calibration auxiliary equipment for the internal and external parameters of each fisheye camera, thereby reducing the hardware cost of panoramic video synthesis and simplifying The calibration process further improves the efficiency of panoramic video synthesis.
  • the computer readable storage medium and the application embodiment since the method content involved is basically similar to the foregoing method embodiment, the description is relatively simple, and the relevant part of the method embodiment is referred to can.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne, selon un mode de réalisation, un procédé et un appareil permettant de synthétiser une vidéo panoramique, ainsi qu'un dispositif électronique. Le procédé permettant de synthétiser une vidéo panoramique comprend les étapes consistant : à acquérir chaque vidéo d'origine capturée respectivement au moyen de deux caméras très grand angulaire dans un dispositif à double très grand angulaire ; à rechercher une table de mappage de synthèse panoramique préétablie et à générer chaque vidéo à monter correspondant à chaque vidéo d'origine et une relation de montage entre des vidéos à monter, la table de mappage de synthèse panoramique étant obtenue par traitement respectif de chaque image d'étalonnage de paramètre externe sur la base d'un paramètre externe des caméras très grand angulaire étalonnées dans les images d'étalonnage de paramètre externe collectées par chaque caméra très grand angulaire et d'un paramètre interne des caméras très grand angulaire calculé selon chaque image d'étalonnage de paramètre externe ; et selon la relation de montage, à monter chaque vidéo à monter et à synthétiser la vidéo panoramique. La solution selon la présente invention permet de réduire le coût matériel de la synthèse de la vidéo panoramique et d'améliorer l'efficacité de la synthèse de la vidéo panoramique.
PCT/CN2019/079726 2018-04-02 2019-03-26 Procédé et appareil permettant de synthétiser une vidéo panoramique, et dispositif électronique WO2019192358A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810284361.1A CN110351494B (zh) 2018-04-02 2018-04-02 一种全景视频合成方法、装置及电子设备
CN201810284361.1 2018-04-02

Publications (1)

Publication Number Publication Date
WO2019192358A1 true WO2019192358A1 (fr) 2019-10-10

Family

ID=68099910

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079726 WO2019192358A1 (fr) 2018-04-02 2019-03-26 Procédé et appareil permettant de synthétiser une vidéo panoramique, et dispositif électronique

Country Status (2)

Country Link
CN (1) CN110351494B (fr)
WO (1) WO2019192358A1 (fr)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111161142A (zh) * 2019-11-29 2020-05-15 西安思后网络科技有限公司 一种监控设备多角度全景图像处理方法
CN111311693A (zh) * 2020-03-16 2020-06-19 威海经济技术开发区天智创新技术研究院 一种多目摄像头的在线标定方法及系统
CN111429382A (zh) * 2020-04-10 2020-07-17 浙江大华技术股份有限公司 一种全景图像矫正方法、装置以及计算机存储介质
CN111968184A (zh) * 2020-08-24 2020-11-20 北京茵沃汽车科技有限公司 一种在全景环视系统中实现视图随动的方法、装置及介质
CN112581369A (zh) * 2020-12-24 2021-03-30 中国银联股份有限公司 图像拼接方法以及装置
CN113256742A (zh) * 2021-07-15 2021-08-13 禾多科技（北京）有限公司 界面展示方法、装置、电子设备和计算机可读介质
CN113674363A (zh) * 2021-08-26 2021-11-19 龙岩学院 一种全景泊车影像拼接标定方法及其标定物
CN113781373A (zh) * 2021-08-26 2021-12-10 云从科技集团股份有限公司 图像融合方法、设备及计算机存储介质
CN114399557A (zh) * 2021-12-25 2022-04-26 深圳市全景达科技有限公司 一种全景标定方法、系统及计算机设备
CN115396644A (zh) * 2022-07-21 2022-11-25 贝壳找房(北京)科技有限公司 基于多段外参数据的视频融合方法及装置
CN115514751A (zh) * 2022-08-24 2022-12-23 网易（杭州）网络有限公司 用于挖掘机远程控制的图像获取方法及远程控制系统
CN116245748A (zh) * 2022-12-23 2023-06-09 珠海视熙科技有限公司 环视镜头的畸变校正方法、装置、设备、系统及存储介质
CN117201705A (zh) * 2023-11-07 2023-12-08 天津云圣智能科技有限责任公司 一种全景图像的获取方法、装置、电子设备及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110992428B (zh) * 2019-10-29 2024-04-05 惠州市德赛西威智能交通技术研究院有限公司 一种摄像头一致性的智能检测方法和检测装置
CN112884767B (zh) * 2021-03-26 2022-04-26 长鑫存储技术有限公司 图像拟合方法
CN113873259A (zh) * 2021-09-17 2021-12-31 影石创新科技股份有限公司 全景视频插帧方法、装置及计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103617606A (zh) * 2013-11-26 2014-03-05 中科院微电子研究所昆山分所 用于辅助驾驶的车辆多视角全景生成方法
US20170127045A1 (en) * 2015-10-28 2017-05-04 Toppano Co., Ltd. Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof
CN106846409A (zh) * 2016-10-28 2017-06-13 北京鑫洋泉电子科技有限公司 鱼眼相机的标定方法及装置
CN106875339A (zh) * 2017-02-22 2017-06-20 长沙全度影像科技有限公司 一种基于长条形标定板的鱼眼图像拼接方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103136720A (zh) * 2013-03-12 2013-06-05 中科院微电子研究所昆山分所 车载360度全景拼接方法
CN103871071B (zh) * 2014-04-08 2018-04-24 北京经纬恒润科技有限公司 一种用于全景泊车系统的摄像头外参标定方法
CN104240236B (zh) * 2014-08-26 2017-10-24 中山大学 一种鱼眼镜头标定后鱼眼图像校正的方法
CN104240258A (zh) * 2014-09-30 2014-12-24 苏州智华汽车电子有限公司 一种基于车联网的全景环视系统标定方法、装置及系统
CN105678787A (zh) * 2016-02-03 2016-06-15 西南交通大学 一种基于双目鱼眼摄像头的载重货车行驶障碍物检测及跟踪方法
CN106683045A (zh) * 2016-09-28 2017-05-17 深圳市优象计算技术有限公司 一种基于双目像机的全景图像拼接方法
CN106952219B (zh) * 2017-03-14 2020-11-06 成都通甲优博科技有限责任公司 一种基于外参数修正鱼眼像机的图像生成方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103617606A (zh) * 2013-11-26 2014-03-05 中科院微电子研究所昆山分所 用于辅助驾驶的车辆多视角全景生成方法
US20170127045A1 (en) * 2015-10-28 2017-05-04 Toppano Co., Ltd. Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof
CN106846409A (zh) * 2016-10-28 2017-06-13 北京鑫洋泉电子科技有限公司 鱼眼相机的标定方法及装置
CN106875339A (zh) * 2017-02-22 2017-06-20 长沙全度影像科技有限公司 一种基于长条形标定板的鱼眼图像拼接方法

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111161142A (zh) * 2019-11-29 2020-05-15 西安思后网络科技有限公司 一种监控设备多角度全景图像处理方法
CN111311693A (zh) * 2020-03-16 2020-06-19 威海经济技术开发区天智创新技术研究院 一种多目摄像头的在线标定方法及系统
CN111311693B (zh) * 2020-03-16 2023-11-14 威海经济技术开发区天智创新技术研究院 一种多目摄像头的在线标定方法及系统
CN111429382A (zh) * 2020-04-10 2020-07-17 浙江大华技术股份有限公司 一种全景图像矫正方法、装置以及计算机存储介质
CN111429382B (zh) * 2020-04-10 2024-01-19 浙江大华技术股份有限公司 一种全景图像矫正方法、装置以及计算机存储介质
CN111968184A (zh) * 2020-08-24 2020-11-20 北京茵沃汽车科技有限公司 一种在全景环视系统中实现视图随动的方法、装置及介质
CN111968184B (zh) * 2020-08-24 2024-04-02 北京茵沃汽车科技有限公司 一种在全景环视系统中实现视图随动的方法、装置及介质
CN112581369A (zh) * 2020-12-24 2021-03-30 中国银联股份有限公司 图像拼接方法以及装置
CN113256742A (zh) * 2021-07-15 2021-08-13 禾多科技（北京）有限公司 界面展示方法、装置、电子设备和计算机可读介质
CN113674363A (zh) * 2021-08-26 2021-11-19 龙岩学院 一种全景泊车影像拼接标定方法及其标定物
CN113674363B (zh) * 2021-08-26 2023-05-30 龙岩学院 一种全景泊车影像拼接标定方法及其标定物
CN113781373A (zh) * 2021-08-26 2021-12-10 云从科技集团股份有限公司 图像融合方法、设备及计算机存储介质
CN114399557A (zh) * 2021-12-25 2022-04-26 深圳市全景达科技有限公司 一种全景标定方法、系统及计算机设备
CN115396644A (zh) * 2022-07-21 2022-11-25 贝壳找房(北京)科技有限公司 基于多段外参数据的视频融合方法及装置
CN115396644B (zh) * 2022-07-21 2023-09-15 贝壳找房(北京)科技有限公司 基于多段外参数据的视频融合方法及装置
CN115514751A (zh) * 2022-08-24 2022-12-23 网易（杭州）网络有限公司 用于挖掘机远程控制的图像获取方法及远程控制系统
CN116245748A (zh) * 2022-12-23 2023-06-09 珠海视熙科技有限公司 环视镜头的畸变校正方法、装置、设备、系统及存储介质
CN116245748B (zh) * 2022-12-23 2024-04-26 珠海视熙科技有限公司 环视镜头的畸变校正方法、装置、设备、系统及存储介质
CN117201705A (zh) * 2023-11-07 2023-12-08 天津云圣智能科技有限责任公司 一种全景图像的获取方法、装置、电子设备及存储介质
CN117201705B (zh) * 2023-11-07 2024-02-02 天津云圣智能科技有限责任公司 一种全景图像的获取方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN110351494A (zh) 2019-10-18
CN110351494B (zh) 2020-06-26

Similar Documents

Publication Publication Date Title
WO2019192358A1 (fr) Procédé et appareil permettant de synthétiser une vidéo panoramique, et dispositif électronique
US11012620B2 (en) Panoramic image generation method and device
JP6721065B2 (ja) 撮像装置、画像処理装置および方法
KR101666959B1 (ko) 카메라로부터 획득한 영상에 대한 자동보정기능을 구비한 영상처리장치 및 그 방법
WO2018214365A1 (fr) Procédé, appareil, dispositif et système de correction d'image, dispositif de prise de vues et dispositif d'affichage
EP2328125B1 (fr) Procédé et dispositif de raccordement d'images
WO2018068719A1 (fr) Procédé et appareil de collage d'image
CN106485753B (zh) 用于无人驾驶汽车的摄像机标定的方法和装置
CN103942754B (zh) 全景图像补全方法及装置
WO2023045147A1 (fr) Procédé et système d'étalonnage d'une caméra binoculaire, dispositif électronique et support d'enregistrement
JPWO2018235163A1 (ja) キャリブレーション装置、キャリブレーション用チャート、チャートパターン生成装置、およびキャリブレーション方法
CN110519528B (zh) 一种全景视频合成方法、装置及电子设备
JP2001134751A (ja) 画像処理方法及び装置、デジタルカメラ、画像処理システム、並びに、画像処理プログラムを記録した記録媒体
TWI587241B (zh) Method, device and system for generating two - dimensional floor plan
CN102096923A (zh) 鱼眼标定方法和装置
US10482571B2 (en) Dual fisheye, hemispherical image projection and stitching method, device and computer-readable medium
US20200294269A1 (en) Calibrating cameras and computing point projections using non-central camera model involving axial viewpoint shift
WO2021004416A1 (fr) Procédé et appareil permettant d'établir une carte de balises sur la base de balises visuelles
WO2020151268A1 (fr) Procédé de génération pour carte dynamique astéroïde 3d et terminal portable
WO2021208486A1 (fr) Procédé de transformation de coordonnées de caméra, terminal et support de stockage
CN111383264A (zh) 一种定位方法、装置、终端及计算机存储介质
CN103489165A (zh) 一种面向视频拼接的小数查找表生成方法
WO2022011560A1 (fr) Procédé et appareil de recadrage d'image, dispositif électronique et support de stockage
JP2006338167A (ja) 画像データ作成方法
CN109089100B (zh) 一种双目立体视频的合成方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19782346

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19782346

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19782346

Country of ref document: EP

Kind code of ref document: A1