CN114007014A - Method and device for generating panoramic image, electronic equipment and storage medium - Google Patents
Method and device for generating panoramic image, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114007014A CN114007014A CN202111272121.8A CN202111272121A CN114007014A CN 114007014 A CN114007014 A CN 114007014A CN 202111272121 A CN202111272121 A CN 202111272121A CN 114007014 A CN114007014 A CN 114007014A
- Authority
- CN
- China
- Prior art keywords
- angle
- image
- panoramic image
- background
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Abstract
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for generating a panoramic image, an electronic device, and a storage medium. The method comprises the steps of driving a camera to rotate at a constant speed by controlling a cloud deck according to a preset rotating speed so as to shoot a panoramic video within a preset angle range, enabling the interval angle of each adjacent frame in the panoramic video to be the same, extracting key frames within a preset interval to serve as background images, obtaining a universal translation transformation matrix only by performing translation transformation once, transforming and splicing the background images at all angle positions by utilizing the translation transformation matrix, and outputting a target panoramic image. This application has greatly reduced the calculation volume of panoramic picture concatenation process through using at the uniform velocity pivoted cloud platform camera, has shortened the concatenation time to promote the convenient impression that the user acquireed panoramic picture.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for generating a panoramic image, an electronic device, and a storage medium.
Background
In the prior art, a camera is mostly used to sweep a target area in sequence, images in the target area are obtained by a shooting method one by one, and then the images are spliced by professional software such as Photoshop to generate a panoramic image. However, this method of generating a panoramic image is computationally complex and requires a long stitching time.
Therefore, there is a need for a method, an apparatus, an electronic device and a storage medium for generating a panoramic image to solve the above problems.
Disclosure of Invention
The application provides a method, a device, an electronic device and a storage medium for generating a panoramic image, which can reduce the calculation amount of the panoramic image splicing process and shorten the splicing time, thereby improving the convenient feeling of a user for acquiring the panoramic image.
In a first aspect, an embodiment of the present application provides a method for generating a panoramic image, including:
responding to the connection to a cloud deck, sending a shooting instruction carrying shooting angle parameters to the cloud deck so as to drive a camera to rotate at a constant speed by using the cloud deck according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
acquiring a first video acquired by the camera in real time;
taking the key frame of each preset time interval in the first video as a background image of each preset angle interval to obtain background images of a plurality of angle positions; wherein the preset angle interval can be evenly divided by the difference between the ending shooting angle and the starting shooting angle;
and transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a target panoramic image.
In a possible design, before the sending the shooting instruction carrying the shooting angle parameter to the pan/tilt head, the method further includes:
and connecting the cradle head according to the IP address, the port, the user name and the password input by the user.
In one possible design, after the obtaining the background images at the multiple angular positions and before the transforming and stitching the background images at all angular positions by using the same translation transformation matrix, the method further includes:
judging whether the number of the background images reaches a preset number or not; the preset number is the quotient of the difference value between the ending shooting angle and the starting shooting angle and the preset angle interval;
if yes, executing the transformation and splicing of the background images at all the angle positions by using the same translation transformation matrix;
and if not, executing the shooting instruction carrying the shooting angle parameter to the holder.
In one possible design, the transforming and stitching the background images at all the angular positions by using the same translation transformation matrix to output the target panoramic image includes:
transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image;
judging whether the panoramic image to be determined is in a splicing success state or not;
if so, taking the to-be-determined panoramic image as a target panoramic image;
and if not, processing the unqualified background image in the panoramic image to be determined so as to output the target panoramic image.
In one possible design, the transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image includes:
for the background image of the first angular position, performing: extracting the background key points in the first background image; matching the background key points at the adjacent positions in the first background image to estimate a translation transformation matrix by using the matched background key points;
and transforming and splicing the background images at all the angle positions by using the translation transformation matrix, and outputting the to-be-determined panoramic image.
In one possible design, the determining whether the to-be-determined panoramic image is in a successful stitching state includes:
detecting whether a background image of a moving object exists in the to-be-determined panoramic image;
if so, determining that the to-be-determined panoramic image is not in a splicing success state;
and if not, determining that the to-be-determined panoramic image is in a splicing success state.
In one possible design, the processing the unqualified background image in the to-be-determined panoramic image to output a target panoramic image includes:
determining a target angle position corresponding to the unqualified background image in the to-be-determined panoramic image;
sending the target angular position to the holder to rotate to the target angular position by using the holder;
acquiring a second video with preset time length, which is acquired by the camera at the target angle position;
taking each frame of image in the second video as the input of a preset background extraction algorithm, and separating a moving foreground from each frame of image;
obtaining a qualified background image corresponding to the target angle position based on the multi-frame image of the separated moving foreground;
and replacing the qualified background image and the unqualified background image, and performing transformation and splicing on the background images at all angle positions by using the same translation transformation matrix.
In a second aspect, an embodiment of the present application provides an apparatus for generating a panoramic image, including:
the control module is used for being connected to the cloud deck and sending a shooting instruction carrying shooting angle parameters to the cloud deck so as to drive the camera to rotate at a constant speed by using the cloud deck according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
the transmission module is used for acquiring a first video acquired by the camera in real time;
the extraction module is used for taking the key frame of each preset time interval in the first video as the background image of each preset angle interval to obtain background images of a plurality of angle positions; wherein the preset angle interval can be evenly divided by the difference between the ending shooting angle and the starting shooting angle;
and the processing module is used for transforming and splicing the background images at all the angle positions by using the same translation transformation matrix and outputting a target panoramic image.
In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor, where the memory stores a computer program, and the processor, when executing the computer program, implements the method described in any one of the above.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed in a computer, the computer program causes the computer to execute any one of the methods described above.
The embodiment provides a method, a device, an electronic device and a storage medium for generating a panoramic image, wherein a camera is driven to rotate at a constant speed by controlling a holder according to a preset rotating speed, a panoramic video within a preset angle range is shot, the interval angle of each adjacent frame in the panoramic video is the same, so that a key frame within a preset interval is extracted as a background image, a universal translation transformation matrix can be obtained only by performing translation transformation once, then the background images at all angle positions are transformed and spliced through the same translation transformation matrix, and a target panoramic image is output; and if the splicing fails due to the occurrence of a moving object in the background image at a certain angle in the splicing process, performing background extraction on the image at the angle position and then splicing.
Therefore, the pan-tilt camera rotating at a constant speed is used, so that the calculation amount of the panoramic image splicing process is greatly reduced, the splicing time is shortened, and the convenience of acquiring the panoramic image by a user is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic diagram of a method for generating a panoramic image according to an embodiment of the present invention;
FIG. 2 is a diagram of a hardware architecture of a computing device according to an embodiment of the present invention;
fig. 3 is a block diagram of an apparatus for generating a panoramic image according to an embodiment of the present invention.
Detailed Description
The present application will be described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In the description of the embodiments of the present application, the terms "first", "second", and the like, unless expressly specified or limited otherwise, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance; the term "plurality" means two or more unless specified or indicated otherwise; the terms "connected," "fixed," and the like are to be construed broadly and may, for example, be fixedly connected, detachably connected, integrally connected, or electrically connected; may be directly connected or indirectly connected through an intermediate. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
In the description of the present application, it should be understood that the terms "upper" and "lower" used in the description of the embodiments of the present application are used in a descriptive sense only and not for purposes of limitation. In addition, in this context, it will also be understood that when an element is referred to as being "on" or "under" another element, it can be directly on "or" under "the other element or be indirectly on" or "under" the other element via an intermediate element.
In the prior art, a camera is mostly used to sweep a target area in sequence, images in the target area are obtained by a shooting method one by one, and then the images are spliced by professional software such as Photoshop to generate a panoramic image. The method cannot ensure that the angle difference and the rotating shaft between any two adjacent images are the same, so that when the images are spliced, operations of key point extraction, key point matching, homography transformation matrix calculation and the like need to be repeated, the calculation is complex, and the splicing processing time is long.
In order to solve the technical problem, the cloud deck adopting the preset rotating speed can be considered to drive the camera to rotate at a constant speed so as to obtain background images at the same interval, so that the panoramic images are spliced by using the same translation transformation matrix by utilizing the same characteristic of the rotating shaft of the cloud deck, the splicing process is simplified, the splicing time is shortened, and the convenient feeling of acquiring the panoramic images by a user is further improved.
As shown in fig. 1, an embodiment of the present invention provides a method for generating a panoramic image, including the following steps:
step 100: responding to the connection to the cloud deck, sending a shooting instruction carrying shooting angle parameters to the cloud deck so as to drive the camera to rotate at a constant speed by using the cloud deck according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
step 102: acquiring a first video acquired by a camera in real time;
step 104: taking the key frame of each preset time interval in the first video as a background image of each preset angle interval to obtain background images of a plurality of angle positions; wherein the preset angle interval can be evenly divided by the difference value of the ending shooting angle and the starting shooting angle;
step 106: and transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a target panoramic image.
In the embodiment of the invention, the camera is driven to rotate at a constant speed by controlling the pan-tilt head according to a preset rotating speed, the panoramic video within a preset angle range is shot, the interval angle of each adjacent frame in the panoramic video is the same, thus, a universal translation transformation matrix can be obtained by extracting the key frame within the preset interval as a background image and only needing one translation transformation, and the matrix is utilized to transform and splice the background images at all angle positions, and a target panoramic image is output.
According to the invention, through using the pan-tilt camera rotating at a constant speed, the calculated amount in the panoramic image splicing process is greatly reduced, and the splicing time is shortened, so that the convenient feeling of a user for acquiring the panoramic image is improved.
The implementation of each step is described below.
For step 100, in some embodiments, before sending the shooting instruction carrying the shooting angle parameter to the pan/tilt head, the method further includes:
and connecting the cradle head according to the IP address, the port, the user name and the password input by the user.
In this embodiment, after the cradle head is connected to the terminal device, the terminal device may communicate with the cradle head through the HTTP protocol. For example, on one hand, a control command can be sent to the pan-tilt, and meanwhile, the position and rotation speed information of the pan-tilt can be acquired, so that a further control command can be made; on the other hand, the first video collected by the pan-tilt camera can be received and used for splicing the panoramic image.
With reference to step 102, in some embodiments, after acquiring the first video acquired by the camera in real time, the first video needs to be decoded by using a decoding library, for example, h.264/h.265 hardware decoding may be performed, and the decoded first video frame data is used for image stitching.
With respect to step 104, in some embodiments, after obtaining the background images at the multiple angular positions and before transforming and stitching the background images at all angular positions by using the same translation transformation matrix, the method further includes:
judging whether the number of the background images reaches a preset number or not; the preset number is the quotient of the difference value between the ending shooting angle and the starting shooting angle and the preset angle interval;
if yes, performing transformation and splicing on the background images at all the angle positions by using the same translation transformation matrix;
and if not, executing to send a shooting instruction carrying the shooting angle parameter to the holder.
In this embodiment, the required number of frames is determined according to the shooting angle range and the preset angle interval input by the user. It is understood that the larger the photographing angle range, the more the number of frames is required, and conversely, the smaller the photographing angle range, the less the number of frames is required. The number of key frames, namely the number of required background images, can be determined according to the total frame number in the shooting angle range and the preset angle interval.
It can be understood that if the number of the background images is judged to reach the preset number, the first video acquired by the pan-tilt camera is considered to meet the acquisition range required by the user, and the next splicing procedure can be performed; and if the number of the background images is less than the preset number, the first video acquired by the pan-tilt camera is considered to not meet the acquisition range required by the user, at the moment, the angle position information of the pan-tilt is required to be called, and an instruction for continuously rotating and shooting or shooting again is sent according to the angle position information of the pan-tilt. For example, if the number of the background images is less than the preset number because the pan-tilt stops rotating due to faults, after the faults are eliminated, the pan-tilt is controlled to continue rotating from the fault angle to the shooting angle to obtain a complete first video; if the number of the background images is less than the preset number because the rotational speed of the pan-tilt is abnormal, the first video is shot again after the rotational speed of the pan-tilt is adjusted.
For step 106, in some embodiments, transforming and stitching the background images at all angular positions by using the same translation transformation matrix, and outputting the target panoramic image, including:
transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image;
judging whether the panoramic image to be determined is in a splicing success state or not;
if so, taking the panoramic image to be determined as a target panoramic image;
and if not, processing the unqualified background image in the panoramic image to be determined so as to output the target panoramic image.
In the embodiment, the pan-tilt head drives the camera to rotate at a constant speed according to a preset rotating speed, so that the corresponding angle differences of any adjacent frames in the first video shot by the camera are the same, and the corresponding angle differences between any adjacent key frames in the key frames extracted according to the preset sequence are also the same; based on the method, the background images at all the angle positions can be transformed and spliced by using the same translation transformation matrix, so that the calculated amount in the image splicing process can be reduced, and the image splicing speed is increased.
It should be noted that, in the image stitching process, there are various factors that may cause the stitching procedure to be terminated, for example, when a background image at a certain angle position contains a moving object, image stitching may fail, at this time, the output to-be-determined panoramic image cannot meet the user requirement, at this time, it is necessary to perform background extraction on the unqualified background image in the to-be-determined panoramic image, and then continue stitching with the image subjected to background extraction to output a target background image; and if the splicing process is not interfered by any factor, the to-be-determined panoramic image can meet the user requirement, and the to-be-determined panoramic image is output as a target panoramic image.
It should be further noted that, in the image stitching process, image stitching may be performed by using a single thread, or image stitching may be performed by using multiple threads; for example, in the embodiment, the key frames are merged by using multiple threads, and the advantage of multiple cores of the processor is fully utilized, so that the processing efficiency of the merged panoramic image is improved, and the waiting time of a user is shortened.
For step 106, in some embodiments, transforming and splicing the background images at all angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image, including:
for the background image of the first angular position, performing: extracting the background key points in the first background image; matching the background key points at adjacent positions in the first background image to estimate a translation transformation matrix by using the matched background key points;
and transforming and splicing the background images at all the angle positions by using the translation transformation matrix, and outputting the to-be-determined panoramic image.
In the embodiment, the general translation transformation matrix can be calculated by only performing key point extraction, key point matching of adjacent images and local variant description for the first background image once and then applying a random sampling consistency algorithm. Therefore, a large number of time-consuming calculation steps such as multiple times of key point detection, local variant description, key point matching in each frame and the like can be skipped, so that the calculation amount is greatly reduced, and particularly for hardware with limited resources, the smooth operation of processing equipment can be ensured.
It should be noted that the first angular position may be a starting angular position, or may be an angular position corresponding to a first background image used by any one of the stitching threads, and as long as one thread calculates a translation transformation matrix, other threads may participate in image stitching by using the translation transformation matrix as a transformation reference, without repeatedly calculating a translation transformation matrix for any background image.
It should be further noted that, the method for extracting the key points of the background image is not specifically limited in the present application, for example, the method for extracting the key points may adopt an ORB method, an SIFT method or a SURF method, and researchers may select the method for extracting the key points according to the difficulty of operation, the operation time, the stability of the detection result, and the like of each algorithm.
For step 106, in some embodiments, determining whether the to-be-determined panoramic image is in a successful stitching state includes:
detecting whether a background image of a moving object exists in the to-be-determined panoramic image or not;
if so, determining that the to-be-determined panoramic image is not in a splicing success state;
if not, determining that the panoramic image to be determined is in a splicing success state.
In this embodiment, the background image at each angle is subjected to template matching to determine whether or not a moving object exists in the background image by calculating the sum of absolute differences between pixels of adjacent images in the same scene in a short time. If the sum of the absolute differences of the pixels is larger than a set threshold value, representing that a moving object exists in the background image, recording the image again aiming at the angle position corresponding to the image so as to extract the background; if the sum of the absolute differences between the pixels is less than the set threshold, the representative image is substantially stationary and there are no moving objects in the background.
With respect to step 106, in some embodiments, processing the unqualified background image in the to-be-determined panoramic image to output the target panoramic image includes:
determining a target angle position corresponding to an unqualified background image in the panoramic image to be determined;
sending the target angle position to a holder so as to rotate to the target angle position by using the holder;
acquiring a second video with preset time duration, which is acquired by a camera at a target angle position;
taking each frame of image in the second video as the input of a preset background extraction algorithm, and separating the moving foreground from each frame of image;
obtaining a qualified background image corresponding to the angle position of the target based on the multi-frame image of the separated moving foreground;
and replacing the qualified background image and the unqualified background image, and performing transformation and splicing on the background images at all angle positions by using the same translation transformation matrix.
In this embodiment, the moving foreground can be separated from the background image by using a background extraction method, so as to obtain a background image of a target angle position, which is used for stitching the panoramic image. By carrying out background extraction operation on the background image with the moving foreground, the repeated recording of the first video can be avoided, and the splicing speed of the panoramic high image is further improved.
It should be noted that, the method for extracting the background is not specifically limited in the present application, for example, a background difference method of a CMG algorithm that combines statistical background image estimation and pixel-by-pixel bayesian segmentation may be used, an inter-frame difference method, a ViBe algorithm, or a ViBe + algorithm may also be used, and a technician may select the method for extracting the background according to the time complexity of each algorithm and the accuracy of segmenting the background image.
It should be noted that, for a complex foreground, if the purpose of separating the foreground cannot be achieved by acquiring the second video only for the target angle position, it is necessary to acquire the third video and the fourth video at the angle positions adjacent to the target angle position at the same time, and extract the background image at the target angle position by fusing each frame image in the second video, the third video and the fourth video and using the image extraction method.
It should be noted that the background image with moving objects may be a frame or multiple frames, and in the actual splicing process, the above method is used to perform background extraction on each frame of image according to the number of actual background image frames with moving objects.
It should be noted that, although it is preferable to perform background extraction on a background image in which a moving object appears to avoid repeated recording of the second video, this method requires a fixed point rotation function in the pan/tilt head. For some clients whose holders cannot rotate to a specific angle and only support uniform-speed movement, a method of re-recording the first video and re-splicing the panoramic images can be adopted. Although the method may need to record the first video for multiple times, the method can achieve higher splicing efficiency and splicing quality by only calculating a translation transformation matrix once in the splicing process due to the adoption of the function of uniform rotation of the pan-tilt. The method for processing the background image after the moving object appears is not specifically limited, and both methods belong to the protection scope of the present application.
As shown in fig. 2 and 3, an embodiment of the present invention provides an apparatus for generating a panoramic image. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. From a hardware aspect, as shown in fig. 2, for a hardware architecture diagram of a computing device in which an apparatus for generating a panoramic image according to an embodiment of the present invention is located, in addition to the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 2, the computing device in which the apparatus is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing a packet, and the like. Taking a software implementation as an example, as shown in fig. 3, as a logical apparatus, a CPU of a computing device in which the apparatus is located reads a corresponding computer program in a non-volatile memory into a memory to run.
As shown in fig. 3, the present embodiment provides an apparatus for generating a panoramic image, including:
the control module 300 is used for being connected to a cradle head, and sending a shooting instruction carrying shooting angle parameters to the cradle head so as to drive the camera to rotate at a constant speed by using the cradle head according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
the transmission module 302 is used for acquiring a first video acquired by a camera in real time;
the extraction module 304 is configured to use the key frame of each preset time interval in the first video as a background image of each preset angle interval to obtain background images of multiple angle positions; wherein the preset angle interval can be evenly divided by the difference value of the ending shooting angle and the starting shooting angle;
and the processing module 306 is configured to transform and splice the background images at all angle positions by using the same translation transformation matrix, and output a target panoramic image.
In an embodiment of the present invention, the control module 300 may be configured to perform the step 100 in the above-described method embodiment, the transmission module 302 may be configured to perform the step 102 in the above-described method embodiment, the extraction module 304 may be configured to perform the step 104 in the above-described method embodiment, and the processing module 306 may be configured to perform the step 106 in the above-described method embodiment.
In one embodiment of the present invention, before the control module 300 is executed, the following operations need to be performed:
and connecting the cradle head according to the IP address, the port, the user name and the password input by the user.
In one embodiment of the present invention, before the control module 304 is executed, the following operations need to be performed:
judging whether the number of the background images reaches a preset number or not; the preset number is the quotient of the difference value between the ending shooting angle and the starting shooting angle and the preset angle interval;
if yes, executing the transformation and splicing of the background images at all the angle positions by using the same translation transformation matrix;
and if not, executing the shooting instruction carrying the shooting angle parameter to the holder.
In one embodiment of the present invention, before the control module 306 is executed, the following operations need to be performed:
transforming and splicing the background images at all angle positions by using the same translation transformation matrix, and outputting a target panoramic image, wherein the method comprises the following steps:
transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image;
judging whether the panoramic image to be determined is in a splicing success state or not;
if so, taking the to-be-determined panoramic image as a target panoramic image;
and if not, processing the unqualified background image in the panoramic image to be determined so as to regenerate the target panoramic image.
In one embodiment of the present invention, before the control module 306 is executed, the following operations are also executed:
the background images of all angle positions are transformed and spliced by using the same translation transformation matrix, and the undetermined panoramic image is output, including:
for the background image of the first angular position, performing: extracting the background key points in the first background image; matching the background key points at the adjacent positions in the first background image to estimate a translation transformation matrix by using the matched background key points;
and transforming and splicing the background images at all the angle positions by using the translation transformation matrix, and outputting the to-be-determined panoramic image.
In one embodiment of the present invention, before the control module 306 is executed, the following operations are also executed:
judging whether the to-be-determined panoramic image is in a splicing success state or not, wherein the judging step comprises the following steps:
detecting whether a background image of a moving object exists in the to-be-determined panoramic image;
if so, determining that the to-be-determined panoramic image is not in a splicing success state;
and if not, determining that the to-be-determined panoramic image is in a splicing success state.
In one embodiment of the present invention, before the control module 306 is executed, the following operations are also executed:
processing unqualified background images in the panoramic image to be determined to output a target panoramic image, and the method comprises the following steps:
determining a target angle position corresponding to the unqualified background image in the to-be-determined panoramic image;
sending the target angular position to the holder to rotate to the target angular position by using the holder;
acquiring a second video with preset time length, which is acquired by the camera at the target angle position;
taking each frame of image in the second video as the input of a preset background extraction algorithm, and separating a moving foreground from each frame of image;
obtaining a qualified background image corresponding to the target angle position based on the multi-frame image of the separated moving foreground;
and replacing the qualified background image and the unqualified background image, and performing transformation and splicing on the background images at all angle positions by using the same translation transformation matrix.
It is to be understood that the configuration illustrated in the embodiment of the present invention does not constitute a specific limitation to an apparatus for generating a panoramic image. In other embodiments of the present invention, an apparatus for generating a panoramic image may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Because the content of information interaction, execution process, and the like among the modules in the device is based on the same concept as the method embodiment of the present invention, specific content can be referred to the description in the method embodiment of the present invention, and is not described herein again.
The embodiment of the invention also provides a computing device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the method for generating the panoramic image in any embodiment of the invention.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, causes the processor to execute a method for generating a panoramic image according to any embodiment of the present invention.
Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or CPU or GPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion module to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an …" does not exclude the presence of other similar elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of generating a panoramic image, comprising:
responding to the connection to a cloud deck, sending a shooting instruction carrying shooting angle parameters to the cloud deck so as to drive a camera to rotate at a constant speed by using the cloud deck according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
acquiring a first video acquired by the camera in real time;
taking the key frame of each preset time interval in the first video as a background image of each preset angle interval to obtain background images of a plurality of angle positions; wherein the preset angle interval can be evenly divided by the difference between the ending shooting angle and the starting shooting angle;
and transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a target panoramic image.
2. The method according to claim 1, wherein before the sending the shooting instruction carrying the shooting angle parameter to the pan/tilt head, the method further comprises:
and connecting the cradle head according to the IP address, the port, the user name and the password input by the user.
3. The method of claim 1, further comprising, after the obtaining the background images at the plurality of angular positions and before the transforming and stitching the background images at all angular positions by using the same translation transformation matrix:
judging whether the number of the background images reaches a preset number or not; the preset number is the quotient of the difference value between the ending shooting angle and the starting shooting angle and the preset angle interval;
if yes, executing the transformation and splicing of the background images at all the angle positions by using the same translation transformation matrix;
and if not, executing the shooting instruction carrying the shooting angle parameter to the holder.
4. The method of claim 1, wherein transforming and stitching the background images at all angular positions by using the same translation transformation matrix to output a target panoramic image comprises:
transforming and splicing the background images at all the angle positions by using the same translation transformation matrix, and outputting a to-be-determined panoramic image;
judging whether the panoramic image to be determined is in a splicing success state or not;
if so, taking the to-be-determined panoramic image as a target panoramic image;
and if not, processing the unqualified background image in the panoramic image to be determined so as to output the target panoramic image.
5. The method of claim 4, wherein the transforming and stitching the background images at all angular positions by using the same translation transformation matrix to output a to-be-determined panoramic image comprises:
for the background image of the first angular position, performing: extracting the background key points in the first background image; matching the background key points at the adjacent positions in the first background image to estimate a translation transformation matrix by using the matched background key points;
and transforming and splicing the background images at all the angle positions by using the translation transformation matrix, and outputting the to-be-determined panoramic image.
6. The method of claim 4, wherein the determining whether the to-be-determined panoramic image is in a stitching success state comprises:
detecting whether a background image of a moving object exists in the to-be-determined panoramic image or not;
if so, determining that the to-be-determined panoramic image is not in a splicing success state;
and if not, determining that the to-be-determined panoramic image is in a splicing success state.
7. The method of claim 4, wherein said processing the unqualified background image in the to-be-determined panoramic image to output a target panoramic image comprises:
determining a target angle position corresponding to the unqualified background image in the to-be-determined panoramic image;
sending the target angular position to the holder to rotate to the target angular position by using the holder;
acquiring a second video with preset time length, which is acquired by the camera at the target angle position;
taking each frame of image in the second video as the input of a preset background extraction algorithm, and separating a moving foreground from each frame of image;
obtaining a qualified background image corresponding to the target angle position based on the multi-frame image of the separated moving foreground;
and replacing the qualified background image and the unqualified background image, and performing transformation and splicing on the background images at all angle positions by using the same translation transformation matrix.
8. An apparatus for generating a panoramic image, comprising:
the control module is used for being connected to the cloud deck and sending a shooting instruction carrying shooting angle parameters to the cloud deck so as to drive the camera to rotate at a constant speed by using the cloud deck according to a preset rotating speed; the shooting angle parameters comprise an initial shooting angle and a final shooting angle;
the transmission module is used for acquiring a first video acquired by the camera in real time;
the extraction module is used for taking the key frame of each preset time interval in the first video as the background image of each preset angle interval to obtain background images of a plurality of angle positions; wherein the preset angle interval can be evenly divided by the difference between the ending shooting angle and the starting shooting angle;
and the processing module is used for transforming and splicing the background images at all the angle positions by using the same translation transformation matrix and outputting a target panoramic image.
9. A computing device comprising a memory having stored therein a computer program and a processor that, when executing the computer program, implements the method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored which, when executed in a computer, causes the computer to carry out the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111272121.8A CN114007014B (en) | 2021-10-29 | 2021-10-29 | Method and device for generating panoramic image, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111272121.8A CN114007014B (en) | 2021-10-29 | 2021-10-29 | Method and device for generating panoramic image, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114007014A true CN114007014A (en) | 2022-02-01 |
| CN114007014B CN114007014B (en) | 2023-06-16 |
Family
ID=79925164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111272121.8A Active CN114007014B (en) | 2021-10-29 | 2021-10-29 | Method and device for generating panoramic image, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114007014B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150548A (en) * | 2022-06-09 | 2022-10-04 | 山东信通电子股份有限公司 | Method, equipment and medium for outputting panoramic image of power transmission line based on holder |
| CN116248992A (en) * | 2023-02-09 | 2023-06-09 | 广州市玄武无线科技股份有限公司 | Dynamically configurable shelf image processing method and device, electronic equipment and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002057879A (en) * | 2000-08-10 | 2002-02-22 | Ricoh Co Ltd | Apparatus and method for image processing, and computer readable recording medium |
| US20040042685A1 (en) * | 2002-08-28 | 2004-03-04 | Lingxiang Zhou | Image warping correction in forming 360 degree panoramic images |
| CN101621634A (en) * | 2009-07-24 | 2010-01-06 | 北京工业大学 | Method for splicing large-scale video with separated dynamic foreground |
| CN101853524A (en) * | 2010-05-13 | 2010-10-06 | 北京农业信息技术研究中心 | Method for generating corn ear panoramic image by using image sequence |
| US20120274739A1 (en) * | 2009-12-21 | 2012-11-01 | Huawei Device Co.,Ud. | Image splicing method and apparatus |
| US20140375762A1 (en) * | 2012-02-17 | 2014-12-25 | Sony Corporation | Information processing apparatus and method, image processing apparatus and method, and program |
| CN104469279A (en) * | 2013-09-25 | 2015-03-25 | 北京环境特性研究所 | Panoramic mosaic device and method based on space coordinates of image |
| CN105894451A (en) * | 2016-03-30 | 2016-08-24 | 沈阳泰科易科技有限公司 | Method and device for splicing panoramic image |
| CN107580175A (en) * | 2017-07-26 | 2018-01-12 | 济南中维世纪科技有限公司 | A kind of method of single-lens panoramic mosaic |
| CN108389157A (en) * | 2018-01-11 | 2018-08-10 | 江苏四点灵机器人有限公司 | A kind of quick joining method of three-dimensional panoramic image |
| CN109087244A (en) * | 2018-07-26 | 2018-12-25 | 贵州火星探索科技有限公司 | A kind of Panorama Mosaic method, intelligent terminal and storage medium |
| CN110782394A (en) * | 2019-10-21 | 2020-02-11 | 中国人民解放军63861部队 | Panoramic video rapid splicing method and system |
-
2021
- 2021-10-29 CN CN202111272121.8A patent/CN114007014B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002057879A (en) * | 2000-08-10 | 2002-02-22 | Ricoh Co Ltd | Apparatus and method for image processing, and computer readable recording medium |
| US20040042685A1 (en) * | 2002-08-28 | 2004-03-04 | Lingxiang Zhou | Image warping correction in forming 360 degree panoramic images |
| CN101621634A (en) * | 2009-07-24 | 2010-01-06 | 北京工业大学 | Method for splicing large-scale video with separated dynamic foreground |
| US20120274739A1 (en) * | 2009-12-21 | 2012-11-01 | Huawei Device Co.,Ud. | Image splicing method and apparatus |
| CN101853524A (en) * | 2010-05-13 | 2010-10-06 | 北京农业信息技术研究中心 | Method for generating corn ear panoramic image by using image sequence |
| US20140375762A1 (en) * | 2012-02-17 | 2014-12-25 | Sony Corporation | Information processing apparatus and method, image processing apparatus and method, and program |
| CN104469279A (en) * | 2013-09-25 | 2015-03-25 | 北京环境特性研究所 | Panoramic mosaic device and method based on space coordinates of image |
| CN105894451A (en) * | 2016-03-30 | 2016-08-24 | 沈阳泰科易科技有限公司 | Method and device for splicing panoramic image |
| CN107580175A (en) * | 2017-07-26 | 2018-01-12 | 济南中维世纪科技有限公司 | A kind of method of single-lens panoramic mosaic |
| CN108389157A (en) * | 2018-01-11 | 2018-08-10 | 江苏四点灵机器人有限公司 | A kind of quick joining method of three-dimensional panoramic image |
| CN109087244A (en) * | 2018-07-26 | 2018-12-25 | 贵州火星探索科技有限公司 | A kind of Panorama Mosaic method, intelligent terminal and storage medium |
| CN110782394A (en) * | 2019-10-21 | 2020-02-11 | 中国人民解放军63861部队 | Panoramic video rapid splicing method and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115150548A (en) * | 2022-06-09 | 2022-10-04 | 山东信通电子股份有限公司 | Method, equipment and medium for outputting panoramic image of power transmission line based on holder |
| CN115150548B (en) * | 2022-06-09 | 2024-04-12 | 山东信通电子股份有限公司 | Method, equipment and medium for outputting panoramic image of power transmission line based on cradle head |
| CN116248992A (en) * | 2023-02-09 | 2023-06-09 | 广州市玄武无线科技股份有限公司 | Dynamically configurable shelf image processing method and device, electronic equipment and storage medium |
| CN116248992B (en) * | 2023-02-09 | 2024-01-16 | 广州市玄武无线科技股份有限公司 | Dynamically configurable shelf image processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114007014B (en) | 2023-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114007014B (en) | Method and device for generating panoramic image, electronic equipment and storage medium | |
| US9554049B2 (en) | Guided video capture for item listings | |
| US8405734B2 (en) | Image capturing apparatus, image capturing method and program for the image capturing apparatus | |
| EP1117262A2 (en) | Image, processing apparatus and method, and storage medium | |
| US9420181B2 (en) | Electronic camera, computer readable medium recording imaging control program thereon and imaging control method | |
| EP1045316A2 (en) | Image processing method and apparatus | |
| US20140369627A1 (en) | Systems and Methods for Image Editing | |
| CN105049695A (en) | Video recording method and device | |
| CN102640486B (en) | Image capturing device and image capturing method | |
| CN113115095B (en) | Video processing method, video processing device, electronic equipment and storage medium | |
| CN113382177A (en) | Multi-view-angle surrounding shooting method and system | |
| US10924637B2 (en) | Playback method, playback device and computer-readable storage medium | |
| CN111881734A (en) | Method and device for automatically intercepting target video | |
| US20240048716A1 (en) | Image processing method and device, storage medium and electronic device | |
| CN113556438B (en) | Scanning control method, system, electronic device and storage medium | |
| CN114745537A (en) | Sound and picture delay testing method and device, electronic equipment and storage medium | |
| CN114694007A (en) | Video structuring processing method and video structuring engine | |
| CN113068024A (en) | Real-time snap analysis method and storage medium | |
| EP3579113A1 (en) | Systems and methods for conducting makeup consultation sessions | |
| CN117478824B (en) | Conference video generation method and device, electronic equipment and storage medium | |
| CN111371992A (en) | Image capturing quality control method, system, device, and storage medium | |
| CN110766611A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN105794193A (en) | Image processing apparatus, image processing method and program | |
| CN113117341B (en) | Picture processing method and device, computer readable storage medium and electronic equipment | |
| US20220207851A1 (en) | System and method for automatic video reconstruction with dynamic point of interest |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |