WO2019134516A1 - Procédé et dispositif de génération d'image panoramique, support d'informations et appareil électronique - Google Patents

Procédé et dispositif de génération d'image panoramique, support d'informations et appareil électronique Download PDF

Info

Publication number
WO2019134516A1
WO2019134516A1 PCT/CN2018/122029 CN2018122029W WO2019134516A1 WO 2019134516 A1 WO2019134516 A1 WO 2019134516A1 CN 2018122029 W CN2018122029 W CN 2018122029W WO 2019134516 A1 WO2019134516 A1 WO 2019134516A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame image
image
key frame
feature points
panoramic
Prior art date
Application number
PCT/CN2018/122029
Other languages
English (en)
Chinese (zh)
Inventor
陈岩
刘耀勇
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019134516A1 publication Critical patent/WO2019134516A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof

Definitions

  • the present application relates to the field of computer technologies, and in particular, to a panoramic image generating method, apparatus, storage medium, and electronic device.
  • mobile terminals have become one of the necessary tools for people's daily life, which not only provides call and text messaging functions, but also has multimedia playback and camera functions.
  • panoramic shooting mode a series of small-view images with partially overlapping boundaries can be matched and aligned according to the characteristics of different images, thereby splicing into a wide viewing angle.
  • the panoramic image but the existing panoramic shooting technology can only be spliced for some relatively simple scene images, and it is difficult to adapt to complex scenes, such as close-up shooting of buildings, murals, etc., with limited limitations and poor stitching effect.
  • the embodiment of the present invention provides a method, a device, a storage medium, and an electronic device for generating a panoramic image, which can be adapted to various shooting scenes, and the stitching effect is good.
  • the embodiment of the present application provides a method for generating a panoramic image, including:
  • the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction;
  • a panoramic image is generated based on the partial image.
  • the embodiment of the present application further provides a panoramic image generating apparatus, which is applied to an electronic device, and includes:
  • An acquisition module configured to acquire a panoramic shooting instruction
  • a shooting module configured to start a camera for video shooting according to the panoramic shooting instruction, and display the shooting content in a preview frame;
  • a scanning module configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting process to obtain a plurality of sets of scanned images, where the first direction is perpendicular to the second direction ;
  • a first generating module configured to generate a corresponding partial image in the second direction according to each set of scanned images
  • a second generating module configured to generate a panoramic image according to the partial image.
  • the first generating module specifically includes:
  • a splicing sub-module configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.
  • the splicing submodule is used to:
  • the corresponding key frame image group is spliced by the first global transformation matrix.
  • the splicing submodule is specifically configured to:
  • a first homography matrix is calculated based on the location information.
  • determining submodule is specifically configured to:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
  • determining submodule is specifically configured to:
  • determining submodule is specifically configured to:
  • the second generating module is specifically configured to:
  • the partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
  • the embodiment of the present application further provides a storage medium, where the storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by a processor to execute any of the above-mentioned panoramic image generating methods.
  • the embodiment of the present application further provides an electronic device, including a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in any one of the foregoing The steps in the panoramic image generation method described.
  • FIG. 1 is a schematic flowchart diagram of a method for generating a panoramic image according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a serpentine scan in a preview frame according to an embodiment of the present application.
  • FIG. 3 is another schematic flowchart of a method for generating a panoramic image according to an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of step 203 according to an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a panoramic image generating apparatus according to an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a first generation module according to an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the embodiment of the present application provides a method, an apparatus, a storage medium, and an electronic device for generating a panoramic image.
  • a panoramic image generating method is applied to an electronic device, comprising: acquiring a panoramic shooting instruction; starting a camera for video shooting according to the panoramic shooting instruction, and displaying the captured content in a preview frame; in the video shooting process, in the second direction Performing a serpentine scan on the preview frame at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction; generating a corresponding partial image in the second direction according to each set of scanned images Generating a panoramic image from the partial image.
  • the generating a corresponding partial image in the second direction according to each set of scanned images comprises:
  • the splicing the key frame image group according to the feature point comprises:
  • the corresponding key frame image group is spliced by the first global transformation matrix.
  • the calculating the corresponding first homography matrix according to the feature points of the adjacent two frame key frame images in the key frame image group includes:
  • a first homography matrix is calculated based on the location information.
  • the splicing the corresponding key frame image group by using the first global transformation matrix comprises:
  • each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.
  • the determining the corresponding key frame image group according to each set of scanned images comprises:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
  • determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image includes:
  • determining whether the target frame image satisfies a preset condition according to the number of the successfully matched feature points and the total number includes:
  • the generating the panoramic image according to the partial image comprises:
  • the partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
  • FIG. 1 is a schematic flowchart of a method for generating a panoramic image according to an embodiment of the present application, which is applied to an electronic device, and the specific process may be as follows:
  • the electronic device may generate a panoramic shooting instruction.
  • the size of the preview frame may be manually set, or may be determined according to the size of the display screen of the electronic device, such as three-quarters of the size of the display screen.
  • the preview frame is subjected to a serpentine scan in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.
  • the second direction and the first direction are usually set in advance, which may be the direction in which the preview frame length and width are located, or may be an artificially defined horizontal and vertical direction, and may of course be other settings. the way.
  • the first direction overlap ratio refers to a scan track overlap ratio in the first direction, which can be artificially set, such as not less than 50%. After each scan, a corresponding set of scanned images is generated.
  • the scan length of the serpentine scan can be a fixed value or a variable, and the scan trajectory overlap rate depends on the starting position of two adjacent serpentine scans, such as Referring to FIG. 2, for two adjacent serpentine scans with a fixed scan length, if the scan start points are a1 and a2, respectively, and the scan length is b, the scan trace overlap ratio is [b-(a2-a1)]/b. .
  • the partial image is formed by splicing in the second direction. Since each group of scanned images is not scanned for the entire preview frame, but a certain longitudinal or horizontal region of the preview frame is scanned, correspondingly formed
  • the stitched image is also not a panoramic image but a partial image.
  • the foregoing step 104 may specifically include:
  • the foregoing step 1-1 may specifically include:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to perform the feature point according to the key frame image and the feature point of the target frame image to determine whether the target frame image is satisfied.
  • the operation of the preset condition is not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to perform the feature point according to the key frame image and the feature point of the target frame image to determine whether the target frame image is satisfied.
  • the feature point refers to a point where the gray value of the image changes drastically or a point with a large curvature on the edge of the image (ie, the intersection of the two edges), and each feature point may include a texture.
  • Information such as features and color characteristics.
  • step "determining whether the target frame image satisfies a preset condition according to the feature point of the key frame image and the feature point of the target frame image” may specifically include:
  • Whether the target frame image satisfies a preset condition is determined according to the number of the feature points that are successfully matched and the total number.
  • the preset condition may be manually set.
  • the similarity between the key frame image and the target frame image may be within a certain range. Generally, the more the number of feature points that are successfully matched, the higher the similarity.
  • step "determining whether the target frame image satisfies the preset condition according to the number of the feature points and the total number of the matching successes" may specifically include:
  • the ratio between the number of feature points matching the success and the total number can be calculated, and the ratio is taken as the similarity.
  • the first preset threshold and the second preset threshold may be manually set.
  • the first preset threshold may be 70%
  • the second preset threshold may be 20.
  • steps 1-3 may specifically include:
  • step 1-3-1 may specifically include:
  • a first homography matrix is calculated based on the location information.
  • the homography matrix (the first homography matrix) is a conversion matrix from one image to another image mapping. Specifically, the two adjacent frames may be determined first. Which feature points are successfully matched in the frame image, and then a certain number of matching feature points are selected to calculate the corresponding homography matrix, so that the feature points in one frame of the key frame image are transformed by the homography matrix, It is in the same spatial coordinate system as the feature points in another frame of keyframe images.
  • the first homography matrix of the adjacent frame key frame image may be calculated in order, for example, for a certain key frame image group M1 Mn Mn, between M1 and M2, between M2 and M3, and M3 may be calculated.
  • the first homography matrix between M4 and Mn-1 and Mn, each homography matrix is equivalent to the local transformation matrix of the key frame image group, and finally, all the first homography matrices Uniform to the same coordinate system to obtain the first global transformation matrix.
  • each key frame image in the key frame image group can be projected into the same coordinate space system through the global transformation matrix, and the obtained image is a spliced partial image.
  • the foregoing step 105 may specifically include:
  • the partial image is spliced in the first direction by the second global transformation matrix to obtain a panoramic image.
  • the serpentine scan is an overlapping scan in the first direction, that is, the scan trajectories of the adjacent two scan operations partially overlap, when the scan image completes the splicing in the second direction, Further splicing in the first direction, that is, the partial image spliced by each column (row) of the scanned image is spliced again in the direction in which the row (column) is located, thereby realizing two-dimensional mesh splicing, which can carry the scanned image
  • the image information is fully utilized, the method is simple, the stitching effect is good, and the video image stitching suitable for various complicated scenes is strong, and the flexibility is strong.
  • the panoramic image generating method provided by the embodiment is applied to an electronic device, by acquiring a panoramic shooting instruction, and then, according to the panoramic shooting instruction, the camera is started to perform video shooting, and the captured content is displayed in the preview box, and During the video shooting, the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction, and according to each set of scanned images The corresponding partial image is generated in the second direction, and then the panoramic image is generated according to the partial image, so that the image mosaic of various complex scenes can be applied, the method is simple, the flexibility is high, and the stitching effect is good.
  • the description will be made from the perspective of the panoramic image generating device, and the detailed description will be made by taking the panoramic image generating device integrated in the electronic device as an example.
  • a method for generating a panoramic image is applied to an electronic device, and the specific process may be as follows:
  • the electronic device acquires a panoramic shooting instruction, and starts a camera to perform video shooting according to the panoramic shooting instruction, and then displays the captured content in a preview frame.
  • the panoramic shooting button of the shooting application can be clicked.
  • the electronic device starts the camera for video shooting, and displays the shooting content in the preview box in real time.
  • the electronic device scans the preview frame in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.
  • the first direction may be a horizontal direction
  • the second direction may be a vertical direction, that is, the electronic device scans the content displayed in the preview frame in a longitudinal direction, as shown in FIG. 2, for two adjacent scan lengths.
  • the horizontal scan overlap ratio is [b-(a2-a1)]/b, for example, b/a may be not less than 50%.
  • the electronic device determines a corresponding key frame image group according to each group of scanned images.
  • the foregoing step 203 may specifically include:
  • the electronic device uses the first frame scanned image as a key frame image
  • the electronic device acquires a next frame scan image adjacent to the key frame image as a target frame image.
  • the electronic device determines, according to the feature point of the key frame image and the feature point of the target frame image, whether the target frame image satisfies a preset condition, and if yes, uses the target frame image as a key frame image, and returns to perform the above steps. 2032. If no, the next frame scan image adjacent to the target frame image is used as the target frame image, and the process returns to step 2033.
  • the above step 2033 can further include:
  • the second preset threshold such as 20 o'clock. It can be considered that the similarity between the two is not lower than the minimum.
  • the ratio of the number of matching feature points to the total number is less than the first preset threshold, such as 70%, the similarity between the two can be considered not higher than the maximum. Only the target frame image that satisfies both of these conditions can be considered as a key frame image if it is considered to satisfy the preset condition.
  • the electronic device extracts feature points of the key frame image in each key frame image group, and matches feature points of adjacent two frame key frame images in the key frame image group.
  • the electronic device acquires location information of the feature point that is successfully matched, and calculates a first homography matrix according to the location information.
  • the electronic device calculates a first global transformation matrix according to the first homography matrix, and splices the corresponding key frame image group by using the first global transformation matrix to obtain a partial image.
  • each key frame image group is obtained by a single vertical scan
  • the partial images of each key frame image group are corresponding to the vertical scan.
  • the image information included in the region may be calculated by first calculating the first homography matrix of the adjacent key frame image, and then normalizing all the first homography matrices to obtain a global transformation matrix, and using the global The transformation matrix projects each key frame image in the key frame image group into the same coordinate space, completing the vertical splicing of each key frame image group.
  • the electronic device calculates a corresponding second homography matrix according to the two adjacent partial images, and calculates a second global transformation matrix according to the second homography matrix, and then uses the second global transformation matrix to use the partial image. Splicing is performed in the first direction to obtain a panoramic image.
  • the serpentine scan has a certain lateral overlap ratio, that is, the same portion exists in two adjacent partial images
  • the partial image can be further spliced horizontally, and the splicing method can be second to two.
  • the homography matrix then normalizes the second homography matrix to obtain a second global transformation matrix, and then uses the second global transformation matrix to project all partial images into a coordinate space to complete the lateral direction of the partial image. splice.
  • the panoramic image generating method provided in this embodiment is applied to an electronic device, wherein the electronic device can acquire a panoramic shooting instruction, and start a camera to perform video shooting according to the panoramic shooting instruction, and then display the captured content in a preview box.
  • the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction, and then, according to each The group scan image determines a corresponding key frame image group, and then extracts feature points of the key frame image in each key frame image group, and matches feature points of adjacent two frame key frame images in the key frame image group, and then Obtaining location information of the matching feature point, and calculating a first homography matrix according to the location information, and then calculating a first global transformation matrix according to the first homography matrix, and using the first global transformation matrix pair
  • the corresponding key frame image group is spliced to obtain a partial image, and then, the pair is calculated according to two adjacent partial images.
  • the two-dimensional grid is used to generate a panoramic image, which is suitable for image mosaic of various complicated scenes, and has the advantages of simple method, high flexibility and good stitching effect.
  • the panoramic image generating device may be implemented as an independent entity or integrated in an electronic device, such as a terminal.
  • the terminal may include a mobile phone, a tablet computer, a personal computer, and the like.
  • the embodiment of the present application provides a panoramic image generating apparatus, which is applied to an electronic device, and includes:
  • An acquisition module configured to acquire a panoramic shooting instruction
  • a shooting module configured to start a camera for video shooting according to the panoramic shooting instruction, and display the shooting content in a preview frame;
  • a scanning module configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting process to obtain a plurality of sets of scanned images, where the first direction is perpendicular to the second direction ;
  • a first generating module configured to generate a corresponding partial image in the second direction according to each set of scanned images
  • a second generating module configured to generate a panoramic image according to the partial image.
  • the first generating module specifically includes:
  • a splicing sub-module configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.
  • the splicing sub-module is used to:
  • the corresponding key frame image group is spliced by the first global transformation matrix.
  • the splicing sub-module is specifically configured to:
  • a first homography matrix is calculated based on the location information.
  • the splicing sub-module is specifically configured to:
  • each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.
  • the determining sub-module is specifically configured to:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
  • the determining sub-module is specifically configured to:
  • the determining sub-module is specifically configured to:
  • the second generating module is specifically configured to:
  • the partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
  • FIG. 5 specifically describes a panoramic image generating apparatus according to an embodiment of the present application, which is applied to an electronic device, and the panoramic image generating apparatus may include: an acquiring module 10, a shooting module 20, a scanning module 30, and a first generating module. 40 and a second generation module 50, wherein:
  • the obtaining module 10 is configured to acquire a panoramic shooting instruction.
  • the electronic device may generate a panoramic shooting instruction.
  • the shooting module 20 is configured to start a camera for video shooting according to the panoramic shooting instruction, and display the captured content in a preview frame.
  • the size of the preview frame may be manually set, or may be determined according to the size of the display screen of the electronic device, such as three-quarters of the size of the display screen.
  • the scanning module 30 is configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.
  • the second direction and the first direction are usually set in advance, which may be the direction in which the preview frame length and width are located, or may be an artificially defined horizontal and vertical direction, and may of course be other settings.
  • the first direction overlap ratio refers to a scan track overlap ratio in the first direction, which can be artificially set, such as not less than 50%. After each scan, a corresponding set of scanned images is generated.
  • the scan length of the serpentine scan can be a fixed value or a variable, and the scan trajectory overlap rate depends on the starting position of two adjacent serpentine scans, such as Referring to FIG. 2, for two adjacent serpentine scans with a fixed scan length, if the scan length is a and the overlap length in the first direction is b, the scan track overlap ratio is b/a.
  • the first generating module 40 is configured to generate a corresponding partial image in the second direction according to each set of scanned images.
  • the partial image is formed by splicing in the second direction. Since each group of scanned images is not scanned for the entire preview frame, but a certain longitudinal or horizontal region of the preview frame is scanned, correspondingly formed
  • the stitched image is also not a panoramic image but a partial image.
  • the first generating module 40 may specifically include a determining submodule 41, an extracting submodule 42 and a splicing submodule 43, where:
  • the determining sub-module 41 is configured to determine a corresponding key frame image group according to each set of scanned images.
  • the determining sub-module 41 can be specifically used to:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to perform the feature point according to the key frame image and the feature point of the target frame image to determine whether the target frame image is satisfied.
  • the operation of the preset condition is not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to perform the feature point according to the key frame image and the feature point of the target frame image to determine whether the target frame image is satisfied.
  • the feature point refers to a point where the gray value of the image changes drastically or a point with a large curvature on the edge of the image (ie, the intersection of the two edges), and each feature point may include a texture.
  • Information such as features and color characteristics.
  • the determining sub-module 41 can be specifically used to:
  • Whether the target frame image satisfies a preset condition is determined according to the number of the feature points that are successfully matched and the total number.
  • the preset condition may be manually set.
  • the similarity between the key frame image and the target frame image may be within a certain range. Generally, the more the number of feature points that are successfully matched, the higher the similarity.
  • determining sub-module 41 is specifically configured to:
  • the ratio between the number of feature points matching the success and the total number can be calculated, and the ratio is taken as the similarity.
  • the first preset threshold and the second preset threshold may be manually set.
  • the first preset threshold may be 70%
  • the second preset threshold may be 20.
  • the extraction sub-module 42 is configured to extract feature points of the key frame image in each key frame image group.
  • the splicing sub-module 43 is configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.
  • the splicing submodule 43 can be specifically used to:
  • the splicing sub-module 43 can be specifically used to:
  • a first homography matrix is calculated based on the location information.
  • the homography matrix (the first homography matrix) is a conversion matrix from one image to another image mapping. Specifically, the two adjacent frames may be determined first. Which feature points are successfully matched in the frame image, and then a certain number of matching feature points are selected to calculate the corresponding homography matrix, so that the feature points in one frame of the key frame image are transformed by the homography matrix, It is in the same spatial coordinate system as the feature points in another frame of keyframe images.
  • the first homography matrix of the adjacent frame key frame image may be calculated in order, for example, for a certain key frame image group M1 Mn Mn, between M1 and M2, between M2 and M3, and M3 may be calculated.
  • the first homography matrix between M4 and Mn-1 and Mn, each homography matrix is equivalent to the local transformation matrix of the key frame image group, and finally, all the first homography matrices Uniform to the same coordinate system to obtain the first global transformation matrix.
  • each key frame image in the key frame image group can be projected into the same coordinate space system through the global transformation matrix, and the obtained image is a spliced partial image.
  • the second generating module 50 is configured to generate a panoramic image according to the partial image.
  • the second generating module 50 can be specifically configured to:
  • the partial image is spliced in the first direction by the second global transformation matrix to obtain a panoramic image.
  • the serpentine scan is an overlapping scan in the first direction, that is, the scan trajectories of the adjacent two scan operations partially overlap, when the scan image completes the splicing in the second direction, Further splicing in the first direction, that is, the partial image spliced by each column (row) of the scanned image is spliced again in the direction in which the row (column) is located, thereby realizing two-dimensional mesh splicing, which can carry the scanned image
  • the image information is fully utilized, the method is simple, the stitching effect is good, and the video image stitching suitable for various complicated scenes is strong, and the flexibility is strong.
  • the foregoing units may be implemented as a separate entity, or may be implemented in any combination, and may be implemented as the same or a plurality of entities.
  • the foregoing method embodiments and details are not described herein.
  • the panoramic image generating method provided in this embodiment is applied to an electronic device, and the panoramic shooting instruction is acquired by the acquiring module 10. Then, the shooting module 20 starts the camera to perform video shooting according to the panoramic shooting instruction, and displays the captured content in the In the preview frame, during the video shooting process, the scanning module 30 performs a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second
  • the first generation module 40 generates a corresponding partial image in the second direction according to each set of scanned images, and then the second generation module 50 generates a panoramic image according to the partial image, so that the image mosaic can be applied to various complex scenes.
  • the method is simple, the flexibility is high, and the stitching effect is good.
  • the embodiment of the present application further provides an electronic device, which may be a device such as a smart phone or a tablet computer.
  • the electronic device 900 includes a processor 901, a memory 902, a display screen 903, and a control circuit 904.
  • the processor 901 is electrically connected to the memory 902, the display screen 903, and the control circuit 904, respectively.
  • the processor 901 is a control center of the electronic device 900, and connects various parts of the entire electronic device using various interfaces and lines, executes the electronic by running or loading an application stored in the memory 902, and calling data stored in the memory 902.
  • the various functions and processing data of the device enable overall monitoring of the electronic device.
  • the processor 901 in the electronic device 900 loads the instructions corresponding to the process of one or more applications into the memory 902 according to the following steps, and is stored in the memory 902 by the processor 901.
  • the application thus implementing various functions:
  • the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction;
  • a panoramic image is generated based on the partial image.
  • the generating a corresponding partial image in the second direction according to each set of scanned images comprises:
  • the splicing the key frame image group according to the feature point comprises:
  • the corresponding key frame image group is spliced by the first global transformation matrix.
  • the calculating the corresponding first homography matrix according to the feature points of the adjacent two frame key frame images in the key frame image group includes:
  • a first homography matrix is calculated based on the location information.
  • the splicing the corresponding key frame image group by using the first global transformation matrix comprises:
  • each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.
  • the determining the corresponding key frame image group according to each set of scanned images comprises:
  • the first frame scanned image is used as a key frame image
  • the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
  • determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image includes:
  • determining whether the target frame image satisfies a preset condition according to the number of the successfully matched feature points and the total number includes:
  • the generating the panoramic image according to the partial image comprises:
  • the partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
  • Memory 902 can be used to store applications and data.
  • the application stored in the memory 902 contains instructions executable in the processor.
  • Applications can form various functional modules.
  • the processor 901 executes various functional applications and data processing by running an application stored in the memory 902.
  • the display screen 903 can be used to display information entered by the user or information provided to the user as well as various graphical user interfaces of the terminal, which can be composed of images, text, icons, video, and any combination thereof.
  • the control circuit 904 is electrically connected to the display screen 903 for controlling the display screen 903 to display information.
  • the electronic device 900 further includes a radio frequency circuit 905, an input unit 906, an audio circuit 907, a sensor 908, and a power source 909.
  • the processor 901 is electrically connected to the radio frequency circuit 905, the input unit 906, the audio circuit 907, the sensor 908, and the power source 909, respectively.
  • the radio frequency circuit 905 is used for transmitting and receiving radio frequency signals to establish wireless communication with network devices or other electronic devices through wireless communication, and to transmit and receive signals with network devices or other electronic devices.
  • the input unit 906 can be configured to receive input digits, character information, or user characteristic information (eg, fingerprints), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function controls.
  • the input unit 906 can include a fingerprint identification module.
  • the audio circuit 907 can provide an audio interface between the user and the terminal through a speaker and a microphone.
  • Electronic device 900 may also include at least one type of sensor 908, such as a light sensor, motion sensor, and other sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may close the display panel and/or the backlight when the terminal moves to the ear.
  • the gravity acceleration sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity.
  • the terminal can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.
  • Power source 909 is used to power various components of electronic device 900.
  • the power supply 909 can be logically coupled to the processor 901 through a power management system to enable functions such as managing charging, discharging, and power management through the power management system.
  • the electronic device 900 may further include a camera, a Bluetooth module, and the like, and details are not described herein.
  • an embodiment of the present invention provides a storage medium in which a plurality of instructions are stored, which can be loaded by a processor to perform the steps in any of the panoramic image generating methods provided by the embodiments of the present invention.
  • the storage medium may include: a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
  • ROM read only memory
  • RAM random access memory
  • magnetic disk a magnetic disk or an optical disk.
  • any embodiment of the present invention can be performed by using the instructions stored in the storage medium. Therefore, any method for generating a panoramic image provided by the embodiments of the present invention can be implemented.
  • any method for generating a panoramic image provided by the embodiments of the present invention can be implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé et un dispositif de génération d'une image panoramique, un support d'informations et un appareil électronique. Le procédé consiste : à acquérir une instruction de capture de panorama; à activer, selon l'instruction de capture de panorama, une caméra de façon à effectuer une opération de capture de vidéo, et à afficher un contenu capturé dans une zone d'aperçu; dans un processus de capture de vidéo, à réaliser, en utilisant un rapport de chevauchement de première direction prédéfini, un balayage en serpentin sur la zone d'aperçu dans une seconde direction; à générer des images partielles correspondantes dans la seconde direction sur la base des images balayées respectives; et à générer une image panoramique sur la base des images partielles.
PCT/CN2018/122029 2018-01-05 2018-12-19 Procédé et dispositif de génération d'image panoramique, support d'informations et appareil électronique WO2019134516A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810011559.2A CN110012209B (zh) 2018-01-05 2018-01-05 全景图像生成方法、装置、存储介质及电子设备
CN201810011559.2 2018-01-05

Publications (1)

Publication Number Publication Date
WO2019134516A1 true WO2019134516A1 (fr) 2019-07-11

Family

ID=67143617

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/122029 WO2019134516A1 (fr) 2018-01-05 2018-12-19 Procédé et dispositif de génération d'image panoramique, support d'informations et appareil électronique

Country Status (2)

Country Link
CN (1) CN110012209B (fr)
WO (1) WO2019134516A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257595A (zh) * 2020-10-22 2021-01-22 广州市百果园网络科技有限公司 视频匹配方法、装置、设备及存储介质
CN112991180A (zh) * 2021-03-25 2021-06-18 北京百度网讯科技有限公司 图像拼接方法、装置、设备以及存储介质
CN113269671A (zh) * 2021-04-09 2021-08-17 浙江省交通运输科学研究院 一种基于局部和全局特征的桥梁表观全景图生成方法
CN113609316A (zh) * 2021-07-27 2021-11-05 支付宝(杭州)信息技术有限公司 媒体内容相似度的检测方法和装置
CN116248992A (zh) * 2023-02-09 2023-06-09 广州市玄武无线科技股份有限公司 动态可配置的货架图像处理方法、装置、电子设备及存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110675420B (zh) * 2019-08-22 2023-03-24 华为技术有限公司 一种图像处理方法和电子设备
CN111340707B (zh) * 2020-02-27 2023-07-14 创新奇智(重庆)科技有限公司 基于关键帧的图像拼接系统及拼接方法
CN113747044B (zh) * 2020-05-29 2023-05-02 华为技术有限公司 一种全景拍摄方法及设备
CN111970537B (zh) * 2020-08-04 2023-02-28 威海精讯畅通电子科技有限公司 一种根系扫描图像处理方法及系统
CN111935408B (zh) * 2020-09-02 2022-01-18 Oppo广东移动通信有限公司 全景图像拍摄方法及系统、电子设备及可读存储介质
CN113240582B (zh) * 2021-04-13 2023-12-12 浙江大华技术股份有限公司 一种图像拼接方法及装置
CN113450254B (zh) * 2021-05-20 2022-06-17 北京城市网邻信息技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113362227B (zh) * 2021-06-22 2023-07-21 北京百度网讯科技有限公司 图像处理方法、装置、电子设备及存储介质
CN114007017A (zh) * 2021-11-18 2022-02-01 浙江博采传媒有限公司 一种视频生成方法、装置及存储介质
CN114520879B (zh) * 2022-04-21 2022-09-06 浙江宇视科技有限公司 雷达视觉协同的视频采集方法及装置、系统、存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1875618A (zh) * 2003-10-28 2006-12-06 皇家飞利浦电子股份有限公司 具有全景摄影或镶嵌功能的数字照相机
CN101474076A (zh) * 2008-10-15 2009-07-08 北京汇影互联科技有限公司 一种获得全视野超声扫查图像数据的方法和设备
CN101984463A (zh) * 2010-11-02 2011-03-09 中兴通讯股份有限公司 全景图合成方法及装置
CN102201115A (zh) * 2011-04-07 2011-09-28 湖南天幕智能科技有限公司 无人机航拍视频实时全景图拼接方法
US20120154639A1 (en) * 2000-03-06 2012-06-21 Fisher Clay H System And Method For Creating Composite Images By Utilizing An Imaging Device
CN103139479A (zh) * 2013-02-25 2013-06-05 广东欧珀移动通信有限公司 一种结束全景预览扫描方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102420898A (zh) * 2011-09-27 2012-04-18 惠州Tcl移动通信有限公司 一种基于手机的全景照相实现方法及手机
US10681304B2 (en) * 2012-06-08 2020-06-09 Apple, Inc. Capturing a panoramic image using a graphical user interface having a scan guidance indicator
JP6672082B2 (ja) * 2016-06-08 2020-03-25 キヤノン株式会社 撮像装置、撮像方法およびプログラム
CN107094236A (zh) * 2017-05-19 2017-08-25 努比亚技术有限公司 全景拍摄方法、移动终端及计算机可读存储介质
CN107395971B (zh) * 2017-07-31 2020-06-12 上海新眼光医疗器械股份有限公司 一种图像采集方法、设备和计算机可读存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120154639A1 (en) * 2000-03-06 2012-06-21 Fisher Clay H System And Method For Creating Composite Images By Utilizing An Imaging Device
CN1875618A (zh) * 2003-10-28 2006-12-06 皇家飞利浦电子股份有限公司 具有全景摄影或镶嵌功能的数字照相机
CN101474076A (zh) * 2008-10-15 2009-07-08 北京汇影互联科技有限公司 一种获得全视野超声扫查图像数据的方法和设备
CN101984463A (zh) * 2010-11-02 2011-03-09 中兴通讯股份有限公司 全景图合成方法及装置
CN102201115A (zh) * 2011-04-07 2011-09-28 湖南天幕智能科技有限公司 无人机航拍视频实时全景图拼接方法
CN103139479A (zh) * 2013-02-25 2013-06-05 广东欧珀移动通信有限公司 一种结束全景预览扫描方法及装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112257595A (zh) * 2020-10-22 2021-01-22 广州市百果园网络科技有限公司 视频匹配方法、装置、设备及存储介质
CN112991180A (zh) * 2021-03-25 2021-06-18 北京百度网讯科技有限公司 图像拼接方法、装置、设备以及存储介质
CN112991180B (zh) * 2021-03-25 2023-10-31 北京百度网讯科技有限公司 图像拼接方法、装置、设备以及存储介质
CN113269671A (zh) * 2021-04-09 2021-08-17 浙江省交通运输科学研究院 一种基于局部和全局特征的桥梁表观全景图生成方法
CN113269671B (zh) * 2021-04-09 2024-02-23 浙江省交通运输科学研究院 一种基于局部和全局特征的桥梁表观全景图生成方法
CN113609316A (zh) * 2021-07-27 2021-11-05 支付宝(杭州)信息技术有限公司 媒体内容相似度的检测方法和装置
CN116248992A (zh) * 2023-02-09 2023-06-09 广州市玄武无线科技股份有限公司 动态可配置的货架图像处理方法、装置、电子设备及存储介质
CN116248992B (zh) * 2023-02-09 2024-01-16 广州市玄武无线科技股份有限公司 动态可配置的货架图像处理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN110012209A (zh) 2019-07-12
CN110012209B (zh) 2020-08-14

Similar Documents

Publication Publication Date Title
WO2019134516A1 (fr) Procédé et dispositif de génération d'image panoramique, support d'informations et appareil électronique
CN110555883B (zh) 相机姿态追踪过程的重定位方法、装置及存储介质
CN108495032B (zh) 图像处理方法、装置、存储介质及电子设备
CN110650379B (zh) 视频摘要生成方法、装置、电子设备及存储介质
CN108776822B (zh) 目标区域检测方法、装置、终端及存储介质
CN108632543B (zh) 图像显示方法、装置、存储介质及电子设备
CN111753784A (zh) 视频的特效处理方法、装置、终端及存储介质
CN112036331A (zh) 活体检测模型的训练方法、装置、设备及存储介质
CN110290426B (zh) 展示资源的方法、装置、设备及存储介质
CN105427369A (zh) 移动终端及其三维形象的生成方法
WO2018184260A1 (fr) Procédé et dispositif de correction pour image de document
WO2022121577A1 (fr) Procédé et appareil de traitement d'images
CN112532882B (zh) 图像显示方法和装置
CN113840070A (zh) 拍摄方法、装置、电子设备及介质
CN110807769B (zh) 图像显示控制方法及装置
CN110086998B (zh) 一种拍摄方法及终端
CN109981989B (zh) 渲染图像的方法、装置、电子设备和计算机可读存储介质
CN112788244B (zh) 拍摄方法、拍摄装置和电子设备
CN112511743B (zh) 视频拍摄方法和装置
WO2024061134A1 (fr) Procédé et appareil de photographie, dispositif électronique et support
CN114785957A (zh) 拍摄方法及其装置
CN114390206A (zh) 拍摄方法、装置和电子设备
CN114241127A (zh) 全景图像生成方法、装置、电子设备和介质
CN114245017A (zh) 一种拍摄方法、装置及电子设备
CN114140839A (zh) 用于人脸识别的图像发送方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18898624

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18898624

Country of ref document: EP

Kind code of ref document: A1