WO2019134516A1

WO2019134516A1 - Method and device for generating panoramic image, storage medium, and electronic apparatus

Info

Publication number: WO2019134516A1
Application number: PCT/CN2018/122029
Authority: WO
Inventors: 陈岩; 刘耀勇
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-01-05
Filing date: 2018-12-19
Publication date: 2019-07-11
Also published as: CN110012209A; CN110012209B

Abstract

A method and device for generating a panoramic image, a storage medium, and an electronic apparatus. The method comprises: acquiring a panorama capture instruction; activating, according to the panorama capture instruction, a camera to perform a video capture operation, and displaying a captured content in a preview box; in a video capture process, performing, using a preset first-direction overlapping ratio, serpentine scanning on the preview box in a second direction; generating corresponding partial images in the second direction on the basis of the respective scanned images; and generating a panoramic image on the basis of the partial images.

Description

Panoramic image generation method, device, storage medium and electronic device

The present application claims priority to Chinese Patent Application No. 201810011559.2, entitled "Panoramic Image Generation Method, Apparatus, Storage Medium, and Electronic Apparatus", which is filed on January 5, 2018, the entire contents of which are incorporated by reference. Combined in this application.

Technical field

The present application relates to the field of computer technologies, and in particular, to a panoramic image generating method, apparatus, storage medium, and electronic device.

Background technique

With the continuous development of mobile communication technology, mobile terminals have become one of the necessary tools for people's daily life, which not only provides call and text messaging functions, but also has multimedia playback and camera functions.

Most of the current mobile terminals have a panoramic shooting mode. In the panoramic shooting mode, a series of small-view images with partially overlapping boundaries can be matched and aligned according to the characteristics of different images, thereby splicing into a wide viewing angle. The panoramic image, but the existing panoramic shooting technology can only be spliced for some relatively simple scene images, and it is difficult to adapt to complex scenes, such as close-up shooting of buildings, murals, etc., with limited limitations and poor stitching effect.

Summary of the invention

The embodiment of the present invention provides a method, a device, a storage medium, and an electronic device for generating a panoramic image, which can be adapted to various shooting scenes, and the stitching effect is good.

The embodiment of the present application provides a method for generating a panoramic image, including:

Obtain a panoramic shooting instruction;

Activating a camera for video shooting according to the panoramic shooting instruction, and displaying the captured content in a preview frame;

During the video shooting, the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction;

Generating a corresponding partial image in the second direction according to each set of scanned images;

A panoramic image is generated based on the partial image.

The embodiment of the present application further provides a panoramic image generating apparatus, which is applied to an electronic device, and includes:

An acquisition module, configured to acquire a panoramic shooting instruction;

a shooting module, configured to start a camera for video shooting according to the panoramic shooting instruction, and display the shooting content in a preview frame;

a scanning module, configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting process to obtain a plurality of sets of scanned images, where the first direction is perpendicular to the second direction ;

a first generating module, configured to generate a corresponding partial image in the second direction according to each set of scanned images;

And a second generating module, configured to generate a panoramic image according to the partial image.

Further, the first generating module specifically includes:

Determining a sub-module for determining a corresponding key frame image group according to each set of scanned images;

Extracting a sub-module for extracting feature points of key frame images in each key frame image group;

And a splicing sub-module, configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.

Further, the splicing submodule is used to:

Calculating a corresponding first homography matrix according to feature points of adjacent two frame key frame images in the key frame image group;

Calculating a first global transformation matrix according to the first homography matrix;

The corresponding key frame image group is spliced by the first global transformation matrix.

Further, the splicing submodule is specifically configured to:

Matching feature points of adjacent two frame key frame images in the key frame image group;

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

Further, the determining submodule is specifically configured to:

In each set of scanned images, the first frame scanned image is used as a key frame image;

Obtaining a next frame scan image adjacent to the key frame image as a target frame image;

Determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image;

If yes, using the target frame image as a key frame image, and returning to perform an operation of acquiring a next frame scan image adjacent to the key frame image;

If not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.

Further, the determining submodule is specifically configured to:

Matching feature points of the key frame image with feature points of the target frame image;

Counting the number of feature points that are successfully matched in the key frame image, and the total number of feature points in the key frame image;

Determining whether the target frame image satisfies a preset condition according to the number of feature points that are successfully matched, and the total number.

Further, the determining submodule is specifically configured to:

Calculating the similarity according to the number of the feature points that are successfully matched and the total number;

When the similarity is less than the first preset threshold, and the number of the successfully matched feature points is greater than the second preset threshold, determining that the target frame image satisfies the preset condition.

Further, the second generating module is specifically configured to:

Calculating a corresponding second homography matrix according to two adjacent partial images;

Calculating a second global transformation matrix according to the second homography matrix;

The partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.

The embodiment of the present application further provides a storage medium, where the storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by a processor to execute any of the above-mentioned panoramic image generating methods.

The embodiment of the present application further provides an electronic device, including a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in any one of the foregoing The steps in the panoramic image generation method described.

DRAWINGS

The technical solutions and other advantageous effects of the present application will be apparent from the detailed description of the embodiments of the present application.

FIG. 1 is a schematic flowchart diagram of a method for generating a panoramic image according to an embodiment of the present application.

FIG. 2 is a schematic diagram of a serpentine scan in a preview frame according to an embodiment of the present application.

FIG. 3 is another schematic flowchart of a method for generating a panoramic image according to an embodiment of the present application.

FIG. 4 is a schematic flowchart of step 203 according to an embodiment of the present application.

FIG. 5 is a schematic structural diagram of a panoramic image generating apparatus according to an embodiment of the present application.

FIG. 6 is a schematic structural diagram of a first generation module according to an embodiment of the present application.

FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present application without creative efforts are within the scope of the present application.

The embodiment of the present application provides a method, an apparatus, a storage medium, and an electronic device for generating a panoramic image.

A panoramic image generating method is applied to an electronic device, comprising: acquiring a panoramic shooting instruction; starting a camera for video shooting according to the panoramic shooting instruction, and displaying the captured content in a preview frame; in the video shooting process, in the second direction Performing a serpentine scan on the preview frame at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction; generating a corresponding partial image in the second direction according to each set of scanned images Generating a panoramic image from the partial image.

In some embodiments, the generating a corresponding partial image in the second direction according to each set of scanned images comprises:

Determining a corresponding key frame image group according to each set of scanned images;

Extracting feature points of key frame images in each key frame image group;

And splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.

In some embodiments, the splicing the key frame image group according to the feature point comprises:

In some embodiments, the calculating the corresponding first homography matrix according to the feature points of the adjacent two frame key frame images in the key frame image group includes:

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

In some embodiments, the splicing the corresponding key frame image group by using the first global transformation matrix comprises:

And each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.

In some embodiments, the determining the corresponding key frame image group according to each set of scanned images comprises:

In some embodiments, determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image includes:

In some embodiments, determining whether the target frame image satisfies a preset condition according to the number of the successfully matched feature points and the total number includes:

In some embodiments, the generating the panoramic image according to the partial image comprises:

As shown in FIG. 1 , FIG. 1 is a schematic flowchart of a method for generating a panoramic image according to an embodiment of the present application, which is applied to an electronic device, and the specific process may be as follows:

101. Obtain a panoramic shooting instruction.

In this embodiment, when the user clicks a certain button of the photographing application, the electronic device may generate a panoramic shooting instruction.

102. Start the camera for video shooting according to the panoramic shooting instruction, and display the shooting content in the preview box.

In this embodiment, the size of the preview frame may be manually set, or may be determined according to the size of the display screen of the electronic device, such as three-quarters of the size of the display screen.

103. During the video shooting process, the preview frame is subjected to a serpentine scan in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.

In this embodiment, the second direction and the first direction are usually set in advance, which may be the direction in which the preview frame length and width are located, or may be an artificially defined horizontal and vertical direction, and may of course be other settings. the way. The first direction overlap ratio refers to a scan track overlap ratio in the first direction, which can be artificially set, such as not less than 50%. After each scan, a corresponding set of scanned images is generated. The scan length of the serpentine scan can be a fixed value or a variable, and the scan trajectory overlap rate depends on the starting position of two adjacent serpentine scans, such as Referring to FIG. 2, for two adjacent serpentine scans with a fixed scan length, if the scan start points are a1 and a2, respectively, and the scan length is b, the scan trace overlap ratio is [b-(a2-a1)]/b. .

104. Generate a corresponding partial image in the second direction according to each set of scanned images.

In this embodiment, the partial image is formed by splicing in the second direction. Since each group of scanned images is not scanned for the entire preview frame, but a certain longitudinal or horizontal region of the preview frame is scanned, correspondingly formed The stitched image is also not a panoramic image but a partial image.

For example, the foregoing step 104 may specifically include:

1-1. Determine a corresponding key frame image group according to each set of scanned images.

For example, the foregoing step 1-1 may specifically include:

If not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to perform the feature point according to the key frame image and the feature point of the target frame image to determine whether the target frame image is satisfied. The operation of the preset condition.

In the embodiment, in the image processing, the feature point refers to a point where the gray value of the image changes drastically or a point with a large curvature on the edge of the image (ie, the intersection of the two edges), and each feature point may include a texture. Information such as features and color characteristics. Before the splicing of the scanned image, it is necessary to filter out the scanned image without splicing, for example, for the image with higher similarity, only one of the splicing is required, and for the image with very low similarity, the image can be discarded. , etc., so that the number of stitched images can be reduced and the stitching efficiency can be improved while ensuring the integrity of the image information.

Further, the above step "determining whether the target frame image satisfies a preset condition according to the feature point of the key frame image and the feature point of the target frame image" may specifically include:

Counting the number of feature points in the key frame image that match successfully, and the total number of feature points in the key frame image;

Whether the target frame image satisfies a preset condition is determined according to the number of the feature points that are successfully matched and the total number.

In this embodiment, the preset condition may be manually set. For example, the similarity between the key frame image and the target frame image may be within a certain range. Generally, the more the number of feature points that are successfully matched, the higher the similarity.

Further, the above step "determining whether the target frame image satisfies the preset condition according to the number of the feature points and the total number of the matching successes" may specifically include:

Calculating the similarity according to the number of the feature points and the total number of the matching successes;

When the similarity is less than the first preset threshold, and the number of successfully matched feature points is greater than the second preset threshold, determining that the target frame image satisfies the preset condition.

In this embodiment, the ratio between the number of feature points matching the success and the total number can be calculated, and the ratio is taken as the similarity. The first preset threshold and the second preset threshold may be manually set. For example, the first preset threshold may be 70%, and the second preset threshold may be 20.

1-2. Extract feature points of key frame images in each key frame image group.

1-3. Splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.

For example, the foregoing steps 1-3 may specifically include:

1-3-1. Calculate a corresponding first homography matrix according to feature points of adjacent two frame key frame images in the key frame image group.

For example, the foregoing step 1-3-1 may specifically include:

Matching feature points of adjacent two key frame images in the key frame image group;

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

In this embodiment, the homography matrix (the first homography matrix) is a conversion matrix from one image to another image mapping. Specifically, the two adjacent frames may be determined first. Which feature points are successfully matched in the frame image, and then a certain number of matching feature points are selected to calculate the corresponding homography matrix, so that the feature points in one frame of the key frame image are transformed by the homography matrix, It is in the same spatial coordinate system as the feature points in another frame of keyframe images.

1-3-2. Calculating a first global transformation matrix according to the first homography matrix.

In this embodiment, the first homography matrix of the adjacent frame key frame image may be calculated in order, for example, for a certain key frame image group M1 Mn Mn, between M1 and M2, between M2 and M3, and M3 may be calculated. The first homography matrix between M4 and Mn-1 and Mn, each homography matrix is equivalent to the local transformation matrix of the key frame image group, and finally, all the first homography matrices Uniform to the same coordinate system to obtain the first global transformation matrix.

1-3-3. splicing the corresponding key frame image group by using the first global transformation matrix.

In this embodiment, each key frame image in the key frame image group can be projected into the same coordinate space system through the global transformation matrix, and the obtained image is a spliced partial image.

105. Generate a panoramic image according to the partial image.

For example, the foregoing step 105 may specifically include:

The partial image is spliced in the first direction by the second global transformation matrix to obtain a panoramic image.

In this embodiment, since the serpentine scan is an overlapping scan in the first direction, that is, the scan trajectories of the adjacent two scan operations partially overlap, when the scan image completes the splicing in the second direction, Further splicing in the first direction, that is, the partial image spliced by each column (row) of the scanned image is spliced again in the direction in which the row (column) is located, thereby realizing two-dimensional mesh splicing, which can carry the scanned image The image information is fully utilized, the method is simple, the stitching effect is good, and the video image stitching suitable for various complicated scenes is strong, and the flexibility is strong.

It can be seen from the above that the panoramic image generating method provided by the embodiment is applied to an electronic device, by acquiring a panoramic shooting instruction, and then, according to the panoramic shooting instruction, the camera is started to perform video shooting, and the captured content is displayed in the preview box, and During the video shooting, the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction, and according to each set of scanned images The corresponding partial image is generated in the second direction, and then the panoramic image is generated according to the partial image, so that the image mosaic of various complex scenes can be applied, the method is simple, the flexibility is high, and the stitching effect is good.

In the present embodiment, the description will be made from the perspective of the panoramic image generating device, and the detailed description will be made by taking the panoramic image generating device integrated in the electronic device as an example.

Referring to FIG. 3, a method for generating a panoramic image is applied to an electronic device, and the specific process may be as follows:

201. The electronic device acquires a panoramic shooting instruction, and starts a camera to perform video shooting according to the panoramic shooting instruction, and then displays the captured content in a preview frame.

For example, when the electronic device enters the shooting interface, the panoramic shooting button of the shooting application can be clicked. At this time, the electronic device starts the camera for video shooting, and displays the shooting content in the preview box in real time.

202. During the video shooting process, the electronic device scans the preview frame in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.

For example, the first direction may be a horizontal direction, and the second direction may be a vertical direction, that is, the electronic device scans the content displayed in the preview frame in a longitudinal direction, as shown in FIG. 2, for two adjacent scan lengths. For a serpentine scan, if the scan start points are a1 and a2, respectively, and the scan length is b, the horizontal scan overlap ratio is [b-(a2-a1)]/b, for example, b/a may be not less than 50%.

203. The electronic device determines a corresponding key frame image group according to each group of scanned images.

For example, referring to FIG. 4, the foregoing step 203 may specifically include:

2031. In each group of scanned images, the electronic device uses the first frame scanned image as a key frame image;

2032. The electronic device acquires a next frame scan image adjacent to the key frame image as a target frame image.

The electronic device determines, according to the feature point of the key frame image and the feature point of the target frame image, whether the target frame image satisfies a preset condition, and if yes, uses the target frame image as a key frame image, and returns to perform the above steps. 2032. If no, the next frame scan image adjacent to the target frame image is used as the target frame image, and the process returns to step 2033.

In this embodiment, before the splicing of the scanned image, it is necessary to filter out the scanned image without using for splicing, for example, for the image with higher similarity, only one of the splicing is required, and for the similarity is extremely low. The image can be discarded, and the like, so that the number of the spliced images can be reduced and the splicing efficiency can be improved, that is, the above step 2033 can further include:

In this embodiment, the more the feature points are successfully matched, the higher the similarity between the two is represented, and the lower the similarity is, the lower the number of feature points that are successfully matched is greater than the second preset threshold, such as 20 o'clock. It can be considered that the similarity between the two is not lower than the minimum. When the ratio of the number of matching feature points to the total number is less than the first preset threshold, such as 70%, the similarity between the two can be considered not higher than the maximum. Only the target frame image that satisfies both of these conditions can be considered as a key frame image if it is considered to satisfy the preset condition.

204. The electronic device extracts feature points of the key frame image in each key frame image group, and matches feature points of adjacent two frame key frame images in the key frame image group.

205. The electronic device acquires location information of the feature point that is successfully matched, and calculates a first homography matrix according to the location information.

206. The electronic device calculates a first global transformation matrix according to the first homography matrix, and splices the corresponding key frame image group by using the first global transformation matrix to obtain a partial image.

For example, since each key frame image group is obtained by a single vertical scan, when the key frame image group is processed by image stitching, the partial images of each key frame image group are corresponding to the vertical scan. The image information included in the region may be calculated by first calculating the first homography matrix of the adjacent key frame image, and then normalizing all the first homography matrices to obtain a global transformation matrix, and using the global The transformation matrix projects each key frame image in the key frame image group into the same coordinate space, completing the vertical splicing of each key frame image group.

207. The electronic device calculates a corresponding second homography matrix according to the two adjacent partial images, and calculates a second global transformation matrix according to the second homography matrix, and then uses the second global transformation matrix to use the partial image. Splicing is performed in the first direction to obtain a panoramic image.

For example, since the serpentine scan has a certain lateral overlap ratio, that is, the same portion exists in two adjacent partial images, the partial image can be further spliced horizontally, and the splicing method can be second to two. The homography matrix, then normalizes the second homography matrix to obtain a second global transformation matrix, and then uses the second global transformation matrix to project all partial images into a coordinate space to complete the lateral direction of the partial image. splice.

It can be seen from the above that the panoramic image generating method provided in this embodiment is applied to an electronic device, wherein the electronic device can acquire a panoramic shooting instruction, and start a camera to perform video shooting according to the panoramic shooting instruction, and then display the captured content in a preview box. Then, during the video shooting, the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction, and then, according to each The group scan image determines a corresponding key frame image group, and then extracts feature points of the key frame image in each key frame image group, and matches feature points of adjacent two frame key frame images in the key frame image group, and then Obtaining location information of the matching feature point, and calculating a first homography matrix according to the location information, and then calculating a first global transformation matrix according to the first homography matrix, and using the first global transformation matrix pair The corresponding key frame image group is spliced to obtain a partial image, and then, the pair is calculated according to two adjacent partial images. a second homography matrix, and calculating a second global transformation matrix according to the second homography matrix, and then splicing the partial image in the first direction by using the second global transformation matrix to obtain a panoramic image, thereby enabling The two-dimensional grid is used to generate a panoramic image, which is suitable for image mosaic of various complicated scenes, and has the advantages of simple method, high flexibility and good stitching effect.

According to the method described in the foregoing embodiment, the present embodiment will be further described from the perspective of the panoramic image generating device. The panoramic image generating device may be implemented as an independent entity or integrated in an electronic device, such as a terminal. The terminal may include a mobile phone, a tablet computer, a personal computer, and the like.

The embodiment of the present application provides a panoramic image generating apparatus, which is applied to an electronic device, and includes:

An acquisition module, configured to acquire a panoramic shooting instruction;

In some embodiments, the first generating module specifically includes:

In some embodiments, the splicing sub-module is used to:

In some embodiments, the splicing sub-module is specifically configured to:

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

In some embodiments, the splicing sub-module is specifically configured to:

In some embodiments, the determining sub-module is specifically configured to:

In some embodiments, the second generating module is specifically configured to:

Referring to FIG. 5 , FIG. 5 specifically describes a panoramic image generating apparatus according to an embodiment of the present application, which is applied to an electronic device, and the panoramic image generating apparatus may include: an acquiring module 10, a shooting module 20, a scanning module 30, and a first generating module. 40 and a second generation module 50, wherein:

(1) Acquisition module 10

The obtaining module 10 is configured to acquire a panoramic shooting instruction.

(2) shooting module 20

The shooting module 20 is configured to start a camera for video shooting according to the panoramic shooting instruction, and display the captured content in a preview frame.

(3) Scanning module 30

The scanning module 30 is configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction.

In this embodiment, the second direction and the first direction are usually set in advance, which may be the direction in which the preview frame length and width are located, or may be an artificially defined horizontal and vertical direction, and may of course be other settings. the way. The first direction overlap ratio refers to a scan track overlap ratio in the first direction, which can be artificially set, such as not less than 50%. After each scan, a corresponding set of scanned images is generated. The scan length of the serpentine scan can be a fixed value or a variable, and the scan trajectory overlap rate depends on the starting position of two adjacent serpentine scans, such as Referring to FIG. 2, for two adjacent serpentine scans with a fixed scan length, if the scan length is a and the overlap length in the first direction is b, the scan track overlap ratio is b/a.

(4) First generation module 40

The first generating module 40 is configured to generate a corresponding partial image in the second direction according to each set of scanned images.

For example, referring to FIG. 6, the first generating module 40 may specifically include a determining submodule 41, an extracting submodule 42 and a splicing submodule 43, where:

The determining sub-module 41 is configured to determine a corresponding key frame image group according to each set of scanned images.

For example, the determining sub-module 41 can be specifically used to:

Further, the determining sub-module 41 is specifically configured to:

The extraction sub-module 42 is configured to extract feature points of the key frame image in each key frame image group.

The splicing sub-module 43 is configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.

For example, the splicing submodule 43 can be specifically used to:

At this time, the splicing sub-module 43 can be specifically used to:

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

(5) Second generation module 50

The second generating module 50 is configured to generate a panoramic image according to the partial image.

For example, the second generating module 50 can be specifically configured to:

In the specific implementation, the foregoing units may be implemented as a separate entity, or may be implemented in any combination, and may be implemented as the same or a plurality of entities. For the specific implementation of the foregoing, refer to the foregoing method embodiments, and details are not described herein.

It can be seen from the above that the panoramic image generating method provided in this embodiment is applied to an electronic device, and the panoramic shooting instruction is acquired by the acquiring module 10. Then, the shooting module 20 starts the camera to perform video shooting according to the panoramic shooting instruction, and displays the captured content in the In the preview frame, during the video shooting process, the scanning module 30 performs a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second The first generation module 40 generates a corresponding partial image in the second direction according to each set of scanned images, and then the second generation module 50 generates a panoramic image according to the partial image, so that the image mosaic can be applied to various complex scenes. The method is simple, the flexibility is high, and the stitching effect is good.

In addition, the embodiment of the present application further provides an electronic device, which may be a device such as a smart phone or a tablet computer. As shown in FIG. 7, the electronic device 900 includes a processor 901, a memory 902, a display screen 903, and a control circuit 904. The processor 901 is electrically connected to the memory 902, the display screen 903, and the control circuit 904, respectively.

The processor 901 is a control center of the electronic device 900, and connects various parts of the entire electronic device using various interfaces and lines, executes the electronic by running or loading an application stored in the memory 902, and calling data stored in the memory 902. The various functions and processing data of the device enable overall monitoring of the electronic device.

In this embodiment, the processor 901 in the electronic device 900 loads the instructions corresponding to the process of one or more applications into the memory 902 according to the following steps, and is stored in the memory 902 by the processor 901. In the application, thus implementing various functions:

Obtain a panoramic shooting instruction;

Starting the camera to perform video shooting according to the panoramic shooting instruction, and displaying the shooting content in the preview box;

A panoramic image is generated based on the partial image.

Extracting feature points of key frame images in each key frame image group;

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.

Memory 902 can be used to store applications and data. The application stored in the memory 902 contains instructions executable in the processor. Applications can form various functional modules. The processor 901 executes various functional applications and data processing by running an application stored in the memory 902.

The display screen 903 can be used to display information entered by the user or information provided to the user as well as various graphical user interfaces of the terminal, which can be composed of images, text, icons, video, and any combination thereof.

The control circuit 904 is electrically connected to the display screen 903 for controlling the display screen 903 to display information.

In some embodiments, as shown in FIG. 7, the electronic device 900 further includes a radio frequency circuit 905, an input unit 906, an audio circuit 907, a sensor 908, and a power source 909. The processor 901 is electrically connected to the radio frequency circuit 905, the input unit 906, the audio circuit 907, the sensor 908, and the power source 909, respectively.

The radio frequency circuit 905 is used for transmitting and receiving radio frequency signals to establish wireless communication with network devices or other electronic devices through wireless communication, and to transmit and receive signals with network devices or other electronic devices.

The input unit 906 can be configured to receive input digits, character information, or user characteristic information (eg, fingerprints), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function controls. The input unit 906 can include a fingerprint identification module.

The audio circuit 907 can provide an audio interface between the user and the terminal through a speaker and a microphone.

Electronic device 900 may also include at least one type of sensor 908, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may close the display panel and/or the backlight when the terminal moves to the ear. . As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the terminal can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.

Power source 909 is used to power various components of electronic device 900. In some embodiments, the power supply 909 can be logically coupled to the processor 901 through a power management system to enable functions such as managing charging, discharging, and power management through the power management system.

Although not shown in FIG. 7, the electronic device 900 may further include a camera, a Bluetooth module, and the like, and details are not described herein.

It will be understood by those skilled in the art that all or part of the steps of the various methods in the above embodiments may be completed by instructions or controlled by related hardware, which may be stored in a computer readable storage medium. And loaded and executed by the processor. To this end, an embodiment of the present invention provides a storage medium in which a plurality of instructions are stored, which can be loaded by a processor to perform the steps in any of the panoramic image generating methods provided by the embodiments of the present invention.

The storage medium may include: a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The steps in the method for generating a panoramic image according to any embodiment of the present invention can be performed by using the instructions stored in the storage medium. Therefore, any method for generating a panoramic image provided by the embodiments of the present invention can be implemented. For the beneficial effects of the implementation, please refer to the previous embodiments, and details are not described herein again.

For the specific implementation of the foregoing operations, refer to the foregoing embodiments, and details are not described herein again.

In the above, although the present application has been disclosed in the above preferred embodiments, the preferred embodiments are not intended to limit the application, and those skilled in the art can make various modifications without departing from the spirit and scope of the application. The invention is modified and retouched, and the scope of protection of the present application is determined by the scope defined by the claims.

Claims

A panoramic image generating method is applied to an electronic device, including:

Obtain a panoramic shooting instruction;

Activating a camera for video shooting according to the panoramic shooting instruction, and displaying the captured content in a preview frame;

During the video shooting, the preview frame is subjected to a serpentine scan at a preset first direction overlap ratio in the second direction to obtain a plurality of sets of scanned images, the first direction being perpendicular to the second direction;

Generating a corresponding partial image in the second direction according to each set of scanned images;

A panoramic image is generated based on the partial image.
The panoramic image generating method according to claim 1, wherein the generating a corresponding partial image in the second direction according to each set of scanned images comprises:

Determining a corresponding key frame image group according to each set of scanned images;

Extracting feature points of key frame images in each key frame image group;

And splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.
The method of generating a panoramic image according to claim 2, wherein the splicing the key frame image group according to the feature point comprises:

Calculating a corresponding first homography matrix according to feature points of adjacent two frame key frame images in the key frame image group;

Calculating a first global transformation matrix according to the first homography matrix;

The corresponding key frame image group is spliced by the first global transformation matrix.
The method for generating a panoramic image according to claim 3, wherein the calculating the corresponding first homography matrix according to the feature points of the adjacent two frame key frame images in the key frame image group comprises:

Matching feature points of adjacent two frame key frame images in the key frame image group;

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.
The method of generating a panoramic image according to claim 3, wherein the splicing the corresponding key frame image group by using the first global transformation matrix comprises:

And each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.
The panoramic image generating method according to claim 2, wherein the determining the corresponding key frame image group according to each set of scanned images comprises:

In each set of scanned images, the first frame scanned image is used as a key frame image;

Obtaining a next frame scan image adjacent to the key frame image as a target frame image;

Determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image;

If yes, using the target frame image as a key frame image, and returning to perform an operation of acquiring a next frame scan image adjacent to the key frame image;

If not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
The panoramic image generating method according to claim 6, wherein the determining whether the target frame image satisfies a preset condition according to the feature point of the key frame image and the feature point of the target frame image comprises:

Matching feature points of the key frame image with feature points of the target frame image;

Counting the number of feature points that are successfully matched in the key frame image, and the total number of feature points in the key frame image;

Determining whether the target frame image satisfies a preset condition according to the number of feature points that are successfully matched, and the total number.
The panoramic image generating method according to claim 7, wherein the determining whether the target frame image satisfies a preset condition according to the number of the feature points and the total number of the matching successful points includes:

Calculating the similarity according to the number of the feature points that are successfully matched and the total number;

When the similarity is less than the first preset threshold, and the number of the successfully matched feature points is greater than the second preset threshold, determining that the target frame image satisfies the preset condition.
The panoramic image generating method according to claim 1, wherein the generating the panoramic image according to the partial image comprises:

Calculating a corresponding second homography matrix according to two adjacent partial images;

Calculating a second global transformation matrix according to the second homography matrix;

The partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
A panoramic image generating device is applied to an electronic device, including:

An acquisition module, configured to acquire a panoramic shooting instruction;

a shooting module, configured to start a camera for video shooting according to the panoramic shooting instruction, and display the shooting content in a preview frame;

a scanning module, configured to perform a serpentine scan on the preview frame in a second direction at a preset first direction overlap ratio during the video shooting process to obtain a plurality of sets of scanned images, where the first direction is perpendicular to the second direction ;

a first generating module, configured to generate a corresponding partial image in the second direction according to each set of scanned images;

And a second generating module, configured to generate a panoramic image according to the partial image.
The panoramic image generating apparatus according to claim 10, wherein the first generating module specifically comprises:

Determining a sub-module for determining a corresponding key frame image group according to each set of scanned images;

Extracting a sub-module for extracting feature points of key frame images in each key frame image group;

And a splicing sub-module, configured to splicing the key frame image group according to the feature point to generate a corresponding partial image in the second direction.
The panoramic image generating apparatus according to claim 11, wherein said splicing sub-module is used to:

Calculating a corresponding first homography matrix according to feature points of adjacent two frame key frame images in the key frame image group;

Calculating a first global transformation matrix according to the first homography matrix;

The corresponding key frame image group is spliced by the first global transformation matrix.
The panoramic image generating apparatus according to claim 12, wherein the splicing sub-module is specifically configured to:

Matching feature points of adjacent two frame key frame images in the key frame image group;

Obtaining location information of the feature points that match the success;

A first homography matrix is calculated based on the location information.
The panoramic image generating apparatus according to claim 12, wherein the splicing sub-module is specifically configured to:

And each key frame image in the key frame image group is projected into the same coordinate space system by the first global transformation matrix to splicing the key frame image group.
The panoramic image generating apparatus according to claim 11, wherein the determining submodule is specifically configured to:

In each set of scanned images, the first frame scanned image is used as a key frame image;

Obtaining a next frame scan image adjacent to the key frame image as a target frame image;

Determining whether the target frame image satisfies a preset condition according to a feature point of the key frame image and a feature point of the target frame image;

If yes, using the target frame image as a key frame image, and returning to perform an operation of acquiring a next frame scan image adjacent to the key frame image;

If not, the next frame scan image adjacent to the target frame image is used as the target frame image, and returning to performing the feature point according to the key frame image and the feature point of the target frame image to determine the target Whether the frame image satisfies the preset condition.
The panoramic image generating apparatus according to claim 15, wherein the determining submodule is specifically configured to:

Matching feature points of the key frame image with feature points of the target frame image;

Counting the number of feature points that are successfully matched in the key frame image, and the total number of feature points in the key frame image;

Determining whether the target frame image satisfies a preset condition according to the number of feature points that are successfully matched, and the total number.
The panoramic image generating apparatus according to claim 16, wherein the determining submodule is specifically configured to:

Calculating the similarity according to the number of the feature points that are successfully matched and the total number;

When the similarity is less than the first preset threshold, and the number of the successfully matched feature points is greater than the second preset threshold, determining that the target frame image satisfies the preset condition.
The panoramic image generating apparatus according to claim 10, wherein the second generating module is specifically configured to:

Calculating a corresponding second homography matrix according to two adjacent partial images;

Calculating a second global transformation matrix according to the second homography matrix;

The partial image is spliced in the first direction by using the second global transformation matrix to obtain a panoramic image.
A storage medium, wherein the storage medium stores a plurality of instructions adapted to be loaded by a processor to perform the panoramic image generating method of claim 1.
An electronic device comprising a processor and a memory, the processor being electrically connected to the memory, the memory for storing instructions and data, the processor for performing the panoramic image generating method of claim 1. The steps in .