WO2023029113A1

WO2023029113A1 - Image splicing method, terminal device, and computer-readable storage medium

Info

Publication number: WO2023029113A1
Application number: PCT/CN2021/119198
Authority: WO
Inventors: 华杰
Original assignee: 广东艾檬电子科技有限公司
Priority date: 2021-08-31
Filing date: 2021-09-18
Publication date: 2023-03-09
Also published as: CN115760561A

Abstract

An image splicing method, a terminal device, and a computer-readable storage medium. The method comprises: first, obtaining two continuous frames of a first image and a second image, determining a first image region of the first image and a second image region of the second image, and matching the first image region and the second image region to obtain a first matching degree; then, when the first matching degree is less than or equal to a matching degree threshold, correcting the first matching degree by means of one or more adjustments of a matching region of the first image region and the second image region until the first matching degree is corrected successfully or the number of matchings is greater than a threshold of the number of times, and obtaining a matching result; and finally, when the matching result shows that matching is successful, determining first relative displacement of the first image and the second image according to the first image region and the second image region that are matched successfully, and splicing the first image and the second image according to the first relative displacement. The method can improve the image splicing accuracy and the splicing effect.

Description

Image stitching method, terminal device and computer-readable storage medium

technical field

The present application relates to the technical field of image processing, and in particular to an image stitching method, a terminal device, and a computer-readable storage medium.

Background technique

It has been found in practice that the process of image stitching mostly includes: matching the two images to be stitched, obtaining the matching degree and relative displacement, and when the matching degree of the two images reaches the preset matching degree, splicing according to the relative displacement .

In the prior art, the two images to be stitched are often matched based on templates or feature points, and the obtained matching results are usually not accurate enough, and the stitching effect is not good.

Contents of the invention

Embodiments of the present application provide an image stitching method, a terminal device, and a computer-readable storage medium, which can improve the accuracy of image stitching and improve the effect of image stitching.

The first aspect of the embodiment of the present application provides an image stitching method, including:

acquiring a first image and a second image, and determining a first image area of the first image and a second image area of the second image;

matching the first image area and the second image area to obtain a first matching degree;

If the first matching degree is less than or equal to the matching degree threshold, adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area, and continue to execute the Describe the step of matching the first image area and the second image area to obtain a first matching degree, until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold, and a matching result is obtained; wherein, The first image area after adjustment is smaller than the first image area before adjustment, the second image area after adjustment is smaller than the second image area before adjustment, and the first image area for matching is the same size as the second image area;

If the matching result is a successful match, then determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area;

Stitching the first image and the second image according to the first relative displacement.

As an optional implementation manner, in the first aspect of the embodiment of the present application, the adjusting of the first image area and the second image area respectively obtains a new first image area and a new second image area, including:

Cutting off the first area from the edge of the first side of the first image area to obtain a new first image area, and cutting off the second area from the edge of the second side of the second image area to obtain A new second image area; wherein, the edge of the first side and the edge of the second side are both perpendicular to the horizontal line, and the size of the first area and the second area are the same.

As an optional implementation manner, in the first aspect of the embodiment of the present application, according to the successfully matched first image area and the second image area, determining the first image of the first image and the second image Relative displacement, including:

Obtaining a second relative displacement of the successfully matched first image area and the second image area;

A first relative displacement between the first image and the second image is determined according to the second relative displacement and, the first region and the number of the first regions.

Taking the central point of the first image area as a reference, cropping the middle area of the first image area according to a first preset ratio, and using the middle area of the first image area obtained by cutting as; The central point of the second image area is used as a reference, the middle area of the second image area is cut according to the first preset ratio, and the middle area of the second image area obtained by cutting is used as the new second image area. Two image areas.

The second relative displacement is used as a first relative displacement between the first image and the second image.

As an optional implementation manner, in the first aspect of the embodiments of the present application, the method further includes:

If the matching result indicates a matching failure, acquiring a third relative displacement between a third image and the first image; the third image is an image successfully matched with the first image;

Stitching the first image and the second image according to the third relative displacement.

As an optional implementation manner, in the first aspect of the embodiment of the present application, the determining the first image area of the first image and the second image area of the second image includes:

respectively performing edge detection on the first image and the second image to obtain detection results;

When the detection result indicates that both the first image and the second image have valid content, determine a first image area of the first image and a second image area of the second image.

As an optional implementation manner, in the first aspect of the embodiments of the present application, when the detection result indicates that both the first image and the second image have valid content, determining the first image the first image area and the second image area of the second image, comprising:

When the detection result indicates that both the first image and the second image have valid content, respectively perform preset compression operations on the first image and the second image;

A first image area of the compressed first image and a second image area of the compressed second image are determined.

As an optional implementation manner, in the first aspect of the embodiments of the present application, the stitching the first image and the second image according to the first relative displacement includes:

Stitching the first image and the second image according to the first relative displacement to obtain a stitched image to be processed;

Smoothing is performed on the image to be processed to obtain a target spliced image.

The second aspect of the embodiment of the present application provides a terminal device, including:

an acquiring unit, acquiring a first image and a second image, and determining a first image area of the first image and a second image area of the second image;

a matching unit, configured to match the first image area and the second image area to obtain a first matching degree;

An adjustment unit, configured to adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area if the first matching degree is less than or equal to a matching degree threshold , and continue to execute the step of matching the first image area and the second image area to obtain the first matching degree until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold, obtaining Matching result; wherein, the first image area after adjustment is smaller than the first image area before adjustment, the second image area after adjustment is smaller than the second image area before adjustment, and the first image area and the second image area for matching are the same size;

A determining unit, configured to determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area if the matching result is successful;

A stitching unit, configured to stitch the first image and the second image according to the first relative displacement.

The third aspect of the embodiment of the present application provides a terminal device, which may include:

a memory storing executable program code;

and a processor to which the memory is coupled;

The processor invokes the executable program code stored in the memory, and when the executable program code is executed by the processor, the processor implements the method described in the first aspect of the embodiment of the present application .

The fourth aspect of the embodiment of the present application provides a computer-readable storage medium, on which executable program code is stored. When the executable program code is executed by a processor, the method as described in the first aspect of the embodiment of the present application is implemented. .

The fifth aspect of the embodiment of the present application discloses a computer program product. When the computer program product is run on a computer, the computer is made to execute any one of the methods disclosed in the first aspect of the embodiment of the present application.

The sixth aspect of the embodiment of the present application discloses an application distribution platform. The application distribution platform is used to distribute computer program products, wherein, when the computer program product runs on a computer, the computer executes the application disclosed in the first aspect of the embodiment of the present application. any of the described methods.

It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

In the embodiment of the present application, firstly, two consecutive frames of the first image and the second image are obtained, and the first image area of the first image and the second image area of the second image are determined, and the first image area and the second image area are determined. Two image areas are matched to obtain the first matching degree; then, when the first matching degree is less than or equal to the matching degree threshold, the first image area and the second image area are respectively adjusted to obtain a new first image area and a new second image area. Two image areas, and continue to perform the step of matching the first image area and the second image area to obtain the first matching degree, until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold, and the matching result is obtained; finally, When the matching result is a successful match, determine the first relative displacement of the first image and the second image according to the successfully matched first image area and the second image area, and stitch the first image and the second image according to the first relative displacement. image. By implementing this method, when the first matching degree between the first image area and the second image area is small, the first matching can be further corrected by adjusting the matching area between the first image area and the second image area one or more times. The accuracy of image stitching can be improved, and the effect of image stitching can be further improved.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings that are required in the description of the embodiments and prior art. Obviously, the accompanying drawings in the following description are only some implementations of the present application For example, other drawings can also be obtained from these drawings.

FIG. 1 is a schematic diagram of a scene disclosed in an embodiment of the present application;

FIG. 2 is a schematic flow diagram of an image stitching method disclosed in an embodiment of the present application;

FIG. 3A is a schematic flowchart of another image stitching method disclosed in the embodiment of the present application;

Fig. 3B is a schematic diagram of acquiring a new first image area and a new second image area disclosed in the embodiment of the present application;

Fig. 3C is another schematic diagram of obtaining a new first image area and a new second image area disclosed in the embodiment of the present application;

FIG. 4 is a structural block diagram of a terminal device disclosed in an embodiment of the present application;

FIG. 5 is a block diagram of a partial structure of a scanning pen related to a terminal device disclosed in an embodiment of the present application;

Fig. 6 is another structural block diagram of a terminal device disclosed in an embodiment of the present application.

Detailed ways

Embodiments of the present application provide an image stitching method, a terminal device, and a computer-readable storage medium, which can improve image matching accuracy and image stitching effects.

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be described below in conjunction with the drawings in the embodiment of the application. Obviously, the described embodiment is only a part of the application Examples, but not all examples. Based on the embodiments in this application, all should belong to the protection scope of this application.

It can be understood that the terminal devices involved in the embodiments of the present application may include general handheld screen electronic terminal devices, such as scanning pens, smart phones, portable terminals, terminals, personal digital assistants (Personal Digital Assistant, PDA), portable Multimedia player (Personal Media Player, PMP) device, notebook computer, notebook (Note Pad), wireless broadband (Wireless Broadband, Wibro) terminal, tablet computer (Personal Computer, PC), smart PC, sales terminal (Point of Sales, POS) and on-board computer, etc.

Terminal devices may also include wearable devices. A wearable device is a portable electronic device that can be worn directly on the user, or integrated into the user's clothing or accessories. Wearable device is not only a hardware device, but also can realize powerful intelligent functions through software support, data interaction and cloud interaction, such as: calculation function, positioning function, alarm function, and can also connect scanning pen and various terminals . Wearable devices may include, but are not limited to, wrist-supported watches (such as watches, wrists, etc.), feet-supported shoes (such as shoes, socks, or other leg-wearing products), head-supported Glass (such as glasses, helmets, headbands, etc.) and smart clothing, school bags, crutches, accessories and other non-mainstream product forms.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of a scene disclosed in an embodiment of the present application. The schematic diagram shown in FIG. 1 may include a scanning pen 10 and a paper book 20 .

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of an image stitching method disclosed in an embodiment of the present application. Can include the following steps:

201. Acquire a first image and a second image.

In the embodiment of the present application, the first image and the second image may be two consecutive or discontinuous images, which is not limited in the embodiment of the present application. The first image and the second image can be obtained by photographing the content of the paper book with a camera installed on the terminal device. Wherein, an acceleration sensor may be installed on the terminal device, and whether the terminal device moves on the paper book can be detected by the acceleration sensor, and when the terminal device is detected to move, multiple frames of images are collected continuously through the camera.

202. Determine a first image area of the first image and a second image area of the second image.

In the embodiment of the present application, the first image area and the second image area have the same size, the first image area is a partial area of the first image, and the second image area is a partial area of the second image.

In some embodiments, determining the first image area of the first image and the second image area of the second image may include: taking the center point of the first image as a reference, and aligning the center point of the first image according to a second preset ratio The area is cropped, and the middle area of the first image obtained by cutting is used as the first image area; taking the center point of the second image as a reference, the middle area of the second image is cropped according to the second preset ratio, and the cropped The middle area of the obtained second image is used as the second image area. It should be noted that the second preset ratio may represent the ratio of the first image area to the first image (the ratio of the second image area to the second image). Exemplarily, the second preset ratio may be 80%, 90%, 95% and so on. In practice, it is found that the effective content of the images collected by the terminal device is mostly concentrated in the middle area of the image. Therefore, before matching, the boundary area between the first image and the second image should be removed first. In this way, the matching can effectively reduce the The amount of image data helps to improve matching efficiency.

203. Match the first image area and the second image area to obtain a first matching degree.

In the embodiment of the present application, the first matching degree is used to characterize the similarity between the first image region and the second image region.

In some embodiments, matching the first image area and the second image area to obtain the first matching degree may include but not limited to the following methods:

Mode 1. Obtain frequency domain information of the first image region and the second image region, and obtain a first matching degree according to the frequency domain information.

Wherein, in the embodiment of the present application, the frequency domain information of the first image region may include a spectrogram of the first image region, and the frequency domain information of the second image region may include a spectrogram of the second image region. Wherein, acquiring the spectrogram of the first image region may include: transforming the first image region from time domain space to frequency domain space through Fourier transform, so as to obtain the spectrogram of the first image region. Similarly, acquiring the spectrogram of the second image region may include: transforming the second image region from time-domain space to frequency-domain space by Fourier transform to obtain the spectrogram of the second image region. Further, obtaining the first matching degree according to the frequency domain information may include: obtaining a matching degree between the spectrogram of the first image region and the spectrogram of the second image region.

Mode 2, extracting the first feature point information of the first image area, and extracting the second feature point information of the second image; calculating the similarity between the first feature point information and the second feature point information, and determining the second feature point information according to the similarity A degree of matching.

Wherein, the first feature point information may include coordinates and descriptors of multiple first feature points corresponding to the first image area, and the second feature point information includes coordinates and descriptors of multiple second feature points corresponding to the second image area . The feature points (the first feature point and the second feature point) represent points with relatively rich information in the first/second area image, such as edge contour points, bright spots in darker areas, dark points in brighter areas, etc. . A descriptor can be a data structure used to characterize feature points, and the dimension of a descriptor can be one-dimensional or multi-dimensional, which is not limited here. The similarity is proportional to the first matching degree, that is, the greater the similarity, the greater the first matching degree.

In some embodiments, calculating the similarity between the first feature point information and the second feature point information may include: determining a matching feature point pair according to the descriptor of the first feature point and the descriptor of the second feature point; The number of point pairs determines the similarity between the first feature point information and the second feature point information. It should be noted that the number of matching feature point pairs is proportional to the similarity, that is, the greater the number of matching feature point pairs, the greater the similarity.

204. Determine whether the first matching degree is greater than a matching degree threshold.

It should be noted that, when the judgment result of step 204 is yes, continue to execute step 207; when the judgment result of step 204 is no, continue to execute step 205.

In the embodiment of the present application, the matching threshold is a value used to define whether the first image area and the second image area are similar. It can be understood that if the first matching degree is greater than the matching threshold, the first image area is similar to the second image area; otherwise, the first image area is not similar to the second image area.

In some embodiments, the matching degree threshold may be represented by a percentage. Exemplarily, the matching degree threshold may be 85%, 90%, 93% or 92%.

205. Determine whether the number of matching times is greater than the number of times threshold.

It should be noted that, when the judgment result of step 205 is yes, continue to execute step 207; when the judgment result of step 205 is no, continue to execute step 206.

In the embodiment of the present application, the number of times threshold can be obtained through a large number of experiments. Exemplarily, the number of times threshold may be 5 times, 6 times or 7 times, which is not limited in this embodiment of the present application.

206. Adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area.

It should be noted that after step 206, step 203-step 204 are continuously executed.

Wherein, the first image area after each adjustment is smaller than the first image area before adjustment, the second image area after each adjustment is smaller than the second image area before adjustment, and the first image area after each adjustment is the same as the adjusted After the second image area is the same size.

207. Obtain a matching result.

In the embodiment of the present application, when the first matching degree is greater than the matching degree threshold, the corresponding matching result is matching success; when the number of matching times is greater than the number threshold, the corresponding matching result is matching failure.

208. If the matching result is successful, determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area.

In the embodiment of the present application, the first relative displacement may represent the moving distance and moving direction of the second image relative to the first image in the horizontal direction; or, the movement of the first image relative to the second image in the horizontal direction The distance and moving direction are not limited in this embodiment of the application.

209. Stitch the first image and the second image according to the first relative displacement.

In the embodiment of the present application, stitching the first image and the second image according to the first relative displacement may include: determining the stitching position on the first image according to the first relative displacement; Move to the splicing bit position to realize the splicing of the first image and the second image. It should be noted that the first side edges of the second image are all perpendicular to the horizontal line.

In some embodiments, when the corresponding matching result indicates that the matching fails, a third relative displacement between the third image and the first image may also be obtained, and the third image is an image successfully matched with the first image; according to the third Relative displacement, stitching the first image and the second image.

Among them, it has been found in practice that since there is often a small difference in the effective content corresponding to the images of adjacent multiple frames collected by the terminal device, when the matching between the first image and the second image fails, the first image and the second image can be directly used. Matching the first image and the second image with the matching result of the successfully matched image is beneficial to improve the success rate of image splicing.

By implementing the above method, when the first matching degree between the first image area and the second image area is small, the first matching degree can be adjusted by adjusting the matching area mode of the first image area and the second image area one or more times. The correction is beneficial to improve the accuracy of image stitching and improve the effect of image stitching.

Please refer to FIG. 3A . FIG. 3A is a schematic flowchart of another image stitching method disclosed in an embodiment of the present application. Can include the following steps:

301. Acquire a first image and a second image.

Wherein, for the detailed description of step 301, please refer to the introduction of step 201 in the embodiment shown in FIG. 2 , which will not be repeated here.

302. Perform edge detection on the first image and the second image respectively to obtain a detection result.

In the embodiment of the present application, a preset edge detection operator may be used to perform edge detection on the first image and the second image respectively to obtain detection results. Wherein, the default edge detection operator can include any of the following: Robert operator, Prewitt operator, Sobel operator (Sobel operator, sobel operator and Canny operator. It should be noted that the Roberts operator is An operator that uses local differential operators to find edges; Prewitt operator is a first-order differential operator; Canny operator is a multi-level edge detection operator.

It can be understood that, performing edge detection on the first image and the second image may be detecting an outline of effective content in the first image and an outline of effective content in the second image.

303. When the detection result indicates that both the first image and the second image have valid content, determine a first image area of the first image and a second image area of the second image.

In some embodiments, determining the first image area of the first image and the second image area of the second image may include:

Perform preset compression operations on the first image and the second image respectively; determine the compressed first image area of the first image and the compressed second image area of the second image.

Further, in some embodiments, determining the first image area of the compressed first image and the second image area of the compressed second image includes: taking the center point of the compressed first image as a reference, according to the first Two preset ratios are used to cut out the middle area of the compressed first image, and use the cut out middle area of the compressed first image as the first image area; taking the center point of the reduced second image as a reference, The middle area of the shrunken second image is cropped according to a second preset ratio, and the cut and compressed middle area of the second image is used as the second image area. By implementing this method, before determining the first image area in the first image and the second image area in the second image, the first image and the second image are compressed, so that the determined first image can be reduced. The amount of image data in the area and the second image area is beneficial to improve the efficiency of image matching.

304. Match the first image area and the second image area to obtain a first matching degree.

305. Determine whether the first matching degree is greater than a matching degree threshold.

It should be noted that, in the embodiment of the present application, if the judgment result of step 305 is yes, continue to execute step 308 , and if the judgment result of step 305 is no, continue to execute step 306 .

306. Determine whether the number of times of matching is greater than the number of times threshold.

It should be noted that, in the embodiment of the present application, if the judgment result of step 306 is yes, continue to execute step 308 , and if the judgment result of step 306 is no, continue to execute step 307 .

307. Adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area.

It should be noted that, after step 307 is executed, step 304 is continued to be executed.

In some embodiments, adjusting the first image area and the second image area respectively to obtain a new first image area and a new second image area may include but not limited to the following methods:

Method 1. Cut off the first area from the edge of the first side of the first image area to obtain a new first image area, and cut off the second area from the edge of the second side of the second image area to obtain a new The second image area of ; wherein, the edge of the first side and the edge of the second side are both perpendicular to the horizontal line, and the size of the first area and the second area are the same.

Please refer to FIG. 3B . FIG. 3B is a schematic diagram of acquiring a new first image area and a new second image area disclosed in an embodiment of the present application. As shown in FIG. 3B , it includes a first image area 01 , a new first image area 02 , a second image area 03 , and a new second image area 04 . It should be noted that the first side may be the left side of the first image area 01, the second side may be the right side of the second image area 03, the height of the first area is the same as the height of the first image area 01, and the second side may be the right side of the second image area 03. The height of the area is the same as the height of the second image area 03 .

Method 2. Taking the central point of the first image area as a reference, crop the middle area of the first image area according to a first preset ratio, and use the cut middle area of the first image area as a new first image area ; Based on the center point of the second image area, crop the middle area of the second image area according to the first preset ratio, and use the middle area of the second image area obtained by cropping as a new second image area.

Please refer to FIG. 3C . FIG. 3C is another schematic diagram of acquiring a new first image area and a new second image area disclosed in an embodiment of the present application. As shown in FIG. 3C , it includes a first image area 05 , a new first image area 06 , a second image area 07 and a new second image area 08 . It should be noted that the first preset ratio may represent the ratio of the new first image area to the first image area before cropping (the ratio of the new second image area to the second image area before cropping).

308. Obtain a matching result.

Wherein, in the embodiment of the present application, for the detailed description of step 305-step 306 and step 308, please refer to the introduction of step 204-step 205 and step 207 in the embodiment shown in FIG. .

309. If the matching result is successful, determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area.

Further, in this embodiment of the present application, determining the first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area may include but not limited to the following methods:

If the acquisition method of the new first image area and the new second image area is the above method (1), then obtain the second relative displacement of the successfully matched first image area and the second image area; according to the second relative displacement, The first area and the number of the first areas determine a first relative displacement between the first image and the second image.

Wherein, it can be understood that each time the first image area is adjusted, the first area needs to be cut out, that is, if the first graphic area is adjusted n times, and n is an integer greater than or equal to 1, then n first image areas need to be cut out. region, it should be noted that the size of the first region cropped each time may be the same or different, which is not limited in this embodiment of the present application. Regarding the adjustment of the second image area, it is similar and will not be repeated here.

In some embodiments, obtaining the second relative displacement of the successfully matched first image area and the second image area may include: obtaining the successfully matched The second relative displacement of the first image area and the second image area.

In some embodiments, determining the first relative displacement between the first image and the second image according to the second relative displacement, the first area and the number of the first area may include: according to the width of the first area and the number of the first area The first distance value is obtained by the number, and the first relative displacement between the first image and the second image is obtained according to the first distance value and the second distance value indicated by the second relative displacement.

If the acquisition mode of the new first image area and the new second image area is the above-mentioned mode (2), then obtain the second relative displacement of the first image area and the second image area of the matching success; the second relative displacement is used as A first relative displacement of the first image and the second image.

It should be noted that, for the manner of obtaining the second relative displacement, refer to the above description, which will not be repeated here.

310. Stitch the first image and the second image according to the first relative displacement.

In some embodiments, splicing the first image and the second image according to the first relative displacement may include: splicing the first image and the second image according to the first relative displacement to obtain a spliced image to be processed; smoothing the image to be processed , to get the target stitched image.

In some embodiments, the image to be processed can be smoothed through a preset fusion algorithm to obtain the target spliced image. Optionally, the preset fusion algorithm may include at least one of the following: a weighted average fusion algorithm, a feathering fusion algorithm, a Laplacian pyramid fusion algorithm, a Gaussian pyramid fusion algorithm, and the like.

By implementing the method, after the spliced image to be processed is obtained by splicing the first image and the second image, the spliced image to be processed can also be smoothed, which helps to realize seamless splicing of images, and further improves the splicing effect.

By implementing the above method, when the first matching degree between the first image area and the second image area is small, the first matching degree can be adjusted by adjusting the matching area mode of the first image area and the second image area for the first time or multiple times. Make corrections to improve the accuracy of image stitching and improve the image stitching effect. Further, before determining the first image area and the second image area, the effective content detection is performed on the first image and the second image, and if there is no effective content in any of the first image and the second image, then no longer Image matching is beneficial to reduce ineffective power consumption.

Please refer to FIG. 4. FIG. 4 is a structural block diagram of a terminal device disclosed in an embodiment of the present application. It may include an acquisition unit 401, a matching unit 402, an adjustment unit 403, a determination unit 404, and a splicing unit 405; wherein:

An acquiring unit 401, acquiring a first image and a second image, and determining a first image area of the first image and a second image area of the second image;

A matching unit 402, configured to match the first image area and the second image area to obtain a first matching degree;

An adjustment unit 403, configured to adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area if the first matching degree is less than or equal to the matching degree threshold, and continue to execute the The step of matching the first image area and the second image area to obtain the first matching degree, until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold, the matching result is obtained; wherein, the adjusted first image area is less than The first image area before adjustment, the second image area after adjustment is smaller than the second image area before adjustment, and the size of the first image area for matching is the same as that of the second image area;

A determining unit 404, configured to determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area if the matching result is successful;

The stitching unit 405 is configured to stitch the first image and the second image according to the first relative displacement.

In some embodiments, the adjustment unit 403 is used to adjust the first image area and the second image area respectively, and the manner of obtaining a new first image area and a new second image area may specifically include: the adjustment unit 403 , which is used to crop the first area from the edge of the first side of the first image area to obtain a new first image area, and to crop the second area from the edge of the second side of the second image area to obtain a new The second image area of ; wherein, the edge of the first side and the edge of the second side are both perpendicular to the horizontal line, and the size of the first area and the second area are the same.

Further, the manner in which the determining unit 404 is used to determine the first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area may specifically include: the determining unit 404 is configured to obtain a matching successful The second relative displacement of the first image area and the second image area; according to the second relative displacement, the first area and the number of the first area, determine the first relative displacement of the first image and the second image.

In some embodiments, the adjustment unit 403 is used to adjust the first image area and the second image area respectively, and the manner of obtaining a new first image area and a new second image area may specifically include: the adjustment unit 403 , using the central point of the first image region as a reference, crop the middle region of the first image region according to a first preset ratio, and use the cut middle region of the first image region as a new first Image area; using the central point of the second image area as a reference, crop the middle area of the second image area according to the first preset ratio, and use the middle area of the second image area obtained by cutting as The new second image area.

Further, the manner in which the determining unit 404 is used to determine the first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area may specifically include: the determining unit 404 is configured to obtain a matching successful The second relative displacement of the first image area and the second image area; using the second relative displacement as the first relative displacement of the first image and the second image.

In some embodiments, the determining unit 404 is further configured to acquire a third relative displacement between the third image and the first image if the matching result indicates that the matching fails; the third image is an image successfully matched with the first image ; Stitching the first image and the second image according to the third relative displacement.

In some embodiments, the manner for the acquiring unit 401 to determine the first image area of the first image and the second image area of the second image may specifically include: the acquiring unit 401 is configured to respectively determine the first image and the second image Perform edge detection to obtain a detection result; when the detection result indicates that both the first image and the second image have valid content, determine the first image area of the first image and the second image area of the second image.

In some embodiments, the method for determining the first image area of the first image and the second image area of the second image by the acquiring unit 401 when the detection result indicates that both the first image and the second image have valid content may specifically be It includes: an acquisition unit 401, configured to perform a preset compression operation on the first image and the second image respectively when the detection result indicates that both the first image and the second image have valid content; determine the first image of the compressed first image The image area and the second image area of the compressed second image.

In some embodiments, the stitching unit 405 is configured to stitch the first image and the second image according to the first relative displacement. The stitching unit 405 is configured to stitch the first image and the second image according to the first relative displacement. image to obtain the spliced image to be processed; smoothing the image to be processed to obtain the target spliced image.

As shown in FIG. 5 , it is a schematic diagram of another embodiment of the terminal device in the embodiment of the present application. FIG. 5 shows a block diagram of a partial structure of the scanning pen related to the terminal device provided in the embodiment of the present application. The scanning pen may include: a camera module 510 , a memory 520 , a processor 530 , a sensor 540 and an audio circuit 550 . Those skilled in the art can understand that the structure shown in Figure 5 does not constitute a limitation on the structure of the scanning pen, and the scanning pen may include more or less components than shown in the illustration, or combine some components, or arrange different components .

The following is a specific introduction to each component of the scanning pen in combination with Figure 5:

The camera module 510 can be used to collect images, and send the collected images to the processor 530 and the memory 520 .

The memory 520 can be used to store software programs and modules, and the processor 530 executes various functional applications and data processing of the scanning pen by running the software programs and modules stored in the memory 520 . The memory 520 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playback function, an image playback function, etc.); The data created by the use of the scanning pen (such as audio data, phonebook, etc.) and the like. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

The processor 530 is the control center of the scanning pen, which uses various interfaces and lines to connect various parts of the entire scanning pen, runs or executes software programs and/or modules stored in the memory 520, and calls data stored in the memory 520 , execute various functions of the scanning pen and process data, so as to monitor the scanning pen as a whole. Optionally, the processor 530 may include one or more processing units; preferably, the processor 530 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, etc. , the modem processor mainly handles wireless communications. It can be understood that the foregoing modem processor may not be integrated into the processor 530 .

The wand may also include at least one sensor 540, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Motion sensors may include accelerometer sensors. Furthermore, the scanning pen can also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

Audio circuitry 550 may provide an audio interface between the user and the wand. The audio circuit 550 can transmit the electrical signal converted from the received audio data to the speaker, and the speaker converts it into an audio signal for output; on the other hand, the microphone converts the collected audio signal into an electrical signal, which is converted by the audio circuit 550 After processing by the audio data output processor 530, the RF circuit 510 sends the audio data to other terminals such as mobile phones, or outputs the audio data to the memory 520 for further processing.

Although not shown, the scanning pen may also include a Bluetooth module, etc., which will not be repeated here.

In the embodiment of the present application, the processor 530 included in the scanning pen also has the following functions:

Optionally, the processor 530 also has the following functions:

Acquiring two consecutive frames of the first image and the second image, and determining the first image area of the first image and the second image area of the second image;

If the first matching degree is less than or equal to the matching degree threshold, then adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area, and continue to execute the process of converting the first image area and the second image area. The step of matching the two image areas to obtain the first matching degree, until the number of matching times is greater than the number of times threshold or the first matching degree is greater than the matching degree threshold, and the matching result is obtained; wherein, the first image area after adjustment is smaller than the first image before adjustment area, the adjusted second image area is smaller than the pre-adjusted second image area, and the matched first image area is the same size as the second image area;

If the matching result is that the matching is successful, then according to the successfully matched first image area and the second image area, determine the first relative displacement of the first image and the second image;

According to the first relative displacement, the first image and the second image are stitched together.

Optionally, the processor 530 also has the following functions:

Cut off the first area from the edge of the first side of the first image area to obtain a new first image area, and cut off the second area from the edge of the second side of the second image area to obtain a new second image area Image area; wherein, the edge of the first side and the edge of the second side are both perpendicular to the horizontal line, and the size of the first area and the second area are the same.

Optionally, the processor 530 also has the following functions:

Acquire the second relative displacement of the successfully matched first image area and the second image area; determine the first relative displacement of the first image and the second image according to the second relative displacement, the first area, and the number of the first areas.

Optionally, the processor 530 also has the following functions:

Taking the central point of the first image area as a reference, the middle area of the first image area is cropped according to a first preset ratio, and the middle area of the first image area obtained by cutting is used as a new first image area; The central point of the second image area is used as a reference, the middle area of the second image area is cropped according to the first preset ratio, and the middle area of the second image area obtained by cutting is used as a new second image area.

Optionally, the processor 530 also has the following functions:

The second relative displacement is used as the first relative displacement between the first image and the second image.

Optionally, the processor 530 also has the following functions:

If the matching result indicates that the matching fails, the third relative displacement of the third image and the first image is obtained; the third image is an image successfully matched with the first image; according to the third relative displacement, the first image and the first image are stitched together Describe the second image.

Optionally, the processor 530 also has the following functions:

performing edge detection on the first image and the second image respectively to obtain detection results;

When the detection result indicates that both the first image and the second image have valid content, determine the first image area of the first image and the second image area of the second image.

Optionally, the processor 530 also has the following functions:

Stitching the first image and the second image according to the first relative displacement to obtain the spliced image to be processed;

The image to be processed is smoothed to obtain the target spliced image.

Please refer to FIG. 6. FIG. 6 is another structural block diagram of a terminal device disclosed in an embodiment of the present application. Can include:

A memory 601 storing executable program codes;

and a processor 602 coupled to the memory 601;

The processor 602 invokes the executable program code stored in the memory 601, and when the executable program code is executed by the processor 602, the processor 602 implements the image stitching method.

The embodiment of the present application discloses a computer-readable storage medium, which stores a computer program, wherein, when the computer program is executed by a processor, the methods described in the above-mentioned embodiments are implemented.

The embodiment of the present application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the foregoing embodiments.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be realized through computer programs to instruct related hardware, and the programs can be stored in a non-volatile computer-readable storage medium When the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a ROM, or the like.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, a Solid State Disk (SSD)).

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions described in each embodiment are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application.

Claims

An image mosaic method, characterized in that, comprising:

acquiring a first image and a second image, and determining a first image area of the first image and a second image area of the second image;

matching the first image area and the second image area to obtain a first matching degree;

If the first matching degree is less than or equal to the matching degree threshold, adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area, and continue to execute the Describe the step of matching the first image area and the second image area to obtain a first matching degree, until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold, and a matching result is obtained; wherein, The first image area after adjustment is smaller than the first image area before adjustment, the second image area after adjustment is smaller than the second image area before adjustment, and the first image area for matching is the same size as the second image area;

If the matching result is a successful match, then determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area;

Stitching the first image and the second image according to the first relative displacement.
The method according to claim 1, wherein said adjusting said first image area and said second image area respectively to obtain a new first image area and a new second image area comprises:

Cutting off the first area from the edge of the first side of the first image area to obtain a new first image area, and cutting off the second area from the edge of the second side of the second image area to obtain A new second image area; wherein, the edge of the first side and the edge of the second side are both perpendicular to the horizontal line, and the size of the first area and the second area are the same.
The method according to claim 2, wherein the determining the first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area comprises:

Obtaining a second relative displacement of the successfully matched first image area and the second image area;

A first relative displacement between the first image and the second image is determined according to the second relative displacement, the first region, and the number of the first regions.
The method according to claim 1, wherein the adjusting the first image area and the second image area respectively to obtain a new first image area and a new second image area comprises:

Taking the central point of the first image area as a reference, cropping the middle area of the first image area according to a first preset ratio, and using the cut out middle area of the first image area as a new second image area. An image area; using the center point of the second image area as a reference, crop the middle area of the second image area according to the first preset ratio, and use the cropped image area of the second image area The middle area serves as a new second image area.
The method according to claim 4, wherein the determining the first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area comprises:

Obtaining a second relative displacement of the successfully matched first image area and the second image area;

The second relative displacement is used as a first relative displacement between the first image and the second image.
The method according to any one of claims 1-5, wherein the method further comprises:

If the matching result indicates a matching failure, acquiring a third relative displacement between a third image and the first image; the third image is an image successfully matched with the first image;

Stitching the first image and the second image according to the third relative displacement.
The method according to any one of claims 1-5, wherein the determining the first image area of the first image and the second image area of the second image comprises:

respectively performing edge detection on the first image and the second image to obtain detection results;

When the detection result indicates that both the first image and the second image have valid content, determine a first image area of the first image and a second image area of the second image.
The method according to claim 7, wherein when the detection result indicates that both the first image and the second image have valid content, determining the first image area and the first image area of the first image The second image area of the second image includes:

When the detection result indicates that both the first image and the second image have valid content, respectively perform preset compression operations on the first image and the second image;

A first image area of the compressed first image and a second image area of the compressed second image are determined.
The method according to any one of claims 1-5, wherein the splicing the first image and the second image according to the first relative displacement comprises:

Stitching the first image and the second image according to the first relative displacement to obtain a stitched image to be processed;

Smoothing is performed on the image to be processed to obtain a target spliced image.
A terminal device, characterized in that it includes:

an acquiring unit, acquiring a first image and a second image, and determining a first image area of the first image and a second image area of the second image;

a matching unit, configured to match the first image area and the second image area to obtain a first matching degree;

An adjustment unit, configured to adjust the first image area and the second image area respectively to obtain a new first image area and a new second image area if the first matching degree is less than or equal to a matching degree threshold , and continue to execute the step of matching the first image area and the second image area to obtain the first matching degree until the number of matching times is greater than the number threshold or the first matching degree is greater than the matching degree threshold to obtain Matching result; wherein, the first image area after adjustment is smaller than the first image area before adjustment, the second image area after adjustment is smaller than the second image area before adjustment, and the matched first image area and the second image area are the same size;

A determining unit, configured to determine a first relative displacement between the first image and the second image according to the successfully matched first image area and the second image area if the matching result is successful;

A stitching unit, configured to stitch the first image and the second image according to the first relative displacement.
A terminal device, characterized in that it includes:

a memory storing executable program code;

and a processor to which the memory is coupled;

The processor invokes the executable program code stored in the memory, and when the executable program code is executed by the processor, the processor implements the method described in any one of claims 1-9. method.
A computer-readable storage medium, on which executable program codes are stored, wherein, when the executable program codes are executed by a processor, the method according to any one of claims 1-9 is implemented.