CN115174811A

CN115174811A - Camera shake detection method, device, equipment, storage medium and program product

Info

Publication number: CN115174811A
Application number: CN202210763970.1A
Authority: CN
Inventors: 邹涵江
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2022-10-11

Abstract

The application relates to a camera shake detection method, a device, equipment, a storage medium and a program product, and belongs to the technical field of electronic equipment. The method comprises the following steps: acquiring a video stream shot by a camera, extracting candidate frames from the video stream at intervals of n frames to obtain a frame sequence, and acquiring a target reference frame of each candidate frame from the frame sequence; for each candidate frame, performing feature point matching processing on the candidate frame and a target reference frame thereof to obtain a plurality of mutually matched feature point pairs, mapping pixels on the candidate frame to the target reference frame thereof according to the plurality of feature point pairs, and determining a motion quantity parameter according to the position change of the pixels before and after mapping, wherein the motion quantity parameter is used for representing the motion change of an object in the candidate frame compared with the object in the target reference frame of the candidate frame; and determining the shaking state of the camera according to the motion quantity parameter of each candidate frame. The method can be used for detecting the camera shake.

Description

Camera shake detection method, device, equipment, storage medium and program product

Technical Field

The present application relates to the field of electronic device technologies, and in particular, to a camera shake detection method, apparatus, device, storage medium, and program product.

Background

Currently, shooting through a camera provided in an electronic device such as a mobile phone has become more and more common in people's daily life, wherein the shooting quality of the camera is a hot issue that all parties are concerned about at present. In practical applications, a problem affecting the shooting quality of a camera is that the camera is unstable to hold, which causes camera shake, for example, shooting while walking, pressing a shutter when the camera is not stable, and the like, which greatly affects the shooting quality of the camera.

In the shooting process, detecting whether the camera shakes is a precondition for improving the shooting quality of the camera, and therefore, a method for detecting whether the camera shakes is urgently needed.

Disclosure of Invention

In view of the above, it is necessary to provide a camera shake detection method, apparatus, device, storage medium, and program product capable of detecting whether a camera shakes, in view of the above technical problems.

In a first aspect, a camera shake detection method is provided, where the method includes:

acquiring a video stream shot by a camera, extracting candidate frames from the video stream at intervals of n frames to obtain a frame sequence, and acquiring a target reference frame of each candidate frame from the frame sequence, wherein the frame sequence comprises a plurality of candidate frames, and n is a positive integer; for each candidate frame, performing feature point matching processing on the candidate frame and a target reference frame of the candidate frame to obtain a plurality of mutually matched feature point pairs, mapping pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and determining a motion quantity parameter according to the position change of the pixels before and after mapping, wherein the motion quantity parameter is used for representing the motion change of an object in the candidate frame compared with the object in the target reference frame of the candidate frame; and determining the shaking state of the camera according to the motion quantity parameter of each candidate frame.

In a second aspect, there is provided a camera shake detection apparatus, the apparatus comprising:

the acquisition module is used for acquiring a video stream shot by the camera;

the extraction module is used for extracting candidate frames from the video stream at intervals of n frames to obtain a frame sequence, and acquiring a target reference frame of each candidate frame from the frame sequence, wherein the frame sequence comprises a plurality of candidate frames, and n is a positive integer;

a first determining module, configured to perform feature point matching processing on the candidate frame and a target reference frame of the candidate frame for each candidate frame to obtain a plurality of mutually matched feature point pairs, map pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and determine a motion parameter according to a change in positions of the pixels before and after mapping, where the motion parameter is used to characterize a change in motion of an object in the candidate frame as compared with an object in the target reference frame of the candidate frame;

and the second determining module is used for determining the shaking state of the camera according to the motion quantity parameter of each candidate frame.

In a third aspect, the application further provides an electronic device. The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the first aspect when executing the computer program.

In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the first aspect described above.

In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program that when executed by a processor performs the steps of the first aspect described above.

The camera shake detection method, the device, the equipment, the storage medium and the program product can realize the improvement of the camera shooting quality by acquiring the video stream shot by the camera, extracting the candidate frame from the video stream at intervals of n frames to obtain the frame sequence, acquiring the target reference frame of each candidate frame from the frame sequence, then carrying out feature point matching processing on the candidate frame and the target reference frame of the candidate frame for each candidate frame to obtain a plurality of mutually matched feature point pairs, then mapping the pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, determining the motion quantity parameter according to the position change of the pixels before and after mapping, finally determining the shake state of the camera according to the motion quantity parameter of each candidate frame, and finally determining the shake state of the camera in such a way.

Drawings

FIG. 1 is a flow chart illustrating a method for detecting camera shake according to an embodiment;

FIG. 2 is a diagram of candidate frame extraction from a video stream in one embodiment;

FIG. 3 is a diagram illustrating input data and output data of an embodiment of computing logic of an electronic device in determining a camera shake state;

FIG. 4 is a flow diagram illustrating a process for determining a target reference frame for a candidate frame in one embodiment;

FIG. 5 is a flow diagram that illustrates processing of the electronic device to perform feature point matching on candidate sample frames and reference sample frames, according to an embodiment;

FIG. 6 is an exemplary diagram of a first feature point detected in a reference sample frame in one embodiment;

FIG. 7 is a flow diagram illustrating a search among candidate sample frames in one embodiment;

FIG. 8 is a diagram illustrating matching of feature points in a reference sample frame and a candidate sample frame, according to an embodiment;

FIG. 9 is a schematic flow chart illustrating the determination of perspective transformation parameters in one embodiment;

fig. 10 is a flowchart illustrating a process of determining a parameter of an amount of motion according to an embodiment;

FIG. 11 is a flow chart illustrating another method for detecting camera shake according to one embodiment;

FIG. 12 is a block diagram showing the structure of a camera shake detection apparatus according to an embodiment;

FIG. 13 is a diagram illustrating the internal architecture of an electronic device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Currently, photographing through a camera provided in an electronic device such as a mobile phone has become more and more common in people's daily life. In poor shooting evaluation fed back by a user, pixels are low, and shooting is unclear, which accounts for a large proportion. Generally, if the brightness of a shooting scene is normal, no moving object exists in the shooting picture, and the handheld shooting is stable, then a camera can shoot a better-quality image without difficulty, but a blurred image can be easily shot in a shooting scene with weak light and dark light or in the situation of unstable handheld shooting (for example, shooting while walking, pressing a shutter when the camera is not stable yet).

In a current technology, the definition of an image shot by a camera can be improved and the probability of blurring can be reduced by reducing the exposure time and combining a specific image algorithm processing mode, and this mode can significantly improve the image shooting quality of the camera under the condition of unstable handheld shooting (i.e. camera shake), however, under the condition of stable handheld shooting (i.e. camera shake), the exposure time is reduced, and on the contrary, noise is increased, so that the image shooting quality is reduced, and if details in the image are recovered by a subsequent image noise reduction algorithm, the calculation cost and time are additionally increased.

Based on the above description, it can be known that accurately determining the camera shake in real time has important practical significance for improving the image capturing quality, and therefore, a method for detecting whether the camera shakes is urgently needed.

Based on this, embodiments of the present application provide a camera shake detection method, apparatus, device, storage medium, and program product capable of detecting whether a camera shakes, and it should be noted that an execution subject of the camera shake detection method provided by embodiments of the present application may be a camera shake detection apparatus. The camera shake detection device can be realized by software, hardware or a combination of software and hardware to become part or all of the electronic equipment. The electronic equipment is provided with a camera, the electronic equipment can be but not limited to various notebook computers, smart phones, tablet computers, internet of things equipment and portable wearable equipment, and the internet of things equipment can be smart sound boxes, smart televisions, smart air conditioners, smart vehicle-mounted equipment and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like.

In an embodiment, as shown in fig. 1, a camera shake detection method is provided, and in the following, an execution subject of the method is taken as an example for description of an electronic device, and it should be understood by readers that, in some cases, the execution subject may also be a server, and the camera shake detection method provided in the embodiment of the present application is implemented through interaction with the electronic device, as shown in fig. 1, the camera shake detection method includes the following steps:

step 101, the electronic device acquires a video stream shot by a camera, extracts candidate frames from the video stream at intervals of n frames to obtain a frame sequence, and acquires a target reference frame of each candidate frame from the frame sequence.

Wherein the frame sequence comprises a plurality of candidate frames, and n is a positive integer.

In practical applications, the camera often enters a preview state before shooting an image, and in the preview state, the camera continues to shoot. In step 101, the electronic device can obtain a video stream obtained by continuously shooting the camera in the preview state.

In practical applications, the preview frame rate of the camera is usually 30fps, that is, if the camera is turned on for 1 second, 30 frames of images in YUV format or RGB format can be obtained, and therefore, the data volume of the video stream is large (30 frames of images per second, and 1800 frames of images are obtained if the camera is turned on for previewing for 1 minute), and if each frame of image is calculated, the calculation amount, calculation time consumption, power consumption and the like of the electronic device are increased, therefore, a frame skipping method is designed in step 101, that is, candidate frames are extracted from every n frames in the video stream, and in the subsequent steps, only the extracted candidate frames are calculated, which can be understood as sampling the video stream, so as to improve the calculation real-time performance and reduce the power consumption.

Please refer to fig. 2, which shows a schematic diagram of extracting candidate frames from a video stream in case of n = 2. In practical applications, n may be set to be a positive integer smaller than 6, which is also an exemplary illustration and is not used to limit the scope of the technical solutions provided by the embodiments of the present application.

After obtaining the frame sequence, the electronic device may obtain the target reference frame of each candidate frame from the frame sequence, in other words, each candidate frame in the frame sequence has a corresponding target reference frame, in practical applications, the target reference frame corresponding to the candidate frame is also located in the frame sequence, and optionally, the target reference frame corresponding to the candidate frame is located before the candidate frame in time sequence.

And 102, for each candidate frame, the electronic device performs feature point matching processing on the candidate frame and a target reference frame of the candidate frame to obtain a plurality of mutually matched feature point pairs, the electronic device maps pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and the electronic device determines a motion quantity parameter according to the position change of the pixels before and after mapping.

The motion parameter is used to characterize the motion change of the object in the candidate frame compared to the object in the target reference frame of the candidate frame, and in practical applications, the motion parameter may be a displacement vector.

And 103, the electronic equipment determines the shaking state of the camera according to the motion quantity parameter of each candidate frame.

In view of the above, in step 103, the electronic device may determine the shake state of the camera according to the motion amount parameter of each candidate frame according to the motion amount parameter of the candidate frame.

In an optional embodiment of the present application, the electronic device may obtain a motion quantity parameter threshold, and the electronic device may determine a shake state of the camera according to a size relationship between a motion quantity parameter of each candidate frame and the motion quantity parameter threshold, where the motion quantity parameter threshold may be set by a technician, and the embodiment of the present application does not specifically limit the shake state.

Please refer to fig. 3, which is a schematic diagram of input data and output data of an electronic device in a computing logic for determining a camera shake state.

In an alternative embodiment of the present application, in executing step 102, the electronic device may determine a perspective transformation parameter between the candidate frame and its target reference frame according to the plurality of feature point pairs, and map pixels on the candidate frame onto its target reference frame based on the perspective transformation parameter.

In the embodiment of the present application, the perspective transformation parameter may specifically be a perspective transformation matrix, and the electronic device may implement perspective transformation based on the perspective transformation matrix.

For example, if pixel (x 1, y 1) is a pixel in the candidate frame, H is the obtained perspective transformation matrix, and the position coordinates of pixel (x 1, y 1) after mapping onto the target reference frame are (x 2, y 2), there is the following mathematical relationship:

X＝m ₁₁ *x1+m ₁₂ *y1+m ₁₃

Y＝m ₂₁ *x1+m ₂₂ *y1+m ₂₃

Z＝m ₃₁ *x1+m ₃₂ *y1+m ₃₃

according to the above mathematical relation, on the basis of obtaining the perspective transformation matrix H by calculation, for any pixel on the candidate frame, the position coordinate of the pixel mapped onto the target reference frame can be obtained by calculation.

Generally, a perspective transformation matrix is calculated by using 4 pairs of feature points, so in step 102, the electronic device may select 4 pairs of feature points from the plurality of pairs of feature points (for example, randomly), and then calculate the perspective transformation matrix by using the selected 4 pairs of feature points, and in practical applications, iteratively calculate the perspective transformation matrix by using an optimization algorithm RANSAC (RANdom SAmple Consensus, chinese).

In an alternative embodiment of the present application, the electronic device may not map every pixel in the candidate frame to its target reference frame, but rather map pixels within the region of interest to its target reference frame.

The region of interest may be a central region of the candidate frame, where the size of the central region may be arbitrarily specified according to experience, for example, the region of interest may be a 1/2 candidate frame region, a 3/4 candidate frame region, and the like, which may reduce the amount of computation, power consumption, and the like on one hand, and may avoid the influence of pixels around the candidate frame on the operation result of the motion amount parameter on the other hand.

Referring to fig. 4, based on the embodiment shown in fig. 1, a flowchart of determining the target reference frame of each candidate frame in step 101 is shown, and as shown in fig. 4, determining the target reference frame of each candidate frame includes the following steps:

step 401, for each candidate frame, if the candidate frame does not meet the first preset condition, the target reference frame of the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

Step 402, if the candidate frame meets the first preset condition, the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

As described above, for each candidate frame in the frame sequence, the electronic device needs to perform feature point matching processing on the candidate frame and the target reference frame of the candidate frame, and in the process of feature point matching, a technical process of feature point identification on the target reference frame is involved.

In view of the above, in an optional embodiment of the present application, a plurality of candidate frames may share one target reference frame, so as to reduce the calculation amount, calculation delay, power consumption, and the like of the electronic device.

Optionally, for each candidate frame, the electronic device needs to determine whether the candidate frame meets a first preset condition, where the first preset condition may be a condition for changing a reference frame, for example, in a possible implementation manner, the electronic device may detect whether a duration between a computing time of the candidate frame and a time at which feature point identification is performed on a target reference frame last time is less than a preset duration threshold, if so, the candidate frame does not meet the first preset condition, if not, the candidate frame meets the first preset condition, and in another possible implementation manner, the electronic device may determine a number of frames spaced between the candidate frame and a target reference frame of a previous candidate frame in the frame sequence, if the number of frames is less than a preset frame number threshold, the candidate frame does not meet the first preset condition, and if the number of frames is not less than a preset frame number threshold, the candidate frame meets the first preset condition.

In case the first preset condition is not satisfied, the candidate frame needs to share the same target reference frame with a previous candidate frame in the frame sequence, in which case the electronic device may use the target reference frame of the previous candidate frame in the frame sequence as the target reference frame of the candidate frame.

In the case that the first preset condition is met, it is described that the target reference frame of the previous candidate frame in the frame sequence is greatly different from the candidate frame, and a deviation or even an error may occur in calculation using the target reference frame.

For example, assuming that the candidate frame to be allocated with the target reference frame is the kth candidate frame in the frame sequence, the electronic device may detect whether the kth candidate frame satisfies the first preset condition, and if not, the electronic device may use the target reference frame of the kth candidate frame as the target reference frame of the kth candidate frame, so that the kth-1 candidate frame and the kth candidate frame share the same target reference frame, otherwise, if so, the electronic device may use the kth-1 candidate frame as the target reference frame of the kth candidate frame, in other words, the electronic device may reallocate a new target reference frame for the kth candidate frame, and the reallocated target reference frame is a previous candidate frame of the kth candidate frame in the frame sequence.

In an optional embodiment of the present application, in step 102, before performing the feature point matching process, the electronic device may perform a downsampling process on the candidate frame and the target reference frame thereof, where the downsampling process may be an antialiasing downsampling process (anti-aliasing) to obtain a candidate sample frame and a reference sample frame, and then the electronic device may perform the feature point matching process on the candidate sample frame and the reference sample frame.

The anti-aliasing downsampling process may reduce aliasing effects on the downsampled image while downsampling. Optionally, the anti-aliasing downsampling process may include: 1. and 2, performing filtering processing on the image by adopting a box filter to obtain a smoother filtering image, and performing downsampling on the filtering image by adopting a Nearest neighbor interpolation method (English). In an alternative embodiment of the present application, after antialiasing downsampling, the image length and width dimensions are scaled to 1/2 of the original, in other words, the reference sample frame is half the length and half the width of the target reference frame, the candidate sample frame is half the length and half the width of the candidate frame.

Referring to fig. 5, a flowchart of the electronic device performing a feature point matching process on a candidate sample frame and a reference sample frame is shown, and as shown in fig. 5, the feature point matching process includes the following steps:

step 501, the electronic device obtains a plurality of first feature points obtained by performing corner detection on a reference sample frame.

A corner point is usually defined as the intersection of two edges, more strictly speaking, a local neighborhood of a corner point should have boundaries of two different regions in different directions. In the field of image processing, most corner detection detects image points with specific features (i.e., feature points), rather than merely "corners". These feature points have specific coordinates in the image and have certain mathematical features such as local maximum or minimum gray levels, certain gradient features, etc.

In practical application, the common corner detection method includes Harris corner detection and Shi-Tomasi corner detection, taking Shi-Tomas corner detection as an example: in step 501, the electronic device may first calculate gradient values I of the reference sample frame in x and y directions by using a Sobel operator _x 、I _y Then calculating the product of the two directional gradients

I _x *I _y Then, to

I _x *I _y And performing mean filtering, finally calculating the characteristic value of the local characteristic result matrix M, and if the smaller of the two characteristic values is greater than the minimum threshold value, obtaining a first characteristic point. Fig. 6 is an exemplary diagram of a first feature point detected in a reference sample frame.

Step 502, the electronic device searches in the candidate sampling frame according to each first feature point to obtain a second feature point matched with each first feature point.

In view of the fact that the amount of computation required for corner detection is large, in the embodiment of the present application, when performing feature point matching, the electronic device only performs corner point detection on the reference sample frame to identify the first feature points in the reference sample frame, and the electronic device does not perform corner point detection on the candidate sample frame, otherwise, the electronic device may combine the sparse optical flow method and the image pyramid to search for the second feature points matching the first feature points in the candidate sample frame based on the first feature points identified in the reference sample frame, so as to obtain the feature point pairs (a feature point pair includes one first feature point and one second feature point) described in step 102.

The sparse optical flow method is an algorithm for tracking a specified group of points (for the embodiment of the present application, the specified group of points are a plurality of first feature points) without performing point-by-point calculation on each pixel of an image, and the calculation is convenient. In addition, in the embodiment of the present application, the multi-scale sparse optical flow calculation is performed in combination with the image pyramid algorithm, considering that the sparse optical flow calculation easily reaches the local minimum value when the camera shake is fast.

Referring to fig. 7, it shows a flowchart of an electronic device performing a search in a candidate sample frame based on each first feature point identified in a reference sample frame in combination with a sparse optical flow method and an image pyramid, as shown in fig. 7, the search process includes the following steps:

step 5021, the electronic equipment performs reduction processing on the candidate sampling frame to obtain a frame pyramid.

The frame pyramid comprises a plurality of reduced frames with different resolutions and candidate sampling frames obtained by reduction processing, and the frames in the frame pyramid are arranged in the order of the resolution from small to large.

For example, if the candidate sampling frames are narrowed down to obtain 3 narrowed frames, where the resolution of narrowed frame a is the smallest and the resolution of narrowed frame B is the next, and the resolution of narrowed frame C is the largest, in the obtained frame pyramid, the top frame is narrowed frame a, the next layer is narrowed frame B, the next layer is narrowed frame C, and the bottom layer is the candidate sampling frame.

Step 5022, the electronic device searches in each frame in the frame pyramid in sequence according to each first feature point by adopting a sparse optical flow method, and determines second feature points matched with each first feature point according to a search result in the last frame in the frame pyramid.

Specifically, the electronic device may search the topmost frame as a search initial value of the topmost frame (with the least image details) of the frame pyramid, and then the electronic device may search downward along the frame pyramid by using a search result of the previous layer as a search initial value of the next layer until the bottommost layer (with the most image details) of the frame pyramid is reached, and the electronic device may obtain the second feature points matched with the first feature points according to the search result of the bottommost frame of the frame pyramid.

As the sparse optical flow searching mode from coarse to fine is adopted, when the camera shake is serious, the pixel motion in the upper layers of frames of the frame pyramid is small, so that the situation that the pixel motion falls into a local minimum value can be avoided.

Please refer to fig. 8, which is a schematic diagram illustrating matching of feature points in a reference sample frame and a candidate sample frame.

As described above, after obtaining a plurality of characteristic point pairs in step 102, the electronic device may determine a perspective transformation parameter between the candidate frame and the target reference frame according to the plurality of characteristic point pairs, please refer to fig. 9, which shows a flowchart for determining the perspective transformation parameter, and as shown in fig. 9, determining the perspective transformation parameter may include the following steps:

step 901, for each first feature point in the plurality of feature point pairs, the electronic device determines a second feature point with the highest matching degree with the first feature point from a plurality of second feature points matched with the first feature point.

Step 902, for each second feature point in the plurality of feature point pairs, the electronic device determines a first feature point with the highest matching degree with the second feature point from the plurality of first feature points matched with the second feature point.

Step 903, the electronic device determines a target feature point pair from the plurality of feature point pairs.

And the first characteristic point and the second characteristic point in the target characteristic point pair are the characteristic points with the highest matching degree.

And 904, the electronic equipment determines perspective transformation parameters according to the target feature point pairs.

In general, the first feature points and the second feature points are not in a one-to-one correspondence relationship, but in a many-to-many correspondence relationship, in other words, one first feature point may correspond to a plurality of second feature points, and in turn, one second feature point may correspond to a plurality of first feature points. Based on this situation, after obtaining a plurality of characteristic point pairs, the electronic device may further screen the plurality of characteristic point pairs in a cross validation manner, so as to retain the characteristic point pairs that are strongly matched (i.e. the target characteristic point pairs in the foregoing), so that the accuracy of the perspective transformation parameters obtained through subsequent calculation may be ensured.

Specifically, the electronic device may determine a second feature point having a highest matching degree with each first feature point from among a plurality of second feature points corresponding to each first feature point, and conversely, the electronic device may determine a first feature point having a highest matching degree with each second feature point from among a plurality of first feature points corresponding to each second feature point, and then the electronic device may determine a pair of feature points that are feature points having a highest matching degree with each other as the target pair of feature points.

For example, assuming that the first feature point a1 corresponds to the second feature points a2, b2, and c2, and the second feature point b2 corresponds to the first feature points a1, d1, and e1, and the second feature point with the highest matching degree with the first feature point a1 is the second feature point b2, and the first feature point with the highest matching degree with the second feature point b2 is the second feature point a1, the feature point pair including the first feature point a1 and the second feature point b2 is the target feature point pair.

In the embodiment of the present application, if the number of target feature point pairs is less than a preset value (e.g., 10), the electronic device may consider that the feature point matching fails, in which case the process may be ended, and if the number of target feature point pairs is not less than the preset value, the electronic device may determine the perspective transformation parameter according to the target feature point pairs, for example, as described above, the electronic device may randomly select 4 target feature point pairs to calculate the perspective transformation matrix.

As described above, after acquiring the perspective transformation parameter in step 102, the electronic device may map the pixels on the candidate frame to its target reference frame according to the perspective transformation parameter, and after mapping, in step 102, the electronic device may determine the motion amount parameter according to the position change of the pixels before and after mapping, please refer to fig. 10, which shows a flowchart of determining the motion amount parameter according to the position change of the pixels before and after mapping, and as shown in fig. 10, the process of determining the motion amount parameter includes the following steps:

in step 1001, for each pixel, the electronic device determines a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping.

Step 1002, the electronic device uses the mean value of the displacement vectors corresponding to the pixels as the motion parameter.

In an optional embodiment of the application, after obtaining the motion parameter of each candidate frame, the electronic device may perform filtering and smoothing processing (for example, mean filtering with kernel _ size = 3) on the motion parameter of each candidate frame to obtain a smoothed motion parameter of each candidate frame, so as to improve a transition condition of an output result, and then the electronic device may determine a shake state of the camera according to the smoothed motion parameter of each candidate frame.

Further, the electronic device may determine the motion parameter of the non-candidate frame not extracted in the video stream according to the smooth motion parameter of each candidate frame, for example, the electronic device may use the smooth motion parameter of the candidate frame closest in time sequence to the non-candidate frame as the motion parameter of the non-candidate frame, and then the electronic device may determine the shake state of the camera according to the smooth motion parameter of each candidate frame and the motion parameter of each non-candidate frame.

For example, assuming that a frame with frame index idx =1 in the video stream is extracted as a candidate frame, and frames with frame index idx =2 and frame index idx =3 are non-candidate frames that are not extracted, the electronic device may use the smooth motion amount parameter of the frame with idx =1 as the motion amount parameter of the frame with idx =2, and at the same time, the electronic device may use the smooth motion amount parameter of the frame with idx =1 as the motion amount parameter of the frame with idx = 3.

According to the foregoing calculation, each frame i in the video stream can output a corresponding motion amount parameter shake _ i, and the shake _ i of each frame i is compared with a preset threshold (including threshold _ stable and threshold _ mode), so as to obtain a real-time camera shake condition judgment, where:

if the shake _ i < threshold _ stable, the camera is shown to be in a stable state at the moment;

if threshold _ stable < = shake _ i < threshold _ mode, it indicates that the camera is jittered at the moment;

if shake _ i > = threshold _ mode, it indicates that the camera is severely jittered at the moment.

In the subsequent steps, the electronic equipment can transmit the judged camera shaking condition to the camera application in real time, and the camera application can determine the algorithm with the best performance and effect based on the real-time camera shaking condition (such as the algorithm for shortening the exposure time and enhancing AI post-processing when shaking violently; the camera is stable and triggering normal exposure, the common flow algorithm and the like).

In order to make the reader easily understand the technical solution provided by the embodiment of the present application, in the following, the present application will perform an exemplary complete description on the camera shake detection method provided by the embodiment of the present application with reference to the flowchart shown in fig. 11, it should be noted that the here is only a relative completeness, and in an actual application, the camera shake detection method may include more or fewer steps than those shown in fig. 11, as shown in fig. 11, the camera shake detection method includes the following steps:

step 1101, the electronic device acquires a video stream shot by a camera, and extracts candidate frames from the video stream every n frames to obtain a frame sequence.

For each candidate frame, the electronic device determines a target reference frame of the candidate frame, step 1102.

For each candidate frame, if the candidate frame does not meet a first preset condition, the target reference frame of the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame, and if the candidate frame meets the first preset condition, the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

Step 1103, for each candidate frame, the electronic device performs antialiasing downsampling processing on the candidate frame and the target reference frame thereof to obtain a candidate sample frame and a reference sample frame.

Step 1104, the electronic device obtains a plurality of first feature points obtained by performing corner detection on the reference sample frame.

Step 1105, the electronic device performs reduction processing on the candidate sampling frame to obtain a frame pyramid.

And step 1106, the electronic device searches in each frame in the frame pyramid in sequence according to each first feature point by adopting a sparse optical flow method, and determines a second feature point matched with each first feature point according to a search result in the last frame in the frame pyramid, so as to obtain a plurality of feature point pairs.

Step 1107, for each first feature point in the plurality of feature point pairs, the electronic device determines a second feature point with the highest matching degree with the first feature point from a plurality of second feature points matched with the first feature point.

Step 1108, for each second feature point in the plurality of feature point pairs, the electronic device determines a first feature point with the highest matching degree with the second feature point from the plurality of first feature points matched with the second feature point.

Step 1109, the electronic device determines a target characteristic point pair from the plurality of characteristic point pairs.

Step 1110, the electronic device determines a perspective transformation parameter between the candidate frame and the target reference frame according to the target feature point pair.

Step 1111, the electronic device maps pixels in the region of interest on the candidate frame to the target reference frame based on the perspective transformation parameters.

Step 1112, for each pixel in the region of interest, the electronic device determines a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping, and uses the mean value of the displacement vectors corresponding to the pixels as the motion amount parameter of the candidate frame.

And step 1113, the electronic equipment performs filtering smoothing processing on the motion quantity parameter of each candidate frame to obtain a smooth motion quantity parameter of each candidate frame.

Step 1114, the electronic device determines motion parameters of non-candidate frames in the video stream that are not extracted according to the smooth motion parameters of each candidate frame.

And step 1115, the electronic equipment determines the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame and the motion quantity parameter of each non-candidate frame.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the application also provides a camera shake detection device for realizing the camera shake detection method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the camera shake detection apparatus provided below can be referred to the limitations on the camera shake detection method in the foregoing, and details are not described herein again.

In one embodiment, as shown in fig. 12, there is provided a camera shake detection apparatus 1200 including: an obtaining module 1201, an extracting module 1202, a first determining module 1203, and a second determining module 1204.

The acquiring module 1201 is configured to acquire a video stream captured by a camera.

The extracting module 1202 is configured to extract candidate frames from the video stream every n frames to obtain a frame sequence, and obtain a target reference frame of each candidate frame from the frame sequence, where the frame sequence includes multiple candidate frames, and n is a positive integer.

The first determining module 1203 is configured to perform feature point matching processing on the candidate frame and the target reference frame of the candidate frame for each candidate frame to obtain a plurality of mutually matched feature point pairs, map pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and determine a motion parameter according to a position change of the pixels before and after mapping, where the motion parameter is used to represent a motion change of an object in the candidate frame compared with a motion change of an object in the target reference frame of the candidate frame.

The second determining module 1204 is configured to determine a shake state of the camera according to the motion parameter of each candidate frame.

In an alternative embodiment of the present application, the first determining module 1203 is specifically configured to: determining a perspective transformation parameter between the candidate frame and the target reference frame of the candidate frame according to a plurality of the characteristic point pairs, and mapping the pixels on the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameter.

In an optional embodiment of the present application, the first determining module 1203 is specifically configured to: and mapping the pixels of the interest area in the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameter.

In an alternative embodiment of the present application, the extracting module 1202 is specifically configured to: for each candidate frame, if the candidate frame does not meet a first preset condition, the target reference frame of the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame, and if the candidate frame meets the first preset condition, the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

In an optional embodiment of the present application, the first determining module 1203 is specifically configured to: respectively carrying out downsampling processing on the candidate frame and the target reference frame of the candidate frame to obtain a candidate sampling frame and a reference sampling frame; and performing feature point matching processing on the candidate sampling frame and the reference sampling frame.

In an alternative embodiment of the present application, the first determining module 1203 is specifically configured to: acquiring a plurality of first characteristic points obtained by performing corner detection on the reference sampling frame; and searching in the candidate sampling frame according to the first characteristic points to obtain second characteristic points matched with the first characteristic points.

In an alternative embodiment of the present application, the first determining module 1203 is specifically configured to: carrying out reduction processing on the candidate sampling frame to obtain a frame pyramid, wherein the frame pyramid comprises a plurality of reduced frames with different resolutions obtained by reduction processing and the candidate sampling frame, and the frames in the frame pyramid are arranged according to the sequence of the resolution from small to large; and searching in each frame in the frame pyramid in sequence by adopting a sparse optical flow method according to each first feature point, and determining the second feature point matched with each first feature point according to a search result in the last frame in the frame pyramid.

In an optional embodiment of the present application, the first determining module 1203 is specifically configured to: for each first feature point in a plurality of feature point pairs, determining a second feature point with the highest matching degree with the first feature point from a plurality of second feature points matched with the first feature point; for each second feature point in a plurality of feature point pairs, determining a first feature point with the highest matching degree with the second feature point from a plurality of first feature points matched with the second feature point; determining a target characteristic point pair from a plurality of characteristic point pairs, wherein a first characteristic point and a second characteristic point in the target characteristic point pair are characteristic points with the highest matching degree; and determining the perspective transformation parameters according to the target characteristic point pairs.

In an alternative embodiment of the present application, the first determining module 1203 is specifically configured to: for each pixel, determining a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping; and taking the mean value of the displacement vectors corresponding to the pixels as the motion quantity parameter.

In an optional embodiment of the present application, the second determining module 1204 is specifically configured to: carrying out filtering smoothing treatment on the motion quantity parameter of each candidate frame to obtain the smooth motion quantity parameter of each candidate frame; and determining the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame.

In an optional embodiment of the present application, the second determining module 1204 is specifically configured to: determining motion parameters of non-candidate frames which are not extracted in the video stream according to the smooth motion parameters of the candidate frames; and determining the shaking state of the camera according to the smooth motion parameter of each candidate frame and the motion parameter of each non-candidate frame.

The modules in the camera shake detection apparatus described above may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the electronic device, or can be stored in a memory in the electronic device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, an electronic device is provided, which may be a terminal, and an internal structure thereof may be as shown in fig. 13. The electronic device includes a processor, a memory, a communication interface, a display screen, an input device, and a camera connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used for carrying out wired or wireless communication with an external electronic device, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a camera shake detection method. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the structure shown in fig. 13 is a block diagram of only a portion of the structure relevant to the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.

In one embodiment, an electronic device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

In one embodiment, the processor, when executing the computer program, further performs the steps of: determining a perspective transformation parameter between the candidate frame and the target reference frame of the candidate frame according to a plurality of the characteristic point pairs, and mapping the pixels on the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameter.

In one embodiment, the processor when executing the computer program further performs the steps of: and mapping the pixels of the interest region in the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameters.

In one embodiment, the processor when executing the computer program further performs the steps of: for each candidate frame, if the candidate frame does not meet a first preset condition, the target reference frame of the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame, and if the candidate frame meets the first preset condition, the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

In one embodiment, the processor, when executing the computer program, further performs the steps of: respectively carrying out downsampling processing on the candidate frame and the target reference frame of the candidate frame to obtain a candidate sampling frame and a reference sampling frame; and performing feature point matching processing on the candidate sampling frame and the reference sampling frame.

In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring a plurality of first characteristic points obtained by carrying out corner detection on the reference sampling frame; and searching in the candidate sampling frame according to each first feature point to obtain a second feature point matched with each first feature point.

In one embodiment, the processor, when executing the computer program, further performs the steps of: carrying out reduction processing on the candidate sampling frame to obtain a frame pyramid, wherein the frame pyramid comprises a plurality of reduced frames with different resolutions obtained by reduction processing and the candidate sampling frame, and the frames in the frame pyramid are arranged according to the sequence of the resolution from small to large; and searching in each frame in the frame pyramid in sequence by adopting a sparse optical flow method according to each first feature point, and determining the second feature point matched with each first feature point according to a search result in the last frame in the frame pyramid.

In one embodiment, the processor, when executing the computer program, further performs the steps of: for each first feature point in a plurality of feature point pairs, determining a second feature point with the highest matching degree with the first feature point from a plurality of second feature points matched with the first feature point; for each second feature point in a plurality of feature point pairs, determining a first feature point with the highest matching degree with the second feature point from a plurality of first feature points matched with the second feature point; determining a target characteristic point pair from a plurality of characteristic point pairs, wherein a first characteristic point and a second characteristic point in the target characteristic point pair are characteristic points with the highest matching degree; and determining the perspective transformation parameter according to the target characteristic point pair.

In one embodiment, the processor when executing the computer program further performs the steps of: for each pixel, determining a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping; and taking the mean value of the displacement vectors corresponding to the pixels as the motion quantity parameter.

In one embodiment, the processor, when executing the computer program, further performs the steps of: carrying out filtering smoothing treatment on the motion quantity parameter of each candidate frame to obtain the smooth motion quantity parameter of each candidate frame; and determining the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame.

In one embodiment, the processor when executing the computer program further performs the steps of: determining motion parameters of non-candidate frames which are not extracted in the video stream according to the smooth motion parameters of the candidate frames; and determining the shaking state of the camera according to the smooth motion parameter of each candidate frame and the motion parameter of each non-candidate frame.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of: determining a perspective transformation parameter between the candidate frame and the target reference frame of the candidate frame according to a plurality of the characteristic point pairs, and mapping the pixels on the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameter.

In one embodiment, the computer program when executed by the processor further performs the steps of: and mapping the pixels of the interest region in the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameters.

In one embodiment, the computer program when executed by the processor further performs the steps of: for each candidate frame, if the candidate frame does not meet a first preset condition, the target reference frame of the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame, and if the candidate frame meets the first preset condition, the previous candidate frame in the frame sequence is used as the target reference frame of the candidate frame.

In one embodiment, the computer program when executed by the processor further performs the steps of: respectively carrying out downsampling processing on the candidate frame and a target reference frame of the candidate frame to obtain a candidate sampling frame and a reference sampling frame; and performing feature point matching processing on the candidate sampling frame and the reference sampling frame.

In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a plurality of first characteristic points obtained by performing corner detection on the reference sampling frame; and searching in the candidate sampling frame according to each first feature point to obtain a second feature point matched with each first feature point.

In one embodiment, the computer program when executed by the processor further performs the steps of: carrying out reduction processing on the candidate sampling frame to obtain a frame pyramid, wherein the frame pyramid comprises a plurality of reduced frames with different resolutions obtained by reduction processing and the candidate sampling frame, and the frames in the frame pyramid are arranged according to the sequence of the resolution from small to large; and searching in each frame in the frame pyramid in sequence by adopting a sparse optical flow method according to each first feature point, and determining the second feature point matched with each first feature point according to a search result in the last frame in the frame pyramid.

In one embodiment, the computer program when executed by the processor further performs the steps of: for each first feature point in a plurality of feature point pairs, determining a second feature point with the highest matching degree with the first feature point from a plurality of second feature points matched with the first feature point; for each second feature point in a plurality of feature point pairs, determining a first feature point with the highest matching degree with the second feature point from a plurality of first feature points matched with the second feature point; determining a target characteristic point pair from a plurality of characteristic point pairs, wherein a first characteristic point and a second characteristic point in the target characteristic point pair are characteristic points with the highest matching degree; and determining the perspective transformation parameter according to the target characteristic point pair.

In one embodiment, the computer program when executed by the processor further performs the steps of: for each pixel, determining a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping; and taking the average value of the displacement vectors corresponding to the pixels as the motion quantity parameter.

In one embodiment, the computer program when executed by the processor further performs the steps of: carrying out filtering smoothing processing on the motion quantity parameter of each candidate frame to obtain a smooth motion quantity parameter of each candidate frame; and determining the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame.

In one embodiment, the computer program when executed by the processor further performs the steps of: determining motion parameters of non-candidate frames which are not extracted in the video stream according to the smooth motion parameters of the candidate frames; and determining the shaking state of the camera according to the smooth motion parameter of each candidate frame and the motion parameter of each non-candidate frame.

In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of: respectively carrying out downsampling processing on the candidate frame and the target reference frame of the candidate frame to obtain a candidate sampling frame and a reference sampling frame; and performing feature point matching processing on the candidate sampling frame and the reference sampling frame.

In one embodiment, the computer program when executed by the processor further performs the steps of: carrying out filtering smoothing treatment on the motion quantity parameter of each candidate frame to obtain the smooth motion quantity parameter of each candidate frame; and determining the shake state of the camera according to the smooth motion quantity parameter of each candidate frame.

In one embodiment, the computer program when executed by the processor further performs the steps of: determining motion parameters of non-candidate frames which are not extracted in the video stream according to the smooth motion parameters of the candidate frames; and determining the shake state of the camera according to the smooth motion parameter of each candidate frame and the motion parameter of each non-candidate frame.

It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A camera shake detection method, characterized in that the method comprises:

acquiring a video stream shot by a camera, extracting candidate frames from the video stream at intervals of n frames to obtain a frame sequence, and acquiring a target reference frame of each candidate frame from the frame sequence, wherein the frame sequence comprises a plurality of candidate frames, and n is a positive integer;

for each candidate frame, performing feature point matching processing on the candidate frame and a target reference frame of the candidate frame to obtain a plurality of mutually matched feature point pairs, mapping pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and determining a motion quantity parameter according to the position change of the pixels before and after mapping, wherein the motion quantity parameter is used for representing the motion change of an object in the candidate frame compared with the object in the target reference frame of the candidate frame;

and determining the shaking state of the camera according to the motion quantity parameter of each candidate frame.

2. The method of claim 1, wherein the mapping pixels on the candidate frame onto a target reference frame of the candidate frame according to the plurality of pairs of eigen points comprises:

determining a perspective transformation parameter between the candidate frame and a target reference frame of the candidate frame according to the plurality of characteristic point pairs, and mapping the pixels on the candidate frame to the target reference frame of the candidate frame based on the perspective transformation parameter.

3. The method of claim 2, wherein the mapping pixels on the candidate frame to a target reference frame of the candidate frame based on the perspective transformation parameters comprises:

mapping pixels on the region of interest in the candidate frame onto a target reference frame of the candidate frame based on the perspective transformation parameters.

4. The method of claim 1, wherein the obtaining the target reference frame of each candidate frame from the sequence of frames comprises:

for each candidate frame, if the candidate frame does not meet a first preset condition, taking a target reference frame of a previous candidate frame in the frame sequence as a target reference frame of the candidate frame, and if the candidate frame meets the first preset condition, taking the previous candidate frame in the frame sequence as the target reference frame of the candidate frame.

5. The method of claim 2, wherein before the feature point matching processing of the candidate frame and the target reference frame of the candidate frame, the method further comprises:

respectively carrying out downsampling processing on the candidate frame and the target reference frame of the candidate frame to obtain a candidate sampling frame and a reference sampling frame;

the performing feature point matching processing on the candidate frame and the target reference frame of the candidate frame includes:

and performing feature point matching processing on the candidate sampling frame and the reference sampling frame.

6. The method of claim 5, wherein the performing the feature point matching process on the candidate sample frame and the reference sample frame comprises:

acquiring a plurality of first characteristic points obtained by performing corner detection on the reference sampling frame;

and searching in the candidate sampling frame according to the first characteristic points to obtain second characteristic points matched with the first characteristic points.

7. The method according to claim 6, wherein the searching in the candidate sample frame according to each first feature point to obtain a second feature point matching each first feature point comprises:

reducing the candidate sampling frames to obtain a frame pyramid, wherein the frame pyramid comprises a plurality of reduced frames with different resolutions obtained by reduction and the candidate sampling frames, and the frames in the frame pyramid are arranged in the order of the resolution from small to large;

and searching in each frame in the frame pyramid in sequence by adopting a sparse optical flow method according to each first feature point, and determining second feature points matched with each first feature point according to a search result in the last frame in the frame pyramid.

8. The method of claim 6, wherein said determining a perspective transformation parameter between the candidate frame and a target reference frame of the candidate frame according to the plurality of pairs of feature points comprises:

for each first feature point in a plurality of feature point pairs, determining a second feature point with the highest degree of matching with the first feature point from a plurality of second feature points matching with the first feature point;

for each second feature point in a plurality of feature point pairs, determining a first feature point with the highest degree of matching with the second feature point from a plurality of first feature points matching with the second feature point;

determining a target characteristic point pair from the plurality of characteristic point pairs, wherein a first characteristic point and a second characteristic point in the target characteristic point pair are characteristic points with the highest matching degree;

and determining the perspective transformation parameters according to the target characteristic point pairs.

9. The method according to any of claims 1 to 8, wherein said determining said motion quantity parameter from the position change of the pixel before and after mapping comprises:

for each pixel, determining a displacement vector corresponding to the pixel according to the position change of the pixel before and after mapping;

and taking the average value of the displacement vectors corresponding to the pixels as the motion quantity parameter.

10. The method according to claim 9, wherein determining the shake state of the camera according to the motion amount parameter of each of the candidate frames comprises:

filtering and smoothing the motion quantity parameter of each candidate frame to obtain a smooth motion quantity parameter of each candidate frame;

and determining the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame.

11. The method of claim 10, wherein determining the shake state of the camera according to the smooth motion amount parameter of each of the candidate frames comprises:

determining motion parameters of non-candidate frames which are not extracted in the video stream according to the smooth motion parameters of the candidate frames;

and determining the shaking state of the camera according to the smooth motion quantity parameter of each candidate frame and the motion quantity parameter of each non-candidate frame.

12. A camera shake detection apparatus, comprising:

the acquisition module is used for acquiring a video stream shot by the camera;

a first determining module, configured to perform feature point matching processing on the candidate frame and a target reference frame of the candidate frame for each candidate frame to obtain a plurality of mutually matched feature point pairs, map pixels on the candidate frame to the target reference frame of the candidate frame according to the plurality of feature point pairs, and determine a motion quantity parameter according to a position change of the pixels before and after the mapping, where the motion quantity parameter is used to characterize a motion change of an object in the candidate frame compared with an object in the target reference frame of the candidate frame;

13. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 11 when executed by a processor.