KR20180095059A - An image fusion method and apparatus, and a terminal device - Google Patents

Abstract

Embodiments of the present invention disclose an image fusion method and apparatus, and a terminal device. This method is applied to the terminal device, and the first camera and the second camera are provided on the terminal device. The method includes capturing at least two frames of the first images at the same exposure time by using a first camera and capturing at least two frames of the second images at different exposure times by using a second camera The capturing time of the first images being the same as the capturing time of the second images; Determining a first reference image from at least two frames of the first images; Moving target detection is performed on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain a first reference image for at least two frames of the first images, Obtaining a local motion size of an image other than an image; And performing image fusion on the second images according to the local motion magnitude. By using embodiments of the present invention, the accuracy of moving target detection can be improved, ghosting is eliminated, and the accuracy of image fusion is relatively high.

Description

An image fusion method and apparatus, and a terminal device

The present invention relates to the field of image processing technology, and more particularly to an image fusion method and apparatus and a terminal device.

High Dynamic Range (HDR) images are obtained by fusing Low Dynamic Range (LDR) images to different exposure times. HDR images can provide more dynamic range and image detail and better reflect visual effects in a real-world environment. Since HDR technology needs to capture LDR images at different exposure times, if an object in a hand shake or scenario moves during the capture of LDR images, ghosting in the HDR image obtained by fusing LDR images Occurs.

In a conventional image fusion method, at least two frames of images are successively captured by using a camera. An image registration and de-ghosting algorithm is performed on the captured image to obtain the processed images to reduce the effects caused by hand movement or moving scenes in image fusion. The HDR image is obtained by fusing the processed images by using an image fusion algorithm. Because there is an exposure difference when at least two frames of images are successively captured by using a camera, moving target detection is inaccurate and the accuracy of image fusion is relatively low.

The present application provides a data transmission method and apparatus and a terminal device to improve the accuracy of moving target detection and eliminate ghosting, so that the accuracy of image fusion is relatively high.

According to a first aspect, an image fusion method is provided, the method being applied to a terminal device, wherein a first camera and a second camera are provided on the terminal device, the method comprising:

Capturing at least two frames of the first images at the same exposure time by using a first camera and capturing at least two frames of the second images at different exposure times by using a second camera, The capturing time of the first images being the same as the capturing time of the second images;

Determining a first reference image from at least two frames of the first images;

Moving target detection is performed on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain a first reference image for at least two frames of the first images, Acquiring an image local motion size other than the image, the local motion magnitude being used to indicate information about the amount of pixels occupied by the moving target detected by the moving target detection; And

And performing image fusion on the second images according to the local motion magnitude.

In this technical solution, the terminal device can synchronously capture at least two frames of images by using a first camera and a second camera, and at least two frames of the first images captured by the first camera Of exposure time are the same. The terminal device performs moving target detection on the first images, so that the accuracy of moving target detection can be improved. The terminal device then performs image fusion on the second images captured by the second camera in accordance with the first images captured by the first camera so that ghosting can be eliminated and the image fusion The accuracy is relatively high.

In a first possible implementation, the terminal device may determine whether the local motion size is greater than or equal to a first predetermined threshold value, and if the local motion magnitude is greater than or equal to a first predetermined threshold value, And outputs an image whose exposure time is the first predetermined time in two frames.

In this technical solution, if the local motion size is greater than or equal to the first preset threshold, the terminal device determines that the motion magnitude of the first reference image for the image other than the first reference image in the first images is relatively large . To avoid ghosting, the terminal device may directly output a second image with an exposure time of a first predetermined time.

Referring to the first aspect, in a second possible implementation, if the local motion size is less than a second predetermined threshold, the terminal device determines a second reference image from at least two frames of the second images, Performing image registration on an image other than the second reference image in at least two frames of the second images according to the reference image to obtain a registered second image, and using the image fusion algorithm, Lt; RTI ID = 0.0 > image < / RTI >

In this technical solution, if the local motion size is smaller than the second predetermined threshold, the terminal device may determine that the motion magnitude of the first reference image for the image other than the first reference image in the first images is ignored . The terminal device can perform direct image fusion on the second reference image and the registered second image, which does not affect the accuracy of the image fusion.

Referring to the first aspect, in a third possible implementation, if the local motion size is greater than or equal to a second predetermined threshold value and less than a first predetermined threshold value, the terminal device determines from at least two frames of second images Determine a second reference image, perform image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image, Performs motion compensation on the registered second image to obtain a motion compensated second image and performs image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm can do.

In this technical solution, if the local motion size is greater than or equal to a second preset threshold value and less than a first predetermined threshold value, the terminal device determines that the first reference image for images other than the first reference image in the first images Can be determined to be relatively small. The terminal device performs motion compensation on the second image registered according to the second reference image to obtain a motion compensated second image, and by using the image fusion algorithm, the second reference image and the motion compensated second image It is possible to perform image fusion with respect to the image, thereby improving the accuracy of the image fusion.

Referring to either the first aspect or any one of the first through third possible implementations of the first aspect, in a fourth possible implementation, the terminal device is configured to generate at least two frames of the first images in accordance with the first reference image, To generate motion compensation matrices for images other than the first reference image in at least two frames of the first images for the first reference image, Performing image registration on an image other than the first reference image in at least two frames of the first images according to a matrix to obtain a registered first image and determining a local motion of the registered first image Size can be obtained.

Referring to a fourth possible implementation of the first aspect, in a fifth possible implementation, moving target detection is performed on an image other than the first reference image of at least two frames of the first images according to the first reference image After generating a motion compensation matrix for an image other than the first reference image in at least two frames of the first images for the first reference image, In the process of performing image registration on an image other than the reference image, the terminal device performs image registration on an image other than the second reference image in at least two frames of the second images according to the above-described motion compensation matrix, A second image can be obtained.

Since the capturing time of the first images is the same as the capturing time of the second images, the motion size of the image other than the first reference image in at least two frames of the first images for the first reference image, Is equal to the motion size of the image other than the second reference image, for the second reference images in at least two frames of the images. The terminal device performs moving target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image and performs motion detection on the at least two frames of the first images, Using the generated motion compensation matrix of an image other than the first reference image as a motion compensation matrix of an image other than the second reference image in at least two frames of the second images and using at least two of the second images according to the motion compensation matrix Image registration is performed on images other than the second reference image in the frames, the image processing efficiency can be improved, and the resource utilization is improved.

According to a second aspect, a computer storage medium is provided, the computer storage medium storing a program, and when the program is executed, some or all of the steps of the first aspect are included.

According to a third aspect, an image fusion apparatus is provided, and the apparatus includes an image capturing unit, a reference image determining unit, a motion size obtaining unit and an image fusion unit, and the apparatus can perform some or all steps . ≪ / RTI >

According to a fourth aspect, there is provided a terminal device comprising a processor, a first camera, a second camera, and a memory, the memory storing a set of program codes, wherein the processor, the first camera and the second camera are stored in a memory The program code is called to perform the following operations:

The first camera captures at least two frames of the first images at the same exposure time; The second camera captures at least two frames of the second images at different exposure times, the capturing time of the first images and the capturing time of the second images being the same; The processor determines a first reference image from at least two frames of the first images; The processor performs motion target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image to generate motion detection information for at least two frames of the first images, 1 obtain a local motion size of an image other than the reference image; The processor performs image fusion on the second images according to the local motion size and the local motion magnitude is used to indicate information about the amount of pixels occupied by the moving target detected by the moving target detection.

In a first possible implementation, the processor performs image fusion on the second images according to the local motion magnitude, in particular: if the local motion magnitude is greater than or equal to the first predetermined threshold value, And outputting an image whose exposure time is at a first predetermined time in at least two frames of two images.

Referring to the fourth aspect, in a second possible implementation, the processor performs image fusion on the second images according to the local motion size, in particular: if the local motion size is less than the second predetermined threshold, Determining, by the processor, a second reference image from at least two frames of the second images; Performing image registration by the processor on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image; And performing image fusion on the second reference image and the registered second image by using an image fusion algorithm.

Referring to the fourth aspect, in a third possible implementation, the processor performs image fusion on the second images according to the local motion size, in particular: if the local motion size is greater than or equal to the second preset threshold Determining a second reference image from at least two frames of the second images by the processor if the first reference image is smaller than a first predetermined threshold; Performing image registration by the processor on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image; Performing, by the processor, motion compensation on the second image registered according to the second reference image to obtain a motion compensated second image; And performing image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm.

Referring to any one of the first to third possible implementations of the fourth aspect or the fourth aspect, in a fourth possible implementation, the processor is configured to determine, in at least two frames of the first images according to the first reference image Obtaining a local motion size of an image other than the first reference image in at least two frames of the first images, for a first reference image, by performing motion target detection on an image other than the first reference image, The processor performs motion target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain at least two frames of the first images Generating a motion compensation matrix for an image other than the first reference image; Performing image registration by the processor on an image other than the first reference image in at least two frames of the first images in accordance with the motion compensation matrix to obtain a registered first image; And obtaining, by the processor, the local motion size of the registered first image for the first reference image.

Referring to a fourth possible implementation of the fourth aspect, in a fifth possible implementation, the processor performs image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image Acquiring a second image registered by the processor is specifically performed by the processor to perform image registration on an image other than the second reference image in at least two frames of the second images in accordance with the motion compensation matrix, ≪ / RTI >

BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly describe the technical solutions in embodiments of the present invention, the following detailed description of the accompanying drawings is provided to explain the embodiments. Obviously, the appended drawings in the following description merely illustrate some embodiments of the invention, and those of ordinary skill in the art can still derive other figures from these attached drawings without creative effort.
1A and 1B are schematic flow diagrams of an image fusion method according to an embodiment of the present invention;
2A is a schematic structural view of a camera according to an embodiment of the present invention;
Figure 2b is a schematic diagram of an interface with a first image according to an embodiment of the invention;
2C is a schematic diagram of an interface with a second image according to an embodiment of the present invention;
3 is a schematic structural view of an image fusion apparatus according to an embodiment of the present invention; And
4 is a schematic structural view of a terminal device according to an embodiment of the present invention.

In the following, technical solutions in embodiments of the present invention are clearly described with reference to the accompanying drawings in the embodiments of the present invention.

Referring to FIGS. 1A and 1B, FIGS. 1A and 1B are schematic flow charts of an image fusion method according to an embodiment of the present invention. In this embodiment of the present invention, the image fusion method shown in the drawings may include at least the following steps.

S101. Capturing at least two frames of the first images at the same exposure time by using a first camera, capturing at least two frames of the second images at different exposure times by using a second camera, The capture time of the first images and the capture time of the second images are the same.

The first camera and the second camera are provided on the terminal device. A schematic structure of the camera shown in Fig. 2A is used as an example. The first camera and the second camera are independent of each other and the first camera and the second camera can be integrated on the same side of the terminal device so that the two cameras can have the same view range as possible. The terminal device may control the first camera and the second camera by using an image signal processor (ISP) to simultaneously capture at least two frames of images. Thus, the capturing time at which the first camera captures the first images and the capturing time at which the second camera captures the second images are the same. For example, the terminal device may be a dual camera smart phone. Dual camera Two cameras on a smart phone can shoot the same picture. Wherein the first images captured by the first camera include a first frame of first images and a second frame of first images, and wherein the second images captured by the second camera include a second frame of the first images and a second frame of the first images. 2 images, the capture time of the first frame of the first images and the capture time of the first frame of the second images are the same, and the capture time of the second frame of the first images and The capture time of the second frame of the second images is the same; Also, the first frame of the first images and the first frame of the second images include the same picture, and the second frame of the first images and the second frame of the second images contain the same picture. Thus, the local motion magnitude of the first frame of the first images for the second frame of the first images and the local motion magnitude of the first frame of the second images for the second frame of the second images are the same.

The exposure times of the first images captured by the first camera are the same and the exposure times of the second images captured by the second camera are different. For example, a first camera captures a first frame of first images and a second frame of first images with an exposure time of a first predetermined time, and the second camera captures a second frame of first images, 2 images, and captures a second frame of second images with an exposure time of a second predetermined time. The capture time of the first frame of the first images and the capture time of the first frame of the second images are the same and the capture time of the second frame of the first images and the capture time of the second frame of the second images Are the same. The first predetermined time may be the normal exposure time obtained by calculation by the auto exposure apparatus (AE) in the current shooting environment. For example, when the aperture is F8, the shutter speed is 1/125 seconds, so the first preset time is 1/125 seconds; When the aperture is F5.6, the shutter speed is 1/250 second, so the first predetermined time is 1/250 seconds. The second predetermined time may be longer or less than the first predetermined time, for example, 1/120 seconds or 1/200 seconds.

The first images may be gray scale images or color images, and the second images may be gray scale images or color images. This is not particularly limited in this embodiment of the present invention.

A schematic view of the interface with the first image shown in Fig. 2B is used as an example. The terminal device can capture three frames of the first images by using the first camera. M1 is the first frame of the first images, M2 is the second frame of the first images, and M3 is the third frame of the first images. A schematic view of the interface with the second image shown in Fig. 2C is used as an example. The terminal device can synchronously capture three frames of the second images by using the second camera. C1 is the first frame of the second images, C2 is the second frame of the second images, and C3 is the third frame of the second images. The exposure times of M1, M2 and M3 may be a first predetermined time, and the exposure times of C1, C2 and C3 may not be completely equal. For example, the exposure time of C1 is a second preset time, the exposure time of C2 is a first predetermined time, and the exposure time of C3 is a third preset time. The first predetermined time and the second predetermined time are different, and the first predetermined time and the third predetermined time are different, but the second predetermined time and the third predetermined time may be the same. Optionally, the second predetermined time and the third predetermined time may also be different. The capturing time of M1 and the capturing time of C1 are the same, and M1 and C1 contain the same picture. The capturing time of M2 and the capturing time of C2 are the same, and M2 and C2 contain the same picture. The capturing time of M3 is the same as the capturing time of C3, and M3 and C3 contain the same picture.

S102. A first reference image is determined from at least two frames of the first images.

The terminal device may determine a first reference image from at least two frames of the first images. In certain implementations, the terminal device may determine a first image with the same capture time as the capture time of the second image, the exposure time of which is a first predetermined time, as a first reference image. For example, a second image with a first predetermined time of exposure is C1, a first image with a capture time of C1 equal to the capturing time is M1, and the terminal device can use M1 as the first reference image have.

S103. Moving target detection is performed on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain a first reference image for at least two frames of the first images, Obtain the local motion size of the image other than the image.

The terminal device performs a moving target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image to generate at least two frames of the first images The local motion size of the image other than the first reference image can be obtained.

를 생성하여, M2에 대한

의 로컬 모션 크기 T1을 획득할 수 있다. 단말 디바이스는 추가로 M2에 따라 M3에 대해 이동 타겟 검출을 수행하여 M2에 대한 M3의 모션 보상 매트릭스 H2를 생성하고, H2에 따라 M3에 대해 이미지 등록을 수행하여 등록된 제1 이미지

를 생성하여, M2에 대한

의 로컬 모션 크기 T2를 획득할 수 있다. 단말 디바이스는 T1 및 T2로부터 최소 로컬 모션 크기를 획득할 수 있고, 제1 참조 이미지에 대한, 제1 이미지들의 적어도 2개의 프레임에서 제1 참조 이미지 이외의 이미지의 로컬 모션 크기로서 최소 로컬 모션 크기를 사용할 수 있다.For example, the terminal device may perform mobile target detection separately for M1 and M3 according to M2 to obtain the local motion magnitude of M1 for M2 and the local motion magnitude of M3 for M2. Specifically, the terminal device performs the mobile target detection on M1 according to M2 to generate the motion compensation matrix H1 of M1 for M2, and performs image registration on M1 according to H1 to obtain the registered first image

, And generates

Lt; RTI ID = 0.0 > T1. ≪ / RTI > The terminal device further performs a moving target detection for M3 according to M2 to generate a motion compensation matrix H2 of M3 for M2 and performs an image registration for M3 according to H2 to obtain a registered first image

, And generates

Lt; RTI ID = 0.0 > T2. ≪ / RTI > The terminal device can obtain the minimum local motion size from T1 and T2 and calculate the minimum local motion size for the first reference image as the local motion size of the image other than the first reference image in at least two frames of the first images Can be used.

The moving target detection method may include an interframe difference method, an optical flow method, a background difference method, and the like. The inter-frame difference method is a method of obtaining a moving target profile by performing a difference operation on two adjacent frames of images. If there is an abnormal object motion in the shooting scenario, there is a relatively clear difference between the frames. The motion feature of the image sequence is analyzed by determining whether the difference between two neighboring frames of images is greater than a threshold value to determine if object motion is present in the image sequence. Optical flow is a simple and practical representation of image motion and is defined as the apparent motion of the image brightness mode in the image sequence. That is, the motion speed of the point on the surface of the spatial object is represented on the imaging surface of the visual sensor. The pixels of the moving target in the picture are detected by using the vector difference between the pixel occupied by the moving target in the image and the background pixel. The background difference method is a method of detecting a motion object by comparing the current frame with the background reference model in the image sequence. For example, the terminal device performs mobile target detection for M1 according to M2 to generate the motion compensation matrix H1 of M2 for M2: Specifically, the minutiae points of M1 and M2 are determined by using feature extraction algorithms The matching is performed on the minutiae points of M1 and M2, and the coordinate transformation parameter between M1 and M2, i.e., H1, is fitted by using the least squares method based on a plurality of matching pairs of successfully matched minutiae points. Feature extraction algorithms may include Scale-invariant feature transform (SIFT), Speed Up Robust Features (SURF), and the like. The motion compensation matrix may be used to represent a motion relationship between two frames of images, e.g., translation or rotation of M2 relative to M1.

를 생성하는 것은 구체적으로: H1에 따라 M1에 대해 좌표 변환을 수행하여

를 획득한다.Image registration may be used to match at least two frames of images obtained at different times, by different sensors (imaging devices), or under different conditions (climate, illumination, shooting position or angle, etc.) Refers to the overlapping process. The image registration method may include a gray scale based method, a transform domain based method, a feature based method, and the like. In the gray scale based image registration method, the similarity between images is measured according to the gray scale statistics of the images. The gray scale based image registration method may include a Sequential Similarity Detection Algorithm (SSDA). In the process of calculating the residual sum of two frames of images in a pixel, if the residual sum is greater than a fixed threshold, the current pixel is considered not to be a matching pixel. Thus, the calculation of the current residual sum is ended, another pixel is selected to calculate the residual sum, and finally the pixel whose residual sum is the slowest increase is considered to be the matching pixel. The transform domain-based image registration method may include a Fourier transform method. The phase-related technique is based on the conversion characteristics of the Fourier transform. Since the difference between the two frames of images is the displacement, the phase difference between the two frames of images is equal to the phase of the cross power spectrum, and an inverse Fourier transform is performed on the phase difference to obtain the pulse function. The pulse function is nonzero at the conversion position and 0 at other positions. A non-zero position is the registered position to be determined. The feature-based image registration method is used for pre-processing images to be registered, i.e., in the process of image segmentation and feature extraction, to match features between two frames of images, Thereby establishing a registration mapping relationship between the images. For example, if the terminal device performs image registration for M1 according to H1 and the registered first image

Specifically, it is possible to perform coordinate transformation on M1 according to H1

.

의 로컬 모션 크기 T1을 획득하는 것은 구체적으로:

에서의 각각의 픽셀이 M2에서의 대응하는 픽셀로부터 감산되고, 감산에 의해 획득된 값의 절대 값이 미리 설정된 값보다 큰 경우, 단말 디바이스는 픽셀이 이동 타겟에 의해 점유된 픽셀인 것으로 결정할 수 있다.The local motion size is used to indicate information about the amount of pixels occupied by the moving target detected by the moving target detection and the information about the amount of pixels occupied by the moving target is used to determine the amount of pixels occupied by the moving target Amount or the ratio of the amount of pixels occupied by the moving target to the total amount of pixels of the image. If the terminal device

Obtaining the local motion size T1 of < RTI ID = 0.0 >

Is subtracted from the corresponding pixel at M2 and the absolute value of the value obtained by the subtraction is greater than a predetermined value, the terminal device can determine that the pixel is the pixel occupied by the moving target .

S104. And determines whether the local motion size is greater than or equal to a first predetermined threshold.

The terminal device may determine whether the local motion size is greater than or equal to a first preset threshold value. If the local motion size is greater than or equal to the first preset threshold, the terminal device may further perform step S105; Or if the local motion size is smaller than the first preset threshold value, the terminal device can additionally perform step S106.

S105. If the local motion size is greater than or equal to a first predetermined threshold, the exposure time in at least two frames of the second images is a first predetermined time.

의 모션 크기가 비교적 큰 것으로 결정할 수 있다. 고스팅을 피하기 위해, 단말 디바이스는 C2를 출력할 수 있다. 선택적으로, 로컬 모션 크기가 제1 미리 설정된 임계값보다 크거나 같을 때, 가장 작은 로컬 모션 크기가 T2이면, 단말 디바이스는 M2에 대한

의 모션 크기가 상대적으로 큰 것을 결정할 수 있다. 고스팅을 피하기 위해, 단말 디바이스는 C2를 출력할 수 있다.If the local motion size is greater than or equal to a first preset threshold, the terminal device may output an image with an exposure time of at least a first predetermined time in at least two frames of the second images. For example, in the second images captured by the terminal device by using the second camera, the image whose exposure time is the first predetermined time is C2. If the local motion size is greater than or equal to the first preset threshold value and the smallest local motion size is T1,

Can be determined to be relatively large. To avoid ghosting, the terminal device can output C2. Alternatively, when the local motion size is greater than or equal to the first preset threshold value, if the smallest local motion size is T2,

Can be determined to be relatively large. To avoid ghosting, the terminal device can output C2.

S106. And determines a second reference image from at least two frames of the second images if the local motion size is less than a first predetermined threshold value.

If the local motion size is smaller than the first predetermined threshold, the terminal device may determine the second reference image from at least two frames of the second images. In certain implementations, the terminal device may determine a second image with a first predetermined time of exposure time as a second reference image. For example, the second images captured by the terminal device by using the second camera include the first frame C1 of the second images, the second frame C2 of the second images, and the third frame C3 of the second images And the exposure time of C2 is the first predetermined time, the terminal device can use C2 as the second reference image.

S107. And performs image registration for images other than the second reference image in at least two frames of the second images according to the second reference image to obtain the registered second image.

를 획득할 수 있고, 단말 디바이스는 C2에 따라 C3에 대해 이미지 등록을 추가로 수행하여 등록된 제2 이미지

를 획득할 수 있다. 구체적으로, 단말 디바이스는 H1에 따라 C1에 대해 이미지 등록을 수행하여, 즉, H1에 따라 C1에 대해 좌표 변환을 수행하여

를 획득할 수 있다. 단말 디바이스는 H2에 따라 C3에 대해 이미지 등록을 추가로 수행하여, 즉, C3에 대해 H2에 따라 좌표 변환을 수행하여

를 획득할 수 있다.The terminal device performs image registration for images other than the second reference image in at least two frames of the second images according to the second reference image to obtain the registered second image. For example, the terminal device performs image registration for C1 according to C2,

, And the terminal device additionally performs image registration on C3 according to C2 to obtain the registered second image < RTI ID = 0.0 >

Can be obtained. Specifically, the terminal device performs image registration for C1 according to H1, that is, performs coordinate transformation on C1 according to H1

Can be obtained. The terminal device performs image registration additionally on C3 according to H2, i.e., performs coordinate transformation on C3 according to H2

Can be obtained.

S108. And determines whether the local motion size is less than a second predetermined threshold.

After obtaining the registered second image, the terminal device may further determine whether the local motion size is less than a second preset threshold value. If the local motion size is smaller than the second preset threshold value, step S109 is additionally performed; Or if the local local motion size is equal to or greater than the second preset threshold value, step S110 is additionally performed. For example, if the minimum local motion size is T1, the terminal device may determine if T1 is less than a second predetermined threshold; Or the minimum local motion size is T2, the terminal device may determine if T2 is less than a second predetermined threshold. The second predetermined threshold value is smaller than the first predetermined threshold value, and the first predetermined threshold value and the second predetermined threshold value are preset motion size threshold values. The preset motion size thresholds may be set by the user or may be configured in the terminal device before factory delivery.

S109. And performs image fusion on the second reference image and the second registered image by using an image fusion algorithm when the local motion size is smaller than a second preset threshold value.

의 모션 크기가 무시될 수 있는 것으로 결정하고, 그 후 이미지 융합 알고리즘을 사용하는 것에 의해 C2,

및

에 대해 이미지 융합을 수행하고, 융합된 이미지를 출력할 수 있다. 선택적으로, 최소 로컬 모션 크기가 T2이면, 로컬 모션 크기가 제2 미리 설정된 임계값보다 작을 때, 단말 디바이스는 M2에 대한

의 모션 크기가 0인 것으로 결정하고, 그 후 이미지 융합 알고리즘을 사용하는 것에 의해 C2,

및

에 대해 이미지 융합을 수행하고, 융합된 이미지를 출력할 수 있다.If the local motion size is less than a second preset threshold, the terminal device may perform image fusion on the second reference image and the second registered image by using an image fusion algorithm. The image fusion algorithm is as follows: Image processing is performed on images captured over multi-source channels that are of the same target, and useful information in each channel is extracted at a maximum, and finally, a high- Lt; / RTI > The image fusion algorithm may include a logic filtering method, a grayscale weighted average method, a wavelet transform method, a Bayesian method, and the like. For example, if the minimum local motion size is T1, then when the local motion size is less than the second predetermined threshold value,

And then uses the image fusion algorithm to determine the motion size of C2,

And

, And output the fused image. Optionally, when the minimum local motion size is T2, when the local motion size is less than a second preset threshold value,

Is determined to be 0, and then the image fusion algorithm is used to determine C2,

And

, And output the fused image.

S110. If the local motion size is greater than or equal to a second preset threshold value, motion compensation is performed on the second image registered according to the second reference image to obtain a motion compensated second image.

의 모션 크기가 비교적 작은 것으로 결정하고, 그 후 C2에 따라

에 대해 모션 보상을 수행하여 모션 보상된 제2 이미지

를 획득하고, C2에 따라

에 대해 모션 보상을 수행하여 모션 보상된 제2 이미지

를 획득할 수 있다. 선택적으로, 최소 로컬 모션 크기가 T2이면, 로컬 모션 크기가 제2 미리 설정된 임계값보다 작을 때, 단말 디바이스는 M2에 대한

의 모션 크기가 0인 것으로 결정하고, C2에 따라

에 대해 모션 보상을 수행하여 모션 보상된 제2 이미지

을 획득하고, C2에 따라

에 대해 모션 보상을 수행하여 모션 보상된 제2 이미지

를 획득할 수 있다.If the local motion size is greater than or equal to a second preset threshold value, the terminal device may perform motion compensation on the registered second image according to the second reference image to obtain the motion compensated second image. For example, if the minimum local motion size is T1, then when the local motion size is greater than or equal to a second preset threshold value,

Is determined to be relatively small, and thereafter, according to C2

To compensate the motion compensated second image < RTI ID = 0.0 >

, And according to C2

To compensate the motion compensated second image < RTI ID = 0.0 >

Can be obtained. Optionally, when the minimum local motion size is T2, when the local motion size is less than a second preset threshold value,

It is determined that the motion size of < RTI ID = 0.0 >

To compensate the motion compensated second image < RTI ID = 0.0 >

, And according to C2

To compensate the motion compensated second image < RTI ID = 0.0 >

Can be obtained.

에 대해 모션 보상을 수행하여 모션 보상된 제2 이미지

를 획득하는 것은 구체적으로: 단말 디바이스가 C2에 따라

에 대해 이동 타겟 검출을 수행하고, 검출에 의해 획득된 로컬 모션 영역에 대해 보상을 수행한다, 즉, C2에서의 대응한 픽셀로부터

에서의 픽셀이 감산된다. 감산에 의해 획득된 값의 절대 값이 미리 설정된 값보다 클 경우, 단말 디바이스는 픽셀이 움직이고 있는 것으로 결정하고,

에서의 움직이고 있는 모든 픽셀을 로컬 모션 영역으로서 사용할 수 있다. 단말 디바이스는

에서의 로컬 모션 영역을 C2에서의 대응하는 픽셀들로 조정하여

에 대해 모션 보상을 구현할 수 있다. 선택적으로, 단말 디바이스는

에서의 로컬 모션 영역의 밝기를 C2에서의 대응하는 픽셀들의 밝기로 조정하여

에 대해 모션 보상을 구현할 수 있다.Motion compensation is a method of describing the difference between two neighboring frames of images. That is, the current image is predicted and compensated according to the reference image. The motion compensation methods may include global motion compensation and block motion compensation. Depending on the terminal device C2

To compensate the motion compensated second image < RTI ID = 0.0 >

Specifically, the terminal device acquires < RTI ID = 0.0 >

And performs compensation for the local motion region obtained by the detection, that is, from the corresponding pixel at C2

Lt; / RTI > is subtracted. If the absolute value of the value obtained by the subtraction is larger than the preset value, the terminal device determines that the pixel is moving,

All the moving pixels in the local motion region can be used as the local motion region. The terminal device

Lt; RTI ID = 0.0 > C2 < / RTI > to the corresponding pixels in C2

Lt; RTI ID = 0.0 > motion compensation. ≪ / RTI > Optionally, the terminal device

Lt; RTI ID = 0.0 > C2 < / RTI > to the brightness of the corresponding pixels at C2

Lt; RTI ID = 0.0 > motion compensation. ≪ / RTI >

S111. And performs image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm.

및

에 대해 이미지 융합을 수행하고, 융합된 이미지를 출력할 수 있다.The terminal device may perform image fusion for the second reference image and the motion compensated second image by using an image fusion algorithm. For example, a terminal device may use C2,

And

, And output the fused image.

If the second image is a grayscale image, the terminal device performs image fusion on the second image according to the first image, and the image output by the terminal device is a fused second image, i.e., a grayscale image; Or the second image is a color image, the terminal device performs image fusion for the second image according to the first image, and the image output by the terminal device is the fused second image, i.e., the color image.

In the image fusion method shown in FIGS. 1A and 1B, at least two frames of the first images at the same exposure time are captured by using the first camera, and at least two frames of the second images at different exposure times Is captured by using the second camera, and the capturing time of the first images and the capturing time of the second images are the same. The first reference image is determined from at least two frames of the first images and the moving target detection is performed on the images other than the first reference image in at least two frames of the first images according to the first reference image, Obtaining a local motion size for an image other than the first reference image in at least two frames of the first images for the reference image and performing image fusion on the second image in accordance with the local motion magnitude. Thus, accuracy of moving target detection can be improved, ghosting is eliminated, and the accuracy of image fusion is relatively high.

Referring to FIG. 3, FIG. 3 is a schematic structural view of an image fusion apparatus according to an embodiment of the present invention. The image fusion apparatus provided in this embodiment of the present invention can be configured to implement some or all of the procedures in the embodiment of the image fusion method described in the present invention with reference to FIGS. 1A and 1B. As shown in the figure, in this embodiment of the present invention, the image fusion apparatus includes at least an image capturing unit 301, a reference image determination unit 302, a motion size acquisition unit 303, and an image fusion unit 304 .

The image capturing unit 301 captures at least two frames of the first images at the same exposure time by using the first camera and captures at least two frames of the second images at different exposure times by using the second camera And the capturing time of the first images and the capturing time of the second images are the same.

The reference image determination unit 302 is configured to determine a first reference image from at least two frames of the first images.

The motion size acquisition unit 303 performs movement target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image, To obtain a local motion size of an image other than the first reference image in at least two frames of the first reference image.

The image fusion unit 304 is configured to perform image fusion on the second images according to the local motion magnitude.

In an alternate embodiment, the image fusion unit 304 in this embodiment of the present invention specifically includes:

And to output an image in which the exposure time is at a first predetermined time in at least two frames of the second images if the local motion size is greater than or equal to a first preset threshold value.

In an alternate embodiment, the image fusion unit 304 in this embodiment of the present invention specifically includes:

Determine a second reference image from at least two frames of the second images if the local motion size is less than a second preset threshold;

Performing image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image;

And to perform image fusion for the second reference image and the registered second image by using an image fusion algorithm.

In an alternate embodiment, the image fusion unit 304 in this embodiment of the present invention specifically includes:

Determine a second reference image from at least two frames of the second images if the local motion size is greater than or equal to a second predetermined threshold and less than a first predetermined threshold;

Performing image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image;

Performing motion compensation on the second image registered according to the second reference image to obtain a motion compensated second image;

And to perform image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm.

In an alternative embodiment, in this embodiment of the present invention, the motion size acquisition unit 303 specifically includes:

Moving target detection is performed on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain a first reference image for at least two frames of the first images, Generate a motion compensation matrix for an image other than an image;

For at least two frames of the first images, for an image other than the first reference image, in accordance with a motion compensation matrix of the image other than the first reference image, for at least two frames of the first images, To obtain a registered first image;

And to obtain a local motion size of the registered first image for the first reference image.

In an alternative embodiment, the motion size obtaining unit 303 performs image registration for images other than the second reference image in at least two frames of the second images according to the second reference image to obtain the registered second image Specifically:

According to a motion compensation matrix for an image other than the first reference image in at least two frames of the first images, for at least two frames of the second images, for an image other than the second reference image, And performing registration to obtain a registered second image.

In the image fusion apparatus shown in Fig. 3, the image capturing unit 301 captures at least two frames of the first images at the same exposure time by using the first camera, and by using the second camera At least two frames of the second images are captured at different exposure times, and the capture time of the first images and the capture time of the second images are the same. The reference image determination unit 302 determines a first reference image from at least two frames of the first images. The motion size acquisition unit 303 performs movement target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image, The local motion size of an image other than the first reference image in at least two frames. The image fusion unit 304 performs image fusion on the second image in accordance with the local motion size. Thus, accuracy of moving target detection can be improved, ghosting is eliminated, and the accuracy of image fusion is relatively high.

Referring to FIG. 4, FIG. 4 is a schematic structural view of a terminal device according to an embodiment of the present invention. 4, the terminal device may include a processor 401, a memory 402, a first camera 403, a second camera 404, and an output device 405. [ The processor 401 is connected to the memory 402, the first camera 403, the second camera 404 and the output device 405. For example, the processor 401 may be connected to the memory 402, the first camera 403, the second camera 404, and the output device 405 by using a bus.

The processor 401 may be a central processing unit (CPU), a network processor (NP), or the like.

The memory 402 may be specifically configured to store a first image, a second image, and so on. Memory 402 may include volatile memory, such as, for example, random-access memory (RAM); The memory can be, for example, a non-volatile memory such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid- - can also include non-volatile memory; The memory may further comprise a combination of the foregoing types of memories.

The first camera 403 is configured to capture a first image.

The second camera 404 is configured to capture a second image.

Output device 405 is configured to output an image; For example, the output device 405 may be a display screen.

The processor 401, the first camera 403, and the second camera 404 call the program stored in the memory 402 to perform the following operations;

The first camera 403 is configured to capture at least two frames of the first images at the same exposure time;

The second camera 404 is configured to capture at least two frames of the second images at different exposure times and the capture time of the first images and the capture time of the second images are the same.

Processor 401 is configured to determine a first reference image from at least two frames of the first images;

The processor 401 performs moving target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain at least two of the first images for the first reference image Further configured to obtain a local motion size of an image other than the first reference image in the frame;

The processor 401 is further configured to perform image fusion on the second images in accordance with the local motion magnitude.

In an alternate embodiment, the processor 401 performs image fusion on the second images in accordance with the local motion size, in particular:

If the local motion size is greater than or equal to a first preset threshold, causing the processor 401 to output an image in which the exposure time is at a first predetermined time in at least two frames of the second images.

In an alternate embodiment, the processor 401 performs image fusion on the second images in accordance with the local motion size, in particular:

Determining, by the processor (401), a second reference image from at least two frames of the second images if the local motion size is less than a second preset threshold;

Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image by the processor 401 to obtain a registered second image; And

And performing image fusion on the second reference image and the registered second image by using the image fusion algorithm by the processor 401. [

In an alternate embodiment, the processor 401 performs image fusion on the second images in accordance with the local motion size, in particular:

Determining a second reference image from at least two frames of the second images by the processor (401) if the local motion size is greater than or equal to a second predetermined threshold and less than a first predetermined threshold;

Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image by the processor 401 to obtain a registered second image;

Performing motion compensation on the second image registered according to the second reference image by the processor 401 to obtain a motion compensated second image; And

And performing image fusion on the second reference image and the motion compensated second image by using the image fusion algorithm by the processor 401. [

In an alternative embodiment, the processor 401 performs motion target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image, Obtaining the local motion size of an image other than the first reference image in at least two frames of images is specifically:

The processor 401 performs motion target detection on an image other than the first reference image in at least two frames of the first images according to the first reference image to obtain at least two images of the first images Generating motion compensation matrices for images other than the first reference image in two frames;

Performing image registration for an image other than the first reference image in at least two frames of the first images according to a motion compensation matrix by the processor 401 to obtain a registered first image; And

By the processor 401, the local motion size of the registered first image for the first reference image.

In an alternative embodiment, the processor 401 performs image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain the registered second image, :

By the processor 401, perform image registration on an image other than the second reference image in at least two frames of the second images according to the motion compensation matrix to obtain the registered second image.

Specifically, the terminal device described in this embodiment of the present invention can be configured to implement some or all of the procedures in the embodiment of the image fusion method described in the present invention with reference to Figs. 1A and 1B.

In the description herein, any reference to such reference terms such as "example", "some embodiments", "examples", "specific examples" and "some examples" Features, structures, materials, or characteristics are included in at least one embodiment or example of the present invention. In the specification, the foregoing examples of expressions of terms are not necessarily referring to the same embodiment or example. In addition, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more of the embodiments or examples. In addition, one of ordinary skill in the art can incorporate or combine the features of the different embodiments or examples and other embodiments or examples described herein as long as they do not conflict with one another.

Also, the terms " first " and " second " are intended for illustrative purposes only and should not be construed as an indication of relative importance or an implicit indication of the number of implied or displayed technical features. Thus, a feature that is limited by " first " or " second " may explicitly or implicitly include at least one of the features. In the description of the present invention, " plurality " means at least two, for example two or three unless otherwise specifically limited.

The program listing of executable instructions that are considered to be implemented to implement the logic functions and / or steps, e.g., logic functions shown or described in the flowcharts in other ways herein, For example, a computer-based system, a system that includes a processor, or an instruction-execution system, apparatus, or device that is capable of acquiring and executing instructions from, Or may be embodied specifically in any computer-readable medium for use in combination with a device. As used herein, " computer-readable medium " may be any device capable of storing, communicating, propagating, or transmitting a program such that the program may be used by the instruction- - Can be used in combination with an execution system, device or device. More specific examples (non-exhaustive list) of computer readable media include: an electrically connected portion (electronic device) having one or more lay wires, a portable computer disk box (magnetic device), a random access memory, a read only memory, Capable read-only memory, a fiber optic device, and a portable compact disc read-only memory. In addition, the computer-readable medium may even be paper or other suitable medium upon which the program may be printed. This can be accomplished electronically by performing optical scanning on a paper or other medium and then performing editing and decoding, or by performing processing in other suitable ways, if necessary, Because it is stored in memory.

It is to be understood that portions of the present invention may be implemented by using hardware, software, firmware, or a combination thereof. In the above-described implementations, a number of steps or methods may be implemented by using software or firmware stored in memory and executed by an appropriate instruction execution system. For example, when hardware is used in an implementation similar to the implementation in other implementations, the following well known techniques in the art: separate logic with logic gate circuitry used to implement the logic function for the data signal Any combination or any combination of circuitry, a programmable gate array, a field programmable gate array, etc., with suitable combinations of logic circuits may be used in the implementation.

The modules in the embodiments of the present invention may be implemented in the form of hardware, or may be implemented in the form of a software function module. When an integration module is implemented in the form of a software function module and sold or used as an independent product, the integration unit can be stored in a computer-readable storage medium.

While embodiments of the present invention have been shown and described, it is to be understood that the above-described embodiments are examples and can not be construed as limitations on the present invention. Within the scope of the present invention, one of ordinary skill in the art can make changes, modifications, substitutions and variations to the above-described embodiments.

Claims (19)

As an image fusion method,
The method is applied to a terminal device, wherein a first camera and a second camera are provided on the terminal device, the method comprising:
Capturing at least two frames of the first images at the same exposure time by using the first camera and capturing at least two frames of the second images at different exposure times by using the second camera Wherein the capturing time of the first images and the capturing time of the second images are the same;
Determining a first reference image from at least two frames of the first images;
Performing movement target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image to determine at least two of the first images for the first reference image Obtaining a local motion size of the image other than the first reference image in a frame, the local motion magnitude being used to indicate information about the amount of pixels occupied by the moving target detected by the moving target detection -; And
Performing image fusion on the second images according to the local motion magnitude
/ RTI > The method according to claim 1,
Wherein performing the image fusion for the second images according to the local motion magnitude comprises:
And outputting an image in which the exposure time is at a first predetermined time in at least two frames of the second images if the local motion magnitude is greater than or equal to a first preset threshold value. The method according to claim 1,
Wherein performing the image fusion for the second images according to the local motion magnitude comprises:
Determining a second reference image from at least two frames of the second images if the local motion size is less than a second predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image; And
Performing image fusion on the second reference image and the registered second image by using an image fusion algorithm
/ RTI > The method according to claim 1,
Wherein performing the image fusion for the second images according to the local motion magnitude comprises:
Determining a second reference image from at least two frames of the second images if the local motion size is greater than or equal to a second preset threshold and less than a first predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image;
Performing motion compensation on the registered second image in accordance with the second reference image to obtain a motion compensated second image; And
Performing image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm
/ RTI > 5. The method according to any one of claims 1 to 4,
Performing movement target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image to determine at least two of the first images for the first reference image Wherein obtaining a local motion size of the image other than the first reference image in a frame comprises:
Performing movement target detection on the image other than the first reference image in at least two frames of the first images according to the first reference image to obtain at least two of the first images for the first reference image, Generating motion compensation matrices of the image other than the first reference image in frames;
Performing image registration on the image other than the first reference image in at least two frames of the first images according to the motion compensation matrix to obtain a registered first image; And
Obtaining a local motion size of the registered first image for the first reference image
/ RTI > 6. The method of claim 5,
Wherein performing the image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image comprises:
And performing image registration on the image other than the second reference image in at least two frames of the second images according to the motion compensation matrix to obtain the registered second image. An image fusion device,
An image configured to capture at least two frames of the first images at the same exposure time by using a first camera and to capture at least two frames of the second images at different exposure times by using a second camera, A capturing unit, the capturing time of the first images being the same as the capturing time of the second images;
A reference image determination unit configured to determine a first reference image from at least two frames of the first images;
Performing movement target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image to determine at least two of the first images for the first reference image A motion magnitude acquisition unit configured to acquire a local motion magnitude of the image other than the first reference image in a frame, the local motion magnitude comprising information about the amount of pixels occupied by the motion target detected by the motion target detection Used to indicate -; And
An image fusion unit configured to perform image fusion on the second images according to the local motion magnitude;
/ RTI > 8. The method of claim 7,
The image fusion unit specifically includes:
And to output an image whose exposure time is at a first predetermined time in at least two frames of the second images if the local motion magnitude is greater than or equal to a first predetermined threshold value. 8. The method of claim 7,
The image fusion unit specifically includes:
Determine a second reference image from at least two frames of the second images if the local motion size is less than a second predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image;
And to perform image fusion for the second reference image and the registered second image by using an image fusion algorithm. 8. The method of claim 7,
The image fusion unit specifically includes:
Determine a second reference image from at least two frames of the second images if the local motion size is greater than or equal to a second predetermined threshold and less than a first predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image;
Performing motion compensation on the registered second image according to the second reference image to obtain a motion compensated second image;
And to perform image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm. 11. The method according to any one of claims 7 to 10,
The motion magnitude acquisition unit may be:
Performing movement target detection on the image other than the first reference image in at least two frames of the first images according to the first reference image to obtain at least two of the first images for the first reference image, Generating motion compensation matrices of the image other than the first reference image in frames;
Performing a global image registration on the image other than the first reference image in at least two frames of the first images according to the motion compensation matrix to obtain a registered first image;
And to obtain a local motion size of the registered first image for the first reference image. 12. The method of claim 11,
Wherein the motion size obtaining unit performs image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image,
And perform image registration on the image other than the second reference image in at least two frames of the second images according to the motion compensation matrix to obtain the registered second image. As a terminal device,
A processor comprising a processor, a memory, a first camera and a second camera, the memory storing a program code set, the processor, the first camera and the second camera calling the program code stored in the memory, The following operations:
Wherein the first camera is configured to capture at least two frames of first images at the same exposure time;
Wherein the second camera is configured to capture at least two frames of the second images at different exposure times, the capturing time of the first images being the same as the capturing time of the second images;
The processor being configured to determine a first reference image from at least two frames of the first images;
Wherein the processor performs movement target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image, An operation further configured to obtain a local motion size of the image other than the first reference image in at least two frames, the local motion magnitude being determined by the amount of pixels occupied by the moving target detected by the moving target detection Used to indicate information about -; And
Wherein the processor is further configured to perform image fusion on the second images in accordance with the local motion size
. 14. The method of claim 13,
Wherein the processor performs image fusion on the second images according to the local motion magnitude.
Wherein if the local motion magnitude is greater than or equal to a first predetermined threshold value, outputting by the processor an image in which the exposure time is at a first predetermined time in at least two frames of the second images, device. 14. The method of claim 13,
Wherein the processor performs image fusion on the second images according to the local motion magnitude.
Determining, by the processor, a second reference image from at least two frames of the second images if the local motion size is less than a second predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image by the processor to obtain a registered second image; And
Performing, by the processor, an image fusion on the second reference image and the registered second image by using an image fusion algorithm
. 14. The method of claim 13,
Wherein the processor performs image fusion on the second images according to the local motion magnitude.
Determining a second reference image from at least two frames of the second images by the processor if the local motion size is greater than or equal to a second predetermined threshold and less than a first predetermined threshold;
Performing image registration for an image other than the second reference image in at least two frames of the second images according to the second reference image by the processor to obtain a registered second image;
Performing, by the processor, motion compensation on the registered second image in accordance with the second reference image to obtain a motion compensated second image; And
Performing, by the processor, image fusion on the second reference image and the motion compensated second image by using an image fusion algorithm
. 17. The method according to any one of claims 13 to 16,
Wherein the processor performs movement target detection on an image other than the first reference image in at least two frames of the first images in accordance with the first reference image, Obtaining the local motion size of the image other than the first reference image in at least two frames is specifically:
And performing, by the processor, motion target detection on the image other than the first reference image in at least two frames of the first images according to the first reference image, Generating a motion compensation matrix for the image in at least two frames of images other than the first reference image;
Performing image registration on the image other than the first reference image in at least two frames of the first images according to the motion compensation matrix, thereby obtaining a registered first image; And
Acquiring, by the processor, a local motion size of the registered first image for the first reference image
. 18. The method of claim 17,
Wherein the processor performs image registration on an image other than the second reference image in at least two frames of the second images according to the second reference image to obtain a registered second image,
And performing, by the processor, image registration for the image other than the second reference image in at least two frames of the second images according to the motion compensation matrix to obtain the registered second image. Terminal device. A computer storage medium, comprising computer storage media storing a program and, when the program is executed, comprising the steps of any one of claims 1 to 6.

Images