JP2016201616A - Image processing apparatus, control method therefor, and control program, and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate a motion vector required for deviation correction efficiently at high speed, at the time of image synthesis.SOLUTION: A system control unit 50 sets a first crop region for detecting a motion vector in each image, and detects the motion vector related to a subject in the first crop region as a first motion vector. When a determination is made that the subject is in a predetermined state depending on the analysis results of the first motion vector, the system control unit sets a second crop region wider than the first crop region and detects the motion vector related to the subject as a second motion vector. An image synthesis unit 54 synthesizes a plurality of images depending on the first motion vector or second motion vector.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing device, a control method thereof, a control program, and an imaging device, and in particular, an image processing device that performs image composition processing on a plurality of images obtained as a result of photographing a celestial body moving in a diurnal motion. About.

  In general, when photographing a celestial body, the amount of light from a celestial body such as a star is very small, and thus it is necessary to perform a long exposure such as 30 seconds or 1 minute. And since the celestial body is moving in a diurnal motion according to the rotation of the earth, if a long exposure is performed, the star will not become a point image but become a light trail. For this reason, when photographing a celestial body, photographing is generally performed using an equator dedicated to the celestial body so that the celestial body can be tracked in accordance with the movement of the celestial body moving in a diurnal motion.

  However, the equatorial mount is expensive and heavy, and not only does it need to be set, but also its operation is difficult. Therefore, without using an equatorial mount, a plurality of images are taken with an exposure time that does not cause the celestial object to become a light trace, and the deviation of the celestial object due to diurnal motion occurring between the images is corrected and combined into a single image. Things have been done. As a result, it is possible to take an image with the same exposure as that obtained when the celestial body is exposed to light for a long time without using a light trace.

  In the above image composition method, first, an image is divided into a plurality of blocks, and a celestial motion vector is calculated for each of the blocks. Then, the movement of the celestial body is calculated based on the motion vector to correct the shift between images.

  As such an image synthesizing method, for example, there is a method in which a total exposure time is determined according to the brightness of a subject, and N-time division exposure is performed with an exposure time that does not cause a star to become a light trace (patent). Reference 1). In this case, image deviation correction is performed according to the motion vector between the images, and one image is synthesized. Furthermore, when calculating a motion vector, there is one in which the search range is switched according to the focal length (see Patent Document 2).

JP 2003-259184 A JP-A-4-180487

  However, in both Patent Documents 1 and 2, as a result of detecting the motion vector for the entire image, when the number of pixels is large, not only the time required for correcting the shift between images increases, but also the processing load increases, and further the time required for photographing. It will be long.

  In order to cope with such a situation, for example, a crop area for detecting a motion vector may be set at a predetermined position of an image. However, in the case of astronomical photography, the direction of the diurnal motion differs depending on the direction, so it is difficult to accurately calculate a motion vector unless a crop area is set at an appropriate location.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus, a control method thereof, a control program, and an imaging apparatus capable of calculating a motion vector necessary for deviation correction at high speed and efficiently.

  In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus that combines a plurality of images obtained by photographing a moving subject to form a combined image, and each of the images First setting means for setting a first detection area for detecting a motion vector, and first detection means for detecting a motion vector related to the subject as the first motion vector in the first detection area; Determining means for determining whether or not the subject is in a predetermined state; and if the determination means determines that the subject is in the predetermined state, a second detection that is wider than the first detection region Second setting means for setting a region for each of the images, second detection means for detecting a motion vector related to the subject as a second motion vector in the second detection region, and the first To comprising: an image synthesizing means for the motion vector or the second motion vector the synthesized image by synthesizing processing the image of the plurality in accordance with, the.

  An imaging apparatus according to the present invention includes an imaging unit that captures the plurality of images, the above-described image processing device, and a recording unit that records a composite image obtained by the image processing device. .

  A control method according to the present invention is a control method of an image processing apparatus that combines a plurality of images obtained by photographing a moving subject to form a composite image, and detects a motion vector in each of the images. A first setting step for setting a first detection area for detecting a motion vector related to the subject as a first motion vector in the first detection region; and A determination step for determining whether or not the image is in a state; and when the determination step determines that the subject is in the predetermined state, each of the images has a second detection region wider than the first detection region. A second setting step for detecting a motion vector related to the subject as a second motion vector in the second detection area; and An image synthesizing step an image of the plurality and synthesis processing to the composite image according to come vectors or the second motion vector, comprising a.

  A control program according to the present invention is a control program used in an image processing apparatus that synthesizes a plurality of images obtained by photographing a moving subject to obtain a composite image, and is provided in a computer included in the image processing apparatus. A first setting step of setting a first detection area for detecting a motion vector in each of the images; and a first setting step of detecting a motion vector related to the subject in the first detection area as a first motion vector. 1 detection step, a determination step for determining whether or not the subject is in a predetermined state, and when the determination step determines that the subject is in the predetermined state, A second setting step for setting a wide second detection area for each of the images; and a motion vector relating to the subject in the second detection area. A second detection step of detecting as a motion vector, and an image synthesis step of synthesizing the plurality of images according to the first motion vector or the second motion vector into the synthesized image. It is made to perform.

  According to the present invention, the first detection area or the second detection area is set, and a plurality of sheets are selected according to the motion vectors detected in the first detection area or the second detection area according to the state of the subject. The image is processed. This makes it possible to calculate a motion vector required for deviation correction at high speed and efficiently.

It is a block diagram which shows the structure about an example of an imaging device provided with the image processing apparatus by embodiment of this invention. 3 is a flowchart for explaining calculation of a displacement correction amount performed by the camera shown in FIG. 1. It is a figure for demonstrating an example of the crop area | region set to the image in the camera shown in FIG. It is a figure for demonstrating the process at the time of determining with it being the state which the whole image is rotating in the camera shown in FIG. FIG. 3 is a diagram for explaining processing when it is determined in the camera shown in FIG. 1 that subjects other than the main subject are mixed in the crop area.

  Hereinafter, an example of an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an example of an imaging apparatus including an image processing apparatus according to an embodiment of the present invention.

  The illustrated imaging apparatus 100 is, for example, a digital camera (hereinafter simply referred to as a camera) and includes a photographing lens unit (hereinafter simply referred to as a photographing lens) 103. A mechanical shutter 101 having a diaphragm function is disposed at the subsequent stage of the photographing lens 103, and an optical image (subject image) enters the imaging unit 22 via the photographing lens 103. A barrier 102 for protecting the photographing lens 103, the shutter 101, and the imaging unit 22 is arranged on the front side of the photographing lens 103.

  The imaging unit 22 has an imaging device such as a CCD or CMOS sensor, and the imaging device outputs an electrical signal (analog signal) corresponding to the optical image. The analog signal is converted into a digital signal (image data) by the A / D converter 23.

  The image sensor provided in the imaging unit 22 has, for example, a Bayer array in which R (red), G (green), and B (blue) pixels are regularly arranged. Note that the pixel array of the image sensor is not limited to the Bayer array. Further, the system control unit 50 controls the imaging unit 22, the photographing lens 103, and the shutter 101 by a timing generation circuit (not shown).

  Further, in the illustrated camera, the accumulation time in the image sensor can be controlled by controlling the reset timing of the image sensor (so-called electronic shutter). This electronic shutter is used, for example, for moving image shooting.

  The image processing unit 24 performs various correction processing such as predetermined pixel interpolation processing and shading correction on the image data from the A / D converter 23 or the image data from the memory control unit 15, and also performs white balance processing, γ Correction processing, color conversion processing, and the like are performed. Furthermore, the image processing unit 24 performs image zooming and scaling processing to perform electronic zoom.

  At the time of shading correction, the image processing unit 24 corrects the luminance level in the image in order to correct shading caused by the characteristics of the photographing lens 103 and the characteristics of the imaging unit 22 such as the aberration of the image data. After the shading correction process, the image processing unit 24 performs a white balance (WB) process for matching the white reference in the image to white for the image data.

  Here, when performing shading correction, the image processing unit 24 performs correction by multiplying the gain for each pixel in accordance with the two-dimensional coordinates (position) of the image sensor. In the WB process, the image processing unit 24 performs a process of multiplying different gains for each of R, G (G1 and G2), and B in the Bayer array.

  The image composition unit 54 composes a plurality of image data to generate one image data. In the illustrated example, the image synthesis unit 54 selects not only the simple addition synthesis or addition average synthesis but also selects the brightest or darkest area in each of the areas of the image data to be synthesized, and synthesizes these areas to 1 A comparatively bright combining process or a comparative dark combining process for generating image data of a sheet can be performed. Note that the image composition unit 54 may be configured integrally with the image processing unit 24.

  In addition, the image processing unit 24 performs predetermined arithmetic processing on the image data. The system control unit 50 controls the exposure control unit and the distance measurement control unit (both not shown) based on the calculation result obtained by the image processing unit 24, and performs TTL AF processing, AE processing, and EF processing. Do.

  The memory control unit 15 controls the A / D converter 23, the image processing unit 24, and the memory 32 under the control of the system control unit 50. The image data that is the output of the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15 or directly via the memory control unit 15. The display unit 28 is, for example, a TFT LCD, and the memory control unit 15 displays the image data written in the memory 32 on the display unit 28 as an image.

  If image data obtained by repeatedly performing exposure and reading at a predetermined cycle in the imaging unit 22 is displayed on the display unit 28 via the image processing unit 24 and the memory control unit 15, a live view display and an electronic viewfinder are displayed. Can be realized. The display unit 28 can turn on or off the display under the control of the system control unit 50. When the display is turned off, the power consumption of the camera 100 can be greatly reduced.

  As described above, the memory 32 stores image data (still images and moving images) obtained as a result of shooting. The memory 32 has a sufficient storage capacity for storing a predetermined number of still images and a moving image for a predetermined time. Thus, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, a large amount of image data can be written into the memory 32 at high speed. The memory 32 can be used as a work area for the system control unit 50.

  The system control unit 50 controls the entire camera 100. The non-volatile memory 56 is, for example, a flash ROM, and a control program and the like are stored in the non-volatile memory 56, and the system control unit 50 executes the control program and performs various processes described below. The non-volatile memory 56 is provided with an area for storing system information and an area for storing user setting information. When the camera is activated, various information and settings are read and restored from these areas. .

  For example, a RAM is used as the system memory 52. In the system memory 52, constants and variables for operation of the system control unit 50, a control program read from the nonvolatile memory 56, and the like are expanded. Further, the system control unit 50 performs display control by controlling the memory 32, the display unit 28, and the like. The system timer 53 is a time measuring unit that measures the time used for various controls and the time of a built-in clock.

  The operation unit 70 includes various buttons and a touch panel. For example, as various buttons, there are a shutter switch (SW1 and SW2), a menu button, a set button, a macro button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and the like. Further, as various buttons, there are a menu movement + (plus) button, a menu movement-(minus) button, a photographing image quality selection button, an exposure correction button, a date / time setting button, and the like.

  During the operation of the shutter button, the shutter switch SW1 is turned on, and the system control unit 50 starts AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and the like. Upon completion of the operation of the shutter button, the shutter switch SW2 is turned on, and the system control unit 50 starts a series of processing such as exposure processing, development processing, compression / expansion processing, and recording processing.

  As the power supply unit 30, for example, a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery is used, and the power supply unit 30 is provided with an AC adapter or the like. . The power supply unit 30 is controlled by the power supply control unit 80.

  The recording medium 200 is detachable from the camera 100. For example, a memory card or a hard disk is used. The recording medium 200 is connected to the camera 100 via an interface (I / F) 18.

  The system control unit 50 executes a predetermined calculation by the calculation unit 51 when calculating a motion vector described later.

  When calculating the motion vector, the system control unit 50 sets the first image as a reference image for two images taken continuously in time. Then, the system control unit 50 divides the crop region cut out in each of the two images into a plurality of blocks. Subsequently, the system control unit 50 performs edge extraction for the reference image, and performs template matching between the two images for blocks in which the edge amount is larger than a predetermined amount in a plurality of blocks. As a result, the system control unit 50 detects a motion vector for each block.

  Here, a combining process for combining a plurality of image data performed by the camera 100 shown in FIG. 1 will be described.

  As the synthesis process, the image synthesis unit 54 can perform four synthesis processes including the above-described addition synthesis process, weighted addition synthesis process, comparatively bright synthesis process, and comparative dark synthesis process.

  Now, the pixel value of the image data before synthesis is I_i (x, y) (i = 1 to N, x, y indicates coordinates in the image), and N images (N is an integer of 2 or more) are synthesized. Let the pixel value of the composite image after processing be I (x, y). Note that the pixel values may be R, G1, G2, and B signal values in a Bayer array, and luminance signal values (luminance values) obtained from a group of R, G1, G2, and B signals. May be used.

  Further, the luminance value may be obtained for each pixel after interpolation processing is performed so that the signal values in the Bayer array have R, G, and B signal values for each pixel. When obtaining the luminance value, for example, assuming that the luminance value is Y, calculation is performed by weighted addition of R, G, and B signal values such as Y = 0.3 × R + 0.59 × G + 0.11 × B. A technique is used.

  The image synthesis unit 54 performs addition synthesis processing on the pixel values associated with each other by performing processing such as alignment between a plurality of image data as necessary, by the following equation (1).

I (x, y) = I_1 (x, y) + I_2 (x, y) +... + I_N (x, y) (1)
The image synthesizing unit 54 adds and synthesizes the pixel values of N images for each pixel as synthesized image data.

  In the weighted addition synthesis process, the image synthesis unit 54 performs the weighted addition totaling process according to the following equation (2) using ak as a weighting coefficient.

I (x, y) = (a1 × I_1 (x, y) + a2 × I_2 (x, y) +... + AN × I_N (x, y)) / N (2)
The image synthesizing unit 54 uses, as synthesized image data, a weighted addition synthesis process of pixel values of N images for each pixel.

  In the comparatively bright combining process, the image combining unit 54 performs the comparatively bright combining process according to the following equation (3).

I (x, y) = max (I_1 (x, y), I_2 (x, y),..., I_N (x, y)) (3)
The image composition unit 54 selects the maximum value of the pixel values of the N images for each pixel and sets the maximum value as composite image data.

  In the comparative dark composition process, the image composition unit 54 performs the comparative dark composition process according to the following equation (4).

I (x, y) = min (I_1 (x, y), I_2 (x, y),..., I_N (x, y)) (4)
The image composition unit 54 selects the minimum value of the pixel values of the N images for each pixel and sets the minimum value as composite image data.

  Here, when the comparatively bright combination process is performed, the comparatively bright combination process is performed in the following procedure. First, in response to a user operation, the system control unit 50 starts interval shooting. Then, the system control unit 50 sets the first image of the plurality of images as an initial value in the compositing memory area set in the memory 32.

  Next, under the control of the system control unit 50, the image composition unit 54 compares the luminance of the same pixel in the second image and the first image set in the composition memory area. Then, the image composition unit 54 stores the pixel value having the higher luminance in the composition memory area.

  The above processing is performed for all pixels of the first image and the second image, and the image composition unit 54 stores the composite image of the first image and the second image in the compositing memory area. To do.

  Subsequently, for the third and subsequent images, the image composition unit 54 determines the luminance of the same pixel as the composite image stored in the compositing memory area and the image to be processed (that is, the third image here). Compare Then, the pixel value having the higher luminance is stored in the synthesis memory area. Furthermore, the image composition unit 54 performs the process on the composite image and all pixels of the image to be processed.

  In this way, the image composition unit 54 performs the above-described processing for all the images to be processed. That is, the image composition unit 54 performs the processing shown by the following equation (5).

I2 (x, y) = max (I_1 (x, y), I_2 (x, y))
I3 (x, y) = max (I_2 (x, y), I_3 (x, y))
...
I (x, y) = IN (x, y) = max (IN_1 (x, y), I_N (x, y)) (5)
FIG. 2 is a flowchart for explaining the calculation of the shift correction amount performed by the camera shown in FIG. Note that the processing according to the illustrated flowchart is performed under the control of the system control unit 50.

  When there is a photographing operation by the user, the system control unit 50 starts photographing a celestial body that is a subject (step S201). When a predetermined number of images (a plurality of images) are obtained by continuous shooting, the system control unit 50 sets a first crop area (first detection area) in the image obtained as a result of shooting (step 1). S202).

  FIG. 3 is a diagram showing an example of a crop area set in an image in the camera shown in FIG.

  In the illustrated example, the image includes a celestial body 302 that moves due to a diurnal motion, and the first crop region 301 is set so as to include a part of the celestial body 302. Here, the size of the first crop area 301 is sufficiently smaller than the size of the image, and the system control unit 50 sets the first crop area 301 according to the crop information stored in the nonvolatile memory 56.

  Subsequently, the system control unit 50 detects a celestial motion vector (first motion vector) in the first crop region (step S203). Then, the system control unit 50 analyzes the motion vector obtained in step S203 (step S204). The motion vector analysis will be described later.

  Next, the system control unit 50 determines whether or not the celestial body is in a predetermined state according to the motion vector analysis result (step S205). If the celestial body is in a predetermined state (YES in step S205), the system control unit 50 reduces the image (also referred to as a vector extraction image) by the image processing unit 24 as shown in FIG. 3 (step S206). .

  Thereafter, the system control unit 50 performs a second crop area (second detection area) whose size is larger than that of the first crop area 301 set in step S202 with respect to the image after reduction processing (reduced image). Is reset (step S207). Then, the system control unit 50 detects a motion vector (second motion vector) in the reset second crop region (step S208). Note that the second crop area may be set for the entire reduced image, or may be set for a part of the reduced image. When the second crop area is set as a part of the reduced image, the size needs to be set so as to be larger than the first crop area.

  Subsequently, the system control unit 50 calculates an affine coefficient based on the motion vector detected in step S208, and obtains an image shift correction amount according to the affine coefficient. (Step S209). Then, the system control unit 50 ends the deviation correction amount calculation. Thereafter, under the control of the system control unit 50, the image synthesis unit 54 corrects the positional deviation between the images using the deviation correction amount, and performs image synthesis.

  If the analysis result of the motion vector is not in a predetermined state (NO in step S205), the system control unit 50 proceeds to the process of step S209, and obtains a deviation correction amount according to the motion vector obtained in step S203.

  As described above, since the image is reduced according to the analysis result of the motion vector, even if a large crop area is set in size, the motion vector detection process can be performed at high speed. As a result, the processing time can be shortened also in the displacement correction processing between images, and the tracking target celestial body can be kept at the field angle of the camera for a long time.

  Furthermore, when high-sensitivity shooting is performed or when the temperature during shooting is high, noise is likely to occur in the image, and the possibility of erroneous detection of a motion vector due to the influence of noise increases. Here, since the image is reduced, the influence of noise can be reduced.

  Here, motion vector analysis will be described. As described above, first, the system control unit 50 obtains an analysis result by analyzing a motion vector detected in a crop area having a small size set in an equal-size image (an image that is not reduced). The predetermined state mentioned above refers to, for example, a state where the entire image is rotated or a state where the proportion of subjects other than the main subject is mixed in the crop area is higher than the predetermined proportion.

  For example, in a state where the entire image is rotated, it is difficult to obtain a sufficient motion vector indicating the rotation in a small crop area, and the shift correction amount cannot be obtained with high accuracy. Further, if the ratio of subjects other than the main subject in the crop area is high, the motion vector related to the subjects other than the main subject is strongly influenced. For this reason, the shift correction amount cannot be obtained with high accuracy. That is, it is assumed that the system control unit 50 is in a state where the deviation correction amount cannot be accurately calculated when the motion vector analysis result is in a predetermined state.

  FIG. 4 is a diagram for explaining processing when it is determined that the entire image is rotated in the camera shown in FIG.

  The system control unit 50 performs motion vector grouping in the crop area 301 based on the direction and magnitude of the motion vector. When the entire image is not rotated, the motion vectors belong to the same group. Therefore, when the motion vector is divided into a plurality of groups in the crop area 301, the system control unit 50 determines that the entire image is rotating.

  FIG. 5 is a diagram for explaining processing when it is determined that subjects other than the main subject are mixed in the crop area in the camera shown in FIG.

  In FIG. 5, the crop area 301 includes a foreground such as a tree 501 in addition to the celestial body 302 that is the main subject. While the celestial body 302 moves, the foreground such as the tree 501 does not move. Therefore, grouping motion vectors converges on two peaks. Therefore, the system control unit 50 determines that a subject other than the main subject is mixed in the crop region 301 when there are two groups in which the motion vectors are grouped in accordance with the vector amount and have a peak.

  In this manner, the motion vector is detected from the equal-size image or the reduced image depending on whether or not the motion vector analysis result is in a predetermined state. Thereby, the motion vector can be obtained accurately in a short time.

  In the above example, it is determined whether or not the celestial body that is the subject is in a predetermined state according to the analysis result of the motion vector. However, the determination method is not limited to this, and other methods may be used for determination. Good. For example, by providing the camera 100 with a detection unit that detects the direction of the celestial body that is the imaging target, it is possible to determine whether or not the entire image is rotating from the imaging direction. It is also possible to determine whether a subject other than the main subject is mixed in the image using information such as luminance other than the motion vector of the image.

  In the above-described embodiment, when the celestial body is in a predetermined state based on the motion vector analysis result, the second crop area having a size larger than that of the first crop area is set. The second crop area may be set for the entire image. Furthermore, a plurality of crop areas may be set and the plurality of crop areas may be set as the second crop area.

  As described above, in the embodiment of the present invention, a motion vector required for deviation correction can be calculated at high speed and efficiently.

  As is clear from the above description, in the example shown in FIG. 1, the image processing unit 24 and the system control unit 50 have the first setting unit, the first detection unit, the determination unit, the second setting unit, and the second setting unit. Functions as a detection means. The system control unit 50 and the image composition unit 54 function as image composition means. The imaging lens 103, the shutter 102, the imaging unit 22, and the A / D conversion unit 23 constitute an imaging unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the image processing apparatus. In addition, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the image processing apparatus. The control program is recorded on a computer-readable recording medium, for example.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 15 Memory control part 22 Imaging part 24 Image processing part 28 Display part 32 Memory 50 System control part 51 Calculation part 54 Image composition part 70 Operation part 103 Shooting lens

Claims (10)

An image processing apparatus that combines a plurality of images obtained by photographing a moving subject to form a combined image,
First setting means for setting a first detection region for detecting a motion vector in each of the images;
First detection means for detecting a motion vector related to the subject in the first detection region as a first motion vector;
Determining means for determining whether or not the subject is in a predetermined state;
A second setting means for setting a second detection area wider than the first detection area in each of the images when the determination means determines that the subject is in the predetermined state;
Second detection means for detecting a motion vector related to the subject in the second detection region as a second motion vector;
Image synthesizing means for synthesizing the plurality of images according to the first motion vector or the second motion vector into the synthesized image;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the second setting unit sets the second detection area in the reduced image by reducing each of the images to obtain a reduced image.   The image processing apparatus according to claim 1, wherein the second setting unit sets the entire image as the second detection area.   The image processing apparatus according to claim 1, wherein the second setting unit sets a plurality of detection areas and sets the plurality of detection areas as the second detection areas.   The determination unit performs the grouping of the motion vectors based on the direction and magnitude of the first motion vector, and the subject is detected when the first motion vector is divided into a plurality of groups. The image processing apparatus according to claim 1, wherein the image processing apparatus is determined to be in a predetermined state.   The determination unit is configured to determine that the subject is in the predetermined state when there are two groups having a peak by grouping the first motion vectors according to the vector amount. Item 5. The image processing apparatus according to any one of Items 1 to 4.   The image processing apparatus according to claim 1, wherein the subject is an astronomical object. An imaging means for imaging the plurality of images;
An image processing apparatus according to any one of claims 1 to 7,
Recording means for recording a composite image obtained by the image processing apparatus;
An imaging device comprising: A control method of an image processing apparatus that combines a plurality of images obtained by photographing a moving subject to form a composite image,
A first setting step of setting a first detection region for detecting a motion vector in each of the images;
A first detection step of detecting a motion vector related to the subject in the first detection region as a first motion vector;
A determination step of determining whether or not the subject is in a predetermined state;
A second setting step of setting a second detection area wider than the first detection area in each of the images when it is determined in the determination step that the subject is in the predetermined state;
A second detection step of detecting a motion vector related to the subject in the second detection region as a second motion vector;
An image synthesis step of synthesizing the plurality of images according to the first motion vector or the second motion vector into the synthesized image;
A control method characterized by comprising: A control program used in an image processing apparatus that synthesizes a plurality of images obtained by photographing a moving subject to form a composite image,
In the computer provided in the image processing apparatus,
A first setting step of setting a first detection region for detecting a motion vector in each of the images;
A first detection step of detecting a motion vector related to the subject in the first detection region as a first motion vector;
A determination step of determining whether or not the subject is in a predetermined state;
A second setting step of setting a second detection area wider than the first detection area in each of the images when it is determined in the determination step that the subject is in the predetermined state;
A second detection step of detecting a motion vector related to the subject in the second detection region as a second motion vector;
An image synthesis step of synthesizing the plurality of images according to the first motion vector or the second motion vector into the synthesized image;
A control program characterized by causing