JP4980982B2 - Imaging apparatus, imaging method, focus control method, and program - Google Patents

Description

  The present invention relates to an image pickup apparatus, an image pickup method, a focus control method, and a program, and more particularly to an image pickup apparatus, an image pickup method, a focus control method, and a program that display an image picked up by an image pickup device in real time (display a through image). .

  In Patent Document 1, the image data stored in both CCDs is alternately read when a through image is displayed, and AF processing is performed using the image data of one CCD during the AF process. An image pickup apparatus is described in which image data of a CCD is displayed as a through image, thereby improving the frame rate when displaying a through image and improving the performance during AF processing.

Patent Document 2 divides subject light into two parts, detects a subject image that moves in a horizontal direction perpendicular to the optical axis in each of the divided images, and determines the subject's optical axis direction and light based on the detection result. An automatic focusing device is described that detects the movement of a subject in a direction perpendicular to the axial direction and focuses the subject.
JP 2007-97033 A JP-A-5-93850

  However, the invention described in Patent Document 1 has a problem that separate CCDs must be prepared for the CCD used for AF processing and the CCD used for through image display.

  Further, in the invention described in Patent Document 2, since it is necessary to divide subject light into two, there is a problem that the structure becomes complicated and the cost is high.

  Furthermore, in the invention described in Patent Document 2, in order to detect a subject image that moves in a direction other than the horizontal direction perpendicular to the optical axis and detect the movement of the subject, a subject that moves in a direction other than the horizontal direction perpendicular to the optical axis. There is a problem that focusing accuracy is not good for an image.

  The present invention has been made in view of such circumstances. By performing through image shooting and AF control in accordance with the movement of a subject with a simple configuration, the image quality of the through image and the focusing accuracy are improved. It is an object to provide an imaging apparatus, an imaging method, a focus control method, and a program capable of performing the above.

In order to achieve the above object, an invention according to one aspect of the present invention continuously displays an image of a subject imaged by the imaging unit and an imaging unit that continuously images the subject at a predetermined frame rate. A display means; a subject detection means for detecting a moving subject from at least two images before and after the image taken by the imaging means; and a calculation for calculating a moving speed or a moving distance of the moving subject detected by the subject detection means. Means for calculating a maximum moving speed or moving distance when a plurality of moving objects are detected, and the frame based on the moving speed or moving distance of the moving object calculated by the calculating means. And a frame rate control means for controlling the rate.

According to one aspect of the present invention, a moving subject is detected from continuously captured images (through images), and when a plurality of moving subjects are detected, the subject having the maximum moving speed or moving distance is detected. A through image is taken at a frame rate corresponding to the moving speed. As a result, it is possible to select a subject most suitable for the control of the frame rate and take a through image at a frame rate optimum for the subject. Therefore, a through image with good image quality can be obtained.

According to another aspect of the present invention, an imaging unit that includes an imaging optical system and an imaging device that images a subject image formed by the imaging optical system, and that continuously captures the subject at a predetermined frame rate. Subject detection means for detecting a moving subject from at least two front and rear images taken by the imaging means; calculation means for calculating a moving speed or a movement distance of the moving subject detected by the subject detection means; Automatic focusing control means for continuously performing focusing control for moving a photographing optical system and maintaining a focusing state for a moving subject detected by the subject detecting means, the subject calculated by the calculating means an automatic focus control means for controlling the interval of the continuously performed focusing control based on the moving speed or moving distance of a plurality of subjects detected by the subject detecting means A target subject setting unit configured to set a subject having the maximum moving speed or moving distance calculated by the calculating unit as a target subject, and the automatic focusing control unit includes the subject detecting unit. When a plurality of subjects are detected, focusing control is performed on the target subject set by the target subject setting means.

According to another aspect of the present invention, a moving subject is detected from continuously photographed images (through images), the moving speed or moving distance of the detected moving subject is calculated, and the calculated moving speed is calculated. Alternatively, the interval of the focusing control (continuous AF) continuously performed according to the moving distance is controlled. Thereby, the focusing accuracy with respect to the moving subject can be increased. In addition, when a plurality of moving subjects are detected, the subject having the maximum moving speed or moving distance is set as the target subject, and AF control is performed on the target subject. Thereby, focusing control can be performed on the most suitable subject.

According to still another aspect of the present invention, the imaging means includes an imaging optical system and an imaging element that images a subject image formed by the imaging optical system, and continuously images the subject at a predetermined frame rate. A subject detection unit that detects a moving subject from at least two front and rear images taken by the imaging unit, and whether the moving subject detected by the subject detection unit is moving in a direction approaching a shooting position, Alternatively, continuous movement direction detecting means for detecting whether the object moves away from the photographing position and focusing control for moving the photographing optical system and maintaining the in-focus state for the moving subject detected by the subject detecting means are continuously performed. Automatic focusing control means for automatically determining the search direction of the photographing optical system based on the moving direction of the subject detected by the moving direction detecting means An automatic focus control unit that, a calculation means for calculating a moving speed or moving distance of the moving subject detected by the subject detecting means, when a plurality of subjects are detected by the subject detecting means, by said calculating means Target subject setting means for setting a subject whose calculated moving speed or moving distance is the maximum as a target subject, and the moving direction detecting means detects the subject when the subject detecting means detects a plurality of subjects. Detecting whether the target subject set by the target subject setting means is moving in a direction approaching the shooting position or moving away from the shooting position, and the automatic focusing control means is the subject detection means When a plurality of subjects are detected by the above, focusing control is performed on the target subject set by the target subject setting means, and Serial and determines a search direction of the photographing optical system based on the movement direction of the target object detected by the movement direction detection means.

According to still another aspect of the present invention, a moving subject is detected from continuously shot images (through images), and the moving subject is moving in a direction approaching the shooting position or in a direction away from the shooting position. It detects whether it is moving or not, and controls the search direction of focus control (continuous AF) that is continuously performed according to the detected moving direction. Thereby, the focusing accuracy with respect to the moving subject can be increased. In addition, since the direction in which the imaging optical system is moved can be determined in advance, the focusing control can be speeded up. Further, when a plurality of moving subjects are detected, the subject having the maximum moving speed or moving distance is set as the target subject, and AF control is performed on the target subject. Thereby, focusing control can be performed on the most suitable subject.

According to still another aspect of the present invention, an imaging unit that continuously images a subject at a predetermined frame rate, and a subject detection that detects a moving subject from at least two front and rear images captured by the imaging unit. Means for calculating the moving speed or moving distance of the moving subject detected by the subject detecting means, and the moving speed calculated by the calculating means when a plurality of subjects are detected by the subject detecting means. Alternatively, a target subject setting unit that sets a subject having the maximum moving distance as a target subject, and a subject that predicts the position of the moving subject in the next image to be captured based on the position of the moving subject detected by the subject detection unit a position prediction means, when a plurality of subjects are detected by the subject detecting means, the target object setting means And the object position predicting means for predicting a more set position of the target object, met the automatic focus control means performs a focusing control for a predetermined region including a position of the object predicted by the object position predicting means Automatic focus control means for performing focus control on a predetermined area including the position of the target subject predicted by the subject position prediction means when a plurality of subjects are detected by the subject detection means; , Provided.

According to still another aspect of the present invention, a moving subject is detected from continuously captured images (through images), a subject position is predicted based on the position of the moving subject, and a predetermined including the predicted subject position is included. Focusing control (continuous AF) is performed on the area. Thereby, focusing control can be reliably performed on a moving subject. In addition, when a plurality of moving subjects are detected, the subject having the maximum moving speed or moving distance is set as the target subject, and AF control is performed on the target subject. Thereby, focusing control can be performed on the most suitable subject.

According to still another aspect of the present invention, the image processing apparatus includes a feature point extraction unit that extracts a feature point of an image captured by the imaging unit, and the subject detection unit uses the feature point extracted by the feature point extraction unit Based on the center position of the subject detected based on the center position of the subject detected by the subject detection means, the calculation means calculates the moving speed or moving distance of the subject.

According to still another aspect of the present invention, a subject and a center position thereof are detected by extracting feature points of image data, and a moving speed or a moving distance of the subject is calculated based on the center position. Thereby, it is possible to prevent erroneous recognition of the subject.

According to yet another aspect, an imaging step for imaging continuously the object at a predetermined frame rate, and a display step of displaying an image of a subject captured by the imaging step in succession, the imaging step of the present invention in a calculation step of calculating a subject detection step of detecting respective moving objects from at least the front and rear of the two images captured, the moving speed or moving distance of the object moving said detected by the subject detection step, moving the object When a plurality of movements are detected, a calculation step for calculating the maximum movement speed or movement distance, and a frame rate control for controlling the frame rate based on the movement speed or movement distance of the moving subject calculated in the calculation step And a step.

According to still another aspect of the present invention, an imaging step of continuously imaging a subject at a predetermined frame rate via an imaging element that captures a subject image formed by a photographing optical system; and A subject detection step for detecting a moving subject from at least two captured images, a calculation step for calculating a moving speed or a moving distance of the moving subject detected in the subject detection step, and the photographing optical system. An automatic focusing control step for continuously performing focusing control for moving and maintaining a focusing state with respect to the moving subject detected in the subject detection step , wherein the moving speed or movement of the subject calculated in the calculation step an automatic focus control step of controlling the spacing of said continuously performed focusing control based on the distance, a plurality of the at the subject detecting step A target subject setting step for setting a subject having the maximum moving speed or moving distance calculated in the calculating step as a target subject when the body is detected, and the automatic focusing control step includes: When a plurality of subjects are detected in the detection step, focusing control is performed on the target subject set in the target subject setting step.

According to still another aspect of the present invention, an imaging step of continuously imaging a subject at a predetermined frame rate via an imaging element that captures a subject image formed by a photographing optical system; and A subject detection step for detecting a moving subject from at least two captured images, and a direction in which the moving subject detected in the subject detection step is moving toward or away from the shooting position. A moving direction detecting step for detecting whether or not the subject is moving, and automatic focusing for continuously performing focusing control for moving the photographing optical system and maintaining the in-focus state for the moving subject detected in the subject detecting step. a control step, determining a search direction of the photographing optical system based on the moving direction of the detected object in the moving direction detection step That an automatic focusing control step, the a calculation step of calculating a moving speed or moving distance of the moving subject detected by the subject detection step, when a plurality of subjects in the subject detection step is detected in the calculation step A target subject setting step of setting a subject whose calculated moving speed or moving distance is the maximum as a target subject, and the moving direction detecting step includes a step when the subject detecting step detects a plurality of subjects. Detecting whether the target subject set in the target subject setting step is moving in a direction approaching the shooting position or moving away from the shooting position, and the automatic focusing control step is the subject detection step When a plurality of subjects are detected in step 1, the subject subject set in the subject subject setting step is Performs focusing control, and is characterized in that to determine the search direction of the photographing optical system based on the movement direction of the target object detected by said moving direction detection step.

According to still another aspect of the present invention, an imaging step for continuously imaging a subject at a predetermined frame rate, and a subject detection for detecting a moving subject from at least two images before and after the imaging step. A calculating step for calculating a moving speed or moving distance of the moving subject detected in the subject detecting step, and a moving speed calculated in the calculating step when a plurality of subjects are detected in the subject detecting step. or a target subject setting step of moving distance is set as the target object up to the subject, based on the position of a moving subject detected by the subject detection step, then the subject for predicting the position of a moving object in an image to be captured a position prediction step, when a plurality of subjects are detected by the subject detecting step, the Automatic focusing performed and the object position prediction step, the focusing control with respect to a predetermined area including the location of the predicted subject in the subject position prediction step of predicting the position of the target object set by elephant object setting step In the control step , when a plurality of subjects are detected in the subject detection step, automatic focusing is performed for a predetermined area including the position of the target subject predicted in the subject position prediction step. A focus control step .

According to still another aspect of the present invention, an arithmetic device is caused to execute the imaging method according to claim 6 or the focus control method according to any one of claims 7 to 9 .

  According to the present invention, it is possible to improve the image quality and focusing accuracy of a through image by performing through image capturing and AF control in accordance with the movement of the subject with a simple configuration. A focus control method and program can be provided.

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The best mode for carrying out an imaging apparatus, an imaging method, a focus control method and a program to which the present invention is applied will be described in detail below with reference to the accompanying drawings.

<First Embodiment>
In the first embodiment of the present invention, when one moving subject is detected in a captured subject image, a through image is displayed at an optimum frame rate according to the subject's motion. is there.

  FIG. 1 is a block diagram showing an electrical schematic configuration inside the digital camera 10 of the imaging apparatus according to the first embodiment.

  As shown in FIG. 1, the digital camera 10 mainly includes a CPU 110, a memory 112, a timing generator (TG) 114, motor drivers 116 and 118, an operation unit 120, a zoom lens 122, a focus lens 124, an image sensor 126, and analog processing. Unit (CDS / AMP) 128, A / D converter 130, image input control unit 132, image signal processing unit 134, compression / decompression processing unit 136, AE / AF detection circuit 140, moving object detection circuit 142, media controller 150, display A circuit 154, a storage medium 152, an image display device 156, and the like are included.

  The CPU 110 performs overall control of the entire digital camera 10 according to a predetermined control program based on an operation signal input from the operation unit 120.

  A memory 112 connected to the CPU 110 via the bus stores various control information executed by the CPU 110, various data necessary for control, various setting information regarding the operation of the digital camera 10 such as user setting information, and the like. . The memory 112 is used as a calculation work area for the CPU 110 and also as a temporary storage area for image data and the like.

  The operation unit 120 includes a shutter button, a power / mode switch, a zoom button, and the like.

  The shutter button is composed of a two-stage stroke type switch composed of a so-called “half press” and “full press”. When the shutter button 26 is “half-pressed”, the digital camera 10 activates AE / AF, and “full-press” to execute shooting.

  The power / mode switch has both a function as a power switch for turning on / off the power of the digital camera 10 and a function as a mode switch for setting the mode of the digital camera 10. ”And“ shooting position ”are slidably arranged. The digital camera 10 is turned on by sliding the power / mode switch to the “reproduction position” or “photographing position”, and turned off by setting it to the “OFF position”. Then, the power / mode switch is slid and set to “playback position” to set “playback mode”, and set to “shooting position” to set “shooting mode”.

  The zoom button functions as zoom instruction means for instructing zooming, and includes a zoom tele button for instructing zooming to the telephoto side and a zoom wide button for instructing zooming to the wide angle side. When the zoom tele button and the zoom wide button are operated in the shooting mode, the CPU 110 moves the zoom lens 122 via the motor driver 116 to change the focal length.

  The zoom lens 122 is driven by the zoom motor via the motor driver 116 and moves back and forth on the photographing optical axis, thereby changing the focal length.

  The focus lens 124 is driven by the focus motor via the motor driver 118 and moves back and forth on the photographing optical axis, thereby changing the imaging position.

  The image sensor 126 has a large number of photodiodes (light receiving elements) through, for example, red (R), green (G), and blue (B) color filters arranged in a predetermined arrangement structure (Bayer, G stripe, etc.). Is two-dimensionally arranged on the light receiving surface, and electronically captures an image of a subject formed by the zoom lens 122, the focus lens 124, and the like. A timing generator (TG) 114 outputs a timing signal for driving the image sensor 126 in response to a command from the CPU 110.

  The analog processing unit 128 samples and holds the R, G, and B signals for each pixel with respect to the image signal output from the image sensor 126 (correlation double sampling processing), and amplifies the signal to the A / D converter 130. Output to.

  The A / D converter 130 converts the analog R, G, and B signals output from the analog processing unit 128 into digital R, G, and B signals and outputs them.

  The image input control unit 132 outputs digital R, G, and B signals output from the A / D converter 130 to the memory 112.

  The image signal processing unit 134 includes a synchronization circuit (a processing circuit that converts a color signal into a simultaneous expression by interpolating a spatial shift of the color signal associated with the color filter array of the single CCD), a white balance correction circuit, and a gamma correction. Circuit, contour correction circuit, luminance / color difference signal generation circuit, etc., and according to a command from the CPU 110, the input image signal is subjected to necessary signal processing to obtain luminance data (Y data) and color difference data (Cr, Cb data). ) Is generated.

  The compression / decompression processing unit 136 performs compression processing in a predetermined format on the input image data in accordance with a command from the CPU 110 to generate compressed image data. Further, in accordance with a command from the CPU 110, the input compressed image data is subjected to a decompression process in a predetermined format to generate uncompressed image data.

  The AE / AF detection circuit 140 includes an AE (automatic exposure) detection circuit and an AF (automatic focus) detection circuit. The AF detection circuit includes a high-pass filter that passes only high-frequency components of the G signal, an absolute value processing unit, an AF area detection unit that extracts a signal within a predetermined focus area (for example, the center of the screen), and an absolute value within the AF area. It consists of an integration unit that integrates data. During the AF control, the CPU 110 sequentially moves the focus lens 124 from a closest position to a lens position corresponding to infinity, and at the same time, a high-frequency component of the image signal based on the image signal of the AF area captured via the image sensor 126 for each lens position. Is obtained from the AF detection circuit, a lens position where the evaluation value reaches a peak is obtained, and contrast AF is performed to move the focus lens 124 to the lens position.

  The AE detection circuit calculates a physical quantity necessary for AE control and AWB (automatic white balance) control from the input image signal in accordance with a command from the CPU 110. For example, as a physical quantity required for AE control, one screen is divided into a plurality of areas (for example, 16 × 16), and an integrated value of R, G, and B image signals is calculated for each divided area.

  The moving object detection circuit 142 detects the movement of the subject and determines a frame rate for displaying a through image. As shown in FIG. 2, the moving object detection circuit 142 receives, as parameters, an image at the time of moving object detection (nth frame) and an image one frame before (n-1 frame) from the time of moving object detection. Using the correspondence table of the moving speed and the frame rate recorded in the memory 112, a frame rate for displaying a through image is output. Processing performed by the moving object detection circuit 142 will be described in detail later.

  The media controller 150 controls reading / writing of data with respect to the storage medium 152 loaded in the media slot according to a command from the CPU 110.

  The display circuit 154 controls display on the image display device 156 in accordance with a command from the CPU 110. That is, in accordance with a command from the CPU 110, the input image signal is converted into a video signal (for example, NTSC signal, PAL signal, SCAM signal) for display on the image display device 156, and output to the image display device 156. Predetermined character and graphic information synthesized by the CPU 110 as necessary is output to the image display device 156.

  The image display device 156 includes a liquid crystal display capable of color display. The image display device 156 is used as an image display panel for displaying captured images in the playback mode, and is used as a user interface display panel for performing various setting operations. In the photographing mode, a through image is displayed as necessary, and is used as an electronic viewfinder for checking the angle of view.

  Next, the operation of the digital camera 10 of the present embodiment configured as described above will be described.

  By setting the power / mode switch to the shooting position, the shooting mode is set and shooting is possible. Then, when the photographing mode is set, the lens group is extended and the photographing standby state is set.

  Under this photographing mode, the subject light that has passed through the lens group forms an image on the light receiving surface of the image sensor 126 through a diaphragm (not shown). The subject light that has passed through the lens group is received by each photodiode and converted into a signal charge corresponding to the amount of incident light.

  The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge on the basis of the drive pulse supplied from the timing generator (TG) 114, and an analog processing unit (CDS / AMP). 128.

  Analog R, G, and B signals output from the analog processing unit 128 are converted into digital R, G, and B signals by the A / D converter 130 and added to the image input control unit 132. The image input control unit 132 outputs digital R, G, and B signals output from the A / D converter 130 to the memory 112.

  Digital R, G, and B signals are input from the memory 112 to the moving object detection circuit 142, and the CPU 110 issues a command to the timing generator (TG) 114 and outputs a frame rate for displaying a through image output from the moving object detection circuit 142. To drive the image sensor 126.

  In FIG. 3, a frame rate for displaying a through image is output from the digital R, G, B signal generated based on the image acquired by the image sensor 126 in the moving object detection circuit 142, and the image is acquired at this frame rate. It is a flowchart which shows the flow of the process which displays a through image.

  As shown below, the moving body detection circuit 142 detects a moving subject based on the input digital R, G, and B signals (step S10).

  The moving object detection circuit 142 acquires image data of the nth frame as an image for detecting the movement of the subject. As shown in FIG. 4B, the moving body detection circuit 142 detects a feature point of the subject (for example, an edge of the subject) from the input image data of the nth frame.

  Next, the moving object detection circuit 142 acquires image data one frame before (n−1 frame) of the n frame image. The moving object detection circuit 142 detects a feature point of the subject (for example, an edge of the subject) in the same manner as performed on the image data of the nth frame for the input image of the (n−1) th frame.

  Then, feature points detected from the image data of the (n−1) th frame (see FIG. 4A) and feature points detected from the image data of the nth frame as shown in FIG. 4 (see FIG. 4B). If the positions of the same feature points do not match, it is determined that the subject is moving.

  In addition, the moving object detection circuit 142 detects the contour of the subject based on the feature point (see FIG. 4A) detected from the image data of the (n-1) th frame, and based on that, the subject of the (n-1) th frame. Find the center of. The moving object detection circuit 142 detects the contour of the subject based on the feature points (see FIG. 4B) detected from the image data of the nth frame, and obtains the center of the subject of the nth frame based on the detected contour. . Thus, by calculating the center of the subject based on the feature points, it is possible to prevent the subject from being erroneously recognized.

  Thereby, the subject detection process (step S10) ends. When a moving subject is detected in step S10, the moving object detection circuit 142, as shown in FIG. 4C, the center of the subject of the (n-1) th frame and the center of the subject of the nth frame. Based on the above, the moving speed of the subject in the direction perpendicular to the optical axis direction (x direction and y direction) is calculated (step S11). Thus, by calculating the moving speed of the subject based on the center of the subject, the moving speed of the subject can be accurately calculated.

  The moving object detection circuit 142 calculates the moving speed of the subject based on the calculated moving distance of the subject and the time of one frame (the reciprocal of the frame rate).

  Thereby, the moving speed calculation process (step S11) is terminated. The moving object detection circuit 142 refers to the correspondence table of the moving speed and the frame rate recorded in the memory 112, and selects a through image display frame rate corresponding to the moving speed of the subject calculated in step S11 (step S12). ). A correspondence table between the moving speed and the frame rate is prepared in advance in the memory 112 so that a frame rate corresponding to the moving speed can be selected. In this correspondence table, a frame rate of 30 to 60 frames / second is set according to the moving speed. The frame rate for displaying the through image selected in step S12 is output from the moving object detection circuit 142 to the CPU 110.

  The CPU 110 issues a command to the timing generator (TG) 114, drives the image sensor 126 at a frame rate for displaying a through image output from the moving object detection circuit 14, and periodically captures an image signal from the image sensor 126. The captured image signal processing unit 134 generates a luminance / color difference signal from the inserted image signal, and the display circuit 154 converts the luminance / color difference signal into a signal format for display (for example, an NTSC color composite video signal) to generate an image. The data is output to the display device 156. Thereby, the image imaged with the image pick-up element 126 is displayed on the image display apparatus 156 in real time (step S13).

  As a result, an optimal frame rate is selected for the moving subject, and through image control is performed in accordance with the motion of the subject. The user operates the zoom button while confirming the through image displayed on the image display device 156, thereby determining the angle of view and performing a photographing operation.

  Shooting is performed by pressing the shutter button 26. When the shutter button 26 is half-pressed, an S1 ON signal is input to the CPU 110.

  Based on the integrated value data calculated by the AE / AF detection circuit 140, the CPU 110 controls the lens driving unit and moves the focus lens 124 of the photographing optical system including the lens group at a plurality of AF detection points. A focus evaluation value (AF evaluation value) is calculated, and a lens position where the evaluation value is maximized is determined as a focus position. Then, the lens driving unit is controlled so that the focus lens 124 moves to the obtained in-focus position.

  The CPU 110 detects the brightness of the subject (subject brightness) based on the integrated value obtained from the AE / AF detection circuit 140, and calculates an exposure value (shooting EV value) suitable for shooting. Then, an aperture value and a shutter speed are determined from the obtained shooting EV value and a predetermined program diagram, and an electronic shutter and an aperture drive unit of the image sensor 126 are controlled according to this (so-called electronic shutter) to obtain an appropriate exposure amount. .

  Further, the AE / AF detection circuit 140 calculates an average integrated value for each color of the R, G, and B signals for each divided area during automatic white balance adjustment, and provides the calculation result to the CPU 110.

  The CPU 110 obtains the ratio of R / G and B / G for each divided area from the obtained R accumulated value, B accumulated value, and G accumulated value, and R of the obtained R / G and B / G values. The light source type is discriminated based on the distribution in the color space of / G and B / G. Then, according to the white balance adjustment value suitable for the discriminated light source type, for example, the value of each ratio is approximately 1 (that is, the RGB integration ratio is R: G: B≈1: 1: 1 on one screen). As described above, the gain value (white balance correction value) for the R, G, and B signals of the white balance adjustment circuit is controlled to correct the signal of each color channel.

  As described above, AE / AF processing and AWB processing are performed by half-pressing the shutter button 26. During this time, the user operates the zoom button 38 as necessary to zoom the lens group and adjust the angle of view, and the CPU 110 continues to perform the AE / AF process and the AWB process.

  Thereafter, when the shutter button 26 is fully pressed, an S2ON signal is input to the CPU 110, and the CPU 110 starts photographing and recording processing. That is, the image sensor 126 is exposed at the shutter speed and aperture value determined based on the photometric result.

  The image signal output from the image sensor 126 is taken into the memory 112 via the analog processing unit 128, the A / D converter 130, and the image input control unit 132, and is converted into a luminance / color difference signal by the image signal processing unit 134. After that, it is stored in the memory 112.

  The image data stored in the memory 112 is added to the compression / decompression processing unit 136, compressed in accordance with a predetermined compression format (for example, JPEG format), then stored in the memory 112, and a predetermined image recording format (for example, Exif format). Are recorded on the storage medium 152 via the media controller 150.

  According to the present embodiment, it is possible to switch the frame rate at the time of obtaining a through image based on the moving speed of the subject and display the through image at an appropriate frame rate according to the moving speed of the subject.

  That is, the frame rate is automatically increased as the subject moves faster. When the frame rate is slower than the movement of the subject, problems such as the subject's movement not appearing smoothly occur, but a through image when moving subjects are included by increasing the frame rate according to the subject's movement. Image quality can be improved.

  Also, when the movement of the subject slows down or the subject stops, the frame rate is automatically lowered. When the frame rate is faster than the movement of the subject, power consumption increases and noise increases, resulting in problems such as image quality degradation.However, by reducing the frame rate according to the subject's movement, Can be prevented.

  In the present embodiment, in step S11, as shown in FIG. 4C, the center of the subject is calculated based on the feature point calculated in step S10, and the moving distance of the subject is calculated based on the center of the subject. However, as shown in FIG. 5, the moving distance of the subject may be calculated based on the feature points calculated in step S10. In this case, since it is not necessary to obtain the center of the subject, the processing can be simplified and the time required for the processing can be shortened.

  In the present embodiment, the frame rate for through image display corresponding to the moving speed of the subject calculated in step S11 is selected in step S12 with reference to the correspondence table of moving speed and frame rate. The method for selecting the frame rate for displaying the through image is not limited to this. For example, instead of calculating the moving speed of the subject in step S11, the moving distance between the center of the subject of the (n-1) th frame and the center of the subject of the nth frame is calculated, and the subject stored in the memory 112 in advance is calculated. A through image display frame rate corresponding to the calculated movement distance of the subject may be selected with reference to a correspondence table of movement distance and frame rate.

  In this embodiment, the subject is detected from two consecutive images. However, the subject may be detected from two non-continuous images, or the subject may be detected from two or more images. Also good.

<Second Embodiment>
In the first embodiment of the present invention, the case where only one moving subject is detected in the moving object detection circuit 142 (step S10) has been described. However, only one moving subject is detected. Not exclusively.

  In the second embodiment of the present invention, when a plurality of moving subjects are detected by the moving object detection circuit 142, a through image is displayed at an optimal frame rate according to the motion of the subject. Hereinafter, the digital camera 11 according to the second embodiment of the present invention will be described. In addition, about the part same as 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  Since the electrical schematic configuration inside the digital camera 11 is the same as that of the digital camera 10, a description thereof will be omitted, and the operation of the digital camera 11 will be described.

  By setting the power / mode switch of the digital camera 11 to the photographing position, the photographing mode is set and the photographing standby state is set.

  Under this photographing mode, the subject light that has passed through the lens group forms an image on the light receiving surface of the image sensor 126 through a diaphragm (not shown). The subject light that has passed through the lens group is received by each photodiode and converted into a signal charge corresponding to the amount of incident light.

The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge on the basis of the drive pulse supplied from the timing generator (TG) 114, and an analog processing unit (CDS / AMP). 128.
Analog R, G, and B signals output from the analog processing unit 128 are converted into digital R, G, and B signals by the A / D converter 130 and added to the image input control unit 132. The image input control unit 132 outputs digital R, G, and B signals output from the A / D converter 130 to the memory 112.

  Digital R, G, and B signals are input from the memory 112 to the moving object detection circuit 142, and the CPU 110 issues a command to the timing generator (TG) 114 and outputs a frame rate for displaying a through image output from the moving object detection circuit 142. To drive the image sensor 126.

  FIG. 6 shows a frame rate for displaying a through image based on digital R, G, and B signals generated based on an image acquired by the image sensor 126 when there are a plurality of moving subjects. It is a flowchart which shows the flow of the process which displays the through image acquired at the frame rate.

  As shown below, the moving object detection circuit 142 detects N moving subjects based on the input digital R, G, and B signals (step S20). This means that N moving subjects have been detected as a result of the processing of step S10.

  Next, the subject having the maximum predicted movement distance is selected from the N subjects detected in step S20 (steps S21 to S26).

  First, i is set to 1 (step S21), and N is set to i (step S22).

  As shown in FIG. 7, the moving body detection circuit 142 determines to which direction the subject moves (moving direction of the subject) based on the center of the subject of the (n−1) th frame and the center of the subject of the nth frame. Prediction is made (step S23).

  The moving object detection circuit 142 calculates the moving speed of the subject based on the calculated moving distance of the subject and the time of one frame (step S11).

  Based on the movement direction predicted in step S23 and the movement speed predicted in step S11, the moving object detection circuit 142 indicates where the subject is moving in the image data of the (n + 1) th frame (predicted movement distance of the subject). Is calculated (step S24).

  N is set to i + 1 (step S25), and it is determined whether N is N, that is, whether or not the processes of steps S23, S11, and S24 have been performed on all N subjects detected in step S20 (step S26). ).

  If the processes in steps S23, S11, and S24 have not been performed on all N subjects detected in step S20 (NO in step S26), the process returns to step S23, and steps S23, S11, and S24 are performed again. Do.

  When the processes in steps S23, S11, and S24 have been performed on all N subjects detected in step S30 (YES in step S26), the prediction calculated in step S25 for N subjects. A subject having the maximum moving distance is selected as a target subject (step S27). The reason for calculating the predicted moving distance in this step S27 is to exclude subjects that are out of the shooting range of the image of the (n + 1) th frame due to fast progress. FIG. 8 shows the predicted movement distance of each subject when two subjects are detected in step S20. In FIG. 8, the subject 2 is selected as the target subject because the predicted travel distance of the subject 2 is larger than the predicted travel distance of the subject 1 and the subject 2 remains in the shooting range.

  The moving object detection circuit 142 refers to the correspondence table of the moving speed and the frame rate recorded in the memory 112, and selects a through image display frame rate corresponding to the moving speed of the target subject selected in step S21 (step S21). S28). A correspondence table between the moving speed and the frame rate is prepared in advance in the memory 112 so that a frame rate corresponding to the moving speed can be selected. In this correspondence table, a frame rate of 30 to 60 frames / second is set according to the moving speed. The through image display frame rate selected in step S28 is output from the moving object detection circuit 142 to the CPU 110.

  The CPU 110 drives the image sensor 126 at a through image display frame rate output from the moving object detection circuit 142 and periodically captures an image signal from the image sensor 126. The captured image signal processing unit 134 generates a luminance / color difference signal from the inserted image signal, and the display circuit 154 converts the luminance / color difference signal into a signal format for display (for example, an NTSC color composite video signal) to generate an image. The data is output to the display device 156. Thereby, the image imaged with the image pick-up element 126 is displayed on the image display apparatus 156 in real time (step S13).

  As a result, optimal through image control is performed on a moving subject. Thereafter, photographing and recording processes are performed. Since the photographing and recording processing is the same as that of the first embodiment, description thereof is omitted.

  According to the present embodiment, by selecting the through image frame rate in accordance with the fastest moving subject, even if there are a plurality of moving subjects in the image data, the through speed is determined based on the moving speed of the subject. By switching the frame rate at the time of image acquisition, a through image can be displayed at an appropriate frame rate according to the moving speed of the subject.

  In this embodiment, in step S24, the predicted moving distance of the subject is obtained using the moving direction of the subject predicted in step S23 and the moving speed of the subject calculated in step S11. The method of obtaining the predicted movement distance is not limited to this, and the center position of the subject of the (n + 1) th frame is obtained from the center position of the subject of the (n-1) th frame and the center position of the subject of the nth frame, and the subject of the nth frame. The difference between the center position of the subject and the center position of the subject of the (n + 1) th frame may be used as the predicted movement distance.

  In the present embodiment, the moving distance of the subject in the direction perpendicular to the optical axis direction (x direction and y direction) is calculated in step S24. However, the optical axis direction (z direction) is added to the z direction and the y direction. In addition, the movement distance may be calculated in consideration of all the directions in the z direction. The moving distance in the z direction is calculated based on, for example, the amount of change in the size of the subject, the distance between the subjects, and the size of the subject (for example, about 20 to 30 cm in the case of a human face). can do. In this case, the relationship between the type and size of the subject may be stored in the memory 112 in advance.

  In this embodiment, in step S27, the subject detected in step S20 is selected from the N subjects as the target subject with the maximum predicted movement distance. However, the method of selecting the target subject is not limited to this. Not limited, the subject whose moving speed of the subject calculated in step S11 is the maximum may be set as the target subject, and the subject closest to the center of the image (see steps S20 to S52 in FIG. 17, details will be described later) may be set as the target subject. .

<Third Embodiment>
In the first embodiment and the second embodiment of the present invention, a through image is displayed at a frame rate corresponding to a moving subject, thereby displaying a smooth through image with good image quality. The method of displaying a through image with good image quality is not limited to this.

  The third embodiment of the present invention is a form in which continuous AF is performed on a moving subject in order to obtain a through image with higher image quality. The digital camera 12 according to the third embodiment of the present invention will be described below. In addition, about the part same as 1st Embodiment or 2nd Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  FIG. 9 is a block diagram showing a schematic electrical configuration inside the digital camera 12.

  As shown in FIG. 9, the digital camera 10 mainly includes a CPU 110, a memory 112, a timing generator (TG) 114, motor drivers 116 and 118, an operation unit 120, a zoom lens 122, a focus lens 124, an image sensor 126, and analog processing. Unit (CDS / AMP) 128, A / D converter 130, image input control unit 132, image signal processing unit 134, compression / decompression processing unit 136, AE / AF detection circuit 144, moving object detection circuit 146, media controller 150, display A circuit 154, a storage medium 152, an image display device 156, and the like are included.

  The AE / AF detection circuit 144 includes an AE detection circuit and an AF detection circuit. Since the AE detection circuit is the same as the AE / AF detection circuit 140, description thereof is omitted.

  The AF detection circuit includes a high-pass filter that passes only a high-frequency component of the G signal, an absolute value processing unit, an AF area detection unit that extracts a signal in a predetermined focus area (for example, the center of the screen), and an evaluation value in the AF area It is comprised from the integration part which integrates.

  The AF detection circuit normally calculates an evaluation value by fixing the distance measurement area at the center of the screen, and when the target area of continuous AF is input from the moving object detection circuit 146, the input target area is calculated. An evaluation value is calculated as a distance measurement area.

  During the AF control, the CPU 110 sequentially moves the focus lens 124 from a closest position to a lens position corresponding to infinity, and at the same time, a high-frequency component of the image signal based on the image signal of the AF area captured via the image sensor 126 for each lens position. Is obtained from the AF detection circuit, a lens position where the evaluation value reaches a peak is obtained, and contrast AF is performed to move the focus lens 124 to the lens position.

  In addition, the CPU 110 performs continuous AF in which AF processing by a contrast method is repeatedly performed during through image shooting.

  The moving object detection circuit 146 detects the movement of the subject, and determines the search direction, frame rate (AF processing interval), and target area for continuous AF. As shown in FIG. 10, the moving object detection circuit 146 receives the image when moving object is detected (the nth frame) and the image one frame before (n−1 frame) before the moving object is detected. The AS AF search direction and the target area are output, and the frame rate of the continuous AF is output using the correspondence table of the moving speed and the frame rate recorded in the memory 112 as parameters. The processing performed by the moving object detection circuit 146 will be described in detail later.

  Next, the operation of the digital camera 12 of the present embodiment configured as described above will be described.

  By setting the power / mode switch to the shooting position, the shooting mode is set and shooting is possible. Then, when the photographing mode is set, the lens group is extended and the photographing standby state is set.

  Under this photographing mode, the subject light that has passed through the lens group forms an image on the light receiving surface of the image sensor 126 through a diaphragm (not shown). The subject light that has passed through the lens group is received by each photodiode and converted into a signal charge corresponding to the amount of incident light.

  The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge on the basis of the drive pulse supplied from the timing generator (TG) 114, and an analog processing unit (CDS / AMP). 128.

  Analog R, G, and B signals output from the analog processing unit 128 are converted into digital R, G, and B signals by the A / D converter 130 and added to the image input control unit 132. The image input control unit 132 outputs digital R, G, and B signals output from the A / D converter 130 to the memory 112. As a result, the subject image captured by the image sensor 126 is captured as an image signal.

  The CPU 110 issues a command to the timing generator (TG) 114, drives the image sensor 126 at a predetermined frame rate, and periodically captures image signals from the image sensor 126. The image signal processing unit 134 generates a luminance / color difference signal from the captured image signal, and the display circuit 154 converts the luminance / color difference signal into a display signal format (for example, NTSC color composite video signal) to display an image. Output to the device 156. As a result, an image captured by the image sensor 126 is displayed on the image display device 156 as a through image.

  At the same time, digital R, G, and B signals generated based on the image acquired by the image sensor 126 are taken into the moving object detection circuit 146. The moving object detection circuit 146 outputs the continuous AF target area, frame rate, and search direction, and the CPU 110 performs continuous AF based on the result output from the moving object detection circuit 146. The process flow will be described with reference to FIG.

  The moving object detection circuit 146 acquires the image data of the nth frame and the image data one frame before (n−1th frame) of the nth frame image, and the feature point of the subject (for example, the subject's feature) Edge, etc.) are detected (see FIGS. 12A and 12B), and these feature points are compared. If the positions of the same feature points do not match, it is determined that the subject is moving. Then, the moving object detection circuit 146 detects the moving subject by calculating the center of the subject of the (n−1) th frame and the center of the subject of the nth frame based on the detected feature points. (Step S10).

  Then, as shown in FIG. 12C, the moving object detection circuit 146 predicts where the subject moves (moving direction of the subject) based on these (step S23). In step S23, a moving direction in a direction (x direction and y direction) perpendicular to the optical axis direction is predicted.

  Further, as shown in FIG. 12C, the moving object detection circuit 146 calculates the moving distance of the subject based on the center of the subject of the (n-1) th frame and the center of the subject of the nth frame. The moving speed of the subject is calculated based on the moving distance of the subject and the time of one frame (step S11).

  Further, as shown in FIG. 12D, the moving object detection circuit 146 is one frame after the nth frame (n + 1) based on the center of the subject of the (n−1) th frame and the center of the subject of the nth frame. The direction (x direction and y direction) perpendicular to the optical axis direction of the (frame)), that is, the position of the subject on the image data is predicted (step S30).

  The predicted position of the subject predicted in step S30 is input from the moving object detection circuit 146 to the AE / AF detection circuit 144 as a target area for continuous AF. The AE / AF detection circuit 144 sets the input target area as the distance measurement area of the continuous AF of the (n + 1) th frame (step S31). Thereby, even when the subject is not in the center of the screen, the subject can be reliably focused.

  When the setting of the ranging area of the continuous AF (step S31) is completed, the moving object detection circuit 146 determines the search direction of the continuous AF based on the moving direction of the subject as follows (step S32). .

  First, the moving object detection circuit 146 predicts the moving direction of the subject in the optical axis direction (z direction) based on a change in the size of the subject. That is, as shown in FIG. 14A, the moving object detection circuit 146 determines that the subject is moving away when the subject of the nth frame is smaller than the subject of the (n−1) th frame, As shown in FIG. 14B, when the subject of the nth frame is larger than the subject of the (n−1) th frame, it is determined that the subject is moving in the approaching direction. Note that the size of the subject can be obtained by a method such as connecting the feature points calculated in step S10.

  Then, the moving object detection circuit 146 determines the same direction as the predicted movement direction of the subject in the z direction as the search direction of the continuous AF. That is, when the moving object detection circuit 146 determines that the subject is moving in the direction away from the subject, the moving object detection circuit 146 determines that the direction in which the subject leaves is the search direction of the continuous AF, and moves the subject toward the direction. If it is determined, the direction in which the subject approaches is determined as the search direction for continuous AF. The search direction of the continuous AF thus determined is input from the moving object detection circuit 146 to the CPU 110. As described above, since the search direction of continuous AF, that is, the direction in which the focus lens 124 is moved is determined in advance, the time required to move the focus lens 124 can be shortened and the focus can be immediately adjusted.

  The moving object detection circuit 146 refers to the correspondence table of the moving speed and the frame rate recorded in the memory 112, and selects the continuous AF frame rate corresponding to the moving speed of the target subject selected in Step S11 (Step S11). S33). A correspondence table between the moving speed and the frame rate is prepared in advance in the memory 112 so that a frame rate corresponding to the moving speed can be selected. The frame rate of the continuous AF selected in step S33 is output from the moving object detection circuit 146 to the CPU 110.

  The CPU 110 repeatedly performs the AF process using the contrast detection method at the frame rate of the continuous AF selected in step S33 (step S34). The AF / AF detection circuit 144 calculates an evaluation value for the continuous AF ranging area calculated in step S31 at the continuous AF frame rate selected in step S33. When the evaluation value is lower than the predetermined value, the CPU 110 moves the focus lens 124 in the search direction of the continuous AF calculated in step S32, and the continuous AF calculated in step S31. The lens position at which the evaluation value of the distance measurement area reaches a peak is obtained, and the focus lens 124 is moved to that lens position.

  As a result, the optimum AF process is performed on the moving subject. Thereafter, photographing and recording processes are performed. Since the photographing and recording processing is the same as that of the first embodiment, description thereof is omitted.

  According to the present embodiment, continuous AF can be performed following the speed and position of a moving subject. Therefore, even a subject that moves irregularly, such as a moving child, can be reliably focused. Since the movement of the subject is predicted, the focusing speed can be increased and the subject can be focused immediately.

  In this embodiment, the frame rate of the continuous AF is changed according to the movement of the subject. However, not only the frame rate of the continuous AF but also the frame rate of the through image display according to the movement of the subject. It may be changed.

  In the present embodiment, in step S30, the position of the subject is predicted based on the center of the subject in the (n-1) th frame and the center of the subject in the nth frame. However, the method for predicting the position of the subject is as follows. For example, the calculation may be performed based on the center of the subject of the nth frame and the moving speed of the subject calculated in step S11.

  In this embodiment, the movement direction in the direction (x direction and y direction) perpendicular to the optical axis direction is predicted in step S23, and the movement direction in the optical axis direction (z direction) is predicted in step S32. In step S33, all the movement directions of the x direction, the y direction, and the z direction may be predicted.

  In the present embodiment, in step S32, the movement direction of the subject in the optical axis direction (z direction) is predicted based on the change in the size of the subject. However, the movement direction of the subject in the optical axis direction (z direction) is predicted. The method of predicting is not limited to this. For example, if the vector connecting the center of the subject of the (n−1) th frame and the center of the subject of the nth frame is upward from the horizontal direction of the image, it is determined that the subject is separated, and the n−1th frame If the vector connecting the center of the subject and the center of the nth frame is downward from the horizontal direction of the image, it may be determined that the subject is moving in the approaching direction.

  In the present embodiment, the frame rate of the continuous AF corresponding to the moving speed of the subject calculated in step S11 is selected in step S33 with reference to the correspondence table of moving speed and frame rate. The method for selecting the frame rate of continuous AF is not limited to this. For example, instead of calculating the moving speed of the subject in step S11, the moving distance between the center of the subject of the (n-1) th frame and the center of the subject of the nth frame is calculated, and the subject stored in the memory 112 in advance is calculated. A frame rate of continuous AF corresponding to the calculated moving distance of the subject may be selected with reference to a correspondence table of moving distance and frame rate.

<Fourth embodiment>
In the third embodiment of the present invention, the case where only one moving subject is detected in the moving object detection circuit 146 has been described (step S30). However, only one moving subject is detected. Not exclusively.

  In the fourth embodiment of the present invention, when a plurality of moving subjects are detected by the moving object detection circuit 146, the subject whose moving distance of the subject is the maximum is processed by the same method as in the second embodiment. By using the subject, the optimum continuous AF is performed in accordance with the movement of the main subject. The digital camera 13 according to the fourth embodiment of the present invention will be described below. In addition, about the part same as 3rd Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  Since the electrical schematic configuration inside the digital camera 13 is the same as that of the digital camera 12, a description thereof will be omitted, and the operation of the digital camera 13 will be described.

  By setting the power / mode switch of the digital camera 13 to the photographing position, the photographing mode is set and the photographing standby state is set.

  Under this photographing mode, the subject light that has passed through the lens group forms an image on the light receiving surface of the image sensor 126 through a diaphragm (not shown). The subject light that has passed through the lens group is received by each photodiode and converted into a signal charge corresponding to the amount of incident light.

  The signal charge accumulated in each photodiode is sequentially read out as a voltage signal (image signal) corresponding to the signal charge on the basis of the drive pulse supplied from the timing generator (TG) 114, and an analog processing unit (CDS / AMP). 128.

  Analog R, G, and B signals output from the analog processing unit 128 are converted into digital R, G, and B signals by the A / D converter 130 and added to the image input control unit 132. The image input control unit 132 outputs digital R, G, and B signals output from the A / D converter 130 to the memory 112.

  Digital R, G, and B signals are input from the memory 112 to the moving object detection circuit 146, and the CPU 110 issues a command to the timing generator (TG) 114 and outputs a frame rate for displaying a through image output from the moving object detection circuit 146. To drive the image sensor 126.

  FIG. 15 shows the condition of the continuous AF (search direction, frame) based on digital R, G, B signals generated based on the image acquired by the image sensor 126 when there are a plurality of moving subjects. It is a flowchart which shows the flow of a process which determines a rate and an object area and performs continuous AF on this condition.

  The moving object detection circuit 146 detects N moving subjects based on the input digital R, G, and B signals (step S20). This means that N moving subjects have been detected as a result of the processing of step S10.

  Next, the subject having the maximum predicted movement distance is selected from the N subjects detected in step S20 (steps S21 to S26).

  First, i is set to 1 (step S21), and N is set to i (step S22).

  The moving object detection circuit 146 calculates the center of the subject of the (n-1) th frame and the center of the subject of the nth frame based on the feature points, and predicts the moving direction of the subject based on this (step S23).

  Further, the moving object detection circuit 146 calculates the moving speed of the subject based on the calculated moving distance of the subject and the time of one frame (step S41). Here, the relative movement speed based on a certain subject is calculated. Based on the center of the subject of the (n-1) th frame and the center of the subject of the nth frame, the moving speed of the subject in the direction perpendicular to the optical axis direction (x direction and y direction) is calculated. Further, based on the difference between the size of the subject of the (n-1) th frame and the size of the subject of the nth frame and the ratio between the tele end and the wide end, the moving speed of the subject in the optical axis direction (z direction) is set. calculate. Then, the moving speed of the subject is calculated based on the moving speed in the x and y directions and the moving speed in the z direction.

  The moving object detection circuit 146 determines the image of the (n + 1) th frame based on the movement direction predicted in step S17 and the movement speed of the subject in the direction perpendicular to the optical axis direction (x direction and y direction) predicted in step S41. In the data, it is calculated where the subject is moving (predicted movement distance of the subject) (step S24).

  N is set to i + 1 (step S25), and it is determined whether N is N, that is, whether or not the processes of steps S23, S11, and S24 have been performed on all N subjects detected in step S20 (step S26). ).

  If the processes in steps S23, S11, and S24 have not been performed on all N subjects detected in step S20 (NO in step S26), the process returns to step S23, and steps S23, S11, and S24 are performed again. Do.

  When the processes in steps S23, S11, and S24 have been performed on all N subjects detected in step S30 (YES in step S26), the prediction calculated in step S25 for N subjects. A subject having the maximum moving distance is selected as a target subject (step S27). The reason for calculating the predicted moving distance in this step S27 is to exclude subjects that are out of the shooting range due to fast advancement or the like. In the example shown in FIG. 16, the subject 2 is selected as the target subject.

  Next, the optimum continuous AF condition is obtained for the subject selected in step S27 (steps S30 to S33). Hereinafter, the case where the optimum continuous AF condition is obtained for the subject 2 shown in FIG. 16 will be described as an example.

  Based on the center of the subject of the nth frame and the moving speed of the subject calculated in step S11, the moving object detection circuit 146 detects the subject 2 on the image data after the first frame of the nth frame (n + 1th frame). The positions in the x direction and the y direction are predicted (step S30).

  The predicted position of the subject predicted in step S30 is input from the moving object detection circuit 146 to the AE / AF detection circuit 144 as a target area for continuous AF, and the AE / AF detection circuit 144 outputs the n + 1th frame of continuous AF. Is set as a distance measurement area (step S31). Thereby, even when the subject is not in the center of the screen, the subject can be reliably focused.

  The moving object detection circuit 146 determines the moving direction of the subject in the optical axis direction (z direction) based on the change in the size of the subject, and determines the direction to follow the subject as the search direction of the continuous AF (step S32). . The search direction of the continuous AF thus determined is input from the moving object detection circuit 146 to the CPU 110. As a result, the CPU 110 can determine in advance the direction in which the focus lens 124 is moved, so that the time required to move the focus lens 124 can be reduced.

  The moving object detection circuit 146 refers to the correspondence table of the moving speed and the frame rate recorded in the memory 112, and selects the continuous AF frame rate corresponding to the moving speed of the target subject selected in Step S11 (Step S11). S33). A correspondence table between the moving speed and the frame rate is prepared in advance in the memory 112 so that a frame rate corresponding to the moving speed can be selected. The frame rate of the continuous AF selected in step S33 is output from the moving object detection circuit 146 to the CPU 110.

  Thereby, the steps (steps S30 to S33) for obtaining the condition for continuous AF are completed. Then, the CPU 110 repeatedly performs the AF process using the contrast detection method at the continuous AF frame rate selected in step S33 (step S34).

  As a result, the optimum AF process is performed on the moving subject. Thereafter, photographing and recording processes are performed. Since the photographing and recording processing is the same as that of the first embodiment, description thereof is omitted.

  According to the present embodiment, by performing AF processing in accordance with the fastest moving subject, even when there are a plurality of moving subjects in the image data, the main subject can be reliably focused.

<Fifth embodiment>
According to the fourth embodiment of the present invention, when a plurality of moving subjects are detected by the moving object detection circuit 146, the subject whose maximum movement distance is the target subject, the movement of the main subject However, the method for selecting the main subject is not limited to this.

  In the fifth embodiment of the present invention, when a plurality of moving subjects are detected by the moving object detection circuit 146, the subject closest to the center of the screen is set as the subject subject, so that the motion of the main subject can be adjusted. In this mode, optimum continuous AF is performed. The digital camera 14 according to the fifth embodiment of the present invention will be described below. In addition, about the part same as 3rd Embodiment or 4th Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  Since the electrical schematic configuration inside the digital camera 14 is the same as that of the digital camera 12, a description thereof will be omitted, and the operation of the digital camera 14 will be described.

  FIG. 17 shows the condition of continuous AF (search direction, frame based on digital R, G, and B signals generated based on the image acquired by the image sensor 126 when there are a plurality of moving subjects. It is a flowchart which shows the flow of a process which determines a rate and an object area and performs continuous AF on this condition.

  The moving object detection circuit 146 detects N moving subjects based on the input digital R, G, and B signals (step S20). This means that N moving subjects have been detected as a result of the processing of step S10.

  Next, the subject having the maximum predicted movement distance is selected from the N subjects detected in step S20 (steps S21 to S26).

  First, i is set to 1 (step S21), and N is set to i (step S22).

  The moving object detection circuit 146 calculates the center of the subject of the (n-1) th frame and the center of the subject of the nth frame based on the feature points, and predicts the moving direction of the subject based on this (step S23).

  Further, the moving object detection circuit 146 calculates the moving speed of the subject based on the calculated moving distance of the subject and the time of one frame (step S11).

  Based on the center of the subject of the nth frame and the moving speed of the subject calculated in step S11, the moving object detection circuit 146 determines the x of the subject on the image data one frame after the nth frame (n + 1th frame). The position in the direction and the y direction is predicted (step S30), and the distance from the center of the image data to the subject position predicted in step S30 is calculated (step S51).

  N is set to i + 1 (step S25), and it is determined whether N is N, that is, whether or not the processes of steps S23, S11, and S24 have been performed on all N subjects detected in step S20 (step S26). ).

  If the processes in steps S23, S11, S30, and S51 have not been performed on all N subjects detected in step S20 (NO in step S26), the process returns to step S23, and steps S23, S11, S30 are performed. And S51 are performed again.

  When the processes in steps S23, S11, S30, and S51 are performed on all N subjects detected in step S30 (YES in step S26), the subject from the center of the image data calculated in step S51. The subject having the shortest distance to the position of the subject, that is, the subject closest to the center of the screen is selected as the subject subject (step S52).

  In the example shown in FIG. 18, in the current frame (the nth frame), since the subject 2 is a subject close to the center of the screen, the subject 2 is selected as the target subject. On the other hand, since the subject 1 is close to the center of the screen in the frame (n + 1 frame) after the current frame, the subject 1 is set as the target subject.

  The moving body detection circuit 146 inputs the predicted position of the subject selected in step S52 and predicted in step S30 to the AE / AF detection circuit 144. The AE / AF detection circuit 144 sets the input predicted position as the distance measurement area of the continuous AF of the (n + 1) th frame (step S31). Thereby, it is possible to reliably focus on the subject closest to the center of the screen.

  The moving object detection circuit 146 determines the moving direction of the subject in the optical axis direction (z direction) based on the change in the size of the subject, and determines the direction to follow the subject as the search direction of the continuous AF (step S32). . The search direction of the continuous AF thus determined is input from the moving object detection circuit 146 to the CPU 110.

  The moving object detection circuit 146 refers to the correspondence table of the moving speed and the frame rate recorded in the memory 112, and selects the continuous AF frame rate corresponding to the moving speed of the target subject selected in Step S11 (Step S11). S33). A correspondence table between the moving speed and the frame rate is prepared in advance in the memory 112 so that a frame rate corresponding to the moving speed can be selected. The frame rate of the continuous AF selected in step S33 is output from the moving object detection circuit 146 to the CPU 110.

  Thereby, the steps (steps S30 to S33) for obtaining the condition for continuous AF are completed. Then, the CPU 110 repeatedly performs the AF process using the contrast detection method at the continuous AF frame rate selected in step S33 (step S34).

  As a result, the optimum AF process is performed on the moving subject. Thereafter, photographing and recording processes are performed. Since the photographing and recording processing is the same as that of the first embodiment, description thereof is omitted.

  According to the present embodiment, by performing AF processing on the subject closest to the center of the screen, even when there are a plurality of moving subjects in the image data, the main subject can be reliably focused. .

  In the present embodiment, the subject closest to the center of the screen is selected as the target subject (step S52), and AF processing is performed on the subject according to the movement of the selected subject. A method of selecting a desired subject when a plurality of subjects are detected is a form in which a through image is displayed at an optimal frame rate corresponding to the movement of the subject (for example, the first embodiment or the second embodiment). You may apply.

  Note that the application of the present invention is not limited to a digital camera. The same applies to an imaging device such as a mobile phone with a camera or a video camera, or an imaging device such as a PC that processes an image captured by the imaging device. Can be applied to. It can also be provided as a program applied to a device such as a PC.

1 is a block diagram showing an electrical configuration of a digital camera 10 to which the present invention is applied. It is a figure which shows the function of the moving body detection circuit 142 of the said digital camera 10. FIG. 6 is a flowchart showing a flow of through image display processing of the digital camera 10. It is a figure for demonstrating the processing content of the said digital camera. It is a figure for demonstrating the processing content of the said digital camera. It is a flowchart which shows the flow of a process of the through image display of the digital camera 11 to which this invention was applied. It is a figure for demonstrating the processing content of the said digital camera. It is a figure for demonstrating the processing content of the said digital camera. 1 is a block diagram showing an electrical configuration of a digital camera 10 to which the present invention is applied. It is a figure which shows the function of the moving body detection circuit 146 of the said digital camera 11. FIG. It is a flowchart which shows the flow of AF process of the said digital camera. It is a figure for demonstrating the content of the process of the said digital camera. It is a figure for demonstrating the content of the process of the said digital camera. It is a figure for demonstrating the content of the process of the said digital camera. 3 is a flowchart showing a flow of AF processing of the digital camera 13; It is a figure for demonstrating the content of the process of the said digital camera. 4 is a flowchart showing a flow of through image display processing of the digital camera. It is a figure for demonstrating the content of the process of the said digital camera.

Explanation of symbols

10, 11, 12, 13, 14: Digital camera, 110: CPU, 112: Memory, 114: Timing generator (TG), 116, 118: Motor driver, 120: Operation unit, 122: Zoom lens, 124: Focus lens 126: Image sensor, 128: Analog processing unit (CDS / AMP), 130: A / D converter, 132: Image input control unit, 134: Image signal processing unit, 136: Compression / decompression processing, 140, 144: AE / AF detection circuit, 142, 146: moving object detection circuit, 150: media controller, 152 storage media, 154: display circuit, 156: image display device

Claims (10)

Imaging means for continuously imaging a subject at a predetermined frame rate;
Display means for continuously displaying images of the subject imaged by the imaging means;
Subject detection means for detecting moving subjects from at least two images before and after the image taken by the imaging means;
A calculating means for calculating a moving speed or moving distance of a moving subject detected by the subject detecting means, and when a plurality of moving subjects are detected, a calculating means for calculating a maximum moving speed or moving distance;
Frame rate control means for controlling the frame rate based on the moving speed or moving distance of the moving subject calculated by the calculating means;
An imaging apparatus comprising: An imaging unit that includes an imaging optical system and an imaging device that captures a subject image formed by the imaging optical system, and that continuously images the subject at a predetermined frame rate;
Subject detection means for detecting moving subjects from at least two images before and after the image taken by the imaging means;
Calculating means for calculating the moving speed or moving distance of the moving subject detected by the subject detecting means;
Automatic focusing control means for continuously performing focusing control for moving a photographing optical system and maintaining a focusing state for a moving subject detected by the subject detecting means, the subject calculated by the calculating means Automatic focusing control means for controlling the continuous focusing control interval based on the moving speed or moving distance of
A target subject setting unit configured to set a subject having the maximum moving speed or moving distance calculated by the calculating unit as a target subject when a plurality of subjects are detected by the subject detecting unit;
The automatic focusing control unit performs focusing control on the target subject set by the target subject setting unit when a plurality of subjects are detected by the subject detection unit. . An imaging unit that includes an imaging optical system and an imaging device that captures a subject image formed by the imaging optical system, and that continuously images the subject at a predetermined frame rate;
Subject detection means for detecting moving subjects from at least two images before and after the image taken by the imaging means;
A moving direction detecting means for detecting whether the moving subject detected by the subject detecting means is moving in a direction approaching the photographing position or moving away from the photographing position;
An automatic focusing control means for continuously performing focusing control for moving the photographing optical system and maintaining a focused state for a moving subject detected by the subject detecting means, and is detected by the moving direction detecting means. Automatic focusing control means for determining a search direction of the photographing optical system based on a moving direction of the subject,
Calculating means for calculating the moving speed or moving distance of the moving subject detected by the subject detecting means;
A target subject setting unit configured to set a subject having the maximum moving speed or moving distance calculated by the calculating unit as a target subject when a plurality of subjects are detected by the subject detecting unit;
The movement direction detection means is configured to detect whether the target subject set by the target subject setting means is moving in a direction approaching the shooting position when the subject detection means detects a plurality of subjects, or from the shooting position. Detect if you are moving away,
The automatic focusing control unit performs focusing control on the target subject set by the target subject setting unit when a plurality of subjects are detected by the subject detection unit, and the moving direction detection unit An image pickup apparatus, wherein a search direction of the photographing optical system is determined based on a moving direction of the target subject detected by the step. Imaging means for continuously imaging a subject at a predetermined frame rate;
Subject detection means for detecting moving subjects from at least two images before and after the image taken by the imaging means;
Calculating means for calculating the moving speed or moving distance of the moving subject detected by the subject detecting means;
A target subject setting means for setting a subject having the maximum moving speed or moving distance calculated by the calculating means as a target subject when a plurality of subjects are detected by the subject detecting means;
Subject position prediction means for predicting the position of the moving subject in the next image to be picked up based on the position of the moving subject detected by the subject detection means, wherein a plurality of subjects are detected by the subject detection means. In this case, subject position predicting means for predicting the position of the target subject set by the target subject setting means,
Automatic focus control means for performing focus control on a predetermined area including the position of the subject predicted by the subject position prediction means, and when a plurality of subjects are detected by the subject detection means, Automatic focus control means for performing focus control on a predetermined area including the position of the target subject predicted by the subject position prediction means;
An imaging apparatus comprising: Comprising feature point extraction means for extracting feature points of the image captured by the imaging means;
The subject detection means detects the center position of the subject based on the feature points extracted by the feature point extraction means;
5. The imaging apparatus according to claim 1 , wherein the calculating unit calculates a moving speed or a moving distance of the subject based on a center position of the subject detected by the subject detecting unit. 6. . An imaging step of continuously imaging a subject at a predetermined frame rate;
A display step of continuously displaying images of the subject imaged in the imaging step ;
A subject detection step of detecting respective moving objects from at least the front and rear of the two images captured by the imaging step,
A calculating step of calculating a moving speed or moving distance of the moving subject detected by the subject detection step, when the moving object is more detected, and calculating a maximum movement speed or movement distance,
A frame rate control step for controlling the frame rate based on the moving speed or moving distance of the moving subject calculated in the calculating step ;
An imaging method comprising: An imaging step of continuously imaging a subject at a predetermined frame rate via an imaging element that images a subject image formed by the imaging optical system;
A subject detection step of detecting respective moving objects from at least the front and rear of the two images captured by the imaging step,
A calculating step for calculating the moving speed or moving distance of the moving subject detected in the subject detecting step ;
An automatic focusing control step for continuously performing focusing control for moving the photographing optical system and maintaining a focusing state for the moving subject detected in the subject detecting step, wherein the subject calculated in the calculating step An automatic focusing control step for controlling the continuous focusing control interval based on the moving speed or moving distance of
When a plurality of subjects are detected in the subject detection step, a target subject setting step of setting a subject having the maximum moving speed or moving distance calculated in the calculating step as a target subject,
The automatic focusing control step performs focusing control on the target subject set in the target subject setting step when a plurality of subjects are detected in the subject detection step. Control method. An imaging step of continuously imaging a subject at a predetermined frame rate via an imaging element that images a subject image formed by the imaging optical system;
A subject detection step of detecting respective moving objects from at least the front and rear of the two images captured by the imaging step,
A moving direction detecting step for detecting whether the moving subject detected in the subject detecting step is moving in a direction approaching the shooting position or moving away from the shooting position;
An automatic focusing control step for continuously performing focusing control for moving the photographing optical system and maintaining a focusing state for the moving subject detected in the subject detecting step, and is detected in the moving direction detecting step. An automatic focusing control step for determining a search direction of the photographing optical system based on a moving direction of the subject,
A calculating step for calculating the moving speed or moving distance of the moving subject detected in the subject detecting step;
A target subject setting step for setting a subject having the maximum moving speed or moving distance calculated in the calculating step as a target subject when a plurality of subjects are detected in the subject detecting step;
In the moving direction detecting step, when a plurality of subjects are detected in the subject detecting step, the target subject set in the target subject setting step is moving in a direction approaching the photographing position, or from the photographing position. Detect if you are moving away,
In the automatic focusing control step, when a plurality of subjects are detected in the subject detection step, focusing control is performed on the target subject set in the target subject setting step, and the moving direction detection step A focus control method, comprising: determining a search direction of the photographing optical system based on a moving direction of the target subject detected in step (1). An imaging step of continuously imaging a subject at a predetermined frame rate;
A subject detection step of detecting respective moving objects from at least the front and rear of the two images captured by the imaging step,
A calculating step for calculating the moving speed or moving distance of the moving subject detected in the subject detecting step;
A target subject setting step for setting a subject having the maximum moving speed or moving distance calculated in the calculating step as a target subject when a plurality of subjects are detected in the subject detecting step;
Based on the position of a moving subject said detected by the subject detection step, a next object position prediction step of predicting the position of a moving object in an image to be captured, a plurality of subjects are detected by the subject detection step A subject position predicting step for predicting a position of the target subject set in the target subject setting step;
An automatic focusing control step for performing focusing control on a predetermined area including the position of the subject predicted in the subject position prediction step, and when a plurality of subjects are detected in the subject detection step, An automatic focus control step for performing focus control on a predetermined region including the position of the target subject predicted in the subject position prediction step ;
A focusing control method comprising: A program that causes an arithmetic device to execute the imaging method according to claim 6 or the focus control method according to any one of claims 7 to 9 .

Images