JP2013003471A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically set the best action for focus detection in order to have the focus track the movement of a subject to be photographed and to automatically create the best action for focus detection according to a photographing environment in which the photographer often photographs.SOLUTION: The imaging apparatus includes: focus detection means for detecting a focus state of an imaging optical system; feature quantity extraction means (S15) for extracting the feature quantity in the movement of a subject from a consecutively photographed image; action varying means (S18) for varying the action of the focus detection means based on the featured quantity of the movement of the subject extracted by the feature quantity extraction means.

Description

  The present invention relates to an improvement of an imaging apparatus suitable for a camera or the like that detects a focus state.

  2. Description of the Related Art Conventionally, there is a camera having a shooting mode that automatically sets an optimal shooting operation according to a shooting environment. This type of camera is suitable for sports modes suitable for shooting moving subjects, portrait shooting modes suitable for shooting people, landscape shooting modes suitable for landscape shooting, and night scene shooting. There is a night view shooting mode. As long as the photographer selects a shooting mode suitable for the shooting environment, various preset values of a shutter speed value, an aperture value, and an exposure correction value that are set in advance are automatically set in a lump. In this way, a photographer with a camera having a photographing mode can set various setting values according to the photographing environment without requiring specialized knowledge.

  In recent years, subject detection and recognition technology for detecting a specific subject from a photographed image is being realized, and a technology applied to a camera has been proposed. In the camera disclosed in Patent Document 1, an optimum shooting mode is automatically selected from a plurality of shooting modes such as a sports mode and a portrait mode depending on conditions such as the presence / absence of face detection and subject movement. A camera to set has been proposed.

JP 2003-344891 A

  However, various setting values are employed as shooting environments that can be selected in the shooting mode so as to achieve generally suitable shooting operations. For this reason, the optimum shooting operation is not always performed in more specific shooting in the selected shooting mode. In particular, with respect to a shooting mode suitable for shooting a moving subject, there are movements that approach monotonously, movements that suddenly start, stop movement, and movements that move greatly up, down, left, and right. In other words, unless there is a shooting mode that is suitable for the shooting environment, it is not possible to realize an optimal shooting operation according to the movement of various subjects.

  Also, some photographers may only shoot in a specific shooting environment such as sports, and there is a need for more specific shooting in sport mode, and a mode of shooting operation that is optimal for soccer or swimming.

(Object of invention)
It is an object of the present invention to automatically set an optimum focus detection operation so as to follow the movement of a subject to be photographed, and to automatically perform an optimum focus detection operation in a photographing environment where a photographer often takes a picture. It is an object of the present invention to provide an imaging apparatus that can be generated.

  In order to achieve the above object, an imaging apparatus according to the present invention includes a focus detection unit that detects a focus state of an imaging optical system, a feature amount extraction unit that extracts a feature amount of a subject's movement from continuously shot images, And an operation change unit that changes the operation of the focus detection unit based on the feature amount of the movement of the subject extracted by the feature amount extraction unit.

  According to the present invention, it is possible to automatically set an optimum focus detection operation in order to make a subject follow the movement of a subject.

  Further, it is possible to automatically generate a focus detection operation that is optimal for a shooting environment in which a photographer often takes a picture.

It is a block diagram which shows the structural example of the camera which is an Example of this invention. It is a flowchart explaining the imaging | photography operation | movement of an Example. It is a flowchart explaining the feature-value extraction of an Example. It is a figure explaining a continuous picked-up image. It is a figure explaining the feature-value extracted in various imaging | photography environments. It is a figure explaining the variable plot in a principal component coordinate. It is a figure explaining a principal component score.

  The mode for carrying out the present invention is as described in the following examples.

  FIG. 1 is a block diagram of a camera which is an example of an imaging apparatus according to an embodiment of the present invention. In the figure, 101 is a photographic lens (corresponding to the imaging optical system of the present invention), 107 is a CCD or CMOS sensor that receives subject light that has passed through the photographic lens 101, photoelectrically converts it, and outputs image signal data. It is an element. The image signal output from the image sensor 107 is input to the analog signal processing circuit 109. The A / D converter 110 converts the analog signal into a digital signal.

  Reference numeral 102 denotes a main mirror having a semi-transmissive portion, and FIG. 1 shows a state in which it is inserted into a photographing light beam (mirror down). The main mirror 102 is retracted out of the photographing light beam at the time of photographing, and is obliquely installed in the photographing optical path when the focus is detected. In addition, the main mirror 102 guides a part of the light beam that has passed through the photographing lens 101 to a finder optical system that includes the focus plate 103, the pentaprism 104, and the eyepiece lens 105 while being inclined in the photographing optical path.

  Reference numeral 106 denotes a submirror that can be folded and expanded with respect to the main mirror 102 in synchronization with the operation of the main mirror 102. A part of the light beam that has passed through the semi-transmissive portion of the main mirror 102 is reflected downward by the sub-mirror 106 and enters the phase difference type focus detection unit 108 to detect the focus state of the photographing lens 101.

  A system controller 112 includes a CPU that controls the entire camera, a RAM that is a storage unit, and the like, and appropriately controls the operation of each unit described below.

  Reference numeral 113 denotes a communication circuit that is connected to the system controller 112 and communicates with the photographing lens 101, a lens driving mechanism that drives the lens to perform focus adjustment, and a lens driving unit that includes the driving circuit. Reference numeral 114 denotes a mirror driving unit that is connected to the system controller 112 and drives the main mirror 102 to the outside of the photographing light beam. A sensor control unit 115 is connected to the system controller 112 and controls the focus detection unit 108. Reference numeral 116 denotes an image sensor driving unit that is connected to the system controller 112 and drives the image sensor 107. The focus detection unit 108, the sensor control unit 115, and the system controller 112 correspond to the focus detection unit of the present invention.

  A digital signal processing circuit 111 is connected to the system controller 112 and performs image processing such as shading correction and gamma correction on the signal converted into a digital signal by the A / D converter 110.

  Reference numeral 117 denotes a frame memory connected to the digital signal processing circuit 111 and capable of storing image signal data for a plurality of frames captured by the image sensor 107. The A / D converted signal is temporarily stored in the buffer memory 117. The digital signal processing circuit 111 reads the image signal data stored in the buffer memory 117 and performs the above-described processes, and the processed image signal data is stored in the buffer memory 117 again.

  Reference numeral 118 denotes a recording / reproducing signal processing circuit for recording on an external recording medium 119 such as a memory card. The recording / reproducing signal processing circuit 118 is connected to the digital signal processing circuit 111. When the image signal data subjected to various processes by the digital signal processing circuit 111 is temporarily stored in the buffer memory 117 and then recorded on the external recording medium 119, the image signal data is compressed by, for example, the JPEG method. Compression is performed. On the other hand, when the image signal data is read from the external recording medium 119, the recording / playback signal processing circuit 118 performs an expansion process of the image signal data. The recording / reproduction signal processing circuit 118 also includes an interface for performing data communication with the external recording medium 119.

  Reference numeral 121 denotes a display unit for displaying a captured image, which is also used when reproducing and displaying image signal data recorded on the external recording medium 119. When displaying an image on the display unit 121, the image signal data stored in the buffer memory 117 is read, and the digital image signal data is converted into an analog video signal by the D / A converter 120. Then, an image is displayed on the display unit 121 using the analog video signal.

  There are two modes for displaying the image captured by the image sensor 107 on the display unit 121. One is a display form when the release operation is not performed, and is a display form called a live view that sequentially updates and displays images repeatedly picked up by the image sensor 107. The other is a display form called a freeze image that displays an image captured by the image sensor 107 for a predetermined time after the release operation of the camera.

  Reference numeral 122 denotes an operation member connected to the system controller 112 for operating the camera, such as a power switch for turning on / off the camera, a release button, and a setting button for selecting a shooting mode such as a human shooting mode. Is an operation unit provided. When these switches and buttons are operated, a signal corresponding to the operation is input to the system controller 112. The release button includes a switch SW1 that is turned on by a first stroke operation (half-press operation) of the release button operated by the photographer, and a switch SW2 that is turned on by a second stroke operation (full-press operation) of the release button. Is connected. In addition, as the focus adjustment mode, there are an arbitrary selection mode, an area enlargement mode, and an automatic selection mode, which can be set as appropriate by the photographer's operation.

  Hereinafter, with reference to FIG. 2, the photographing operation according to the embodiment of the present invention will be described.

  In step S11, the operation control information is read from the RAM of the system controller 112 and set. The focus detection is controlled by the system controller 112 based on the operation control information read and set.

  In step S12, it is determined whether the release button switch SW1 of the operation unit 122 is on. If the switch SW1 is on, the process proceeds to step S13, and if the switch SW1 is off, step S12 is repeated.

  In step S13, it is determined whether the release button switch SW2 of the operation unit 122 is on. If the switch SW2 is on, the process proceeds to step S14, and if the switch SW2 is off, the process proceeds to step S15.

  In step S <b> 14, the system controller 112 performs a photographing operation and records an image on the external recording medium 119 via the digital signal processing circuit 111 and the recording / reproducing signal processing 118. The operation control information for focus detection during this photographing operation is set in step S11.

  In step S15, the feature quantity of the movement of the subject is extracted from the continuously shot images. The system controller 112 that executes step S15 corresponds to the feature amount extraction unit of the present invention. Details will be described later.

  In step S16, it is determined whether or not the feature quantity of the subject movement stored for each continuous photographed image is equal to or greater than a predetermined value. If the feature quantity of the subject movement is stored above a predetermined value, the process proceeds to step S17, and if not above the predetermined value, the process proceeds to step S18.

  In step S17, principal component analysis is performed by general multivariate analysis software based on the stored feature values for each continuously shot image. Details will be described later.

  In step S18, the operation control information corresponding to the feature amount extracted in step S15 is updated by storing it in the RAM of the system controller 112. From the next imaging, the focus detection operation is changed based on the operation control information updated in step S18. That is, an operation change is performed.

  Hereinafter, the feature amount extraction in step S15 of FIG. 2 will be described in detail with reference to FIG.

  In step S 21, the start image of the continuously shot image recorded on the external recording medium 119 is read into the RAM of the system controller 112. At this time, the image may be compressed to an image size that can be easily processed in the subsequent processing. Further, it may be converted into a color space such as YUV or binarized.

  In step S22, a template image is created from the start image read in step S21 in order to detect the movement of the subject. The created template is stored in the RAM of the system controller 112. The template is obtained by cutting out an image around the selected AF frame 201 shown in FIG. 4A in which the AF frame 201 is superimposed on the subject. However, if there is a characteristic shape near the selected AF frame 201, this shape may be used as a template. Further, a face image detected by face recognition may be used as a template.

  In step S <b> 23, it is determined whether or not all images of the continuously captured images that have been captured have been read into the RAM of the system controller 112. If all the images have not been read, the process proceeds to step S24, and if all the images have been read, the process proceeds to step S27.

  In step S24, the next image is read into the RAM of the system controller 112. In step S25, template matching between the template created in step S22 and the image read in step S24 is performed.

  In step S26, the feature amount of the subject motion is extracted. Here, in order to explain the extraction of the feature quantity of the movement of the subject, the following four feature quantities are given as examples. However, these four feature values are not all for carrying out the present invention, but may be those that represent the movement of the subject.

  The first time is the time when the subject is removed from the AF frame 201 due to an obstacle entering in front of the subject or passing through. This is a feature amount representing the ease of removing the subject. With reference to FIGS. 4A to 4D, which show four consecutively photographed images, the feature amount of the ease of removing the subject will be described. In the captured images of FIGS. 4A, 4B, and 4D, the subject is captured by the AF frame 201, so the time taken for removal is zero. In the photographed image of FIG. 4C, no obstacle enters the front of the subject, but since the subject is removed from the AF frame 201, the time between the frames from FIGS. 4C to 4D is excluded. . Actually, a subject area (subject) is recognized by image recognition technology, and it is determined whether the position detected by template matching is inside or outside the subject area. As described above, the feature amount representing the ease of removing the subject is extracted by obtaining the time when the subject is removed for each captured image.

  The second is whether or not the currently selected AF frame 201 is immediately focused when an obstacle enters in front of the subject or when the subject is removed from the AF frame 201 due to a gap or the like. This is a feature amount indicating that the subject to be focused is immediately switched to the subject targeted by the AF frame 201. With reference to FIGS. 4 (a) to 4 (d), which show four consecutively photographed images, feature quantities for instantly switching the subject will be described. Although not shown in the background in FIG. 4C, FIGS. 4A to 4D aim at the same subject. For this reason, it is not necessary to switch the subject immediately in the continuously shot images of FIGS. Accordingly, the feature amount is set to zero, and it may be set to 1 if it is necessary to switch immediately. Actually, a subject area (subject) is recognized by image recognition technology, and it is determined whether the position detected by template matching is inside or outside the subject area. As described above, the feature amount indicating whether to switch the subject immediately for each captured image is extracted.

  The third is the amount of change in speed that represents how much the speed of movement of the subject has changed due to sudden movements, sudden stops, and the like. The amount of change in the moving speed of the subject will be described with reference to FIGS. 4 (a) to 4 (d) showing four consecutively shot images. 4A to 4D represent a subject approaching at a constant speed. Therefore, the amount of change in speed is zero. In order to obtain the amount of change in speed, the image plane moving speed obtained from the focus detection result may be used, or the focal length of the lens and the size of the image sensor that are stored in the Exif of the image are read and Gaussian results are obtained. A value obtained by calculating the distance to the subject by the image formula and dividing the distance difference between the previous and next images by the time between frames may be used.

  The fourth is the amount of movement when the subject moves up, down, left and right. The amount of movement when the subject moves up, down, left, and right will be described with reference to FIGS. In step S22, a movement amount Db detected from template matching between the template created from FIG. 4A and FIG. 4B read in step S24 is detected. Similarly, the movement amounts Dc and Dd are also detected by template matching between the template created from FIG. 4A in step S22 and FIG. 4C read in step S24, or (d).

  In step S27, a maximum value is obtained from each of the four feature amounts extracted for each captured image in step S26. A value obtained by normalizing the maximum value with the number of continuously shot images is set as a feature amount of the continuously shot images.

  In step S28, the feature quantity of the continuously shot image determined in step S27 is stored in the RAM of the system controller 112 for each continuously shot image.

  Hereinafter, the principal component analysis in step S17 of FIG. 2 will be described in detail. First, principal component analysis is a computer that obtains a principal component, which is a characteristic index, from data composed of a plurality of parameters (for example, ease of removal, instantaneous switching, speed change, vertical and horizontal movement). That is. This principal component is a composite variable of a plurality of parameters and is a comprehensive index that makes it easy to grasp the relationship between parameters included in data. The main component associates the feature amount of the subject movement with the motion control information for focus detection.

  FIG. 5 shows feature amounts of continuously shot images taken in various shooting environments such as soccer and swimming using the extraction method described in the feature amount extraction in step S15 of FIG. FIG. 6 shows a variable plot in which the principal component analysis is performed based on the feature amount shown in FIG. 5 and the obtained first principal component is on the vertical axis and the second principal component is on the horizontal axis. In this case, the principal component analysis is to convert the feature amount data into the position of the principal component coordinate system where the center of gravity of the feature amount data is the origin. From FIG. 6, the positive direction of the first principal component is highly evaluated for vertical and horizontal movement and speed change, and the negative direction is highly evaluated for immediate switching. The positive direction of the second principal component is highly evaluated for speed change, and the negative direction is highly evaluated for ease of removal.

  Principal component analysis is performed based on the feature values shown in FIG. 5, and the principal component scores are shown in FIG. 7, with the obtained first principal component on the vertical axis and the second principal component on the horizontal axis. In comparison with the variable plot in FIG. 6, it can be said that the group A swimming (butterfly) and swimming (breaststroke) surrounded by a dotted line are easy to remove. Group E's gymnastics (floor) can be said to be a shooting environment with a large change in speed and vertical and horizontal movement. Regarding the other groups, the characteristics of each photographing environment can be understood by comparing with the variable plot of FIG.

  As described above, the motion control information is updated depending on where the feature amount of the continuously captured image captured this time on the principal component score of FIG. 7 is located. The operation control information corresponding to “easy to remove” and “switch instantly” is the time from when the subject is removed to when it is captured by the AF frame 201 again. In a shooting environment that is easy to remove, the time until the AF frame 201 is captured again is lengthened, and in a shooting environment that switches immediately, the shooting time is shortened. Also, the motion control information corresponding to the change in speed is a defocus amount deviation amount that can be allowed from a prediction curve obtained from a past focus detection history. In a shooting environment where the speed change is large, even if the defocus amount deviation allowable from the prediction curve is large, it is not determined as an error, but is determined as a focus detection result obtained by detecting the focus of the subject. Furthermore, the motion control information corresponding to the movement in the up / down / left / right directions is a deviation amount from the defocus amount expected from the past focus detection history. In a shooting environment in which the amount of deviation from the defocus amount predicted from the past focus detection history is large, it is easier to change to the AF frame capturing the subject than the currently selected AF frame.

  The motion control information stored in association with the principal component score may be determined for each divided region obtained by dividing each principal component at equal intervals, or for each group plotted on the principal component score as shown in FIG. You can decide.

  The above-described operation control information is stored in advance in association with the main component score of FIG. As a result, it is possible to set operation control information optimal for the shooting environment shot this time. In addition, since the feature amount of the continuously shot image taken by the photographer is stored, the principal component score based on the stored continuously shot image can be obtained (changed). For example, a photographer who specially shoots athletics in sports can determine the principal component score based on the feature quantities of continuously shot images taken in each shooting environment such as 100 m, long jump, and pole vault. This makes it possible to set finer operation control information in a shooting environment where the photographer often takes pictures.

  As described above, in this embodiment, by extracting the feature amount of the movement of the subject using the image recognition technology, the optimum focus detection operation is automatically performed in order to follow the movement of the subject to be photographed. An imaging apparatus that can be set can be provided.

  In addition, it is possible to provide an imaging apparatus capable of automatically generating a focus detection operation optimal for a shooting environment in which a photographer often takes a picture.

DESCRIPTION OF SYMBOLS 101 Shooting lens 108 Focus detection part 112 System controller 115 Sensor control part 122 Operation part 201 AF frame

Claims (3)

Focus detection means for detecting the focus state of the imaging optical system;
Feature amount extraction means for extracting feature amounts of the movement of the subject from continuously shot images;
An image pickup apparatus comprising: an operation changing unit that changes an operation of the focus detection unit based on a feature amount of a subject motion extracted by the feature amount extraction unit. Storage means for storing the feature quantity of the movement of the subject extracted by the feature quantity extraction means for each of the continuously shot images;
2. The imaging according to claim 1, wherein the operation changing unit reads operation control information corresponding to a feature amount of the movement of the subject stored in the storage unit, and changes the operation of the focus detection unit. apparatus.   The imaging apparatus according to claim 2, wherein the operation changing unit changes a main component that associates the feature amount of the subject motion stored in the storage unit with the operation control information.

Images