JP5602423B2 - Imaging apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a program, and in particular, appropriately identifies color scenes, brightness information, and the like included in a captured image to determine a captured scene, and an appropriate image according to the captured scene. The present invention relates to a technique for performing correction.

  2. Description of the Related Art Conventional imaging devices such as digital cameras and digital video cameras have been proposed in which shooting parameters and image quality correction are performed according to a scene to be shot (see Patent Document 1). In Patent Document 1, in order to perform appropriate image quality correction for each shooting scene, color information and luminance information included in an image are analyzed, and whether the scene is mainly a person of a subject or a scene that is mainly a landscape Etc. are automatically discriminated and image quality correction is performed. For example, if it is determined that the scene contains a blue sky, the noise removal level of the image area that appears to be a blue sky is increased, or if it is determined that the scene is a night scene, the exposure time is reduced by suppressing noise. Control to make it longer.

  In addition, as a product to which the above technology is applied, a scene determination result regarding a background such as a blue sky or a night view is combined with a scene determination result regarding a main subject such as whether the subject is close (macro) or the subject is moving. There are also products that use scene determination results. For example, even if it is determined that the scene is the same blue sky, the shooting parameters and image quality correction may be performed depending on whether a person image is detected, a person image is not detected, or a macro image is detected. Is to change.

JP 2009-116691 A

  In the conventional imaging apparatus, when the zoom operation is performed, the angle of view changes to the wide angle side or the telephoto side during the zoom operation, so that the color information and luminance information included in the image change greatly. Therefore, despite the fact that the position and orientation of the imaging device have not changed, the scene determination result changes frequently, and the shooting parameters and image quality correction frequently change, thereby generating a moving image that is difficult for the user to see. There is a risk that. Therefore, it is conceivable that the scene discrimination is temporarily stopped during the zoom operation and the scene discrimination is resumed after the zoom operation.

  However, when scene discrimination is paused, the user recognizes that a new person's video has appeared in the image or that the person in the image has fallen out due to the zoom operation. The change in the captured image is not reflected in the scene discrimination.

  In addition, it is desirable that the shooting scene determination result can be gradually changed within a range in which shooting parameters and image quality correction do not frequently change, rather than changing the shooting scene at once after the zoom operation.

  The present invention has been made in view of the above problems, and it is possible to eliminate the instability of scene discrimination during the zoom operation and realize more appropriate image adjustment by performing highly accurate scene discrimination. An object is to provide an imaging apparatus, a control method thereof, and a program.

In order to achieve the above object, the present invention provides an imaging device, zoom means for executing a zoom operation for changing the magnification of a subject image on the imaging device surface, and an image of a shooting scene from a video signal obtained from the imaging device. An imaging apparatus comprising: a scene determination unit that performs determination; and a control unit that controls at least one of a shooting parameter at the time of shooting and an image processing parameter for a shot image according to the scene determined by the scene determination unit. First detection means for detecting first information used for determining a background of a shooting scene from the video signal, and second information used for determining a subject of the shooting scene from the video signal. A second detection means for detecting; and a zoom detection means for detecting a zoom operation by the zoom means; When the zoom operation is not detected by the above, when the shooting scene is determined based on the latest detected first information and second information detected, and the zoom operation is detected by the zoom detection unit The photographing scene is discriminated based on first information detected before the zoom operation and second information detected during the zoom operation.

  According to the present invention, it is possible to eliminate the instability of scene determination during a zoom operation and to perform more appropriate image adjustment by performing highly accurate scene determination.

1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. 3 is a flowchart showing a flow of a scene determination operation executed by the imaging apparatus of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention.

  In FIG. 1, reference numeral 101 denotes a lens group in which a plurality of imaging lenses are combined. A diaphragm 102 is disposed behind the lens group 101. The light that has passed through the lens group 101 and the diaphragm 102 is formed as an optical image of a subject on an image forming surface of an image sensor 103 (imaging device) constituted by a CCD sensor or a CMOS sensor. The optical image of the subject imaged by the image sensor 103 is converted into an image signal and output.

  The image signal output from the image sensor 103 is sent to the signal processing unit 104 and the face detection unit 105. In the signal processing unit 104, the received image signal is subjected to predetermined image processing including white balance correction based on the image processing parameter. The video signal output from the signal processing unit 104 is output to the luminance detection unit 106 and the color detection unit 107. The signal processing unit 104 outputs a video signal to the display 108, a recording medium 109 such as a memory card or a DVD disk. On the display 108, a moving image is displayed by the video signal received from the signal processing unit 104. The video signal output to the recording medium 109 is recorded as a moving image file on a memory card or the like.

  The face detection unit 105 (face information detection means) performs face detection processing on the image signal received from the image sensor 103, detects a human face reflected in the frame image, and detects the detected face area as face information. To the scene determination unit 110.

  The luminance detection unit 106 (luminance information detection means) performs luminance detection processing on the video signal received from the signal processing unit 104, divides the frame image on the video into a plurality of regions, and obtains average luminance in each region. . Then, the luminance detection unit 106 uses the average luminance to obtain luminance information such as a luminance difference between the central portion of the frame image and its peripheral portion, a central luminance value, and the like. The luminance information detected by the luminance detection unit 106 is sent to the scene determination unit 110.

  The color detection unit 107 (color information detection unit) performs color detection processing on the video signal received from the signal processing unit 104 to detect color information such as the average saturation and the area of the high saturation region. The color information detected by the color detection unit 107 is sent to the scene determination unit 110.

  The zoom operation unit 111 is a part where the user performs a zoom operation. When the zoom operation is performed by the user, information regarding the zoom operation is sent from the zoom operation unit 111 to the camera microcomputer 112.

  The camera microcomputer 112 (control means) performs drive control such as charge accumulation and image reading in the image sensor 103 through the sensor driving unit 113. In addition, the camera microcomputer 112 performs drive control such as focusing and zooming of the lens group 101 through the lens control unit 114 (zoom means). Thereby, a zoom operation for changing the magnification of the subject image on the image sensor surface is executed.

  Further, the camera microcomputer 112 uses the luminance information of the subject obtained from the video signal received from the signal processing unit 104 to perform exposure control by performing drive control of the aperture 102 and control of the shutter speed by the aperture control unit 115. Do. Furthermore, the camera microcomputer 112 can correct camera shake during shooting by controlling the lens group 101 by the lens control unit 114 or by controlling the signal processing unit 104.

  In addition, the camera microcomputer 112 sends information regarding the zoom operation sent from the zoom operation unit 111 to the zoom detection unit 116, while sending the current zoom position, the position of the focus lens, and focus information to the distance detection unit 117.

  The zoom detection unit 116 (zoom detection means) detects the current zoom position, the presence / absence of a zoom operation, and the like from information regarding the zoom operation received from the camera microcomputer 112. The detected result is sent to the scene determination unit 110.

  The distance detection unit 117 (distance information detection means) detects distance information (second information) indicating the distance to the subject from the current zoom position, focus lens position, and focus information received from the camera microcomputer 112. . The detected distance information is sent to the scene determination unit 110.

  In the scene discriminating unit 110 (scene discriminating means), the signal processing unit 104 processed the information sent from the face detecting unit 105, the luminance detecting unit 106, the color detecting unit 107, and the distance detecting unit 117. The background of the shooting scene and the subject of the shooting scene are determined from the video signal. Information transmitted from the face detection unit 105, the luminance detection unit 106, the color detection unit 107, and the distance detection unit 117 is temporarily stored by the scene determination unit 110 and updated as needed. The video signal processed by the signal processing unit 104 is a video signal for each frame or a predetermined number of frames of the moving image output from the image sensor 103.

  Then, the scene determination unit 110 sends information for changing shooting parameters, image processing parameters, and the like at the time of shooting to the camera microcomputer 112 according to the determined shooting scene. The shooting parameters include, for example, a focus position, an aperture value, ISO sensitivity, and the like.

  The camera microcomputer 112 drives and controls the lens control unit 114, the aperture control unit 115, and the signal processing unit 104 based on information transmitted from the scene determination unit 110.

  Next, the scene determination operation by the scene determination unit 110 will be described.

  The scene discriminating unit 110 discriminates the background of the photographic scene using the luminance information detected by the luminance detecting unit 106 and the color information detected by the color detecting unit 107. Furthermore, using the face information detected by the face detection unit 105 and the distance information detected by the distance detection unit 117, a subject in the shooting scene is determined.

  First, the case of determining the background of the shooting scene will be described.

  The scene determination unit 110 analyzes the luminance information detected by the luminance detection unit 106 and the color information detected by the color detection unit 107, determines whether the area of the blue sky color region on the image is equal to or greater than a threshold, If the area of the blue sky area is equal to or greater than the threshold, it is determined that the background of the shooting scene is blue sky. Then, information for changing shooting parameters and image processing parameters corresponding to the blue sky scene is transmitted to the camera microcomputer 112. The camera microcomputer 112 raises the saturation of the blue sky area on the image based on information for changing the shooting parameters and image processing parameters corresponding to the blue sky scene, and reduces the noise suppression level of the blue sky area. Is sent to the signal processing unit 104. The signal processing unit 104 performs image processing on the image signal based on an instruction received from the camera microcomputer 112.

  The scene determination unit 110 analyzes the luminance information from the luminance detection unit 106 and the color information from the color detection unit 107, and determines that the luminance on the image satisfies a predetermined histogram distribution and dispersion condition. It is determined that the background of the shooting scene is a night view. For example, the brightness on the image at night is a state in which the low-luminance portion occupies most and the high-luminance portion occurs once. When the scene determination unit 110 determines that the background of the shooting scene is a night scene, the camera microcomputer 112 transmits information for changing shooting parameters, image processing parameters, and the like corresponding to the night scene. The camera microcomputer 112 instructs the signal processing unit 104 to apply a gain limit so that the white balance is not excessively drawn based on information for changing shooting parameters, image processing parameters, and the like corresponding to the night scene. send. The signal processing unit 104 performs image processing on the image signal based on an instruction received from the camera microcomputer 112.

  In addition, the scene determination unit 110 analyzes the luminance information from the luminance detection unit 106 and the color information from the color detection unit 107, determines whether both the average saturation area and the high saturation area on the image are equal to or greater than a threshold, If it is equal to or greater than the threshold value, it is determined that the shooting scene is a vivid scene. Then, information for changing the focus position and image processing parameters corresponding to the vivid scene is transmitted to the camera microcomputer 112. The camera microcomputer 112 weights the color difference signals based on information for changing shooting parameters and image processing parameters corresponding to vivid scenes, and increases the saturation and enhances the vividness. Is sent to the signal processing unit 104. The signal processing unit 104 performs image processing on the image signal based on an instruction received from the camera microcomputer 112.

  Next, a case where a subject in a shooting scene is determined will be described.

  The scene determination unit 110 analyzes the face information from the face detection unit 105, and determines that the subject in the shooting scene is a person when a face is detected from the image signal. Then, the determination result is transmitted to the camera microcomputer 112. The camera microcomputer 112 sends an instruction for increasing the noise suppression level in the skin color area to the signal processing unit 104 in addition to the change information such as the image processing parameter according to the background scene determination result described above. The signal processing unit 104 performs image processing on the image signal based on an instruction received from the camera microcomputer 112.

  Further, when the scene determination unit 110 analyzes the distance information from the distance detection unit 117 and determines that the distance to the subject is closer than the threshold, the scene determination unit 110 determines that the shooting scene is a macro scene in which the subject is close. To do. Then, the determination result is transmitted to the camera microcomputer 112. In addition to the change information such as the image processing parameter according to the background scene determination result described above, the camera microcomputer 112 gives an instruction to widen the shift width of the image subject to camera shake compensation (camera shake correction). Send to. The signal processing unit 104 performs image processing on the image signal based on an instruction received from the camera microcomputer 112.

  In the scene determination described above, it is preferable that the determination of the subject in the shooting scene has priority over the determination of the macro scene, and the determination of the macro scene is not performed when it is determined that the subject of the shooting scene is a person. However, the present invention is not limited to this.

  As described above, the scene determination unit 110 determines both the background of the scene and the subject (or macro scene), combines these determination results, and outputs one determination result to the camera microcomputer 112. That is, when the background of the shooting scene is determined to be a blue sky and it is determined that the subject is a person, the shooting scene is determined to be a blue sky + person. On the other hand, if it is determined that the background of the shooting scene is a blue sky, but it is determined that the subject is neither a person nor a macro scene, it is simply determined that the shooting scene is a blue sky.

  Next, the flow of the scene discrimination operation described above will be described with reference to the flowchart of FIG.

  FIG. 2 is a flowchart showing a flow of a scene determination operation executed by the imaging apparatus of FIG.

  In FIG. 2, when shooting is started, detection results are sent from the face detection unit 105, the luminance detection unit 106, the color detection unit 107, the zoom detection unit 116, and the distance detection unit 117 to the scene determination unit 110 ( Step S1). Next, the scene determination unit 110 determines whether or not the zoom operation is being performed based on the detection result from the zoom detection unit 116 (step S2). If it is determined that the zoom operation is being performed, the process proceeds to step S3. On the other hand, if it is determined that the zoom operation is not being performed, the process proceeds to step S4.

  In step S3, the scene determination unit 110 reads the detection results from the luminance detection unit 106 and the color detection unit 107 stored before the zoom operation, and proceeds to step S5.

  In step S4, the scene determination unit 110 reads the current detection results from the latest luminance detection unit 106 and color detection unit 107, and proceeds to step S5.

  In step S5, the scene discrimination unit 110 discriminates the photographic scene based on the detection results from the face detection unit 105 and the distance detection unit 117 received in step S1 and the detection result read out in step S3 or step S4. . Next, the scene determination unit 110 sends information for changing shooting parameters, image processing parameters, and the like corresponding to the determined scene to the camera microcomputer 112 (step S6). Based on the information received from the scene determination unit 110, the camera microcomputer 112 sends a predetermined instruction to the signal processing unit 104, and controls the lens control unit 114, the aperture control unit 115, and the signal processing unit 104 (step S7). .

  Next, a specific example of the flow of the scene determination operation described above will be described.

  When the imaging apparatus is not in a zoom operation, the scene determination unit 110 receives the latest face information, luminance information, color information, and the like from the face detection unit 105, the luminance detection unit 106, the color detection unit 107, and the distance detection unit 117, respectively. And receive distance information. Then, the scene discrimination unit 110 performs a new scene discrimination based on all the latest information. Therefore, if the scene background changes, the luminance information obtained by the luminance detection unit 106 and the color information obtained by the color detection unit 107 change, so that the change of the scene background can be determined. Also, regardless of the background change, if a person is reflected in the image, a scene determination result that is a person scene is obtained, and if there is no person image in the image, a scene determination result that is not a person scene is obtained. can get. Note that the macro scene determination is the same as the determination of the person scene.

  On the other hand, when the imaging apparatus is performing a zoom operation, the scene determination unit 110 receives the latest face information and distance information (second information) from the face detection unit 105 and the distance detection unit 117, respectively. On the other hand, the latest luminance information and color information (first information) are not received from the luminance detection unit 106 and the color detection unit 107, and the stored luminance information and color information before the zoom operation are read out. That is, since the scene determination unit 110 does not receive new luminance information and color information during the zoom operation, the background scene determination result does not change during the zoom operation.

  However, even during the zoom operation, the scene determination unit 110 receives the latest face information and distance information from the face detection unit 105 and the distance detection unit 117. Therefore, if a person appears in the image, the scene determination unit 110 is a person scene. A scene determination result is obtained. On the other hand, when the image of the person disappears from the image, a scene determination result that is not a person scene is obtained. Note that the macro scene determination is the same as the determination of the person scene.

  For example, a case will be described in which a state where a colorful flower blooming on the ground is photographed with zoom-up is switched to a state where a castle against a blue sky along a distant coast is photographed.

  The scene discriminating unit 110 determines that the scene is a vivid scene from the detection results of the luminance detection unit 106 and the color detection unit 107 when the colorful flower blooming on the ground is being photographed with zoom-up, and the distance detection unit 117. It is determined that the scene is a macro scene from the detection result of. On the other hand, when shooting a castle with a blue sky in the background along a distant coast, the scene determination unit 110 determines that the scene is a blue sky scene from the detection results of the luminance detection unit 106 and the color detection unit 107. Then, it is determined from the detection results by the face detection unit 105 and the distance detection unit 117 that the scene is neither a person scene nor a macro scene.

  Here, when switching from the state of shooting colorful flowers on the ground zoomed in to the state of shooting a castle against the blue sky along the distant coast, changing the direction of the camera, A zoom operation is also performed.

  During the zoom operation, the latest detection result by the luminance detection unit 106 and the color detection unit 107 cannot be obtained, and the detection result before the zoom operation is retained, so even if a large blue sky appears in the image, it is bright. There is no change in the determination result determined as a correct scene. However, since the latest detection result is obtained from the distance detection unit 117 even during the zoom operation, the determination result determined as a macro scene before the zoom operation is canceled during the zoom operation. When the zoom operation is completed, the latest detection result is obtained from the luminance detection unit 106 and the color detection unit 107, so that the result determined as the background of the shooting scene is obtained from the bright scene at a timing later than the cancellation of the macro scene. Changed to a blue sky scene.

  In addition, during the zoom operation, the luminance information and the color information detected (first detection) by the luminance detection unit 106 and the color detection unit 107 vary in various patterns due to a change in the subject reflected in the image. The possibility is relatively high. On the other hand, in the face information detected by the face detection unit 105 (second detection), all the faces of the person shown in the image have disappeared, or whether a new face has appeared where no face was originally present. There is only a fluctuation of the degree, and it is unlikely that the scene determination result changes frequently as in the background. The same applies to macro scene determination based on distance information detected (second detection) by the distance detection unit 117.

  Therefore, in this embodiment, the background determination of the shooting scene is temporarily stopped during the zoom operation. However, by configuring so that the determination of the subject in the shooting scene is continued, the fluctuation risk even during the zoom operation. It is possible to execute scene discrimination with less.

  According to the above-described embodiment, it is possible to eliminate the instability of scene determination during the zoom operation and to perform more appropriate image adjustment by performing highly accurate scene determination.

  In the above embodiment, the background of the shooting scene has been described by taking a blue sky, a night view, and a vivid scene as examples. However, the present invention is not limited to these, and a backlight scene, an underwater scene, snow Needless to say, scene determination may be performed using other determination elements such as a scene.

  In the above embodiment, two examples of luminance information and color information are exemplified as the first information related to the image processing parameter used for scene determination. However, any one of them is information other than these. Also good. In addition, although the face information and the distance information are exemplified as the second information related to the shooting parameter used for scene determination, either one or other information may be used.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

103 image sensor 104 signal processing unit 105 face detection unit 106 luminance detection unit 107 color detection unit 110 scene determination unit 111 zoom operation unit 112 camera microcomputer 116 zoom detection unit 117 distance detection unit

Claims (6)

An image pickup device, a zoom means for executing a zoom operation for changing a magnification of a subject image on the image pickup device surface, a scene determination means for determining a shooting scene from a video signal obtained from the image pickup device, and the scene determination means An image pickup apparatus having control means for controlling at least one of a shooting parameter at the time of shooting and an image processing parameter for the shot image according to the scene determined by
First detection means for detecting first information used to determine a background of a shooting scene from the video signal;
Second detection means for detecting second information used for discriminating a subject of a shooting scene from the video signal;
Zoom detection means for detecting a zoom operation by the zoom means,
The scene discrimination means includes
When the zoom operation is not detected by the zoom detection unit, the shooting scene is determined based on the latest detected first information and second information,
When the zoom operation is detected by the zoom detection unit, the shooting scene is determined based on the first information detected before the zoom operation is performed and the second information detected during the zoom operation. An imaging apparatus characterized by performing   The first detection means includes at least one of luminance information detection means for detecting luminance information in the video signal and color information detection means for detecting color information in the video signal, and the luminance information and the color information The imaging apparatus according to claim 1, wherein the first information including at least one of the information is sent to the scene determination unit.   The second detection means analyzes the video signal and detects face information of a person reflected in the frame image, and from the current zoom position, focus lens position, and focus information, At least one of distance information detection means for detecting distance information indicating the distance to the second information, and sending the second information including at least one of the face information and the distance information to the scene determination means, The imaging apparatus according to claim 1. Signal processing means for performing signal processing on the video signal using the image processing parameter;
The imaging apparatus according to claim 1, wherein the signal processing unit performs signal processing based on the image processing parameter changed by the control unit. An image pickup device, a zoom means for executing a zoom operation for changing a magnification of a subject image on the image pickup device surface, a scene determination means for determining a shooting scene from a video signal obtained from the image pickup device, and the scene determination means A control method for an imaging apparatus having control means for controlling at least one of a shooting parameter at the time of shooting and an image processing parameter for a shot image according to the scene determined by
A first detection step of detecting first information used to determine a background of a shooting scene from the video signal;
A second detection step of detecting second information used for discriminating the subject of the shooting scene from the video signal;
A zoom detection step of detecting a zoom operation by the zoom means ;
A scene discrimination step of discriminating a shooting scene by the scene discrimination means ,
The scene discrimination step includes
When the zoom operation is not detected in the zoom detection step, the shooting scene is determined based on the detected latest first information and second information,
When the zoom operation is detected in the zoom detection step, based on the first information detected before the zoom operation is performed and the second information detected during the zoom operation, A control method characterized by performing discrimination. The program for making an imaging device perform the control method of Claim 5.

Images