JP5367129B2 - Imaging apparatus, control apparatus, and control method thereof - Google Patents

Description

  The present invention relates to a technique for detecting a subject region based on an image generated by photoelectrically converting a subject image formed by an imaging optical system.

  Conventionally, a phase difference detection type AF camera having a plurality of focus detection points performs focus detection on a plurality of focus detection points in the object field, and automatically or arbitrarily selects from these focus detection points. Control is performed so that one focused detection point is focused. When a focus detection point is automatically selected, a general camera selects a focus detection point with the shortest distance to a subject based on a focus detection result at each focus detection point and focuses.

  On the other hand, there is an AF camera using a contrast method in which a subject image formed by an imaging lens is picked up by an image pickup device, and an in-focus position is determined using an image pickup signal in a focus area. The contrast-type camera focuses at a focal position where the focus evaluation value in the focus area shows a peak.

  In this type of contrast AF camera, a camera that detects a region of a person's face and focuses on the face is known. Many face detection methods have already been proposed. For example, Patent Document 1 proposes a method of determining a face candidate area corresponding to the shape of a human face and determining the face area from the feature amount of the face candidate area. In the same document, a method for detecting a face candidate region by extracting the outline of a human face from an image and a correlation value with a plurality of templates having various shapes of the face are calculated. A method for detecting a face candidate area based on the above has been proposed.

  However, in any AF focus detection method, it is difficult to focus in a situation where a lens with a long focal length is attached to the camera and the depth of field becomes shallow. In particular, when a plurality of people are present at different distances, it is difficult to focus on the main subject that the photographer wants to focus on because the main subject does not always exist within the depth of field. is there.

  According to the phase difference method camera, when a focus control point is automatically controlled from a plurality of focus detection points, if an object unintended by the photographer enters near the focus detection point, the subject Will be in focus. For this reason, the phase difference type camera has a problem that the main subject intended by the photographer cannot be focused.

  Further, according to the contrast type camera, the focus is detected so as to focus on the subject having the highest contrast peak. For this reason, for example, when the background has a larger contrast peak than the main subject, the background is in focus. In addition, if the contrast peak of the clothes is larger than the face of the subject, the clothes are in focus. As described above, the contrast-type camera has a problem that the position of the focus to be detected becomes unstable, and the position desired by the photographer cannot be focused.

  Furthermore, with a face detection type camera that detects and focuses on the face of the subject, the face of the subject existing within the depth of field at the focal position can be detected, but the main subject does not exist within this depth of field. Sometimes there is a problem that the main subject cannot be found.

In contrast to these methods, Patent Document 2 proposes a method of detecting faces of a plurality of subjects and performing grouping based on shooting parameters such as subject distances and subject brightness with respect to these faces. According to this document, it is possible to perform photographing with the optimum photographing parameter for each subject by setting and photographing the optimum photographing parameter for each group.
JP-A-8-63597 JP 2006-345254 A

  However, in a situation where the depth of field of the camera is shallow, it is difficult for the photographer to focus on the main subject that the photographer wants to focus on when a plurality of people are present at different distances.

  When the main subject does not exist within the depth of field at the current focal position, it is necessary to focus on the main subject within the depth of field at another focal position while driving the focusing lens. At this time, in the case of the phase difference type camera, the closest subject is focused, and in the case of the contrast type, the focus is focused on the highest contrast peak. However, the focused position is not necessarily the main subject.

  There is a method of detecting the faces of a plurality of subjects, grouping them from imaging parameters such as subject distances and subject luminances with respect to these faces, and setting and imaging optimal imaging parameters for each group. Since this method shoots at a plurality of focal positions where the face of the subject is detected with a single release operation, it is possible to shoot a plurality of persons on the screen.

  However, with this method, it is possible to record an image focused on the main subject, but since other images are also recorded, unnecessary images are increased, and a great deal of time is required for organizing images after shooting. Take it. According to this method, when there are a plurality of people on the screen, it is not possible to record an image focusing only on the main subject.

In order to solve the above problems, an imaging apparatus according to an aspect of the present invention includes an image generation unit that photoelectrically converts a subject image formed by an imaging optical system to generate an image, and the image generation unit that generates the image. Subject detection means for detecting a plurality of subject areas on the screen based on the image, wherein the subject detection means moves the imaging optical system within a predetermined drive range and moves the subject areas at a plurality of focal positions. When the subject area is detected, the focal position of the imaging optical system is stored in correspondence with the subject area.

According to another aspect of the present invention, there is provided a control device that selects any one of a plurality of subject areas on a screen detected based on an image and stores the focus of the imaging optical system stored corresponding to the selected subject. In the control device that reads the position, the plurality of subject areas are generated by photoelectrically converting subject images formed by the imaging optical system at a plurality of focal positions where the imaging optical system is moved within a predetermined driving range. The focus position of the imaging optical system when the subject area can be detected is stored in association with the subject area.

According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus, the step of photoelectrically converting a subject image formed by an imaging optical system to generate an image, and moving the imaging optical system within a predetermined driving range. Detecting a subject area on the screen based on the generated image at a plurality of focal positions, and if the subject area can be detected, the focus of the imaging optical system corresponding to the subject area The position is memorized.

According to another aspect of the present invention, there is provided a control method for a control device, comprising: selecting one of a plurality of subject areas on a screen detected based on an image; and storing the corresponding subject corresponding to the selected subject. Reading a focal position of the imaging optical system, and the plurality of subject areas are subjects formed by the imaging optical system at a plurality of focal positions where the imaging optical system is moved within a predetermined driving range. It is detected based on an image generated by photoelectrically converting an image, and the focal position of the imaging optical system when the subject area is detected is stored in association with the subject area. It is characterized by being.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide an apparatus capable of associating a subject region detected based on an image with a focal position.

It is a block diagram of the camera in a present Example. It is a flowchart of the imaging process in the camera of a present Example. It is a flowchart of the face search process in step S102 of a present Example. It is a flowchart of the face list display process in step S104 of the present embodiment. 1 shows a first display format of a face list in the display device of the present embodiment. It shows a second display format of the face list in the display device of the present embodiment. The 3rd display format of the face list in the display apparatus of a present Example is shown. It shows a fourth display format of the face list in the display device of the present embodiment. It shows a fifth display format of the face list in the display device of the present embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 shows a block diagram of a camera 100 (imaging device) in an embodiment of the present invention.

  In FIG. 1, reference numeral 101 denotes an imaging optical system (imaging lens). Reference numeral 107 denotes an image sensor composed of a plurality of pixels. The image sensor 107 is a CCD, CMOS sensor, or the like that forms a subject optical image that has passed through the imaging optical system 101. The image sensor 107 photoelectrically converts a subject image formed by the imaging optical system 101 and outputs an imaging signal.

  Reference numeral 102 denotes a main mirror having a semi-transmissive portion. The main mirror 102 is retracted to the outside of the photographing light beam at the time of photographing, and is inclined in the photographing light beam (in the optical path) at the time of focus detection. FIG. 1 shows a state where the main mirror 102 is inserted in the imaging light beam (mirror down). The main mirror 102 is a finder optical system that includes a focus plate 103, a pentaprism 104, and an eyepiece lens 105 for a part of the light beam that has passed through the imaging optical system 101 while being inclined in the photographing light beam. Lead to.

  Reference numeral 106 denotes a submirror. The sub mirror 106 is configured to be foldable and unfoldable with respect to the main mirror 102 in synchronization with the operation of the main mirror 102. A part of the light beam that has passed through the semi-transmissive portion of the main mirror 102 is reflected downward by the sub mirror 106 and enters the phase difference type focus detection device 108, and the in-focus state of the imaging optical system 101 is detected.

  On the other hand, in the contrast method, the main mirror 102 is always retracted out of the imaging light beam. In synchronization with this operation, the subject optical image that has passed through the imaging optical system 101 is formed on the imaging element 107 in a state in which the sub mirror 106 is folded with respect to the main mirror 102 (mirror up). A focus state is detected.

  Reference numeral 109 denotes an analog signal processing circuit. The analog signal processing circuit 109 performs processing such as noise removal on the imaging signal output from the imaging element 107 in accordance with the amount of light of the subject optical image formed on the imaging optical system 101. Reference numeral 110 denotes an A / D converter. The A / D converter 110 converts the analog signal output from the analog signal processing circuit 109 into a digital signal.

  Reference numeral 111 denotes a digital signal processing circuit. The digital signal processing circuit 111 is connected to a system controller 112 described later, and performs image processing such as shading correction and gamma correction on the digital signal output from the A / D converter 110.

  The image sensor 107, the analog signal processing circuit 109, the A / D converter 110, and the digital signal processing circuit 111 described above constitute an image generation unit 130. The image generation unit 130 photoelectrically converts the subject image formed by the imaging optical system 101 to generate an image.

  Reference numeral 112 denotes a system controller. The system controller 112 includes a CPU that controls the entire camera 100, a RAM that is a storage device (storage means), and the like, and appropriately controls the operation of each unit (described later) such as the lens driving device 113. As will be described later, the storage device of the system controller 112 stores the focus positions and screen positions of a plurality of subject areas detected by the face detection circuit 121.

  Reference numeral 113 denotes a lens driving device (lens driving means). The lens driving device 113 is connected to the system controller 112, communicates with the imaging optical system 101, a lens driving mechanism that drives the imaging optical system 101 to perform focus adjustment, and a drive that drives the driving mechanism. Provide a circuit.

  The system controller 112 and the lens driving device 113 constitute a control unit 140. As will be described later, the control unit 140 performs focus control of the imaging optical system 101.

  Reference numeral 114 denotes a mirror driving device. The mirror driving device 114 is connected to the system controller 112 and drives the main mirror 102 out of the imaging light beam. Reference numeral 115 denotes an image sensor driving device. The image sensor driving device 115 is connected to the system controller 112 and drives the image sensor 107.

  Reference numeral 116 denotes a buffer memory. The buffer memory 116 is a frame memory that is connected to the digital signal processing circuit 111 and can store data for a plurality of frames captured by the image sensor 107. The digital signal A / D converted by the A / D converter 110 is temporarily stored in the buffer memory 116. The digital signal processing circuit 111 reads the data stored in the buffer memory 116 and performs each process described above. Data processed by the digital signal processing circuit 111 is stored in the buffer memory 116 again.

  Reference numeral 117 denotes a recording / reproducing signal processing circuit. The recording / reproducing signal processing circuit 117 is connected to the digital signal processing circuit 111. As described above, the image data that has been subjected to various types of digital processing by the digital signal processing circuit 111 is stored in the buffer memory 116. The recording / reproducing signal processing circuit 117 then records the image data stored in the buffer memory 116 in an external storage medium 118 such as a memory card.

  When the image data is recorded in the external storage medium 118, the image data is compressed, for example, the data is compressed by the JPEG method. On the other hand, when the image data is read from the external storage medium 118, the recording / playback signal processing circuit 117 performs an expansion process of the image data. The recording / reproduction signal processing circuit 117 also includes an interface for performing data communication with the external storage medium 118.

  Reference numeral 120 denotes a display device (display means). The display device 120 displays an image captured by the image sensor 107. The display device 120 is also used when reproducing and displaying image data recorded on the external storage medium 118. When an image is displayed on the display device 120, the image data stored in the buffer memory 116 is read out. The read image data is converted from digital image data to an analog video signal by a D / A converter 119. The display device 120 displays an image using the analog video signal converted by the D / A converter 119. As will be described later, the display device 120 displays a plurality of subject areas detected by the face detection circuit 121.

  When the image captured by the image sensor 107 is displayed on the display device 120, there are two display modes. One is a display form when the release operation is not performed. This display form is a display form called a through image in which images repeatedly picked up by the image sensor 107 are sequentially updated and displayed. The other is a display form called a freeze image that displays an image captured by the image sensor 107 for a predetermined time after the release operation of the camera 100.

  Reference numeral 121 denotes a face detection circuit (subject detection means). The face detection circuit 121 is connected to the digital signal processing circuit 111 and detects whether or not a face (subject area) exists in the image data subjected to various digital processes by the digital signal processing circuit 111. As described above, the face detection circuit 121 detects a plurality of subject areas based on the image generated by the image generation unit 130. As for the face detection method, many methods have already been proposed as disclosed in Patent Document 1, for example.

  Reference numeral 122 denotes an operation unit (selection means). The operation unit 122 is connected to the system controller 112 and operates the camera 100 such as a power switch for turning on / off the camera 100, a release button, and a setting button for selecting a shooting mode such as a person shooting mode. An operation member is provided. When these switches and buttons are operated, a signal corresponding to the operation is input to the system controller 112. As will be described later, the operation unit 122 selects one of a plurality of subject areas displayed on the display device 120. Then, the system controller 112 performs focus control on the subject area selected by the operation unit 122.

  The release button includes a release switch SW1 that is turned on by a first stroke operation (half-pressing operation) of the release button operated by a photographer, and a release switch that is turned on by a second stroke operation (full-pressing operation) of the release button. SW2 is connected.

  Next, imaging processing in the camera 100 of the present embodiment will be described.

  FIG. 2 is a flowchart of imaging processing in the camera of this embodiment. The imaging processing in the present embodiment is to perform shooting with focusing on the face of the main subject by detecting and recognizing the face of the subject.

  First, in step S101, the system controller 112 determines whether or not the release switch SW1, which is one of the operation units 122, has been pressed. If the release switch SW1 is not pressed, the process of step S101 is repeated until the release switch SW1 is pressed. On the other hand, if the release switch SW1 is pressed, the process proceeds to step S102.

  In step S <b> 102, the system controller 112 communicates with the imaging optical system 101 via the lens driving device 113 to drive the imaging optical system 101 within the entire driving range of the imaging optical system 101. By such control, the face detection circuit 121 can detect the face of the subject from the image data that has been subjected to various processes by the digital signal processing circuit 111 at a plurality of focal positions. Details of such face search processing will be described later.

  Step S102 includes an image generation step for photoelectrically converting a subject image formed by the imaging optical system 101 to generate an image, and a detection step for detecting a plurality of subject regions based on the image generated in the image generation step. Including.

  In step S103, it is determined whether face detection is established as a result of the face search process in step S102. If face detection is not established, the photographing process is terminated. On the other hand, if face detection is established, the process proceeds to step S104.

  In step S104, the system controller 112 creates a list of a plurality of faces that have been detected in step S102. The created face list is displayed on the display device 120 via the digital signal processing circuit 111. As described above, step S104 is a display step for displaying a plurality of subject areas detected in step S102. Details of this will be described later.

  In step S105, it is determined whether or not the photographer has selected the face to be focused as the main subject from the list of faces displayed on the display device 120 in step S104 by the operation unit 122. This determination is made by the system controller 112. If no face has been selected by the photographer, the process returns to step S101 and the above process is repeated. On the other hand, if a predetermined face is selected by the photographer, the process proceeds to step S106.

  In step S106, the system controller 112 drives the imaging optical system 101 via the lens driving device 113 to the focal position where the detection of the face selected in step S105 is established. As described above, step S106 is a control step for performing focus control on the selected subject region when one of the plurality of subject regions displayed in step S104 is selected in step S105.

  In step S <b> 106, the lens driving device 113 drives the imaging optical system 101 to the focal position of the subject area selected by the operation unit 122. As will be described later, this operation is executed by the system controller 112 reading out the focal position of the subject area selected by the operation unit 122 from the storage device (storage means) in the system controller 112. The lens driving device 113 (control unit 140) performs focus control based on the focus position of the subject area read from the storage device.

  Thereafter, the face detection process may be performed once again, or the process may directly proceed to step S107.

  In step S107, the system controller 112 determines whether or not the release switch SW2 that is one of the operation units 122 has been pressed. If the release switch SW2 is not pressed, the process returns to step S101 and the above process is repeated. On the other hand, if the release switch SW2 is pressed, the process proceeds to step S108.

  Finally, in step S108, the system controller 112 captures and stores the captured image.

  Next, the face search process in step S102 of FIG. 2 will be described in detail. FIG. 3 is a flowchart of the face search process in step S102.

  In step S <b> 201, the system controller 112 determines whether the face search process has been performed in the entire driving range of the mounted imaging optical system 101 via the lens driving device 113. When the face search process is performed in the entire driving range of the imaging optical system 101, the face search process ends. On the other hand, if face search processing is not performed in the entire drive range of the imaging optical system 101, the process proceeds to step S202.

  In step S <b> 202, the system controller 112 writes the image signal output from the image sensor 107 as image data in the buffer memory 116 via the A / D converter 110 and the digital signal processing circuit 111.

  In step S203, the face detection circuit 121 performs face search processing (face recognition processing) from the image data written in the buffer memory 116 in step S202.

  In step S204, it is determined whether or not the face recognition process is established in step S203. If the face recognition process is not established, the process proceeds to step S206. On the other hand, if the face recognition process is established, the process proceeds to step S205.

  In step S <b> 205, the face, focus position, and screen position for which face recognition processing is established in step S <b> 203 are stored in the storage device in the system controller 112. As described above, the system controller 112 stores the focal positions and screen positions of a plurality of subject areas detected by the face detection circuit 121.

  In step S <b> 206, the system controller 112 performs focus adjustment via the lens driving device 113 that drives the imaging optical system 101. After the focus adjustment is completed, the process returns to step S201, and steps S201 to S206 are repeated until the face search process in the entire drive range of the imaging optical system 101 is completed.

  Next, the face list display process in step S104 of FIG. 2 will be described in detail. FIG. 4 is a flowchart of the face list display process in step S104.

  In step S301, the system controller 112 determines whether or not face recognition is established during the face search process in step S102. If face recognition has not been established, the process proceeds to step S304 to notify the photographer that face recognition has not been established. On the other hand, if face recognition is established, the process proceeds to step S302.

  In step S302, the faces recognized in step S301 are sorted. The sorting order includes a distance order or a composition order. When sorting the faces, information such as the focal positions or screen positions of a plurality of faces detected by the face detection circuit 121 is read from a storage device provided in the system controller 112 of the control means 140. Then, the digital signal processing circuit 111 of the image generation means 130 performs sorting based on these pieces of information.

  When adopting the order of distance as the order of sorting, for example, they can be arranged in the order of close distance. When the composition order is adopted as the sorting order, for example, the screens can be arranged in the order from the center of the screen to the peripheral direction, from the left to the right of the screen, or in the clockwise direction. These directions may be combined and arranged. Moreover, it may be displayed in the order in which face detection is established without sorting, or may be displayed randomly.

  In step S303, the list of faces created in step S302 is displayed on the display device 120. In step S <b> 303, the image generation unit 130 displays a plurality of subject areas on the display device 120 based on information on the plurality of subject areas detected by the face detection circuit 121. An example is shown in FIGS. 5 to 7 show the display format of the face list in the display device of this embodiment.

  As shown in FIG. 5, the first display format for displaying the image 201 and the face list 202 in the live view can be adopted as the face list display format. Here, corresponding to the subjects A to H in the image 201 in live view, the faces a to h of the subjects are displayed, respectively.

  In the first display format of the face list of FIG. 5, the face list 202 (faces a to h) is sorted in order of distance in step S302, and displayed from left to right in the order of the subject A from a short distance to the subject H from a long distance. It is a thing. The faces displayed in the face list 202 may be the same size as the subject area stored in the storage device of the system controller in the face search process, or may be enlarged or reduced to a unified size.

  The image 201 in live view is displayed with the focus on the closest subject A in the face list 202. The image 201 in live view can be an image focused on the closest subject A until the face list 202 is displayed and the photographer selects a face to focus on. Alternatively, an image that is focused in order from subject A to subject H may be displayed every certain time.

  As a display format of the face list, as shown in FIG. 6, a second display format that displays only the face list 203 (faces a to h) of the subject may be adopted. Further, as shown in FIG. 7, a third display format may be employed in which an image 204 in which a face for which face detection is established is superimposed on an image in live view is displayed. Again, faces a to h are displayed corresponding to the subjects A to H, respectively. The display format of the face list may be configured so that the photographer can change it by operating the operation unit 122.

  FIG. 8 shows a fourth display format of the face list in the display device of this embodiment. FIG. 8 shows a scene in which subjects I, J, and K are equidistant from the camera. These subjects I, J, and K are assumed to be within the depth of field at the same focal position.

  An image 301 in live view displays an image focused on all the subjects I, J, and K. The face list 302 displays the faces i, j, and k of the subjects I, J, and K for which face detection has been established in the composition order (counterclockwise from the center of the screen to the periphery).

  FIG. 9 shows a fifth display format of the face list in the display device of this embodiment. FIG. 9 shows a scene where subjects I, J, and K are equidistant from the camera, as in FIG.

  If the subjects I, J, and K exist within the same depth of field at the same focal position, the subjects I, J, and K are grouped as shown in the live view image 401 in FIG. L. In the face list 402, the face l of the representative of the group L can be displayed. At this time, the face l is selected in the order of composition such as the center face on the screen. The grouping may be performed by operating the photographer's operation unit 122 or may be automatically performed according to the number of subjects in the screen. As described above, when the face detection circuit 121 detects a plurality of subject areas within the depth of field, the system controller 112 (image generating unit 130) displays a representative subject area by grouping the plurality of subject areas. It is displayed on the device 120.

  As described above, according to the imaging apparatus of the present embodiment, the main subject can be selected by selecting the face of the subject that the photographer wants to focus on from the list display of the faces of a plurality of subjects for which face detection has been established. Focus control can be performed. Therefore, it is possible to provide an imaging apparatus capable of controlling the focus on the main subject selected by the photographer.

  The embodiment of the present invention has been specifically described above. However, the present invention is not limited to the matters described as the above-described embodiments, and can be appropriately changed without departing from the technical idea of the present invention.

  For example, in the above description, the subject intended by the photographer is described as a person, and the area for focus adjustment is the area where the face is detected. However, the present invention is not limited to this. For example, the subject image may be cut out from the background to detect the subject, and the region corresponding to the subject position may be set as the subject region for focus detection.

DESCRIPTION OF SYMBOLS 100 Camera 101 Imaging optical system 102 Main mirror 106 Sub mirror 111 Digital signal processing circuit 112 System controller 113 Lens drive device 116 Buffer memory 120 Display device 121 Face detection process 122 Operation part 130 Image generation means 140 Control means

Claims (6)

Image generation means for photoelectrically converting a subject image formed by the imaging optical system to generate an image;
Subject detection means for detecting a plurality of subject areas on a screen based on the image generated by the image generation means;
The subject detection means detects the subject region at a plurality of focal positions by moving the imaging optical system within a predetermined driving range, and corresponds to the subject region when the subject region can be detected. An imaging apparatus that stores a focal position of the imaging optical system.   The apparatus includes: a selection unit that selects any one of the plurality of subject areas detected by the subject detection unit and reads a focal position of the imaging optical system stored in association with the selected subject area. The imaging apparatus according to 1.   The imaging apparatus according to claim 1, further comprising a display unit configured to display the plurality of subject areas detected by the subject detection unit. In a control device that selects any one of a plurality of subject areas on a screen detected based on an image and reads a focal position of an imaging optical system stored corresponding to the selected subject.
The plurality of subject areas are detected based on an image generated by photoelectrically converting a subject image formed by the imaging optical system at a plurality of focal positions where the imaging optical system is moved within a predetermined driving range. And a focus position of the imaging optical system when the subject area can be detected is stored in association with the subject area. Photoelectrically converting a subject image formed by the imaging optical system to generate an image;
Detecting a subject area on a screen based on the generated image at a plurality of focal positions obtained by moving the imaging optical system within a predetermined driving range ;
A method for controlling an imaging apparatus, comprising: storing a focal position of the imaging optical system in association with the subject area when the subject area is detected. Selecting one of a plurality of subject areas on a screen detected based on an image;
Reading the focal position of the imaging optical system stored corresponding to the selected subject,
The plurality of subject areas are detected based on an image generated by photoelectrically converting a subject image formed by the imaging optical system at a plurality of focal positions where the imaging optical system is moved within a predetermined driving range. And a focus position of the imaging optical system when the subject area is detected is stored in association with the subject area.