JP2018066889A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately switch a subject focused.SOLUTION: An imaging apparatus includes an imaging part which takes pictures of a subject by means of an optical system and repeatedly outputs picture signals, a recognition part which repeatedly recognizes a first subject picture selected in the pictures by the picture signals, and a control part which controls a focusing state of the optical system with respect to a second subject picture different from the first subject picture.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus.

  An imaging device that tracks a subject to be tracked and focuses on the tracked subject is known (see Patent Document 1). However, the subject that the user desires to focus on is not necessarily the subject to be tracked.

JP 2013-12942 A

  An imaging device includes an imaging unit that captures an image of a subject by an optical system and repeatedly outputs an image signal, a recognition unit that repeatedly recognizes a first subject image selected in an image based on the image signal, and the recognition unit And a control unit that controls the in-focus state of the optical system with respect to a second subject image different from the first subject image recognized by.

1 is a block diagram illustrating an overall configuration of an imaging apparatus according to a first embodiment. It is a figure which shows the state by which the scene where the dribbling is performed in the soccer game is displayed as a through image on the monitor. It is a figure which shows the state by which the scene where the dribbling is performed in the soccer game is displayed as a through image on the monitor. It is a figure which shows the state by which the scene where the pass is performed in the soccer game is displayed on the monitor as a through image. FIG. 5 is a diagram illustrating a state after the scene illustrated in FIG. 4 and a state in which the ball approaches the player. It is a flowchart of the process performed at the time of focus adjustment. It is a figure explaining a modification. It is a block diagram which shows the whole structure of the imaging device of 2nd Embodiment. It is a flowchart of the process performed at the time of focus adjustment of 2nd Embodiment.

--- First embodiment ---
A first embodiment of the imaging apparatus will be described with reference to FIGS. FIG. 1 is a block diagram illustrating the overall configuration of the imaging apparatus according to the first embodiment.
A camera 100 as an imaging apparatus according to the present embodiment includes an operation member 101, a photographing lens 102, an imaging element 103, a memory card slot 105, a monitor 106, and a control device 110. The operation member 101 includes various operation buttons operated by the user. For example, the operation member 101 includes a power button, a release button, a zoom button, a cross button, an enter button, a play button, a delete button, and the like.

  The photographic lens 102 is composed of a plurality of optical lenses, but in FIG. 1, it is represented by a single lens. Further, the lenses constituting the photographing lens 102 include an AF lens (focusing lens) for AF (Auto Focus / automatic focus adjustment). The image sensor 103 is an image sensor such as a CCD or a CMOS, for example, receives a light beam from the photographing lens 102 to generate an image signal and a focus detection signal, and controls the generated image signal and focus detection signal. Output to device 110.

  The control device 110 includes a CPU, a memory, and other peripheral circuits, and controls the camera 100. Note that the memory constituting the control device 110 includes SDRAM and flash memory. The SDRAM is a volatile memory, and is used as a work memory for the CPU to develop a program when the program is executed or as a buffer memory for temporarily recording data. The flash memory is a non-volatile memory in which data of a program executed by the control device 110, various parameters read during program execution, and the like are recorded.

  The control device 110 includes a recognition unit 111, a main subject detection unit 112, a focus detection unit 113, and a focus adjustment unit 114. The control device 110 realizes the functions of these units by executing a program stored in the memory. The contents of the function of each part will be described later.

  The memory card slot 105 is a slot for inserting a memory card as a storage medium, and writes and records the image file output from the control device 110 on the memory card.

  The monitor 106 is a liquid crystal display (LCD) or the like mounted on the back of the camera 100. The monitor 106 displays an image stored in a memory card, a setting menu for setting the camera 100, and the like. The In addition, when the operation member 101 is operated by the user and the start of display of the captured image on the monitor 106 is instructed by the user, the control device 110 generates display image data based on the image signal repeatedly acquired from the image sensor 103. It is generated and output to the monitor 106. As a result, the captured image is displayed in real time on the monitor 106 (so-called live view display of a through image). The monitor 106 is a touch panel type display device and constitutes a part of the operation member 101.

--- About the function of each part of control device 110 ---
The recognition unit 111 of the control device 110 repeatedly detects and tracks a tracking target designated by the user by repeatedly detecting an image corresponding to a specific target from a plurality of images repeatedly obtained from the image sensor 103. The user can specify the tracking target by touching a specific subject image in the image displayed on the monitor 106, for example. The tracking target image is also referred to as a first subject image.
In the present embodiment, as will be described later, a ball used in various ball games is designated as a tracking target. For example, when the tracking target is specified by touching the subject image in the display image of the monitor 106, the recognition unit 111 acquires the tracking target image as a template image, and uses the template image and the image from the imaging element 103 as the template image. Template matching is used to repeatedly recognize and track the tracking target.
If the tracking target is known in advance, such as a ball used in various ball games, an image of the ball is stored in a memory (not shown) in advance, and the ball The image may be read from the memory and used as a template image.
Of course, the recognition unit 111 can also perform tracking by detecting the position of the tracking target by a process other than the template matching process.

The main subject detection unit 112 is a subject that is different from the tracking target and has predetermined characteristics from image data based on image signals repeatedly acquired from the image sensor 103 (hereinafter, a subject having predetermined characteristics is referred to as a target subject). ) Is detected. Further, the main subject detection unit 112 selects a target subject having a predetermined relationship with the tracking target from among the detected one or more target subjects, and sets the selected target subject as the main subject. The focus adjustment unit 114 adjusts the focus of the main subject as described later. Therefore, the main subject detection unit 112 includes a detection unit that recognizes, that is, detects a target subject, and a main subject selection unit that selects a target subject having a predetermined relationship with a tracking target as a main subject from the detected target subject. The The image of the main subject is also called a second subject image.
In the present embodiment, the target subject is a person's face, and thus the main subject detection unit 112 has a face detection unit as the detection unit. The main subject selection unit of the main subject detection unit 112 selects a face having a predetermined relationship with the tracking target from one or more faces detected by the face detection unit.

In the present embodiment, the face having a predetermined relationship with the tracking target is the face closest to the tracking target. More specifically, a face having a predetermined relationship with the tracking target is a face located within a predetermined range from the tracking target, and when there are a plurality of faces within the predetermined range, The face with the smallest distance from the tracking target. Note that the predetermined range is such that if the face is within the predetermined range, the person of the face has a special relationship with the tracking target, and if the face is outside the predetermined range, the person of the face is specially identified as the tracking target. It is selected so that it can be assumed that there is no relationship.
For example, in the present embodiment, a face person (ball game player) located within a predetermined range from a ball game ball such as soccer to be tracked is a person who controls or can control the ball. Here, for example, in soccer, dribbling the ball, kicking the ball into the air, heading the ball, throwing the ball, and the like.
Therefore, in the present embodiment, the main subject detection unit 112 sets the face closest to the tracking target ball among the detected human faces as the main subject. For example, the main subject detection unit 112 determines the face closest to the tracking target based on the tracking target or the position of the detected face on the imaging surface of the image sensor 103, that is, the tracking target or the position of the face in the image obtained by imaging. Set as the main subject.

  In the present embodiment, the main subject detection unit 112 determines the predetermined range described above as follows. The predetermined range is determined as a range in which the person of the face is estimated to have a special relationship with the tracking target ball as described above, and the face detection unit of the main subject detection unit 112 detects the face. In this way, by defining the predetermined range as the range in which the face detection unit of the main subject detection unit 112 detects the face, the face detection unit is compared with the case where the face detection unit detects a face on the entire screen of the image sensor 103. The calculation load can be suppressed. The range in which the face detection unit detects a face in the present embodiment is a predetermined range centered on the position of the tracking target image, and this range is referred to as a face detection range. The main subject detection unit 112 performs face detection processing within the face detection range.

  The size of the face detection range is set to a size proportional to the size of the image to be tracked in an image obtained by imaging with the image sensor 103, for example. Therefore, the face detection range becomes large when the distance between the camera 100 and the tracking target is short, and the face detection range becomes small when the distance between the camera 100 and the tracking target is long. In addition, the face detection range becomes large if the focal length of the photographing lens 102 is long, and the face detection range becomes small if the focal length of the lens 102 is short. Further, if the size of the tracking target itself is large, the face detection range becomes large, and if the size of the tracking target itself is small, the face detection range becomes small.

  The main subject detection unit 112 determines that there is no main subject to be focused near the tracking target when the human face cannot be detected within the face detection range. In this case, the main subject detection unit 112 sets the tracking target itself as the main subject.

  The focus detection unit 113 detects the focus of the main subject image based on the focus detection signal from the image sensor 103 regarding the main subject. The focus detection unit 113 may detect the focus of the main subject image and other subject images based on the focus detection signal from the image sensor 103. The focus detection unit 113 may be a phase detection type focus detection unit or a contrast detection type focus detection unit. When the focus detection unit 113 is a phase difference detection type focus detection unit, the defocus amount of the main subject image can be calculated at any position on the imaging surface of the image sensor 103. It is desirable to provide pixels capable of focus detection in the image sensor 103 over the entire imaging surface of the image sensor 103. In addition to the image sensor 103, a sensor for performing focus detection by the phase difference detection method may be provided, and the focus detection unit 113 may perform focus detection based on a signal from the sensor.

  The focus adjustment unit 114 controls the focus state of the photographing lens 102 with respect to the face set as the main subject by the main subject detection unit 112 based on the focus detection result regarding the main subject detected by the focus detection unit 113. That is, the focus adjustment unit 114 adjusts the focus of the photographing lens 102 so that the face set as the main subject by the main subject detection unit 112 is in focus.

In the camera 100 configured as described above, for example, when shooting a state of a ball game, when the ball is set as a tracking target, the face closest to the ball is the main face when the face exists in the face detection range. The focus state of the taking lens 102 is controlled so that the subject is set and the face is in focus. Therefore, when taking a still image with the camera 100, a still image in which the face closest to the ball is in focus at the time of release is obtained. When a moving image is shot with the camera 100, a moving image in which the face closest to the ball is in focus is obtained according to the movement of the ball or the player. When the face does not exist within the face detection range, the focusing state of the lens 102 is controlled so that the ball is set as the main subject and the ball is in focus.
In the following, the player's face to be focused will be described by taking as an example a scene where a dribbling or passing in a soccer game as an example of a ball game is performed in the camera 100 of the present embodiment.

  FIG. 2 is a diagram showing a state in which a dribbling scene in a soccer game is displayed on the monitor 106 as a through image. In FIG. 2, it is assumed that the player 161 out of a plurality of persons (players) 161 to 166 dominates the ball 151 and dribbles. It is assumed that the ball 151 and the players 161 to 165 are moving in the directions indicated by the arrows 171 and 181 to 185, and the player 166 is substantially stationary. The arrows 171 and 181 to 185 are shown for convenience of explanation and are not displayed on the monitor 106. A rectangular frame 141 indicated by a broken line is the face detection range described above. The face detection range 141 is a rectangle centered on the ball 151 to be tracked, but may be a polygon other than a rectangle, a circle, an ellipse, or the like. In addition, the center of the shape of the face detection range 141 is not necessarily required to coincide with the ball 151 that is the tracking target. Note that the frame 141 of the face detection range 141 may or may not be displayed on the monitor 106.

  For example, when the recognition unit 111 detects that the image of the ball 151 in the image 140 displayed on the monitor 106 is touched by the user, the recognition unit 111 sets the ball 151 as a tracking target and starts tracking the ball 151.

The main subject detection unit 112 detects the face of a person existing in the face detection range 141 in FIG. In the example illustrated in FIG. 2, the faces existing in the face detection range 141 are the face 161 a of the player 161 and the face 162 a of the player 162. Then, the main subject detection unit 112 sets, as the main subject, a face having a predetermined relationship with the tracking target, that is, the face 161a closest to the ball 151 among the two detected faces 161a and 162a. As described above, the reason why the main subject detection unit 112 selects the face 161a closest to the ball 151 is that there is a high possibility that the person 161 of the face 161a controls the ball 151.
If there is only one face in the face detection range 141, the main subject detection unit 112 sets that face as the main subject.

  FIG. 3 is a diagram showing a state where a dribbling scene in a soccer game is displayed as a through image on the monitor 106 as in FIG. 2. The only difference between FIG. 3 and FIG. 2 is whether or not there is a player 167 with the rear side of the head 167 b facing the camera 100 in the image 140. For example, as shown in FIG. 3, when the player 167 with the rear side of the head 167 b facing the camera 100 is present in the image 140, the face of the player 167 is not captured, so the main subject detection unit 112 has the player 167. The face cannot be detected. Therefore, even when the position of the head 167b of the player 167 is closer to the ball 151 than the face 161a of the player 161, the main subject detection unit 112 sets the face 161a as the main subject. Even if the face recognizing unit of the main subject detecting unit 112 can recognize the head 167b of the player 167 with the rear side of the head facing the camera 100 as being similar to the face, it is determined from the setting candidates as the main subject. It is desirable to exclude the head 167b.

FIG. 4 is a diagram showing a state in which a scene in which a pass is made in a soccer game is displayed on the monitor 106 as a through image, and is an example of a scene after the scene shown in FIG.
When the player 161 who has controlled the ball in the scene shown in FIG. 2 passes the ball 151 toward the player 168 on the right side on the screen 140 in the scene shown in FIG. 4, the ball 151 is passed from the player 161 that has made the pass. Leave. A broken-line arrow 172 in FIG. 4 indicates the movement trajectory or predicted movement trajectory of the ball 151 moving in the air. In the state shown in FIG. 4, none of the faces 161 a to 166 a and 168 a exists in the face detection range 141. In this case, the main subject detection unit 112 sets the tracking target ball 151 as the main subject.

FIG. 5 shows a state after the scene shown in FIG. 4 and shows a state where the ball 151 approaches the player 168. As shown in FIG. 5, when the ball 151 approaches the face 168 a of the player 168 within a predetermined distance and the face 168 a of the player 168 exists in the face detection range 141, the main subject detection unit 112 causes the ball 151 to The face 168a of the player 168, which is the face that is closest to, is set as the main subject.
As described above, in the camera 100 according to the present embodiment, the focusing state of the photographing lens 102 is controlled with respect to the face based on the distance between the tracking target and the face.

  As described above, in the camera 100 of the present embodiment, for example, when shooting a soccer game, when the ball 151 is set as a tracking target, a predetermined range including the ball 151 becomes the face detection range 141. Is set. Then, the face closest to the ball 151 within the face detection range 141 is set as the main subject, and the focus state of the photographic lens 102 is controlled so that this face is in focus. Even if the ball 151 moves or the player who controls the ball 151 changes depending on the situation of the soccer game, the camera 100 of this embodiment focuses on the face closest to the ball 151. Therefore, the camera 100 according to the present embodiment can focus on the face of the person who is most closely related to the ball 151 according to the situation of the soccer game, and is therefore most appropriate according to the situation of the game. You can focus on the subject. That is, in the camera 100 of the present embodiment, the subject to be focused can be automatically switched to an appropriate subject in accordance with the shooting scene, the situation of the shooting target, and the like, so that an image intended by the user can be obtained.

--- Flow chart ---
FIG. 6 is a flowchart of processing executed at the time of focus adjustment according to the present embodiment. This flowchart is executed in the control device 110 when a power switch (not shown) of the camera 100 is turned on.

  In step S <b> 11, the recognition unit 111 of the control device 110 waits until the tracking target is designated by touching the monitor 106 or the like. When the tracking target is specified by touching the monitor 106 or the like, the process proceeds to step S13, and the recognition unit 111 sets the subject specified in step S11 as the tracking target and starts tracking the tracking target.

  When step S13 is executed, the process proceeds to step S15, where the main subject detection unit 112 sets a face detection range 141 around the tracking target, and performs face detection processing for detecting a human face within the face detection range 141. Proceed to step S17. In step S <b> 17, the main subject detection unit 112 determines whether or not a face exists within a predetermined range from the tracking target, that is, the face detection range 141 as a result of the face detection process in step S <b> 15.

If a face exists in the face detection range 141, an affirmative determination is made in step S17 and the process proceeds to step S19. In step S19, the main subject detection unit 112 sets the face closest to the tracking target among the plurality of faces in the face detection range 141 as the main subject, and proceeds to step S21. If there is only one face in the face detection range 141, the main subject detection unit 112 sets that face as the main subject and proceeds to step S21.
If no face exists in the face detection range 141, a negative determination is made in step S17 and the process proceeds to step S25. In step S25, the main subject detection unit 112 sets the tracking target as the main subject, and proceeds to step S21.

In step S21, the focus adjustment unit 114 adjusts the focus of the photographing lens 102 so as to focus on the main subject set in step S19 or step S25 based on the focus detection result detected by the focus detection unit 113. Proceed to S23. In step S23, the control device 110 determines whether or not the end of the tracking operation is instructed. In the present embodiment, for example, when a power switch (not shown) is turned off, an affirmative determination is made in step S23.
If a positive determination is made in step S23, the control device 110 ends the program. If a negative determination is made in step S23, the process returns to step S15.

In the present embodiment, the following operational effects are achieved.
(1) The camera 100 captures an image of a subject by the photographing lens 102 and repeatedly outputs an image signal, and a recognition unit 111 that repeatedly recognizes a first subject image selected in the image based on the image signal. And a focus adjustment unit 114 that controls the in-focus state of the taking lens 102 with respect to a second subject image different from the first subject image recognized by the recognition unit 111. Accordingly, it is possible to focus on the second subject image different from the first subject image while tracking the first subject image. Therefore, when the subject to be focused is different from the subject to be tracked, it is possible to focus on an appropriate subject, so that an image intended by the user can be obtained. That is, in the camera 100 of the present embodiment, the subject to be focused can be automatically switched to an appropriate subject according to the shooting scene, the situation of the shooting target, etc., and an image intended by the user can be obtained.

(2) The focus adjustment unit 114 controls the in-focus state of the photographing lens 102 with respect to the second subject image based on the distance between the first subject image and the second subject image. Thus, when the subject to be focused on is different from the subject to be tracked, it is possible to focus on an appropriate subject, so that an image intended by the user can be obtained.

(3) The focus adjustment unit 114 controls the focusing state of the photographing lens 102 with respect to the second subject image when the second subject image exists within the face detection range 141 with respect to the first subject image. Then, when the second subject image does not exist in the face detection range 141, the focusing state of the photographing lens 102 is controlled with respect to the first subject image. Accordingly, even when the second subject image does not exist in the face detection range 141, the first subject image can be focused, and it is possible to suppress obtaining an image that does not match the user's intention.

(4) The camera 100 further includes a main subject detection unit 112 that selects a second subject image close to the first subject image among the plurality of second subject images when there are a plurality of second subject images. . The focus adjustment unit 114 controls the in-focus state of the photographic lens 102 with respect to the second subject image selected by the main subject detection unit 112. Accordingly, when there are a plurality of subjects that are candidates for focusing, an appropriate subject can be focused, and an image intended by the user can be obtained.

--- Modification of the first embodiment ---
(Modification 1) In the first embodiment described above, the main subject detection unit 112 sets the face present at the position closest to the tracking target in the face detection range 141 as the main subject. On the other hand, in the present modification, the main subject detection unit 112 sets the face present at the closest position to the tracking target among the faces in the same direction as the tracking target in the face detection range 141 as the main subject. . That is, in this modified example, the main subject is set in consideration of not only the distance to the tracking target but also the tracking target and the moving direction of the detected face.
Specifically, the main subject detection unit 112 calculates the moving direction of the face detected within the face detection range 141 based on a plurality of images obtained repeatedly from the image sensor 103. The recognizing unit 111 calculates the moving direction of the tracking target based on a plurality of images obtained repeatedly from the image sensor 103. Then, the main subject detection unit 112 sets the face closest to the tracking target among the faces that move in substantially the same direction as the movement direction of the tracking target calculated by the recognition unit 111 as the main subject.
The direction substantially the same as the moving direction of the tracking target referred to here is a direction in which the angle formed with the moving direction of the tracking target is a predetermined angle or less.

This modification will be described with reference to FIG.
FIG. 7 is a diagram showing a state where a dribbling scene in a soccer game is displayed as a through image on the monitor 106 as in FIG. For example, as shown in FIG. 7, there is a player 169 running from the lower right side of the screen 140 toward the ball 151 with respect to the player 161 dribbling on the screen 140 toward the right side. And In FIG. 7, it is assumed that the position of the face 169a of the player 169 is closer to the ball 151 than the position of the face 161a of the player 161. In FIG. 7, players 161 and 162 move in the same direction as the movement direction of the ball 151 indicated by the arrow 171. As indicated by an arrow 189, the moving direction of the player 169 is different from the moving direction of the ball 151.

  In such a situation, the main subject detection unit 112 calculates the moving direction of each face 161a, 162a, 169a detected in the face detection range 141 based on a plurality of images obtained repeatedly from the image sensor 103. The recognizing unit 111 calculates the moving direction of the ball 151 based on a plurality of images obtained repeatedly from the image sensor 103. The main subject detection unit 112 sets the face 161a closest to the ball 151 as the main subject among the face 161a and the face 162a that move in substantially the same direction as the movement direction of the ball 151 calculated by the recognition unit 111.

That is, in this modified example, the moving direction of the player 169 is different from the moving direction of the ball 151. Therefore, even if the position of the face 169a of the player 169 is closer to the ball 151 than the position of the face 161a of the player 161, The detection unit 112 does not set the face 169a of the player 169 as a main subject.
Thereby, in this modification, since an appropriate subject can be focused, an image intended by the user can be obtained.

(Modification 2) In the first embodiment described above, the main subject detection unit 112 detects the tracking target on the imaging surface of the image sensor 103 and the position of the detected face, that is, the tracking target in the image obtained by imaging. Based on the position of the face, the face closest to the tracking target was set as the main subject. That is, in the first embodiment described above, the distance in the depth direction of the tracking target and the detected face is not considered when setting the main subject. On the other hand, in the present modification, the distance in the depth direction of the tracking target and the detected face is taken into account when setting the main subject.
Specifically, for example, the focus detection unit 113 calculates the defocus amount for the face within the face detection range 141 detected by the tracking target or the main subject detection unit 112. The main subject detection unit 112 includes the focal length of the photographing lens 102, the position of the tracking target or the detected face on the imaging surface of the image sensor 103, and the defocus amount calculated by the focus detection unit 113 for the tracking target or the detected face. Based on the above, the actual position of the tracking target or the detected face is calculated. Then, the main subject detection unit 112 identifies the face having the smallest actual distance from the tracking target based on the calculated actual position of the tracking target and the actual position of the detected face, and uses the identified face as the main subject. Set to.
Thereby, in this modified example, since the calculation accuracy of the distance between the tracking target and the detected face is improved, the setting of the main subject can be made more appropriate and the image intended by the user can be obtained.

(Modification 3) In the first embodiment described above, the main subject detection unit 112 detects a human face as a subject image having a predetermined feature from image data based on image signals repeatedly obtained from the image sensor 103. Face detection processing was performed. On the other hand, in this modification, the main subject detection unit 112 performs person detection processing for detecting the whole body of a person as a subject image having a predetermined feature from image data based on image signals repeatedly acquired from the image sensor 103.
In the present modification, the main subject detection unit 112 includes a detector constructed using, for example, HOG (Histograms of Oriented Gradients) as a feature amount and Real AdaBoost as a learning algorithm. Perform detection processing.
Then, the main subject detection unit 112 sets the person closest to the tracking target among the detected persons as the main subject.
Also according to this modification, the same operational effects as those of the first embodiment described above can be obtained.

  In the above description, soccer has been described as an example of a ball game. However, in shooting other ball games such as rugby, football, water polo, ice hockey, etc. The imaging device 100 has the same effects as the effects described above. For ball games such as table tennis and tennis, the ball reciprocates between a plurality of players, as in the scene where a pass in a soccer game in the above description is performed. Therefore, in the imaging apparatus 100 according to the present embodiment, when the ball approaches the player, the face of the player is focused, and when the ball is hit back and separated from the player, the ball is focused. As described above, the imaging apparatus 100 also has the same operational effects as the above-described operational effects in shooting ball games such as playing table tennis and tennis.

--- Second Embodiment ---
A second embodiment of the imaging apparatus will be described with reference to FIGS. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and different points will be mainly described. Points that are not particularly described are the same as those in the first embodiment. The present embodiment is different from the first embodiment in that the movement of the tracking target is mainly analyzed and the setting condition of the main subject is changed based on the analysis result of the tracking target movement.

FIG. 8 is a block diagram illustrating an overall configuration of the imaging apparatus according to the second embodiment. The control device 110A of the camera 100A as the imaging device of the present embodiment includes a determination unit 115 in addition to the recognition unit 111, the main subject detection unit 112, the focus detection unit 113, and the focus adjustment unit 114.
The determination unit 115 analyzes the movement of the tracking target tracked by the recognition unit 111. Based on the analysis result, the determination unit 115 outputs to the main subject detection unit 112 a change signal that changes how the subject having the tracking target is set as the main subject.

Hereinafter, the analysis contents in the determination unit 115 in the present embodiment will be described.
First, the following example is given as a feature of the movement of the ball in the ball game to be photographed.
For example, in a ball game where players compete for a ball like a soccer game, the ball held by the player may be temporarily stopped in the air during a throw-in, and the ball placed on the court may be a court during a corner kick. Rest on top. Thereafter, the ball starts to move by being thrown or kicked. The ball moves by rolling on the court or flying in the air. Generally, when a ball and a player who controls the ball move together like dribbling, the ball often rolls on the court. Also, when passing or shooting, there are cases where the ball rolls on the court and where the ball flies in the air. When the ball flies in the air, the speed of the ball tends to be higher than when the ball rolls on the court.

In general, when shooting a throw-in or corner kick scene in a soccer game, it is desirable to focus on a player who is touching or is about to touch the ball. In other words, it is desirable to focus on a player near the ball. When shooting a dribble scene, it is desirable to focus on the player who controls the ball.
In a pass or shoot scene, when the ball is rolling on the court, the moving speed of the ball tends to be slower than when the ball is flying in the air. Therefore, for example, the time that the ball is separated from each player by a pass by a predetermined distance or more tends to be longer when the ball rolls on the court than when the ball flies in the air. Therefore, when the ball is rolling on the court due to a pass, it is desirable to focus on the ball. When the ball approaches the player who wants to receive the pass, it is desirable to focus on the player who wants to receive the pass.
On the other hand, when the ball is flying in the pass or shot scene, the moving speed of the ball tends to increase as described above. Therefore, the time during which the ball is separated from each player by a predetermined distance or more by the pass tends to be short. Therefore, in such a case, it is desirable to focus on the player who wants to receive the pass.

Also, for example, in ball games in which a ball is shot on the opponent's court, such as table tennis, tennis, and volleyball, the movement speed of the ball before it is hit with a racket at the time of serving is the speed of the ball after being hit with a racket Slower than. Further, the ball thrown up into the air at the time of serving temporarily stops in the air. The movement of the ball changes suddenly when it is hit with a racket. The ball reciprocates between one area of the court and the other area mainly by flying in the air.
Therefore, when shooting a game of table tennis, tennis, or volleyball, it is desirable to focus on a player present near the ball when serving. Further, when the ball is flying in the air by being hit, it is desirable to focus on a player who tries to strike back or touch the flying ball.

Here, paying attention to the relationship between the movement form of the ball and the subject to be focused, if the ball is stationary or can be regarded as stationary, the player who is near the ball is focused. It is desirable.
When the ball is rolling, if the distance between the ball and the player is within a predetermined distance, that is, for example, if the player's face exists in the face detection range 141 in the first embodiment, the closest to the ball It is desirable to focus on the player. Further, when the ball is rolling, if the distance between the ball and the player is equal to or greater than a predetermined distance, that is, if the player's face does not exist within the face detection range 141 in the first embodiment, for example, It is desirable to focus on.
When the ball is flying in the air, it is desirable to focus on a player that exists in the moving direction of the ball, for example, a player that exists near the predicted drop point of the ball.

  Therefore, the determination unit 115 of the present embodiment analyzes the movement of the ball that is the tracking target. Then, the determination unit 115 determines whether the ball is stationary, rolling on the court, or flying in the air. Note that the determination unit 115 determines that the ball is stationary when the moving speed is low, such as when the ball is held by a throwing-in or serving player's hand. It is considered.

The determination unit 115 outputs a first setting signal or a second setting signal described below to the main subject detection unit 112 based on the analysis result.
The first setting signal is a signal that the determination unit 115 outputs to the main subject detection unit 112 when it is determined that the tracking target is stationary or rolling on the court. As in the case of the first embodiment, the first setting signal sets the face closest to the tracking target as the main subject if a face is detected in the face detection range 141, and the face detection range 141 This is a signal that instructs the main subject detection unit 112 to set conditions so that the tracking target is set as the main subject if no face is detected.

  The second setting signal is a signal that the determination unit 115 outputs to the main subject detection unit 112 when it is determined that the tracking target is flying in the air. The second setting signal predicts the falling point of the tracking target flying in the air based on the image signal repeatedly acquired from the image sensor 103, and is closest to the predicted falling point among the faces existing ahead of the predicted falling point. This is a signal that instructs the main subject detection unit 112 to set conditions so that the face is set as the main subject.

The main subject detection unit 112 that has received the first setting signal performs face detection similarly to the first embodiment, and sets the face detected closest to the tracking target as the main subject.
In addition, the main subject detection unit 112 that has received the second signal predicts the fall point of the tracking target. Then, the main subject detection unit 112 detects a face existing beyond the predicted drop point. Then, the main subject detection unit 112 sets the face closest to the predicted drop point among the faces existing ahead of the predicted drop point as the main subject.
Note that the main subject detection unit 112 sets, for example, the tracking target as the main subject when the face existing beyond the predicted drop point cannot be detected.

--- Flow chart ---
FIG. 9 is a flowchart of processing executed at the time of focus adjustment according to the present embodiment. This flowchart is executed in the control device 110A when a power switch (not shown) of the camera 100A is turned on.

  Step S11 and step S13 are the same as step S11 and step S13 in the first embodiment shown in FIG. When step S13 is executed, the process proceeds to step S31, and the determination unit 115 of the control device 110A analyzes the movement of the tracking target from the image data based on the image signal repeatedly acquired from the image sensor 103, and proceeds to step S33. In step S <b> 33, the determination unit 115 determines whether the tracking target is stationary, rolling on the court, or flying in the air.

If it is determined in step S33 that the tracking target is stationary or rolling on the court, the process proceeds to step S35, and the determination unit 115 outputs the first setting signal described above to the main subject detection unit 112.
If it is determined in step S33 that the tracking target is flying in the air, the process proceeds to step S37, and the determination unit 115 outputs the above-described second setting signal to the main subject detection unit 112.

  When step S35 or step S37 is executed, the process proceeds to step S39, and the main subject detection unit 112 performs face detection processing based on the received instruction by the first or second setting signal. That is, when the first setting signal is received, the main subject detection unit 112 performs face detection as in the first embodiment. When the second signal is received, the main subject detection unit 112 predicts the tracking target fall point and detects a face existing ahead of the fall prediction point.

When step S39 is executed, the process proceeds to step S41, and the main subject detection unit 112 determines whether or not there is a face corresponding to the setting condition according to the instruction of the first or second setting signal.
If an affirmative determination is made in step S41, the process proceeds to step S43, and the main subject detection unit 112 sets the face corresponding to the setting condition according to the instruction of the first or second setting signal as the main subject, and the process proceeds to step S21.
If a negative determination is made in step S41, the process proceeds to step S25. Step S25 is the same as step S25 in the first embodiment shown in FIG.

  Step S21 and step S23 are the same as S21 and step S23 in the first embodiment shown in FIG.

In the second embodiment described above, the following functions and effects are obtained in addition to the functions and effects of the first embodiment.
(1) The camera 100A includes a determination unit 115 that determines the state of the first subject image. The focus adjustment unit 114 controls the focusing state of the photographic lens 102 with respect to either the first subject image or the second subject image selected according to the state of the first subject image by the determination unit 115. Thereby, the setting of the main subject can be made more appropriate, and an image intended by the user can be obtained.

--- Modification of the second embodiment ---
(Modification 1) In the second embodiment described above, if it is determined that the ball is rolling, the tracking target is set as the main subject if no face is detected within the face detection range 141. On the other hand, in this modification, when it is determined that the ball is rolling, if a face is not detected in the face detection range 141, a face that exists in the moving direction of the tracking target is set as the main subject.
Specifically, when it is determined that the tracking target is stationary or rolling on the court based on the analysis result, the determination unit 115 uses a third setting signal described below instead of the first setting signal as a main subject. The data is output to the detection unit 112. Note that the determination unit 115 outputs a second setting signal to the main subject detection unit 112 when it is determined that the tracking target is flying in the air based on the analysis result.
The third setting signal is a signal that the determination unit 115 outputs to the main subject detection unit 112 when it is determined that the tracking target is stationary or rolling on the court. The third setting signal is a signal that instructs the main subject detection unit 112 for the following setting conditions.
(1) If a face is detected within the face detection range 141, the face closest to the tracking target is set as the main subject.
(2) If a face is not detected within the face detection range 141, the movement direction of the tracking target is predicted, and a face that exists in the predicted movement direction and is closest to the tracking target is set as the main subject.
(3) In (2) above, if the tracking target movement direction cannot be predicted because the tracking target is stationary, etc., or even if the tracking target movement direction can be predicted, there is a face in the predicted movement direction. If not, the tracking target is set as the main subject.
Thereby, in this modification, when it is determined that the ball is rolling, if a face is not detected in the face detection range 141, a face existing in the moving direction of the tracking target is set as the main subject.

(Modification 2) In the second embodiment described above, the subject selected by the state of the tracking target, that is, the movement of the tracking target is focused. On the other hand, in this modification, the subject selected according to the type of tracking target is focused. For example, the control device 110A stores information and images about the size and shape of the balls used in each ball game in advance in a memory (not shown). Then, for example, the determination unit 115 calculates the size of the subject designated as the tracking subject from the focal length, the defocus amount, the size occupied on the imaging surface of the image sensor 103, and the like for the ball stored in advance. The type of the ball is determined by comparing with the image of. The determination unit 115 outputs to the main subject detection unit 112 a setting signal that instructs the main subject detection unit 112 which subject to set as the main subject according to the determined type of ball.

For example, when the determination unit 115 determines that the ball designated as the tracking subject is a soccer ball, the face closest to the tracking target in the face detection range 141 is set as the main subject in the face detection range 141 as described above. If a face does not exist within the face detection range 141, a setting signal is output to instruct the main subject detection unit 112 to set the tracking target as the main subject.
The main subject detection unit 112 that has received this setting signal sets the face closest to the tracking target in the face detection range 141 as the main subject, and sets the tracking target as the main subject if no face exists in the face detection range 141. Set.

For example, if the determination unit 115 determines that the ball is a tennis ball, the main subject detection unit performs prediction of movement of the tracking target and sets the face existing at the predicted movement destination of the tracking target as the main subject. A setting signal instructing 112 is output.
The main subject detection unit 112 that has received the setting signal performs movement prediction of the tracking target, and sets the face existing at the predicted movement destination of the tracking target as the main subject.

Further, for example, the determination unit 115 may output different setting signals depending on the type of the tracking target, such as whether it is a ball or a Frisbee disk, instead of the type of the ball. For example, the control device 110A stores information and images about the size and shape of the Frisbee disc in addition to information and images about the sizes and shapes of various balls in advance in a memory (not shown). For example, the determination unit 115 estimates the size of the subject designated as the tracking subject from the focal length, the defocus amount, the size occupied on the imaging surface of the image sensor 103, and the like. By comparing, it is determined which type of ball the tracking subject is or a Frisbee disc. When determining the type of ball of the tracking subject, the determination unit 115 outputs the setting signal described above to the main subject detection unit 112 according to the determined type of ball. If the determination unit 115 determines that the tracking subject is a Frisbee disc, the determination unit 115 outputs a setting signal as described below to the main subject detection unit 112, for example.
(1) Predict the flight trajectory of the disc.
(2) If a face exists in the face detection range 141, the movement trajectory of the face is predicted.
(3) If the flight trajectory of the disc and the movement trajectory of the face are predicted to substantially intersect, the face predicted to intersect the earliest is set as the main subject.
(4) In the above (2), when a face does not exist in the face detection range 141, or when it is predicted that the flight trajectory of the disc and the movement trajectory of the face do not intersect, the tracking target is set as the main subject. Set to.
The main subject detection unit 112 that has received such a setting signal sets, for example, the face of a person who wants to catch the thrown Frisbee disc as the main subject in a scene where a person playing with a Frisbee disc is photographed. Thereby, even if there are a plurality of persons who want to catch the disc, the face of the person who is highly likely to catch the disc can be set as the main subject.
In addition to the person's face, the main subject detection unit 112 may be configured to detect an animal face such as a dog face. This makes it possible to set the face of a dog having a high possibility of catching the disc as a main subject among a plurality of dog faces playing with a Frisbee disc.

  Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Moreover, you may combine embodiment and the modification which were mentioned above suitably. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 100,100A; Camera, 102; Shooting lens, 103; Image pick-up element, 110, 110A; Control apparatus, 111; Recognition part, 112; Main subject detection part, 114;

Claims (8)

An imaging unit that captures an image of a subject by an optical system and repeatedly outputs an image signal;
A recognizing unit that repeatedly recognizes the first subject image selected in the image based on the image signal;
An imaging apparatus comprising: a control unit that controls a focused state of the optical system with respect to a second subject image different from the first subject image recognized by the recognition unit. The imaging device according to claim 1,
The control unit is an imaging apparatus that controls a focusing state of the optical system with respect to the second subject image based on a distance between the first subject image and the second subject image. The imaging device according to claim 1 or 2,
The control unit controls a focusing state of the optical system with respect to the second subject image when the second subject image exists within a first range with respect to the first subject image; An imaging apparatus that controls a focusing state of the optical system with respect to the first subject image when a second subject image is outside the first range. In the imaging device according to any one of claims 1 to 3,
The control unit is an imaging apparatus that controls a focusing state of the optical system with respect to the second subject image according to a moving direction of the first subject image and the second subject image. The imaging apparatus according to claim 4,
The control unit is an imaging apparatus that controls an in-focus state of the optical system with respect to the second subject image that moves in a direction in which the first subject image moves. The imaging apparatus according to claim 4,
The control unit is an imaging apparatus that controls an in-focus state of the optical system with respect to the second subject image existing in a direction in which the first subject image moves. In the imaging device according to any one of claims 1 to 6,
A selection unit that selects the second subject image close to the first subject image from the plurality of second subject images when there are a plurality of the second subject images;
The said control part is an imaging device which controls the focus state of the said optical system with respect to the 2nd to-be-photographed image selected by the said selection part. In the imaging device according to any one of claims 1 to 7,
A determination unit for determining the state or type of the first subject image;
The control unit determines a focus state of the optical system with respect to either the first subject image or the second subject image selected by the state or type of the first subject image by the determination unit. An imaging device to be controlled.

Images