JP6899049B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging device.

An imaging device that tracks a subject to be tracked and focuses on the tracked subject is known (see Patent Document 1). However, the subject that the user wants to focus on is not necessarily the subject to be tracked.

Japanese Unexamined Patent Publication No. 2013-12942

The image pickup apparatus includes an imaging unit that repeatedly outputs an image signal, a recognition unit that repeatedly recognizes a first subject detected based on the image signal, and a first subject that is different from the first subject and that is different from the first subject. It is provided with a selection unit that selects a second subject whose distance is variable based on the distance from the first subject, and a focus detection unit that detects the focus based on the second subject.

It is a block diagram which shows the whole structure of the image pickup apparatus of 1st Embodiment. It is a figure which shows the state which the scene where dribbling is performed in a soccer game is displayed as a through image on a monitor. It is a figure which shows the state which the scene where dribbling is performed in a soccer game is displayed as a through image on a monitor. It is a figure which shows the state which the scene where a pass is performed in a soccer game is displayed as a through image on a monitor. It is the scene after the scene shown in FIG. 4, and is the figure which shows the state which the ball approaches a player. It is a flowchart of the process executed at the time of focus adjustment. It is a figure explaining the modification. It is a block diagram which shows the whole structure of the image pickup apparatus of 2nd Embodiment. It is a flowchart of the process executed at the time of the focus adjustment of the 2nd Embodiment.

--- First Embodiment ---
The first embodiment of the image pickup apparatus will be described with reference to FIGS. 1 to 6. FIG. 1 is a block diagram showing an overall configuration of the image pickup apparatus of the first embodiment.
The camera 100 as an image pickup device of the present embodiment includes an operation member 101, a photographing lens 102, an image pickup element 103, a memory card slot 105, a monitor 106, and a control device 110. The operation member 101 includes various operation buttons operated by the user. For example, the operation member 101 includes a power button, a release button, a zoom button, a cross button, a decision button, a play button, a delete button, and the like.

The photographing lens 102 is composed of a plurality of optical lenses, and is represented by one lens as a representative in FIG. Further, the lens constituting the photographing lens 102 includes an AF lens (focusing lens) for AF (Auto Focus / automatic focus adjustment). The image sensor 103 is, for example, an image sensor such as a CCD or CMOS, receives a light flux from the photographing lens 102, generates an image signal and a focus detection signal, and controls the generated image signal and the focus detection signal. Output to device 110.

The control device 110 is composed of a CPU, a memory, and other peripheral circuits, and controls the camera 100. The memory constituting the control device 110 includes an SDRAM and a flash memory. SDRAM is a volatile memory, which is used as a work memory for the CPU to expand a program at the time of program execution, or as a buffer memory for temporarily recording data. Further, the flash memory is a non-volatile memory in which data of a program executed by the control device 110, various parameters read at the time of program execution, and the like are recorded.

The control device 110 includes a recognition unit 111, a main subject detection unit 112, a focus detection unit 113, and a focus adjustment unit 114. The control device 110 realizes the functions of each of these parts by executing a program stored in the memory. The contents of the functions of each part will be described later.

The memory card slot 105 is a slot for inserting a memory card as a storage medium, and writes an image file output from the control device 110 to the memory card for recording.

The monitor 106 is a liquid crystal display (LCD) or the like mounted on the back of the camera 100, and the monitor 106 displays an image stored in a memory card, a setting menu for setting the camera 100, and the like. To. Further, when the operation member 101 is operated by the user to instruct the monitor 106 to start displaying the captured image, the control device 110 outputs display image data based on the image signal repeatedly acquired from the image sensor 103. Generate and output this to the monitor 106. As a result, the captured image is displayed on the monitor 106 in real time (so-called live view display of the through image). The monitor 106 is a touch panel type display device and constitutes a part of the operation member 101.

--- About the functions of each part of the control device 110 ---
The recognition unit 111 of the control device 110 repeatedly recognizes and tracks the tracking target specified by the user by repeatedly detecting an image corresponding to a specific target from a plurality of images repeatedly obtained from the image sensor 103. The user can specify the tracking target by touching, for example, a specific subject image in the image displayed on the monitor 106. The image to be tracked is also referred to as a first subject image.
In the present embodiment, as will be described later, a ball used in various ball games is designated as a tracking target. When the tracking target is specified by touching the subject image in the display image of the monitor 106, for example, the recognition unit 111 acquires the tracking target image as a template image, and obtains the template image and the image from the image sensor 103. Template matching is performed to repeatedly recognize and track the tracking target.
If the tracking target is known in advance, such as a ball used in various ball games, an image of the ball is stored in advance in a memory (not shown), and the ball is tracked during the tracking process. The image may be read from the memory and used as a template image.
Of course, the recognition unit 111 can also detect the position of the tracking target and perform tracking by a process other than the template matching process.

The main subject detection unit 112 refers to a subject different from the tracking target from the image data based on the image signal repeatedly acquired from the image sensor 103 and having a predetermined feature (hereinafter, a subject having a predetermined feature is referred to as a target subject). ) Is detected. Further, the main subject detection unit 112 selects a target subject having a predetermined relationship with the tracking target from the detected one or a plurality of target subjects, and sets the selected target subject as the main subject. The focus of this main subject is adjusted by the focus adjustment unit 114 as described later. Therefore, the main subject detection unit 112 includes a detection unit that recognizes or detects the target subject, and a main subject selection unit that selects the target subject having a predetermined relationship with the tracking target from the detected target subject as the main subject. To. The image of the main subject is also referred to as the second subject image.
In the present embodiment, the target subject is the face of a person, and therefore the main subject detection unit 112 has a face detection unit as a detection unit. The main subject selection unit of the main subject detection unit 112 selects a face having a predetermined relationship with the tracking target from one or a plurality of faces detected by the face detection unit.

In the present embodiment, the face having a predetermined relationship with the tracking target is the face closest to the tracking target. More specifically, the face having a predetermined relationship with the tracking target is a face located within a predetermined range from the tracking target, and when a plurality of faces exist within the predetermined range, the faces are included in the plurality of faces. It is the face with the shortest distance to the tracking target. In the above predetermined range, if the face is within the predetermined range, the person with the face has a special relationship with the tracking target, and if the face is outside the predetermined range, the person with the face is special with the tracking target. It is selected so that it can be inferred that there is no relationship.
For example, in the present embodiment, a person with a face (ball game player) located within a predetermined range from the ball of a ball game such as soccer to be tracked is a person who controls or can control the ball. Here, domination means, for example, in soccer, dribbling the ball, kicking the ball in the air, heading the ball, throwing the ball, and the like.
Therefore, in the present embodiment, the main subject detection unit 112 sets the face of the detected person closest to the tracking target ball as the main subject. For example, the main subject detection unit 112 determines the face closest to the tracking target based on the position of the tracking target or the detected face on the image pickup surface of the image sensor 103, that is, the tracking target or the face position in the image obtained by imaging. Set as the main subject.

In the present embodiment, the main subject detection unit 112 defines the above-mentioned predetermined range as follows. As described above, the predetermined range is defined as a range in which the person with the face is presumed to have a special relationship with the ball to be tracked, and the face detection unit of the main subject detection unit 112 detects the face. In this way, by defining the predetermined range as the range in which the face detection unit of the main subject detection unit 112 detects the face, the face detection unit is compared with the case where the face detection unit detects the face on the full screen of the image sensor 103. It is possible to suppress the calculation load of. In the present embodiment, the range in which the face detection unit detects the face is a predetermined range centered on the position of the tracking target image, and this range is referred to as a face detection range. The main subject detection unit 112 performs face detection processing within the face detection range.

The size of the face detection range is set to a size proportional to the size of the image to be tracked in the image obtained by imaging with the image sensor 103, for example. Therefore, if the distance between the camera 100 and the tracking target is short, the face detection range is large, and if the distance between the camera 100 and the tracking target is long, the face detection range is small. Further, if the focal length of the photographing lens 102 is long, the face detection range becomes large, and if the focal length of the lens 102 is short, the face detection range becomes small. Further, if the size of the tracking target itself is large, the face detection range is large, and if the size of the tracking target itself is small, the face detection range is small.

If the main subject detection unit 112 cannot detect the face of a person within the face detection range, the main subject detection unit 112 determines that there is no main subject to be focused on near the tracking target. In this case, the main subject detection unit 112 sets the tracking target itself as the main subject.

The focus detection unit 113 detects the focus of the main subject image based on the focus detection signal from the image sensor 103 regarding the main subject. The focus detection unit 113 may perform focus detection on a main subject image and other subject images based on the focus detection signal from the image sensor 103. Further, the focus detection unit 113 may be a phase difference detection type focus detection unit or a contrast detection type focus detection unit. When the focus detection unit 113 is a focus detection unit of the phase difference detection method, the defocus amount of the main subject image can be calculated regardless of the position of the main subject image on the image pickup surface of the image sensor 103. It is desirable that the focus-detectable pixels of the image sensor 103 are provided on the entire image pickup surface of the image sensor 103. Further, a sensor for performing the focus detection of the phase difference detection method may be provided separately from the image sensor 103, and the focus detection unit 113 may perform the focus detection based on the signal from this sensor.

The focus adjustment unit 114 controls the focusing state of the photographing lens 102 with respect to the face set as the main subject by the main subject detection unit 112 based on the focus detection result regarding the main subject detected by the focus detection unit 113. That is, the focus adjustment unit 114 adjusts the focus of the photographing lens 102 so that the face set as the main subject by the main subject detection unit 112 is in focus.

In the camera 100 configured in this way, for example, when the ball is set as the tracking target when shooting a state of a ball game, the face closest to the ball is the main one when the face is within the face detection range. It is set as the subject, and the focusing state of the photographing lens 102 is controlled so that the face is in focus. Therefore, when a still image is taken by the camera 100, a still image in which the face closest to the ball is in focus at the time of release can be obtained. When a moving image is taken by the camera 100, a moving image in which the face closest to the ball is in focus can be obtained at any time according to the movement of the ball or the player. When the face does not exist within the face detection range, the focusing state of the lens 102 is controlled so that the ball is set as the main subject and the ball is in focus.
Hereinafter, in the camera 100 of the present embodiment, a scene in which a dribble or a pass is performed in a soccer game as an example of a ball game will be taken as an example, and which player's face will be focused on will be described.

FIG. 2 is a diagram showing a state in which a scene in which dribbling is performed in a soccer game is displayed as a through image on the monitor 106. In FIG. 2, it is assumed that the player 161 of the plurality of persons (players) 161 to 166 controls the ball 151 and dribbles. It is assumed that the ball 151 and each player 161 to 165 are moving in the directions indicated by arrows 171 and arrows 181 to 185, and the player 166 is substantially stationary. The arrows 171 and 181 to 185 are shown for convenience of explanation and are not displayed on the monitor 106. The rectangular frame 141 shown by the broken line is the face detection range described above. The face detection range 141 is a rectangle centered on the ball 151 to be tracked, but may be a polygon other than a rectangle, a circle, an ellipse, or the like. Further, the center of the shape of the face detection range 141 does not necessarily have to match the ball 151 to be tracked. The frame 141 of the face detection range 141 may or may not be displayed on the monitor 106.

When the recognition unit 111 detects that the image of the ball 151 in the image 140 displayed on the monitor 106 is touched by the user, the recognition unit 111 sets the ball 151 as a tracking target and starts tracking the ball 151.

The main subject detection unit 112 detects the face of a person existing in the face detection range 141 in FIG. 2. In the example shown in FIG. 2, the faces existing in the face detection range 141 are the face 161a of the player 161 and the face 162a of the player 162. Then, the main subject detection unit 112 sets the face having a predetermined relationship with the tracking target among the two detected faces 161a and 162a, that is, the face 161a closest to the ball 151 as the main subject. As described above, the reason why the main subject detection unit 112 selects the face 161a at the position closest to the ball 151 is that there is a high possibility that the person 161 of the face 161a dominates the ball 151.
If only one face exists in the face detection range 141, the main subject detection unit 112 sets that face as the main subject.

FIG. 3 is a diagram showing a state in which a scene in which dribbling is performed in a soccer game is displayed as a through image on the monitor 106 as in FIG. 2. The only difference between FIG. 3 and FIG. 2 is whether or not there is a player 167 in the image 140 with the rear side of the head 167b facing the camera 100. For example, as shown in FIG. 3, when the image 140 has a player 167 with the rear side of the head 167b facing the camera 100, the face of the player 167 is not captured, so that the main subject detection unit 112 is the player 167. The face cannot be detected. Therefore, even when the position of the head 167b of the player 167 is closer to the ball 151 than the face 161a of the player 161, the main subject detection unit 112 sets the face 161a as the main subject. Even if the face recognition unit of the main subject detection unit 112 can recognize the head 167b of the player 167 with the rear side of the head facing the camera 100 as being similar to the face, the setting candidates as the main subject can be used. It is desirable to exclude the head 167b.

FIG. 4 is a diagram showing a state in which a scene in which a pass is being performed in a soccer game is displayed as a through image on the monitor 106, and is an example of a scene after the scene shown in FIG. 2, for example.
When the player 161 who controlled the ball in the scene shown in FIG. 2 passes the ball 151 toward the player 168 on the right side of the screen 140 in the scene shown in FIG. 4, the ball 151 is transmitted from the player 161 who issued the pass. Leave. The dashed arrow 172 in FIG. 4 indicates the movement locus or predicted movement locus of the ball 151 moving in the air. In the state shown in FIG. 4, none of the faces 161a to 166a and 168a exist in the face detection range 141. In this case, the main subject detection unit 112 sets the ball 151, which is the tracking target, as the main subject.

FIG. 5 is a scene after the scene shown in FIG. 4, and is a diagram showing a state in which the ball 151 approaches the player 168. As shown in FIG. 5, when the ball 151 approaches the face 168a of the player 168 within a predetermined distance and the face 168a of the player 168 is present within the face detection range 141, the main subject detection unit 112 moves the ball 151. The face 168a of the player 168, which is the face closest to the player 168, is set as the main subject.
As described above, in the camera 100 of the present embodiment, the focusing state of the photographing lens 102 is controlled with respect to the face according to the distance between the tracking target and the face.

As described above, in the camera 100 of the present embodiment, for example, when shooting a soccer game, when the ball 151 is set as the tracking target, a predetermined range including the ball 151 becomes the face detection range 141. Set. Then, the face closest to the ball 151 within the face detection range 141 is set as the main subject, and the focusing state of the photographing lens 102 is controlled so that the face is in focus. Even if the ball 151 moves or the player who controls the ball 151 is replaced depending on the situation of the soccer game, the camera 100 of the present embodiment focuses on the face closest to the ball 151. Therefore, the camera 100 of the present embodiment can focus on the face of the person most closely related to the ball 151 according to the situation of the soccer match, and is most appropriate according to the situation of the match. You can focus on the subject. That is, in the camera 100 of the present embodiment, the subject to be focused can be automatically switched to an appropriate subject according to the shooting scene, the situation of the shooting target, and the like, so that an image intended by the user can be obtained.

--- Flowchart ---
FIG. 6 is a flowchart of processing executed at the time of focus adjustment according to the present embodiment. This flowchart is executed in the control device 110 when the power switch (not shown) of the camera 100 is turned on.

In step S11, the recognition unit 111 of the control device 110 waits until the tracking target is specified by touching the monitor 106 or the like. When the tracking target is specified by touching the monitor 106 or the like, the process proceeds to step S13, the recognition unit 111 sets the subject designated in step S11 as the tracking target, and starts tracking the tracking target.

When step S13 is executed, the process proceeds to step S15, and the main subject detection unit 112 sets a face detection range 141 around the tracking target, and performs face detection processing for detecting a person's face within the face detection range 141. The process proceeds to step S17. In step S17, the main subject detection unit 112 determines, as a result of the face detection process in step S15, whether or not a face exists within a predetermined range from the tracking target, that is, within the face detection range 141.

If a face is present within the face detection range 141, step S17 is positively determined and the process proceeds to step S19. In step S19, the main subject detection unit 112 sets the face closest to the tracking target among the plurality of faces in the face detection range 141 as the main subject, and proceeds to step S21. If only one face exists in the face detection range 141, the main subject detection unit 112 sets the face as the main subject and proceeds to step S21.
If no face exists within the face detection range 141, step S17 is negatively determined and the process proceeds to step S25. In step S25, the main subject detection unit 112 sets the tracking target as the main subject and proceeds to step S21.

In step S21, the focus adjustment unit 114 adjusts the focus of the photographing lens 102 so as to focus on the main subject set in step S19 or step S25 based on the focus detection result detected by the focus detection unit 113. Proceed to S23. In step S23, the control device 110 determines whether or not the end of the tracking operation is instructed. In the present embodiment, for example, when a power switch (not shown) is turned off, step S23 is positively determined.
If the affirmative determination is made in step S23, the control device 110 ends the program. If the negative determination in step S23 is made, the process returns to step S15.

In this embodiment, the following effects are exhibited.
(1) The camera 100 has an image sensor 103 that repeatedly captures an image of a subject by a photographing lens 102 and repeatedly outputs an image signal, and a recognition unit 111 that repeatedly recognizes a first subject image selected in the image based on the image signal. A focus adjusting unit 114 that controls the focusing state of the photographing lens 102 with respect to a second subject image different from the first subject image recognized by the recognition unit 111. As a result, it is possible to focus on a second subject image different from the first subject image while tracking the first subject image. Therefore, when the subject to be focused on is different from the subject to be tracked, the subject can be focused on an appropriate subject, so that an image intended by the user can be obtained. That is, in the camera 100 of the present embodiment, the subject to be focused can be automatically switched to an appropriate subject according to the shooting scene, the situation of the shooting target, and the like, and an image intended by the user can be obtained.

(2) The focus adjusting unit 114 controls the focusing state of the photographing lens 102 with respect to the second subject image according to the distance between the first subject image and the second subject image. As a result, when the subject to be focused on is different from the subject to be tracked, the subject can be focused on an appropriate subject, so that an image intended by the user can be obtained.

(3) The focus adjusting unit 114 controls the focusing state of the photographing lens 102 with respect to the second subject image when the second subject image exists within the face detection range 141 with respect to the first subject image. Then, when the second subject image does not exist within the face detection range 141, the focusing state of the photographing lens 102 is controlled with respect to the first subject image. As a result, even when the second subject image does not exist within the face detection range 141, the first subject image can be focused, so that it is possible to suppress the acquisition of an image that does not meet the user's intention.

(4) The camera 100 further includes a main subject detection unit 112 that selects a second subject image close to the first subject image among the plurality of second subject images when a plurality of second subject images exist. .. The focus adjusting unit 114 controls the focusing state of the photographing lens 102 with respect to the second subject image selected by the main subject detecting unit 112. As a result, when there are a plurality of subjects that are candidates for focusing, the subject can be focused on an appropriate subject, so that an image intended by the user can be obtained.

--- A modified example of the first embodiment ---
(Modification 1) In the first embodiment described above, the main subject detection unit 112 sets the face located closest to the tracking target within the face detection range 141 as the main subject. On the other hand, in this modification, the main subject detection unit 112 sets the face in the face detection range 141 that is closest to the tracking target among the faces in the same moving direction as the tracking target as the main subject. .. That is, in this modification, the main subject is set in consideration of not only the distance to the tracking target but also the tracking target and the detected movement direction of the face.
Specifically, the main subject detection unit 112 calculates the moving direction of the face detected within the face detection range 141 based on a plurality of images repeatedly obtained from the image sensor 103. Further, the recognition unit 111 calculates the moving direction of the tracking target based on a plurality of images repeatedly obtained from the image sensor 103. Then, the main subject detection unit 112 sets the face closest to the tracking target as the main subject among the faces that move in substantially the same direction as the movement direction of the tracking target calculated by the recognition unit 111.
The direction substantially the same as the moving direction of the tracking target referred to here is a direction in which the angle formed with the moving direction of the tracking target is equal to or less than a predetermined angle.

This modification will be described with reference to FIG.
FIG. 7 is a diagram showing a state in which a scene in which dribbling is performed in a soccer game is displayed as a through image on the monitor 106 as in FIG. 2. For example, as shown in FIG. 7, there is a player 169 running from the lower right side of the screen 140 toward the ball 151 with respect to a player 161 dribbling on the screen 140 toward the right side. And. In FIG. 7, the position of the face 169a of the player 169 is closer to the ball 151 than the position of the face 161a of the player 161. In FIG. 7, the players moving in the same direction as the moving direction of the ball 151 indicated by the arrow 171 are the player 161 and the player 162. As shown by the arrow 189, the moving direction of the player 169 is different from the moving direction of the ball 151.

In such a situation, the main subject detection unit 112 calculates the moving directions of the faces 161a, 162a, 169a detected in the face detection range 141 based on a plurality of images repeatedly obtained from the image sensor 103. Further, the recognition unit 111 calculates the moving direction of the ball 151 based on a plurality of images repeatedly obtained from the image sensor 103. Then, the main subject detection unit 112 sets the face 161a closest to the ball 151 among the faces 161a and the faces 162a that move in substantially the same direction as the movement direction of the ball 151 calculated by the recognition unit 111 as the main subject.

That is, in this modification, since the moving direction of the player 169 is different from the moving direction of the ball 151, even if the position of the face 169a of the player 169 is closer to the ball 151 than the position of the face 161a of the player 161, the main subject. The detection unit 112 does not set the face 169a of the player 169 as the main subject.
As a result, in this modified example, it is possible to focus on an appropriate subject, so that an image intended by the user can be obtained.

(Modification 2) In the first embodiment described above, the main subject detection unit 112 determines the tracking target on the image pickup surface of the image pickup device 103 and the position of the detected face, that is, the tracking target in the image obtained by imaging. Based on the position of the face, the face closest to the tracking target was set as the main subject. That is, in the first embodiment described above, the distance in the depth direction with respect to the tracking target and the detected face was not taken into consideration when setting the main subject. On the other hand, in this modified example, when setting the main subject, the distance in the depth direction with respect to the tracking target and the detected face is taken into consideration.
Specifically, for example, the focus detection unit 113 calculates the defocus amount for the face within the face detection range 141 detected by the tracking target or the main subject detection unit 112. The main subject detection unit 112 includes the focal length of the photographing lens 102, the position of the tracking target and the detected face on the image pickup surface of the image sensor 103, and the defocus amount calculated by the focus detection unit 113 for the tracking target and the detected face. Based on, the actual position of the tracking target and the detected face is calculated. Then, the main subject detection unit 112 identifies the face having the shortest actual distance to the tracking target based on the calculated actual position of the tracking target and the detected actual position of the face, and uses this identified face as the main subject. Set to.
As a result, in this modification, the accuracy of calculating the distance between the tracking target and the detected face is improved, so that the setting of the main subject can be made more appropriate, and the image intended by the user can be obtained.

(Modification 3) In the first embodiment described above, the main subject detection unit 112 detects a person's face as a subject image having a predetermined feature from image data based on image signals repeatedly acquired from the image sensor 103. Face detection processing was performed. On the other hand, in this modification, the main subject detection unit 112 performs a person detection process for detecting the whole body of a person as a subject image having a predetermined feature from image data based on image signals repeatedly acquired from the image sensor 103.
In this modification, the main subject detection unit 112 includes, for example, a detector constructed by using HOG (Histograms of Oriented Gradients) as a feature quantity and Real AdaBoost as a learning algorithm, and a person using this detector. Perform detection processing.
Then, the main subject detection unit 112 sets the person closest to the tracking target among the detected people as the main subject.
This modification also has the same effect as that of the first embodiment described above.

In the above explanation, soccer is taken as an example of a ball game, but it is also possible to shoot other ball games such as rugby, football, water polo, and ice hockey that compete for the ball or pack to be tracked in the same way as soccer. The image pickup apparatus 100 exhibits the same effects as those described above. As for ball games such as table tennis and tennis, the reciprocating motion of the ball between a plurality of players is the main, as in the scene where a pass is performed in a soccer game in the above description. Therefore, in the image pickup apparatus 100 of the present embodiment, when the ball approaches the player, the player's face is in focus, and when the ball is hit back and away from the player, the ball is in focus. As described above, the image pickup apparatus 100 exerts the same effect as the above-mentioned effect even in the shooting of a ball game such as table tennis or tennis.

--- Second embodiment ---
A second embodiment of the image pickup apparatus will be described with reference to FIGS. 8 and 9. In the following description, the same components as those in the first embodiment are designated by the same reference numerals, and the differences will be mainly described. The points not particularly described are the same as those in the first embodiment. The present embodiment is different from the first embodiment in that the movement of the tracking target is mainly analyzed and the setting conditions of the main subject are changed based on the analysis result of the movement of the tracking target.

FIG. 8 is a block diagram showing the overall configuration of the image pickup apparatus of the second embodiment. The control device 110A of the camera 100A as the imaging device of the present embodiment includes a recognition unit 111, a main subject detection unit 112, a focus detection unit 113, a focus adjustment unit 114, and a determination unit 115.
The determination unit 115 analyzes the movement of the tracking target to be tracked by the recognition unit 111. Then, the determination unit 115 outputs a change signal for changing what kind of relationship the subject having with the tracking target is set as the main subject to the main subject detection unit 112 based on the analysis result.

Hereinafter, the analysis contents in the determination unit 115 in the present embodiment will be described.
First, the following are examples of the characteristics of the movement of the ball in the ball game to be photographed.
For example, in a ball game such as a soccer game in which the ball is scrambled for the ball, the ball held by the player may temporarily stand still in the air during a throw-in, and the ball placed on the court during a corner kick is courted. Rest on. After that, the ball starts to move when it is thrown or kicked. The ball moves by rolling on the court or flying in the air. In general, when the ball and the player who controls the ball move together as in dribbling, the ball often rolls on the court. In addition, when passing or shooting, the ball may roll on the court or the ball may fly in the air. When the ball flies in the air, the speed of the ball tends to be faster than when the ball rolls on the court.

In general, when shooting a throw-in or corner kick scene in a soccer match, it is desirable to focus on the player who is touching or is about to touch the ball. That is, it is desirable to focus on the player who is near the ball. Also, when shooting a dribble scene, it is desirable to focus on the player who controls the ball.
Further, in a pass or shoot scene, when the ball is rolling on the court, the moving speed of the ball tends to be slower than when the ball is flying in the air. Therefore, for example, the time when the ball is separated from each player by a predetermined distance or more by a pass tends to be longer when the ball rolls on the court than when the ball flies in the air. Therefore, if the ball is rolling on the court due to a pass, it is desirable to focus on the ball. Then, when the ball approaches the player who is going to receive the pass, it is desirable to focus on the player who is going to receive the pass.
On the other hand, when the ball is flying in the air in a pass or shoot scene, the moving speed of the ball tends to increase as described above. Therefore, the time that the ball is separated from each player by a predetermined distance or more due to the pass tends to be short. Therefore, in such a case, it is desirable to focus on the player who is going to receive the pass.

Also, in ball games such as table tennis, tennis, and volleyball, where the ball is hit on the opponent's court, the moving speed of the ball before being hit with a racket at the time of serving is the speed of the ball after being hit with a racket. Slower than. In addition, the ball thrown into the air at the time of serving temporarily stands still in the air. The movement of the ball changes suddenly when it is hit with a racket. The ball reciprocates between one area of the court and the other, mainly by flying in the air.
Therefore, when shooting a table tennis, tennis, or volleyball game, it is desirable to focus on a player who is near the ball when serving. In addition, when the ball is flying in the air by being hit, it is desirable to focus on the player who tries to hit or touch the flying ball.

Here, paying attention to the relationship between the form of movement of the ball and the subject to be focused on, when the ball is stationary or can be regarded as stationary, the player who is near the ball is focused. Is desirable.
When the ball is rolling, it is closest to the ball if the distance between the ball and the player is within a predetermined distance, that is, for example, if the player's face is within the face detection range 141 in the first embodiment. It is desirable to focus on the player. Further, when the ball is rolling, if the distance between the ball and the player is equal to or greater than a predetermined distance, that is, if the player's face does not exist within the face detection range 141 in the first embodiment, for example, the ball It is desirable to focus on.
When the ball is flying in the air, it is desirable to focus on a player who is in the direction of movement of the ball, for example, a player who is near the predicted drop point of the ball.

Therefore, the determination unit 115 of the present embodiment analyzes the movement of the ball to be tracked. Then, the determination unit 115 determines whether the ball is stationary, rolling on the court, or flying in the air. In addition, the determination unit 115 indicates that the ball is stationary when the ball is not completely stationary but the moving speed is low, such as when the ball is held by the hand of the player who throws in or serves. Consider it as.

The determination unit 115 outputs the first setting signal or the second setting signal described below to the main subject detection unit 112 based on the analysis result.
The first setting signal is a signal output by the determination unit 115 to the main subject detection unit 112 when it is determined that the tracking target is stationary or rolling on the court. As in the case of the first embodiment, the first setting signal sets the face closest to the tracking target as the main subject if a face is detected within the face detection range 141, and within the face detection range 141. It is a signal instructing the main subject detection unit 112 of the setting condition so that the tracking target is set as the main subject if the face is not detected.

The second setting signal is a signal output by the determination unit 115 to the main subject detection unit 112 when it is determined that the tracking target is flying in the air. The second setting signal predicts the fall point of the tracking target flying in the air based on the image signal repeatedly acquired from the image sensor 103, and is the closest to the fall prediction point among the faces existing beyond the fall prediction point. This is a signal instructing the main subject detection unit 112 to set the setting conditions so that the face is set as the main subject.

The main subject detection unit 112 that has received the first setting signal performs face detection in the same manner as in the first embodiment, and sets the face detected closest to the tracking target as the main subject.
In addition, the main subject detection unit 112 that has received the second signal predicts the drop point of the tracking target. Then, the main subject detection unit 112 detects the face existing ahead of the fall prediction point. Then, the main subject detection unit 112 sets the face closest to the fall prediction point as the main subject among the faces existing ahead of the fall prediction point.
If the main subject detection unit 112 cannot detect the face existing ahead of the fall prediction point, for example, the main subject detection unit 112 sets the tracking target as the main subject.

--- Flowchart ---
FIG. 9 is a flowchart of processing executed at the time of focus adjustment according to the present embodiment. This flowchart is executed in the control device 110A when the power switch (not shown) of the camera 100A is turned on.

Step S11 and step S13 are the same as steps S11 and S13 in the first embodiment shown in FIG. When step S13 is executed, the process proceeds to step S31, and the determination unit 115 of the control device 110A analyzes the movement of the tracking target from the image data based on the image signal repeatedly acquired from the image sensor 103, and proceeds to step S33. In step S33, the determination unit 115 determines whether the tracking target is stationary, rolling on the court, or flying in the air.

When it is determined in step S33 that the tracking target is stationary or rolling on the court, the process proceeds to step S35, and the determination unit 115 outputs the above-mentioned first setting signal to the main subject detection unit 112.
When it is determined in step S33 that the tracking target is flying in the air, the process proceeds to step S37, and the determination unit 115 outputs the above-mentioned second setting signal to the main subject detection unit 112.

When step S35 or step S37 is executed, the process proceeds to step S39, and the main subject detection unit 112 performs face detection processing based on the instruction given by the received first or second setting signal. That is, when the first setting signal is received, the main subject detection unit 112 performs face detection as in the first embodiment. When the second signal is received, the main subject detection unit 112 predicts the drop point of the tracking target and detects the face existing ahead of the fall prediction point.

When step S39 is executed, the process proceeds to step S41, and the main subject detection unit 112 determines whether or not there is a face corresponding to the setting condition instructed by the first or second setting signal.
If the affirmative determination is made in step S41, the process proceeds to step S43, and the main subject detection unit 112 sets the face corresponding to the setting condition according to the instruction of the first or second setting signal as the main subject and proceeds to step S21.
If the negative determination in step S41 is made, the process proceeds to step S25. Step S25 is the same as step S25 in the first embodiment shown in FIG.

Step S21 and step S23 are the same as S21 and step S23 in the first embodiment shown in FIG.

In the second embodiment described above, the following effects are exhibited in addition to the effects of the first embodiment.
(1) The camera 100A includes a determination unit 115 for determining the state of the first subject image. The focus adjusting unit 114 controls the focusing state of the photographing lens 102 with respect to either the first subject image or the second subject image selected by the state of the first subject image by the determination unit 115. As a result, the setting of the main subject can be made more appropriate, and the image intended by the user can be obtained.

--- A modified example of the second embodiment ---
(Modification 1) In the second embodiment described above, when it is determined that the ball is rolling, if a face is not detected within the face detection range 141, the tracking target is set as the main subject. On the other hand, in this modification, when it is determined that the ball is rolling, if the face is not detected within the face detection range 141, the face existing in the moving direction of the tracking target is set as the main subject.
Specifically, when the determination unit 115 determines that the tracking target is stationary or rolling on the court based on the analysis result, the determination unit 115 uses the third setting signal described below instead of the first setting signal as the main subject. Output to the detection unit 112. The determination unit 115 outputs a second setting signal to the main subject detection unit 112 when it is determined that the tracking target is flying in the air based on the analysis result.
The third setting signal is a signal output by the determination unit 115 to the main subject detection unit 112 when it is determined that the tracking target is stationary or rolling on the court. The third setting signal is a signal that instructs the main subject detection unit 112 of the following setting conditions.
(1) If a face is detected within the face detection range 141, the face closest to the tracking target is set as the main subject.
(2) If no face is detected within the face detection range 141, the moving direction of the tracking target is predicted, and the face existing in the predicted moving direction and closest to the tracking target is set as the main subject.
(3) In (2) above, when the moving direction of the tracking target cannot be predicted due to the tracking target being stationary, or when the moving direction of the tracking target can be predicted, the face exists in the predicted moving direction. If not, the tracking target is set as the main subject.
As a result, in this modification, when it is determined that the ball is rolling, if the face is not detected within the face detection range 141, the face existing in the moving direction of the tracking target is set as the main subject.

(Modification 2) In the second embodiment described above, the focus is on the state of the tracking target, that is, the subject selected by the movement of the tracking target. On the other hand, in this modified example, the subject selected according to the type of the tracking target is focused. For example, the control device 110A stores information and images about the size, shape, and the like of the balls used in each ball game in advance in a memory (not shown). Then, for example, the determination unit 115 calculates the size of the subject designated as the tracking subject from the focal length, the defocus amount, the size occupied on the image pickup surface of the image pickup element 103, and the like, or the ball stored in advance. The type of ball is judged by comparing it with the image of. The determination unit 115 outputs to the main subject detection unit 112 a setting signal instructing the main subject detection unit 112 what kind of subject is set as the main subject according to the type of the determined ball.

When the determination unit 115 determines that the ball designated as the tracking subject is a soccer ball, for example, the face closest to the tracking target within the face detection range 141 is set as the main subject, as in the first embodiment described above. It sets and outputs a setting signal instructing the main subject detection unit 112 to set the tracking target as the main subject if the face does not exist in the face detection range 141.
Upon receiving this setting signal, the main subject detection unit 112 sets the face closest to the tracking target within the face detection range 141 as the main subject, and if there is no face within the face detection range 141, sets the tracking target as the main subject. Set.

Further, when the determination unit 115 determines that the ball is a tennis ball, for example, the determination unit 115 predicts the movement of the tracking target and sets the face existing at the predicted movement destination of the tracking target as the main subject. The setting signal instructed to 112 is output.
Upon receiving this setting signal, the main subject detection unit 112 predicts the movement of the tracking target, and sets the face existing at the movement destination of the predicted tracking target as the main subject.

Further, for example, the determination unit 115 may output different setting signals depending on the type of tracking target, such as whether the ball is a ball, for example, a Frisbee disc, instead of the type of the ball. For example, the control device 110A stores in advance information and images about the size and shape of various balls, as well as information and images about the size and shape of the Frisbee disc, in a memory (not shown). Then, for example, the determination unit 115 estimates the size of the subject designated as the tracking subject from the focal length, the amount of defocus, the size occupied on the image pickup surface of the image sensor 103, and the like, or obtains an image stored in advance. By comparing, it is possible to determine what kind of ball the tracking subject is or whether it is a Frisbee disc. When the determination unit 115 determines the type of the ball of the tracking subject, the determination unit 115 outputs the above-mentioned setting signal to the main subject detection unit 112 according to the determined ball type. Further, when the determination unit 115 determines that the tracking subject is a Frisbee disc, it outputs a setting signal as described below to the main subject detection unit 112, for example.
(1) Predict the flight trajectory of the disc.
(2) If the face exists within the face detection range 141, the movement locus of the face is predicted.
(3) When it is predicted that the flight locus of the disc and the movement locus of the face substantially intersect, the face predicted to intersect earliest is set as the main subject.
(4) In (2) above, if the face does not exist within the face detection range 141, or if it is predicted that the flight locus of the disc and the movement locus of the face do not intersect, the tracking target is the main subject. Set to.
Upon receiving such a setting signal, the main subject detection unit 112 sets the face of the person who is trying to catch the thrown Frisbee disc as the main subject in, for example, in a scene of shooting a person playing with the Frisbee disc. As a result, even if there are a plurality of people trying to catch the disc, the face of the person who is likely to catch the disc can be set as the main subject.
The main subject detection unit 112 may be configured to detect not only the face of a person but also the face of an animal such as the face of a dog. This makes it possible to set the face of a dog that is likely to catch the disc as the main subject among the faces of a plurality of dogs playing with the Frisbee disc.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. In addition, the above-described embodiments and modifications may be combined as appropriate. Other aspects conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

100, 100A; camera, 102; photographing lens, 103; image sensor, 110, 110A; control device, 111; recognition unit, 112; main subject detection unit, 114; focus adjustment unit

Claims (7)

An imaging unit that repeatedly outputs image signals and
A recognition unit that repeatedly recognizes the first subject detected based on the image signal,
A selection unit that selects a second subject that is different from the first subject and has a variable distance to the first subject based on the distance to the first subject.
A focus detection unit that detects the focus based on the second subject,
An imaging device comprising. In the imaging device according to claim 1,
When the second subject exists within the first range with respect to the first subject, the focus detection unit detects the focus based on the second subject, and the second subject is the first subject. An imaging device that detects the focus based on the first subject when it is out of range. In the imaging apparatus according to claim 1 or 2,
The selection unit is an imaging device that selects the second subject according to the moving direction of the first subject. In the imaging device according to claim 3,
The selection unit is an imaging device that selects the second subject that moves in the direction in which the first subject moves. In the imaging device according to claim 3,
The selection unit is an imaging device that selects the second subject existing in the direction in which the first subject moves. In the imaging apparatus according to any one of claims 1 to 5,
The selection unit is an imaging device that selects a second subject close to the first subject among the plurality of second subjects when a plurality of the second subjects are present. In the imaging apparatus according to any one of claims 1 to 6,
A judgment unit for determining the type of the first subject is provided.
The selection unit is an imaging device that selects the second subject according to the type of the first subject.

Images