JP2009229569A - Autofocus system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an autofocus system that can set a predetermined subject as an automatically tracking subject as a target of AF while reproducing the zooming and focusing to preset position. <P>SOLUTION: In the tracking device 16, the zoom and focus preset positions are registered according to pattern data of a face of a person set as the tracking target. When a shot switch 114b is turned on, the zooming and the focusing are set to the preset position and also the face image of the person whose pattern data is registered is set to the tracking target which is tracked as the target of AF. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an autofocus system, and in particular, to focus while tracking a desired object (person) by automatically moving a position (range) targeted for autofocus (AF) within a shooting range of a camera. It can be related to autofocus system.

  Conventionally, when shooting a scene where the subject moves rapidly, such as soccer, figure skating, or horse racing, for example, with a TV camera, the auto focus (AF) area automatically follows the subject's movement to focus on the subject. Tracking systems are known (see, for example, Patent Document 1).

  In an imaging system that converts an image of a subject into an electrical signal (image signal) by an imaging device (CCD or the like) like a television camera, a contrast method is generally employed as an autofocus (AF) method. The contrast method is a method of automatically focusing to the best focus state by detecting the contrast of the subject image from the image signal captured by the image sensor and controlling the focus of the photographic lens so that the contrast becomes the highest. is there.

  In AF such as a contrast method, the entire subject within the shooting range of the camera is not targeted for AF, but only the subject within a part of the shooting range is often targeted. In contrast-based AF, an image signal in a range targeted for AF is extracted from image signals captured by the entire imaging area of the image sensor, and focus is performed so that the contrast of the image signal in the extracted range is the highest. By controlling this, the target of AF is limited to only a certain range of subjects. In this specification, the range of the subject to be AF in the shooting range of the camera or the image range of the subject to be AF in the shot image of the camera is referred to as an AF area, and the outline of the AF area is shown. The frame is called an AF frame.

In a lens system for broadcasting, for example, a cameraman performs pan / tilt (P / T) and zoom operations so that a fast-moving subject such as a figure skate is placed on the screen. In such a situation, the AF area is moved. Is very difficult. In such a case, the AF area automatic tracking system as described above is effective.
JP 2006-267221 A

  However, if there are multiple moving subjects within the shooting range, for example, when shooting a soccer game, simple pattern recognition recognizes a specific subject and tracks that subject all the time. There is a problem that the object to be tracked may be blurred because an object to be tracked may change because the tracking target cannot be continued.

  In recent years, the face recognition technology installed in digital cameras is effective for extracting a person from the shooting range, but it does not recognize a specific person, and there are multiple persons in the frame. In this case, there is a problem that if a face is simply an AF frame, a specific person cannot be recognized, so that the target of AF may be switched to another person.

  Therefore, it is effective to identify and recognize a face (object) of a specific individual by using a face (object) authentication technology currently used for the purpose of security.

  Conventionally, there is known a preset shot function in which preset positions of zoom and focus are registered in advance, and a zoom and focus set to an arbitrary position are moved to a preset position by a predetermined reproduction instruction operation. When this preset shot function is used, the subject is often determined in advance, and it is preferable that the subject to be automatically tracked is set when the zoom or focus is reproduced to the preset position.

  The present invention has been made in view of such circumstances, and provides an autofocus system in which a subject to be automatically tracked is set to a predetermined subject along with reproduction to a preset position of zoom and focus. The purpose is to do.

  In order to achieve the object, an autofocus system according to claim 1 includes an imaging unit that captures a subject image formed by an optical system, and a predetermined AF among captured images captured by the imaging unit. Registered by the pattern registration unit, an autofocus unit that adjusts the focus of the optical system so as to focus on a subject in the area, a pattern registration unit that registers pattern data for specifying a subject to be tracked, and the pattern registration unit. Preset position registration means for registering at least one preset position of zoom and focus of the optical system in association with the subject to be tracked, and automatic tracking of the subject whose pattern data is registered by the pattern registration means. Is instructed, at least one of zoom and focus of the optical system is The preset position registered by the reset position registration means is set, and the subject to be tracked is selected from the subjects within the image range imaged by the imaging means based on the pattern data registered in the pattern registration means. The AF target area is moved in accordance with the movement of the tracking target detection unit to be detected and the tracking target subject detected by the tracking target detection unit, and the auto focusing unit focuses on the tracking target subject. And tracking means for performing focus adjustment as described above.

  The autofocus system according to a second aspect is the invention according to the first aspect, wherein the subject to be tracked is a face of a specific person.

  According to the present invention, a subject to be automatically tracked is set to a predetermined subject along with reproduction to a preset position of zoom or focus. Therefore, the operator can perform such processing with a simple operation.

  Hereinafter, an autofocus system according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the overall configuration of an imaging system to which an autofocus system according to the present invention is applied. The image pickup system shown in FIG. 1 is an image pickup system used for shooting with, for example, a broadcast television camera. In the figure, a camera head 10 with a replaceable lens, and a shooting lens (optical) mounted on the lens mount of the camera head 10 are shown. A lens device 12 having a system), a tracking device 16 including an operation unit 14, and the like are shown.

  The camera head 10 is equipped with an image sensor (for example, a CCD), a required signal processing circuit, and the like, and an image formed by the photographing lens of the lens device 12 is photoelectrically converted by the image sensor and then subjected to signal processing. Necessary signal processing is performed by the circuit. A video signal in a predetermined format generated by the signal processing circuit is output from the video signal output terminal of the camera head 10 to an external device.

  In addition, a viewfinder (monitor) 18 is set in the camera head 10, and a real-time image (captured image) or the like currently captured by the image sensor of the camera head 10 is confirmed on the viewfinder 18. It is supposed to be displayed for. That is, the camera head 10 outputs a video signal of a captured video currently captured by the imaging device, and the video signal is input to the viewfinder 18 via a video synthesis unit 36 described later of the tracking device 16, It is displayed on the screen of the viewfinder 18 as a real-time video image. The viewfinder 18 may be set or arranged at an arbitrary place other than the camera head 10.

  The lens device 12 includes a photographing lens (optical system) (not shown) attached to the lens mount of the camera head 10, and an object is imaged on the imaging surface of the imaging element of the camera head 10 by the photographing lens. The photographing lens is provided with movable parts for adjusting photographing conditions such as a focus lens group, a zoom lens group, and a diaphragm as its constituent parts. These movable parts are electrically driven by a motor (servo mechanism) (not shown). It is designed to be driven. For example, the focus lens group and zoom lens group move in the optical axis direction, and the focus (subject distance) adjustment is performed by moving the focus lens group, and the focal length (zoom magnification) adjustment is performed by moving the zoom lens group. Is done. It should be noted that in a system related to autofocus only control, it is sufficient that at least the focus lens group can be driven electrically, and other movable parts may be driven only manually. Further, when the predetermined movable part is electrically driven in accordance with the operation of the operator, it is based on a control signal output in accordance with the operation of the operator from an operation unit (not shown) (such as an operation unit of a controller connected to the lens device 12). Thus, the operation of the movable part is controlled, but details are omitted.

  As shown in the figure, the lens device 12 is equipped with a lens CPU 20 that performs overall control of the lens device 12, an AF processing unit 22 that executes autofocus (AF) processing, an AF imaging circuit 24, and the like. Yes. The AF image pickup circuit 24 is disposed in the lens device 12 to acquire a video signal for AF processing, and outputs an image pickup device (CCD or the like) or an output signal of the image pickup device as a video signal of a predetermined format. Etc. The image pickup element of the AF image pickup circuit 24 is referred to as an AF image pickup element. The video signal output from the AF imaging circuit 24 is a luminance signal. On the imaging surface of the AF imaging element, subject light branched from the subject light incident on the imaging element of the camera head 10 is imaged by a half mirror or the like disposed on the optical path of the photographing lens. The shooting range and subject distance (distance of the subject in focus) with respect to the imaging area of the AF imaging element are configured to match the shooting range and subject distance with respect to the imaging area of the imaging element of the camera head 10 and are used for AF. The subject image captured by the image sensor is substantially the same as the subject image captured by the image sensor of the camera head 10. Note that it is not necessary for the shooting ranges of the two to be completely the same. For example, the shooting range of the AF imaging device may be a larger range including the shooting range of the imaging device of the camera head 10.

  The AF processing unit 22 acquires a video signal from the AF imaging circuit 24, and calculates a focus evaluation value indicating the level of contrast of the subject image based on the video signal. For example, a high frequency component signal of a video signal obtained from an AF image sensor is extracted by a high-pass filter, and an AF area that is a target range of AF is extracted from the high frequency component signal. Signals in a range corresponding to the subject range to be matched are integrated for one screen (one frame). The integrated value obtained for each screen in this manner indicates the level of contrast of the subject image, and is given to the lens CPU 20 as a focus evaluation value. The AF area range is designated by the lens CPU 20 as will be described later.

  As will be described in detail later, the lens CPU 20 acquires AF frame information (AF frame information) indicating the range (contour) of the AF area from the operation unit 14 of the tracking device 16 and designates the AF frame specified by the AF frame information. This range is designated to the AF processing unit 22 as an AF area. FIG. 2 shows an example of an AF area (AF frame). A square AF area is set within the shooting range, and its outline corresponds to the AF frame. The lens CPU 20 designates such an AF area as the AF processing unit 22, and acquires a focus evaluation value obtained from an image (video signal) in the AF area from the AF processing unit 22. In this way, every time a video signal for one screen is acquired from the AF imaging circuit 24 (each time the focus evaluation value is obtained by the AF processing unit 22), the focus evaluation value is acquired from the AF processing unit 22, and The focus lens group is controlled so that the acquired focus evaluation value is maximum (maximum), that is, the contrast of the subject image in the AF area is maximum. For example, a hill-climbing method is generally known as a control method for a focus lens group based on a focus evaluation value, and the focus evaluation value starts to decrease by moving the focus lens group in a direction in which the focus evaluation value increases. Is detected, the focus lens group is set at that position. Thereby, the subject in the AF area is automatically focused.

  In the present embodiment, as described above, the AF processing unit 22 acquires the video signal for detecting the contrast of the subject image from the AF imaging circuit 24 mounted on the lens device 12. The video signal obtained by photographing with the image sensor of the camera head 10 may be acquired from the camera head 10. Also, the AF method is not limited to the contrast method as described above.

  The tracking device 16 is a device for tracking a subject on the screen so that a predetermined subject (a person's face or the like) is within the AF area, and includes an operation unit 14, an image processing unit 26, and an image input unit 28. A tracking processing unit 30, a pattern data storage unit 32, a frame information generation unit 34, a video composition unit 36, and the like.

In the tracking device 16, as basic processing, an AF area setting process for setting (determining) an AF area (AF frame) used in the AF processing unit 22 of the lens device 12, and an automatic tracking function for automatically tracking an object to be tracked. Tracking processing and recognition (detection) of a person's face image in the captured video, person identification processing for identifying the person of the recognized face image, and pattern registration for registering the face image pattern of the person identified by the person identification processing Processing, frame display processing for displaying the AF frame and tracking frame (face frame) together with the captured image on the viewfinder 18 are performed, and these processing are performed by the tracking processing unit 30 in an integrated manner.

The AF area setting process is a process of setting an AF area (AF frame) range (position, size, shape) and transmitting AF frame information indicating the set AF area range to the lens CPU 20 of the lens device 12. Thereby, the AF area used in the AF processing unit 22 is set by the lens CPU 20 in accordance with the AF frame information.

  As will be described later, when automatic tracking is executed (when automatic tracking is turned on), automatic tracking processing is executed by the tracking processing unit 30, and the position of the AF area is changed according to the position of the tracking target to be AF target. Is set. Then, AF frame information indicating the range of the AF area set by the tracking processing unit 30 is transmitted to the lens CPU 20 of the lens device 12 via the operation unit 14.

  On the other hand, when automatic tracking is not executed (when automatic tracking is turned off), the range of the AF area is fixed unless the operator performs an operation to change the range of the AF area using the operation unit 14. For example, in the initial state, the AF area is set as a square area of a predetermined size at the center of the screen (shooting range).

  When the operator operates the operation member for setting the AF area of the operation unit 14, the range of the AF area is changed according to the manual operation. However, in this embodiment, it is assumed that the operator can change only the position of the AF area. The operation unit 14 is provided with a trackball 14a. When the operator operates the trackball 14a, only the movement direction and the movement amount corresponding to the operation direction and the operation amount with respect to the position of the current AF area. The position of the moved new AF area is obtained. Then, AF frame information indicating the range of the AF area changed to that position is transmitted to the lens CPU 20 of the lens apparatus 12, and the position of the AF area is set according to the operation of the trackball 14a.

  Even when the automatic tracking is on, if the manual operation for changing the range of the AF area is performed by the operation unit 14, the change of the AF area by the manual operation is performed in the tracking processing unit 30. This is prioritized over the AF area change by automatic tracking.

  Although not shown, the operation unit 14 requires AF frame information and the like based on output signals from the detection circuits in addition to detection circuits (sensors, switches, etc.) that detect the operation of each operation member. Although a simple processing unit that generates information and outputs the information to the lens device 12 and the tracking processing unit 30 is provided, such a processing unit is not necessarily provided. That is, all processing performed by the processing unit of the operation unit 14 is performed by the tracking processing unit 30 that has acquired the output of the detection circuit of the operation unit 14, and information necessary for the lens CPU 20 from the tracking processing unit 30 (AF frame information and the like). May be transmitted directly. Alternatively, each of the tracking processing unit 30 and the lens CPU 20 may directly acquire an output signal necessary for its own processing from the detection circuit of the operation unit 14.

  Furthermore, although the operation unit 14 is a component included in the tracking device 16, it may be a separate device from the tracking device 16 and connected to the tracking device 16 by a cable or the like.

  The automatic tracking process is a process of automatically tracking an object set as a tracking target (described later) on the screen when the automatic tracking is on. In the stage before starting the tracking process, the image area of the object to be tracked or the image area including the image of the object to be tracked is determined. For example, the image of the person detected in the person specifying process to be described later is determined. A face image area, an AF area image area, and the like correspond to this. Since the automatic tracking method is well known, detailed description thereof is omitted, but the tracking processing unit 30 sequentially captures frame images (shooting) by the AF imaging circuit 24 of the lens device 12 when executing the automatic tracking process. Image) are sequentially taken into the image processing unit 26 via the AF processing unit 22 and the image input unit 28 of the tracking device 16. Then, the image area of the object to be tracked is sequentially detected from the initial frame image in which the automatic tracking is started, and the position in the image is tracked. Even if there are a plurality of tracking targets in the image, each of them can be tracked.

  The person specifying process is a process of recognizing (detecting) a person's face (face image) included in the photographed image by face recognition processing and specifying an individual of the recognized face. In the present embodiment, there are an area automatic tracking mode and a face recognition automatic tracking mode as types (modes) of automatic tracking. In the area automatic tracking mode, an arbitrary object in the currently set AF area is detected by AF. The face recognition automatic tracking mode is set as a tracking target to be tracked as an AF target in the face recognition automatic tracking mode. The area automatic tracking mode is executed when an area automatic tracking switch (not shown) of the operation unit 14 is turned on, and the face recognition automatic tracking mode is executed when the face recognition automatic tracking switch of the operation unit 14 is turned on. Is done.

  The person specifying process is a process performed in the face recognition automatic tracking mode. When the face recognition automatic tracking mode is turned on, the tracking processing unit 30 sequentially captures frame images taken by the AF imaging circuit 24 of the lens device 12. Is started to be sequentially taken into the image processing unit 26 via the AF processing unit 22 and the image input unit 28 of the tracking device 16. Then, a known face recognition process is performed on the initial frame image captured by the image processing unit 26 to detect a human face (face image) included in the frame image. For example, a face image is detected based on the general shape and color of a person's face.

  When a face image is detected by the face recognition process, an individual of the detected face image is specified by a well-known face authentication process. For example, if the pattern data of an individual face to be specified is registered in the pattern data storage unit 32 in advance, and the face image detected by the face recognition process matches the pattern data registered in the pattern data storage unit 32. It is specified as a face person in the pattern data. In the present embodiment, the pattern data storage unit 32 can register face pattern data of a plurality of persons. If the pattern data matches any person's pattern data, the person of the face image is specified. The

  As described above, when a face image of a specific person is specified in the captured image by the person specifying process, the face image is set as a tracking target to be automatically tracked as an AF target. When face images of a plurality of persons are specified, a face image to be automatically tracked is selected as that of any one person as an AF target. For example, when pattern data of a plurality of persons is stored in the pattern data storage unit 32, when two or more face images of those persons are present in the photographed video at the same time, which person's face image is used for AF. The priority order of selection as a target may be determined in advance, or the operator may designate it by a predetermined operation.

  The pattern registration process is a process of registering the pattern of the individual face image specified by the person specifying process as pattern data in advance as described above, and at least one person before the operator executes the face recognition automatic tracking mode. In order to register the face image pattern, the pattern registration mode switch of the operation unit 14 is turned on to turn on the pattern registration mode.

  When the pattern registration mode is turned on, the tracking processing unit 30 performs image processing on the frame images sequentially captured by the AF imaging circuit 24 of the lens device 12 via the AF processing unit 22 and the image input unit 28 of the tracking device 16. The process of sequentially loading into the unit 26 is started. Then, a known face recognition process is performed on the initial frame image captured by the image processing unit 26 to detect a human face image included in the frame image. The detected face image can be recognized by the operator through a frame (face frame) display that is superimposed on the photographed video together with the photographed video on the viewfinder 18 by a frame display process described later.

  When there are a plurality of detected face images, the operator selects a face image to be registered. For example, in the initial stage, any face image is selected. At this time, the viewfinder 18 can distinguish the selected face image from other face images not selected due to the difference in the color of the face frame. For example, when the operator operates the trackball 14a of the operation unit 14 in the direction of the face image to be registered with respect to the currently selected face image, the face image to be selected may be changed to the face image to be registered. it can. Note that a desired face image may be selected by changing the shooting direction of the camera and aligning the face image to be selected with the AF area (overlaying the face frame to be selected on the AF area). In addition, since it is possible to change the shooting angle of view at any time other than the actual shooting, the pan / tilt operation and zoom operation of the camera are performed, and only the desired face image is included in the shot image. By doing so, only a desired face image may be selected. If the pattern registration mode is turned on while only the face image of the person desired to be registered is photographed from the beginning, the above-described operation of selecting the face image to be registered becomes unnecessary.

  When the face image of the person to be registered is selected in this way, the tracking processing unit 30 samples the pattern data of the face image. At this time, it is possible to register a plurality of pattern data for one person to be registered, and the operator captures the face image of the person to be registered and changes the direction of the face. For example, when it is determined that the pattern data of the face image when the face is photographed from a necessary direction can be sampled, the sampling is terminated by a predetermined operation. As a result, pattern data when the face image of the person to be registered is photographed in three dimensions can be obtained. The pattern data sampled in this way is registered in the pattern data storage unit 32 together with a name for identifying the person.

  The operation unit 14 is provided with a plurality of push button type switches 14b shown in FIG. 1. For example, when the pattern registration mode is turned on, any one of the plurality of switches 14b is turned on. By designating a switch to be associated with a person to register pattern data, it is possible to register pattern data by associating one person with the switch that is turned on. In the face recognition tracking mode, it is possible to select which person's face to be tracked as an AF target among the persons whose pattern data is registered by turning on a switch corresponding to the person. Yes. The registered person may be automatically associated with the switch 14b in the order in which the pattern data is registered.

  When registering the pattern data, information such as the name of the registered person can be registered, and the data is also registered in the pattern data storage unit 32 together with the pattern data. For the input of a name or the like, a device equipped with a character input device such as a personal computer may be connected to the tracking device 16 to capture information input by the device, or a character input device is provided in the operation unit 14. Alternatively, it may be possible to connect and input from the character input device.

  The frame display process is a process of displaying a frame image such as an AF frame or a tracking frame (face frame) on the viewfinder 18 so as to be superimposed on the captured image. When the tracking processing unit 30 determines the position and size of the AF frame and the rectangular frame surrounding the tracking target on the captured video, the tracking processing unit 30 gives the information to the frame information generation unit 34 to generate images of the frames. . Not only the image of the frame but also the color of the frame and character information associated with the frame are generated by the instruction of the tracking processing unit 30. The frame image generated by the frame information generation unit 34 is given to the video composition unit 36. The video composition unit 36 receives a video signal of a photographed video currently captured by the imaging element of the camera head 10, and the video composition unit 36 receives the frame of the captured video and the frame given from the frame information generation unit 34. A video signal obtained by synthesizing the images is generated and output to the viewfinder 18. As a result, images such as an AF frame and a tracking frame (face frame) are displayed at corresponding positions on the viewfinder 18 so as to be superimposed on the captured video.

  For example, a name such as “FUJI # 1” is displayed on the face frame, and a display indicating that AF driving is being performed at the bottom of the screen.

  Next, a basic processing procedure (first embodiment) of the autofocus system shown in FIG. 1 will be described with reference to the flowchart of FIG.

  When the power is turned on, the lens CPU 20 of the lens device 12 and the tracking processing unit 30 of the tracking device 16 first perform initial setting (step S100). Subsequently, the tracking processing unit 30 determines whether or not the face recognition automatic tracking switch is turned on (step S102). When the face recognition automatic tracking switch is turned on, processing in the face recognition automatic tracking mode is executed. The processing procedure at this time will be described later.

  If it is determined as NO in step S102, it is subsequently determined whether or not the pattern registration mode switch is turned on (step S104). When the pattern registration mode switch is turned on, pattern data registration processing (pattern registration processing) is performed. The processing procedure at this time will be described later.

  If it is determined as NO in step S104, an area manual setting process is executed (step S106). That is, when the operator operates the trackball 14a of the operation unit 14, the AF area range is set according to the operation.

  Next, it is determined whether or not the area automatic tracking switch is turned on (step S108). When it determines with YES, the tracking process part 30 starts the automatic tracking process in area automatic tracking mode as mentioned above (step S110), and transfers to step S300. That is, the tracking processing unit 30 sets the subject in the AF area when the area automatic tracking switch is turned on as a tracking target, and changes the position of the AF area according to the position of the tracking target. If it is determined as NO in step S108, the automatic tracking process is turned off (stopped) (step S112), and the process proceeds to step S300.

  Subsequently, in step S300, the lens CPU 20 determines whether AF is on / off. This determination process is determined based on ON / OFF of an AF switch that instructs start and end of AF (not shown).

  If YES is determined in step S300, AF processing is performed (step S132). Accordingly, the focus lens is automatically controlled so that the subject in the AF area set above is in focus.

  If NO is determined in step S300, focus control is performed according to the focus operation manually performed by the operator (step S134). When the process of step S132 or step S134 is completed, the process returns to step S102.

  On the other hand, if YES in step S104, that is, if the pattern registration mode switch is turned on and it is determined that the pattern registration mode is selected, the tracking processing unit 39 executes the pattern registration processing according to the following procedure.

  First, the tracking processing unit 30 of the tracking device 16 starts capturing the frame images sequentially captured by the AF imaging circuit 24 of the lens device 12 into the image processing unit 26 (step S114). Subsequently, the tracking processing unit 30 performs face recognition processing on the image captured by the image processing unit 26, and recognizes (detects) a human face image included in the image (step S116). Even when a plurality of face images exist in the image, the process proceeds to the next step S118 when one face image is recognized. Further, when detecting the face image, the tracking processing unit 30 superimposes and displays the face frame of the recognized face image on the captured image displayed on the viewfinder 18 by the frame display process.

  Next, the tracking processing unit 30 starts the automatic tracking process with the face image recognized in step S116 as a tracking target (step 118). The face frame of the viewfinder 18 is also changed in accordance with the movement of the face image.

  Next, it is determined whether or not to end the face recognition process (step S120). For example, if there is no newly detected face image in the face recognition process in step S116, the face recognition process is terminated.

  If it is determined as NO in step S120, the process returns to step S116 and the face recognition process is repeated.

  If it is determined as YES in step S120, the face image that the operator desires to register is selected from the face images recognized in step S116 (step S122).

  Next, the tracking processing unit 30 samples the pattern data of the face image selected as the face image to be registered (step S124). The sampled pattern data is registered (stored) in the database of the pattern data storage unit 32.

  Subsequently, an automatic tracking process for tracking the selected face image is started (step S126). Then, it is determined whether or not to end sampling (step S128). Whether or not to end the sampling can be instructed by an operator using a predetermined switch, for example.

  If it is determined NO, the process returns to step S124 and sampling is continued. That is, pattern data sampling is repeated until pattern data is obtained when the face of the person to be registered is photographed from as many directions (angles) as possible. Therefore, a plurality of pattern data is registered in the database of the pattern data storage unit 32 for one person.

  If YES is determined in step S128, a name (name, No., etc.) for identifying the person of the registered face image is registered (step S130). If the name is registered, when a registered face image is detected in the face recognition tracking mode, the face frame of the face image is displayed on the display screen of the viewfinder 18 as shown in FIG. "Can be displayed.

  When the above pattern registration process is completed, the tracking processing unit 30 switches the pattern registration mode off (step S132). Then, an AF area is set so as to include the face image in which the pattern data is registered, and an automatic tracking process for automatically tracking the face image as an AF target is started (step S134). That is, the registered face image is automatically tracked as a tracking target, and the AF area is changed according to the movement of the face image. The face frame may be an AF area. After the start of the automatic tracking process in step S134, the process proceeds to step S300.

  If YES in step S102, that is, if it is determined that the face recognition automatic tracking switch is turned on, the processing in the face recognition automatic tracking mode is executed in the following procedure.

  First, the tracking processing unit 30 of the tracking device 16 starts capturing the frame images sequentially captured by the AF imaging circuit 24 of the lens device 12 into the image processing unit 26 (step S136). Subsequently, the tracking processing unit 30 performs face recognition processing on the image captured by the image processing unit 26, and recognizes (detects) a human face image included in the image (step S138). Even when a plurality of face images exist in the image, the process proceeds to the next step S140 when one face image is recognized. Further, when detecting the face image, the tracking processing unit 30 superimposes and displays the face frame of the recognized face image on the captured image displayed on the viewfinder 18 by the frame display process.

  Next, the face image pattern of the person recognized in step 138 is sampled, and the same pattern is searched from the pattern data stored in the database of the pattern data storage unit 32 (step S140).

  Subsequently, the viewfinder 18 determines whether or not to display a name for the detected face image (step S142). Whether or not the name is displayed can be specified by the operator by turning on / off a predetermined switch, for example.

  If NO is determined in step S142, the process proceeds to step S150. If YES is determined, the face detected in the pattern data registered in the database in the search in step S140 is subsequently performed. It is determined whether or not the same pattern as the image has been found (step S144).

  If it is determined as YES in step S144, the name registered corresponding to the pattern data, for example, “FUJI # 1” is displayed on the viewfinder 18 in association with the detected face image. (Step S146). If NO is determined in step S144, the character “??” is displayed on the viewfinder 18 in association with the detected face image (step 148). Thereby, the operator can easily confirm whether or not each face image displayed on the viewfinder 18 is registered as a pattern.

  When the process of step S146 or step S148 ends, or when it is determined NO in step S142, an automatic tracking process for automatically tracking the face image detected in step S138 is started (step S150). At this time, the position of the face frame superimposed on the captured image in the viewfinder 18 is also moved in accordance with the position of the face image detected by the automatic tracking process.

  When the processing in step S150 is completed, it is next determined whether or not the above processing has been completed for all face images included in the image (step S152). If a new face image is not detected in the face recognition process in step S138, it is determined as YES in the determination process in step S152. Otherwise, it is determined as NO and the process returns to step S138, and the face recognition process is performed. Execute repeatedly from.

  If it is determined as YES in step S152, next, it is determined whether or not one or more face images in which pattern data is registered are found among the face images detected in the processes of steps S136 to S152. Determination is made (step S154).

  If it is determined as NO in step S154, the AF process in the lens device 12 is turned off (step S156), and the process proceeds to step S300.

  On the other hand, if it is determined as YES in step S154, it is determined whether or not the preset is turned on (step S158). That is, it is determined whether one of the switches 14b of the operation unit 14 associated with the person for which the pattern data is registered is turned on.

  If NO is determined in step S158, a face image to be subjected to AF is selected according to a predetermined rule (step S160). For example, if a number is assigned to the switch 14b of the operation unit 14 and a plurality of face images of a person registered with pattern data are detected, the switch 14b with which the number is the smallest is assigned to the AF switch. set to target. When only one face image of a person with registered pattern data is detected, the face image is selected as an AF target.

  Next, in step S160, it is determined whether or not the face image selected as the target of AF is different from the person of the face image that has been the target of AF (step S162). At this time, if it is determined as YES, the face image to be subjected to AF is changed to the selected face image (step S164). That is, the face frame of the selected face image or a frame having a predetermined size including the face image is set as the AF frame (AF area). Then, the AF process in the lens device 12 is turned on (step S166), and the process proceeds to step S300.

  If NO is determined in step S162, the process proceeds to step S166 without changing the AF target.

  If YES is determined in step S158, that is, if any one of the switches 14b of the operation unit 14 is turned on, it is determined whether or not a human face image corresponding to the switch exists (step S168). ). If YES is determined, the face image of the person is selected as an AF target (step S170), and the process proceeds to step S162.

  If it is determined as NO in step S168, it is determined that no face image is subject to AF (step S172). After the AF processing in the lens device 12 is turned off (step S174), the process proceeds to step S300. .

  FIG. 4 shows an example of an image displayed on the screen of the viewfinder 118 as a result of performing the face recognition automatic tracking mode. FIG. 4 shows an image in the case of automatically tracking a video of a soccer game. Each face frame is displayed in light blue, for example, with respect to the recognized face image in the screen.

  Moreover, the name (name, No., etc.) of the face image in which pattern data is registered is displayed near the face frame. In the case of FIG. 4, the face images of three people are recognized, and the face images of two of them are face images of persons whose pattern data is registered. Respectively, “XXXX # 1” and “YYYY # 2” are displayed. Also, since the face image of a person who looks like a cameraman at the back of the screen has no pattern data registered, the name is displayed as “#?” For that face image, and these persons are not registered. Is understood by the operator.

  Note that information other than “#?” May be displayed as long as it is information indicating that a face image for which pattern data is not registered is not registered. In addition, the operator can select whether or not to display the name by using a predetermined switch. When the name display is turned on as shown in FIG. Therefore, it is possible to make the subject easier to see by turning off the name display (including the display of information indicating that it is not registered).

  Further, when face images of a plurality of persons are detected on the screen, face frames are displayed for all the face images, so it is difficult to understand whether the tracking target to be tracked is set as an AF target. Therefore, the face frame of the face image set as the tracking target to be tracked as an AF target, that is, the AF frame is displayed in, for example, light blue, and the other face frames are displayed in yellow. However, the color used is not limited to this. Further, a face image set as a tracking target to be tracked as an AF target by another method may be distinguishable from other face images.

  Also, instead of using the face frame of the face image set as the tracking target to be tracked as the AF target as the AF frame, an AF frame different from the face frame is set as the tracking target to be tracked as the AF target. It may be set according to the position of the face frame of the face image and displayed on the viewfinder 18.

  Next, another embodiment of an imaging system to which the autofocus system according to the present invention is applied will be described.

  FIG. 5 is a block diagram showing another configuration example of an imaging system to which the autofocus system according to the present invention is applied.

  The imaging system shown in FIG. 5 is substantially the same in configuration as the imaging system shown in FIG. 1 described above, and the same or similar components as those in the imaging system shown in FIG. Omitted. The tracking device 116 shown in the figure is provided with an automatic tracking function as described in the flowchart of FIG. 3 together with a zoom and focus preset function. For example, a device called a shot box having the preset function is automatically used. This corresponds to a mode incorporating a tracking function.

  The preset function, when the zoom and focus positions in the lens device 12 are registered in advance, the zoom and focus positions set at arbitrary positions are reproduced at the pre-registered positions by a predetermined switch operation. It is a function that can be done. In the figure, the configuration for implementing the preset function is omitted, but the operation unit 114 shows a configuration for implementing the preset function.

  The operation unit 114 shown in the figure includes a cross key 114a instead of the trackball 14a as an operation member for manually specifying the movement direction and movement amount of the AF frame, and provides zoom (Z) and focus (F) preset positions. A shot switch 114b for registering and reproducing or registering and selecting a tracking target (person's face image), a confirmation display panel 114c for registering the name of the tracking target, and a registered zoom (Z) And an indicator 114d indicating the preset position of the focus (F), a slide switch 114e for switching the function (mode) of the tracking device 116, a memo switch 114F for instructing registration of the preset position and pattern data of a person's face, etc. is doing.

  In the tracking device 116 according to the present embodiment, the zoom and focus preset mode (Z + F mode), the face recognition automatic tracking mode (OBJ mode), and the combined mode (Z + F + OBJ mode) are switched by switching the setting position of the slide switch 114e. 3) modes can be selected.

  The Z + F mode is a mode for reproducing zoom and focus to a preset position, which is a normal function as a shot box, and the OBJ mode is the face recognition automatic tracking mode described in the first embodiment. This is a mode for automatically tracking a face image of a specific person as an AF target by processing, and the composite mode is a mode in which the Z + F mode and the OBJ mode are integrated, and performs reproduction to a preset position of zoom and focus. This is a mode for automatically tracking a face image of a specific person as an AF target by processing in the face recognition automatic tracking mode. The Z + F mode may be a mode in which only one of the zoom and the focus is reproduced at the preset position.

  The process regarding each mode in the tracking device 116 will be described using the flowchart of FIG. This flowchart shows a part of the processing performed in the tracking device 116, and is a processing that is repeatedly executed together with other processing.

  First, it is determined whether or not the memo switch 114F is turned on (step S500). If it is determined as YES, it is determined whether or not the slide switch 114e is set to the Z + F mode (step S502). If YES is determined in this step, a zoom and focus preset position registration process is executed (step S504). That is, when the operator turns on one of the four shot switches 114b from No. 1 to No. 4 in FIG. 5, the current zoom position and focus position in the lens device 12 are read by the tracking device 116, These zoom position and focus position are registered (stored) as preset positions associated with the shot switches that are turned on. The registered preset position is displayed on an indicator 114d corresponding to the shot switch in which the preset position is registered. Note that the preset position may be registered in association with the unregistered shot switch 114b in order from the first, without the operator turning on the shot switch 114b. When the process of step S504 ends, the process proceeds to step S520.

  If it is determined as NO in step S502, it is determined whether or not the slide switch 114e is set to the OBJ mode (step S506). If “YES” is determined here, a pattern registration process for registering pattern data of the face image of a specific person is executed (step S508). That is, in the first embodiment, the processing shown in steps S114 to S132 in the flowchart of FIG. 3 is executed. At this time, the pattern data of the face image of a different person can be registered in each of the first to fourth shot switches 114e, but the number of the shot switch 114e to be registered is registered in the same manner as in the Z + F mode. The numbered shot switch 114e may be turned on by the operator, or may be registered in order from the first to the unregistered shot switch 114e. A name for identifying a person who has registered pattern data can also be registered by a character input device (not shown), and the name is displayed on the confirmation display panel 114c corresponding to the registered shot switch. When the process of step S508 ends, the process proceeds to step S520.

If NO is determined in step S506, the slide switch 114e is in the Z + F + OBJ mode, and first, zoom and focus preset position registration processing is executed as in step S504 (step S504). S510). Subsequently, a pattern registration process is executed as in step S508 (step S512).
Thereby, zoom and focus preset positions and face image pattern data are registered. These data are registered in association with the shot switch of the number turned on by the operator when registering the preset position. However, the preset position and face image pattern data may be registered in association with the unregistered shot switch 114e in order from the first without the operator turning on the shot switch 114b. The registered preset position is displayed on the indicator 114d, and the name of the registered person is displayed on the confirmation display panel 114c. When the process of step S512 is completed, the process proceeds to step S520.

  If it is determined NO in step S500, or if any of steps S504, S508, and S512 is completed, it is next determined whether any of the shot switches 114e is turned on (step S500). S520).

  When it is determined YES by this determination process, it is determined whether or not the slide switch 114e is set to the Z + F mode (step S522). If YES is determined in this step, a process of reproducing the zoom and focus of the lens device 12 to a preset position registered in association with the shot switch that is turned on is executed (step S524). When the process of step S524 ends, the process of this flowchart ends, and the process proceeds to another process.

  If it is determined as NO in step S522, it is determined whether or not the slide switch 114e is set to the OBJ mode (step S526). If YES is determined here, the face recognition automatic tracking mode processing as shown in step 136 to step S174 in FIG. 3 is executed, and the pattern data registered in association with the shot switch that has been turned on is executed. The matching face image is set as a tracking target to be automatically tracked as an AF target (step S528). Then, AF processing is executed in the lens device 12 (step S530). In step 136 to step S174 in FIG. 3, all face images included in the captured video are detected, and the detected face images are compared with the face images of all persons whose pattern data is registered. In this embodiment, only the face image of the person registered in association with the shot data is detected, and the face image is tracked as an AF target. You only have to set it. When the process of step S530 ends, the process of this flowchart ends, and the process proceeds to another process.

  If it is determined NO in step S526, the slide switch 114e is set to the Z + F + OBJ mode. First, as in step S524, the zoom and focus of the lens device 12 are set to the shot switch that is turned on. A process of playing back to the preset positions registered in association with each other is executed (step S532). Subsequently, the face recognition automatic tracking mode process is executed in the same manner as in step S528, and the face image that matches the pattern data registered in association with the shot switch that is turned on is set as a tracking target to be automatically tracked as an AF target. (Step S534). Then, AF processing is executed in the lens device 12 (step S536). When the process of step S536 ends, the process of this flowchart ends, and the process proceeds to another process.

  5 shows the case where four shot switches 114e are provided (when the number of registrations is four), the number of shot switches 114e is not limited to four.

  In addition, the tracking device 116 in FIG. 5 can select three types of modes of Z + F mode, OBJ mode, and Z + F + OBJ mode by switching the setting position of the slide switch 114e. Of these, only two modes may be selectable. Further, only the Z + F + OBJ mode may be used.

  In the above embodiment, pattern data of a person's face image is registered, and the case where the registered face image can be automatically tracked as an AF target in the face recognition automatic tracking mode has been described. Even for an object (subject), pattern data can be registered in the same manner as a face image, and the object can be automatically tracked. For example, an object of the same type as the registered object is detected from the photographed image based on a shape pattern or a color shared by the object of that type. Then, a registered object can be detected from the detected objects based on the pattern data, and can be automatically tracked as an AF target.

1 is a block diagram illustrating an overall configuration of an imaging system to which an autofocus system according to the present invention is applied. It is explanatory drawing which shows the example of AF area (AF frame). It is a flowchart which shows the basic process sequence of the auto-focus system of this Embodiment. It is explanatory drawing which shows the example of the image displayed on the screen of a viewfinder. It is a block diagram which shows the other structural example of the imaging system to which the autofocus system which concerns on this invention is applied. It is a flowchart which shows the processing content for every mode with which the tracking apparatus of the autofocus system in FIG. 5 is provided.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Camera head, 12 ... Lens apparatus, 14 ... Operation part, 14a ... Trackball, 14b ... Switch, 16 ... Tracking apparatus, 18 ... Viewfinder, 20 ... Lens CPU, 22 ... AF processing part, 24 ... Imaging for AF Circuit, 26 ... Image processing unit, 28 ... Image input unit, 30 ... Tracking processing unit, 32 ... Pattern data storage unit, 34 ... Frame information generation unit, 36 ... Video composition unit

Claims (2)

Imaging means for imaging a subject image formed by an optical system;
Auto focus means for adjusting the focus of the optical system so that a subject in a predetermined AF area out of the captured image captured by the imaging means is in focus;
Pattern registration means for registering pattern data for specifying a subject to be tracked;
Preset position registration means for registering at least one preset position of zoom and focus of the optical system in association with the subject to be tracked registered by the pattern registration means;
When instructed to automatically track the subject whose pattern data is registered by the pattern registration means, at least one of the zoom and focus of the optical system is set to the preset position registered by the preset position registration means, Tracking target detection means for detecting a subject to be tracked from subjects within an image range imaged by the imaging means based on pattern data registered in the pattern registration means;
Tracking that moves the AF area in accordance with the movement of the subject to be tracked detected by the tracking target detection unit, and performs focus adjustment so that the subject to be tracked is in focus by the autofocus unit. Means,
An autofocus system characterized by comprising   2. The autofocus system according to claim 1, wherein the subject to be tracked is a face of a specific person.

Images