JP2010096963A - Auto focus system with af frame auto-tracking function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an auto focus system with an AF frame auto-tracking function which performs both a face detecting processing and a pattern matching processing to track a face image of a person when the face image of the person is tracked as an auto focus target, thereby more reliably tracking the face image. <P>SOLUTION: When a user sets an AF frame such that an object tracked as an auto focus target is in the AF frame (S100), an auto-tracking processing starts to detect a face image from an imported captured image (S110). Then, it is determined whether the face image is detected from a peripheral range of the AF frame (S112). If the face image is detected, the position of the AF frame is changed on the basis of the position of the face image (S118). At the same time, if the face image is not detected from the peripheral range, the pattern matching processing is performed (S114). The position of the AF frame is changed to the position of the object which is the tracking target detected by the pattern matching processing (S118). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an autofocus system having an AF frame auto-tracking function, and particularly to an AF area (AF frame) that is a target range of a subject to be focused by autofocus (AF) as a predetermined subject that moves on a shooting screen. The present invention relates to an autofocus system having an AF frame automatic tracking function for tracking.

  2. Description of the Related Art Broadcast or commercial television camera systems are known that have an autofocus function that automatically focuses on a desired subject. This type of camera system generally employs contrast AF, and detects the height of the contrast of the captured image and controls the focus of the photographic optical system so that the contrast becomes maximum (maximum). Has been done.

  In addition, the target range of the subject to be focused by AF is usually limited to a part of the range called the AF area, not the entire screen of the captured video, and the contrast of the captured image (subject image) in the AF area. By controlling the focus so as to maximize the focus, the subject in the AF area can be focused.

  In the present specification, a frame indicating the outline of the AF area is referred to as an AF frame.

  Further, Patent Document 1 discloses an autofocus provided with an AF frame automatic tracking function that automatically tracks an AF frame for a desired subject moving within a screen of a captured video and keeps focusing on the subject. The system is described. Automatic tracking of the AF frame is performed by detecting the image of the subject to be tracked from the captured image and moving the AF frame to the detected position. As a method for detecting the image of the subject to be tracked, Patent Document 1 employs a method in which the image of the subject to be tracked is stored as a reference pattern, and an image matching the reference pattern is detected from captured images by pattern matching processing. Has been.

As another method, when the subject to be tracked is a human face, a method of detecting a face image by a well-known face detection process as the image of the subject to be tracked can be employed.
JP 2006-258944 A

  By the way, when tracking a face image detected by the face detection process, if the face changes, for example, in the direction of 30 degrees above, below, left, or right from the front, the face image cannot be recognized, and tracking may not be necessary. There is.

  On the other hand, when a predetermined subject is tracked by pattern matching processing, the processing takes time, and it may be difficult to track a fast moving subject.

  The present invention has been made in view of such circumstances, and when tracking a human face (face image) as an AF target, the AF frame can track the face image more reliably than in the past. An object of the present invention is to provide an autofocus system having an automatic tracking function.

  In order to achieve the above object, an autofocus system having an AF frame automatic tracking function according to claim 1 focuses on a subject within a range corresponding to the AF frame in a range of subjects photographed by an imaging unit. As described above, an autofocus unit that controls the focus of a photographic optical system that forms a subject image on the imaging unit, and a face detection processing unit that detects an image of a person's face from the captured images obtained by the imaging unit Using a reference pattern image stored as an image of the subject to be tracked, a pattern matching processing unit for detecting an image of the subject to be tracked by a pattern matching process from a captured image obtained by the imaging unit; When a face image of a person is included in the AF frame, the face detection processing procedure is performed on the captured image obtained by the imaging unit. First AF frame determining means for determining the position of the AF frame based on the position of the face image detected by the step, and the pattern matching processing means for the captured image obtained by the imaging means. Which of the second AF frame determining means for determining the position of the AF frame based on the position of the image of the subject to be tracked, the first AF frame determining means, and the second AF frame determining means Either one is used as a priority AF frame determination means, the AF frame is changed to the position determined by the priority AF frame determination means, and the position of the AF frame cannot be determined by the priority AF frame determination means. When the situation becomes unsatisfactory, there is provided an AF frame changing means for changing the AF frame to a position determined by the other AF frame determining means.

  The autofocus system having an AF frame auto-tracking function according to claim 2 is the invention according to claim 1, wherein the AF frame changing unit is configured to replace the first AF frame determining unit with the priority AF frame determining unit. It is characterized by that.

  According to the present invention, when a person's face (face image) is a tracking target, the position of the AF frame is determined by a processing method that can appropriately detect the tracking target face image among the face detection processing and the pattern matching processing. Since the change is made, the face image is appropriately tracked even when automatic tracking by one process is impossible.

  According to the present invention, when a person's face (face image) is tracked as an AF target, the face image can be tracked more reliably than in the past.

  Hereinafter, an autofocus system having an AF frame automatic tracking function according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the overall configuration of a video camera system to which the present invention is applied. This video camera system is, for example, an imaging system used for shooting with a broadcast television camera.

  As shown in the figure, the video camera system 1 of the present embodiment includes a television camera 10, an image processing unit 18, and the like.

  The television camera 10 includes a camera body 12 composed of an HD camera compatible with a high-definition television [HD (High Definition) TV] system, and a lens device 14 having a photographing lens (optical system) attached to a lens mount of the camera body 12. It consists of.

  The camera body 12 is equipped with an image sensor (for example, a CCD), a required signal processing circuit, and the like, and an image formed by the photographing lens of the lens device 14 is subjected to signal processing after being photoelectrically converted by the image sensor. Necessary signal processing is performed by the circuit and output as an HDTV video signal (HDTV signal) from the video signal output terminal of the camera body 12 to the outside.

  In addition, the camera body 12 includes a viewfinder 13 so that an image currently captured by the television camera 10 is displayed on the viewfinder 13. Also, various information is displayed on the viewfinder 13, and for example, an AF frame that is a focus target range in autofocus described later is displayed superimposed on the captured video. Yes.

  The lens device 14 includes a photographing lens (zoom lens) 24 that is attached to the lens mount of the camera body 12. The subject 16 is imaged on the imaging surface of the imaging element of the camera body 12 by the photographing lens 24. Although not shown, the photographic lens 24 is provided with movable parts for adjusting photographing conditions such as a focus lens group, a zoom lens group, and a diaphragm as constituent parts thereof. Servo mechanism) is electrically driven. For example, the focus lens group and the zoom lens group move in the optical axis direction, the focus (subject distance) adjustment is performed by moving the focus lens group, and the focal length (zoom magnification) is adjusted by moving the zoom lens group. Adjustments are made.

  In a system related to autofocus (AF), it is sufficient that at least the focus lens group can be driven electrically, and the other movable parts may be driven only manually. Further, when the predetermined movable portion is electrically driven according to the operation of the operator, it is based on a control signal output according to the operation of the operator from an operation means (not shown) such as an operation means of a controller connected to the lens device 14. Thus, the operation of the movable part is controlled, but details are omitted.

  The lens device 14 includes an AF unit 26 and a lens CPU (not shown). The lens CPU controls the entire lens device 14. The AF unit 26 is a processing unit for acquiring information necessary for performing AF focus control (automatic focus adjustment). Although not shown, the AF unit 26 includes an AF processing unit, an AF imaging circuit, and the like. Has been. The AF image pickup circuit is disposed in the lens device 14 to acquire an image signal for AF processing. An image pickup device such as an image pickup device CCD (referred to as an image pickup device for AF) or an output signal of the AF image pickup device is used in a predetermined format. And a processing circuit for outputting as a video signal. The video signal output from the AF imaging circuit is a luminance signal.

  On the image pickup surface of the AF image pickup device, subject light branched from the subject light incident on the image pickup device of the camera main body 12 is imaged by a half mirror or the like disposed on the optical path of the taking lens 24. . The shooting range and subject distance (distance of the subject in focus) with respect to the imaging area of the AF imaging element are configured to match the shooting range and subject distance with respect to the imaging area of the imaging element of the camera body 12. The subject image captured by the image sensor matches the subject image captured by the image sensor of the camera body 12. It should be noted that the shooting ranges of the two do not have to coincide completely, and for example, the shooting range of the AF image sensor may be a larger range including the shooting range of the image sensor of the camera body 12.

  The AF processing unit acquires a video signal from the AF imaging circuit, and calculates a focus evaluation value indicating the level of contrast of the subject image based on the video signal. For example, after extracting a high-frequency component signal of a video signal obtained from an AF image sensor with a high-pass filter, a signal in a range corresponding to the AF area targeted for AF is extracted from the high-frequency component signal. Accumulate one screen (one frame) at a time. The integrated value obtained for each screen in this way indicates the level of contrast of the subject image, and is given to the lens CPU as a focus evaluation value.

  The lens CPU obtains AF frame information (AF frame information) indicating the range (contour) of the AF area from the image processing unit 18 as will be described later, and the range within the AF frame specified by the AF frame information is obtained. The AF area is designated as the AF area. Then, the focus evaluation value obtained from the image (video signal) in the AF area is acquired from the AF processing unit.

  In this way, every time a video signal for one screen is acquired from the AF imaging circuit (each time the focus evaluation value is obtained by the AF processing unit), the focus evaluation value is acquired from the AF processing unit, and the acquired focus is obtained. The focus lens group is controlled so that the evaluation value is maximum (maximum), that is, the contrast of the subject image in the AF frame is maximum. For example, a hill-climbing method is generally known as a control method for a focus lens group based on a focus evaluation value, and the focus evaluation value starts to decrease by moving the focus lens group in a direction in which the focus evaluation value increases. Is detected, the focus lens group is set at that position. Thereby, the subject in the AF frame is automatically focused.

  The above-described AF processing unit obtains a video signal from the AF image sensor mounted on the lens device in order to calculate the focus evaluation value, but the video image captured by the image sensor of the camera body 12 is used. It is good also as a structure which acquires a signal from the camera main body 12. FIG. Further, any AF means for automatically focusing on the subject within the AF frame may be used.

  In addition, the camera body 12 and the lens device 14, and the lens device 14 and an image processing unit 18 described later are connected to each other directly or via a cable or the like. As a result, the camera body 12 and the lens device 14 transmit and receive various kinds of information by serial communication via serial communication interfaces (SCI) 12a and 14a provided respectively. In addition, the lens device 14 and the image processing unit 18 transmit and receive various kinds of information to and from the serial communication interfaces 14a and 30a provided in the respective units by serial communication.

  Further, the video output connector of the camera body 12 and the video input connector of the image processing unit 18 are connected via a down converter 28 with a cable. As a result, the HDTV signal output from the video output connector of the camera body 12 is converted (down-converted) into a standard television [NTSC (National Television System Committee)] video signal (SDTV signal) by the down converter 28. Are input to the image processing unit 18.

  The image processing unit 18 is a device for designating the range (position, size, shape (aspect ratio)) of the AF frame when the AF unit 26 of the lens device 14 performs focus control by AF as described above. Thus, AF frame information for designating the range of the AF frame in the photographed image (photographing screen) of the television camera 10 is given from the image processing unit 18 to the lens device 14 by the serial communication. The AF unit 26 sets the range of the AF frame based on the AF frame information from the image processing unit 18 and executes the AF process as described above.

  The image processing unit 18 mainly includes a main board 30, a pattern matching processing calculation board 32, and a face detection processing calculation board 34. The main board 30, the pattern matching processing calculation board 32, and the face detection processing calculation board 34 are equipped with CPUs 38, 50, and 52, and individual calculation processing is performed for each board, and each CPU 38, 50, 52 are connected by a bus or a control line so that data can be exchanged and arithmetic processing can be synchronized with each other.

  The processing in the image processing unit 18 is performed on the main board 30 in an integrated manner. The main board 30 is equipped with an SCI 30a, a decoder (A / D converter) 36, a superimposer 42, a RAM 40, and the like, in addition to the CPU 38 that performs arithmetic processing.

  The SCI 30a is an interface circuit for performing serial communication with the SCI 14a of the lens device 14 as described above, and transmits the AF frame information and the like to the lens device 14.

  The decoder 36 is a circuit for converting a video signal (SDTV signal) of a captured video input to the image processing unit 18 from the down converter 28 into data that can be digitally processed by the image processing unit 18. A / D conversion processing for converting an analog SDTV signal into a video signal of digital data is performed.

  The RAM 40 is a memory that temporarily stores data used in the arithmetic processing of the CPU 38.

  On the other hand, the pattern matching processing calculation board 32 and the face detection processing calculation board 34 are calculation boards for individually performing pattern matching and face detection processing, and temporarily store image data in addition to the CPUs 50 and 52 that perform calculation processing. VRAMs 54, 56 and the like for storing them are provided.

  The image processing unit 18 is provided with an operation unit 20 integrally, or a part or all of the operation members of the operation unit 20 are provided in a separate device from the image processing unit 18 and are connected by a cable or the like. It is connected.

  Although details are omitted, the operation unit 20 includes a position operation member 60 (for example, a joystick or a trackball) for moving the position of the AF frame up and down and left and right manually by the user, and manually operating the size of the AF frame. A size operation member 62 (for example, a knob) for changing the shape of the AF frame, a shape operation member 64 (for example, the knob) for changing the shape of the AF frame by manual operation, a tracking start switch 68 for instructing the start of automatic tracking, automatic A tracking stop switch 70 for instructing the stop of tracking is provided, and the setting states of these operation members 60, 62, 64, 68, 70 are read by the CPU 38 of the main board 30 in the image processing unit 18. ing.

  Note that the LCD 66 with a touch panel is configured so that settings such as a mode related to automatic tracking of the AF frame can be input by a touch operation, and an image displayed on the LCD 66 by the CPU 38 of the image processing unit 18 is appropriately determined according to the setting contents. It can be switched.

  Further, the image displayed on the LCD 66 with the touch panel is given via the superimposer 42 of the main board 30, and the superimposer 42 takes a video image of the TV camera 10 given from the decoder 36. The video signal and the image signal generated by the CPU 38 can be synthesized. As a result, similar to the viewfinder 13 set in the camera body 12, an image obtained by superimposing the captured image captured by the television camera 10 and the image of the currently set AF frame is displayed on the LCD 66 with a touch panel. The user can perform the same operation as that performed by the operation members 60, 62, 64, 68 and 70 by the touch operation on the screen.

  In the image processing unit 18 of the present embodiment having the above-described configuration, several aspects of the process of setting an AF frame to be designated for the lens device 14 (AF frame setting process) will be described in order.

  A first embodiment of the AF frame setting process will be described with reference to the flowchart of FIG. In this embodiment, when a person's face is tracked as an object to be focused by AF, the range of the AF frame is automatically changed so as to match the face to be tracked. The AF frame range refers to the range within the AF frame determined by the position, size, and shape (aspect ratio) of the AF frame. The change of the AF frame range refers to the position, size, and shape of the AF frame. Indicates that at least one element is to be changed.

  First, the user operates the operation unit 20 to designate an initial AF frame so that a subject to be tracked as an AF target is included. At this time, as shown in FIG. 3A, the user designates the AF frame 100 as a subject for tracking the face of a predetermined person so as to include the face image 102, and one person is included in the AF frame. It is assumed that the AF frame is set so as to include only the face and the AF frame including a plurality of faces is not assumed. Even when a part of the face is included in the AF frame, the face is handled as being included in the AF frame.

  Based on such user operation, the CPU 38 of the main board 30 sets an AF frame to be transmitted to the lens device 14 (step S10). Then, the CPU 38 starts an AF frame automatic tracking process (step S12). Note that the automatic tracking process starts, for example, when the tracking start switch 68 of the operation unit 20 is turned on.

  When the automatic tracking process is started, the CPU 38 converts the image data of the captured image for one frame (one frame) to the face detection processing calculation board 34 based on the video signal of the captured video that is converted into digital data by the decoder 36 and output. Capture (step S14).

  Then, face detection processing for detecting a face image (face image) from the photographed image is performed by the face detection processing of the CPU 52 of the face detection processing calculation board 34, and a face image included in the AF frame is detected (step) S16).

  Subsequently, according to the position, size, and shape of the face image included in the AF frame, an AF frame that includes the entire face and does not include other subjects as much as possible (that is, the smallest AF frame that includes the entire face). ) Is obtained, and the AF frame transmitted to the lens device 14 is updated to the obtained AF frame (step S18). When the user designates an AF frame as shown in FIG. 3A, the AF frame 100 having a size suitable for the size of the face image 102 as shown in FIG. .

  When the AF frame is updated in accordance with the face image in this way, the image data in the AF frame is set as the image data of the reference pattern in the pattern matching process described later (step S20). Note that the image data of the reference pattern need not completely match the size of the AF frame, and may be image data in a range larger or smaller than the AF frame.

  When the above processing is completed, the processing from step S22 to step S36 below is repeated to track the face image in the AF frame and update the AF size (size) according to the size of the face image to be tracked. To do. In the following description, the size of the AF frame includes the shape of the AF frame, the aspect ratio of the AF frame may be updated in accordance with the face image, and the aspect ratio of the AF frame is assumed to be constant. Also good.

  First, in step S <b> 22, the CPU 38 captures image data of a captured image for one frame from the decoder 36 into the pattern matching processing calculation board 32 and the face detection processing calculation board 34.

  Then, the CPU 50 of the pattern matching processing calculation board 32 detects an image that matches the reference pattern from the photographed images, and executes a pattern matching process for detecting the position (step S24).

  The CPU 38 determines whether or not the image of the reference pattern, that is, the face image to be tracked has moved within the captured image as a result of the pattern matching process in step S24 (step S26). Here, the determination that the face image has moved includes a case where the face image has been enlarged or reduced by zooming or moving the subject back and forth.

  When it determines with NO in step S26, it returns to the process of step S22.

  On the other hand, if YES is determined, the CPU 52 of the face detection processing calculation board 34 performs face detection processing for detecting a face image from the photographed image (step S28), and the face image is within the vicinity of the AF frame. It is determined whether or not it has been detected (step S30).

  If it is determined YES in step S30, the size of the AF frame is changed (updated) to a size suitable for the detected face image, that is, the minimum size including the entire face ( In step S32, the position of the AF frame is changed (updated) to the position of the detected face image (step S34). As a result, for example, when the photographed image is changed from FIG. 3B to FIG. 3C and the position and size of the face image to be tracked change, AF is performed as shown in FIG. The position and size of the frame 100 are changed, and the frame 100 is changed to one that matches the position and size of the face image 102.

  If NO is determined in the determination process in step S30, the position of the AF frame is changed (updated) to the position of the image of the reference pattern detected in the pattern matching process in step S24 without changing the size of the AF frame. (Step S34). Note that the position of the AF frame may be changed to the position of the image of the reference pattern detected by the pattern matching process also in the process of step S34 when it is determined YES in step S30.

  When the update of the AF frame is completed by the process in step S34, the image in the AF frame is updated as the image of the reference pattern (step S36), and the process returns to step S22.

  In the process of the above flowchart, the pattern matching process (step S24) and the face detection process (step S28) can be performed in order, and these processes need to be performed individually by a plurality of arithmetic units as shown in FIG. However, both processings may be performed in the main board 30, and an arithmetic board dedicated to image processing other than the main board 30 is provided, and both processes or the main board 30 and the arithmetic board One of each process may be performed.

  Further, the range of the AF frame to be changed according to the size of the face image is not necessarily limited to the minimum range including the entire face image as described above. Any range may be used as long as it is suitable for a range in which the face image can be properly focused by AF targeting the subject in the AF frame, that is, the position and size of the face image.

  According to the first embodiment of the AF frame setting process described above, when the subject in the AF frame is automatically tracked by the pattern matching process, if the AF frame includes a human face, The range of the AF frame is changed to match the position and size. For this reason, even when the user wants to track a person's face, the time and effort for accurately setting the initial AF frame so as not to include the background image as much as possible are not required. Even when the size of the face image to be tracked changes due to the zoom operation, the AF frame range is changed to match the face image, so that AF focusing is appropriately performed using the background image in the AF frame. Inconveniences that are not performed and problems that automatic tracking is not appropriately performed are reduced.

  Next, a second embodiment of the AF frame setting process will be described with reference to the flowcharts of FIGS. In the present embodiment, in the processing shown in the first embodiment, the pattern matching processing and the face detection processing are performed in parallel by the pattern matching processing calculation board 32 and the face detection processing calculation board 34, and the tracking processing is performed at high speed. It is intended to make it easier.

  4 (A), 4 (B), and 4 (C) are processing procedures of processes performed by the CPU 38 of the main board 30, the CPU 50 of the pattern matching processing calculation board 32, and the CPU 52 of the face detection processing calculation board 34, respectively. Is shown. 2 that are the same as those shown in the flowchart of FIG. 2 of the first embodiment are denoted by the same reference numerals as those of the flowchart of FIG.

  First, as in the first embodiment, the user operates the operation unit 20 to specify an initial AF frame so that the subject tracked as an AF target is included. Based on this designation, the CPU 38 of the main board 30 sets an AF frame to be transmitted to the lens device 14 (step S10 in FIG. 4A). Then, the CPU 38 starts an AF frame automatic tracking process (step 12 in FIG. 4A).

  When the automatic tracking process is started, the CPU 38 sets the position, size, and shape of the reference pattern in the pattern matching process based on the position, size, and shape of the AF frame (step S50 in FIG. 4A). The data specifying the position, size, and shape of the reference pattern is hereinafter referred to as reference pattern coordinates.

  Next, the CPU 38 transmits the reference pattern coordinates to the CPU 50 of the pattern matching processing calculation board 32 (step S52 in FIG. 4A).

  On the other hand, when the CPU 50 of the pattern matching processing calculation board 32 receives the reference pattern coordinates from the CPU 38 of the main board 30 (step S70 in FIG. 4B), the following steps S72 to S82 (steps S72, S24, S26, S80). , S82) is repeatedly executed. However, in the first process, based on the reference pattern coordinates received in step S70, the image data of the reference pattern of the captured image is set by the processes in steps S72 and 80.

  The CPU 50 sets the image data of the reference pattern, and then converts the image data of the captured image for one frame (one frame) from the image signal of the captured image that is converted into digital data by the decoder 36 and output the pattern matching processing. The data is taken into the board 32 (step S72). Then, the CPU 50 detects an image matching the reference pattern from the photographed image, and executes a pattern matching process for detecting the position (step S24 in FIG. 4B).

  Subsequently, as a result of the pattern matching process in step S24, the CPU 50 determines whether or not the image of the reference pattern, that is, the face image to be tracked has moved within the captured image (step S26 in FIG. 4B).

  When it determines with NO in step S30, it returns to the process of step S24.

  On the other hand, if YES is determined, the detected reference pattern image is updated as a new reference pattern image (step S80 in FIG. 4B), and the reference pattern coordinates are transmitted to the CPU 38 of the main board 30 (see FIG. 4). Step S82 in FIG. Then, the process returns to step S72.

  The CPU 50 of the pattern matching processing calculation board 32 repeatedly executes the processes in steps S72 to S82 described above.

  Further, the CPU 52 of the face detection processing calculation board 34 starts the processing together with the start of the pattern matching processing in the pattern matching processing calculation board 32 in accordance with an instruction from the CPU 38 of the main board 30 (not shown). ) Steps S90 to S94 (S90, S28, S92, S94) are repeatedly executed.

  That is, the CPU 52 of the face detection processing calculation board 34 converts the image data of one frame (one frame) of the shot image into the face detection processing calculation board based on the video signal of the shot video that is converted into digital data by the decoder 36 and output. 34 (step S90 in FIG. 4C).

  Subsequently, face detection processing for detecting a face image from the captured image is performed (step S28 in FIG. 4C), and it is determined whether or not a face image is detected (step S92 in FIG. 4C). .

  If NO is determined in step S92, the process returns to step S90. If YES is determined, information indicating the position (coordinates) and size of the detected face image is transmitted to the CPU 38 of the main board 30. (Step S94), the process returns to Step S90.

  The CPU 52 of the face detection processing calculation board 34 repeatedly executes the processes in steps S90 to S94 described above.

  The CPU 38 of the main board 30 repeatedly executes the processes of steps S54 to S34 (S54, S56, S30, S32, S34) after the process of step S52 in FIG.

  First, the reference pattern coordinates transmitted by the process of step S82 of FIG. 4B are received from the pattern matching processing calculation board 32 (step S54 of FIG. 4A).

  Subsequently, information on the position and size of the face image transmitted by the process of step S94 in FIG. 4C is received from the face detection processing calculation board 34 (step S56 in FIG. 4A).

  Then, the CPU 38 determines whether or not there is a face image detected in the vicinity of the current AF frame position (step S30 in FIG. 4A).

  If it is determined NO in step S30, the position of the AF frame is changed (updated) based on the reference pattern coordinates acquired in step S54 (step S34 in FIG. 4A).

  On the other hand, if YES is determined in step S30, the size of the AF frame is changed (updated) to match the face image detected in the vicinity of the position of the AF frame (FIG. 4A). ) In step S32), the position of the AF frame is changed (updated) to the position of the face image (step S34 in FIG. 4A).

  When the process of step S34 ends, the process returns to step S54, and the processes of steps S54 to S34 (S54, S56, S30, S32, S34) are repeatedly executed.

  With the above processing, the pattern matching processing in the pattern matching processing calculation board 32 and the face detection processing in the face detection processing calculation board 34 are performed in a distributed manner and in parallel, and time-consuming image processing is performed. The problem that AF frame tracking is delayed is reduced.

  Next, a third embodiment of the AF frame setting process will be described with reference to the flowchart of FIG. In the present embodiment, among AF frame automatic tracking by pattern matching processing and AF frame automatic tracking by face detection processing, priority is given to automatic AF frame tracking by face detection processing, and AF frame tracking by face detection processing is prioritized. When automatic tracking becomes impossible, automatic tracking is performed by pattern matching processing.

  First, as in the first and second embodiments, the user operates the operation unit 20 to specify an initial AF frame so that a subject tracked as an AF target is included. Based on this designation, the CPU 38 of the main board 30 sets an AF frame to be transmitted to the lens device 14 (step S100). Then, the CPU 38 starts an AF frame automatic tracking process (step 102).

  When the automatic tracking process is started, the CPU 38 converts the image data of the captured image for one frame (one frame) to the pattern matching processing calculation board 32 based on the video signal of the captured video that is converted into digital data by the decoder 36 and output. Capture (step S104).

  Then, the image data in the AF frame in the captured image is set as the image data of the reference pattern in the pattern matching process (step S106). Note that the image data of the reference pattern need not completely match the size of the AF frame, and may be image data in a range larger or smaller than the AF frame.

  When the above processing is completed, the following steps S108 to S120 are repeated to track the face image in the AF frame.

  First, in step S <b> 108, the CPU 38 captures image data of a captured image for one frame from the decoder 36 into the pattern matching processing calculation board 32 and the face detection processing calculation board 34.

  Then, the CPU 52 of the face detection processing calculation board 34 performs face detection processing for detecting a face image from the photographed image (step S110), and the face image is detected within the AF frame (or within the vicinity of the AF frame). It is determined whether or not (step S112).

  If YES is determined in the determination process in step S112, it is determined whether or not the position of the face image detected in S110 has moved within the captured image (step S116). If NO is determined, the process returns to step S108. If YES is determined, the position of the AF frame is changed (updated) to the position of the detected face image (step S118). Further, the image in the AF frame is updated as a reference pattern image (step S120), and the process returns to step S108. As in the first and second embodiments, the size of the AF frame may be changed (updated) to a size that matches the detected face image.

  On the other hand, if NO is determined in step S112, the CPU 50 of the pattern matching processing calculation board 32 detects an image that matches the reference pattern from the photographed image, and executes a pattern matching process for detecting the position ( Step S114).

  Then, the CPU 38 determines whether or not the image of the reference pattern has moved within the captured image as a result of the pattern matching process in step S114 (step S116).

  When it determines with NO in step S116, it returns to the process of step S108.

  On the other hand, if YES is determined, the position of the AF frame is changed (updated) to the position of the image of the reference pattern detected by the pattern matching process (step S118). Further, the image in the AF frame is updated as a reference pattern image (step S120), and the process returns to step S108.

  In the process of the flowchart, the pattern matching process (step S114) and the face detection process (step S110) can be performed in order, and these processes need to be performed individually by a plurality of arithmetic units as shown in FIG. However, both processings may be performed in the main board 30, and an arithmetic board dedicated to image processing other than the main board 30 is provided, and both processing, or the main board 30 and the arithmetic board are provided with this arithmetic board. One of each process may be performed.

  Further, as in the second embodiment, the pattern matching process and the face detection process are executed in parallel by the pattern matching process calculation board 32 and the face detection process calculation board 34, and each result is obtained by the CPU 38 of the main board 30. May be obtained when needed.

  In the above process, when a face image is detected by the face detection process, the AF frame is automatically tracked preferentially by the face detection process. However, the reference pattern is detected by the pattern matching process. If it is, the AF frame automatic tracking by the pattern matching process may be executed preferentially, and the AF frame automatic tracking by the face detection process may be executed when the pattern matching process becomes impossible.

  According to the third embodiment of the AF frame setting process described above, for example, the face of the person to be tracked faces in a different direction from the top, bottom, left, and right fronts (camera side), and face detection becomes impossible. Even in such a case, since the automatic tracking is appropriately performed by the pattern matching process, the reliability of the automatic tracking is improved.

  In the third embodiment, similarly to the second embodiment, the pattern matching process and the face detection process are performed in parallel by the pattern matching process calculation board 32 and the face detection process calculation board 34. It is possible to reduce the processing time.

1 is a block diagram showing the overall configuration of a video camera system to which the present invention is applied. The flowchart which showed the process sequence of 1st Embodiment of AF frame setting processing. Explanatory drawing used for description of 1st Embodiment of AF frame setting processing. The flowchart which showed the process sequence of 2nd Embodiment of AF frame setting processing. The flowchart which showed the process sequence of 3rd Embodiment of AF frame setting processing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Video camera system, 10 ... Television camera, 12 ... Camera body, 14 ... Lens apparatus, 16 ... Subject, 18 ... Image processing unit, 20 ... Operation part, 24 ... Shooting lens, 26 ... AF unit, 30 ... Main board 32 ... Pattern matching processing calculation board, 34 ... Face detection processing calculation board, 36 ... Decoder, 38 ... CPU, 40 ... RAM, 42 ... Superimposer

Claims (2)

An autofocus unit that controls the focus of a photographing optical system that forms a subject image on the imaging unit so that a subject within a range corresponding to the AF frame in a range of the subject captured by the imaging unit is focused;
A face detection processing means for detecting an image of a person's face from a photographed image obtained by the imaging means;
Pattern matching processing means for detecting, by pattern matching processing, an image of the subject to be tracked from a captured image obtained by the imaging means using a reference pattern image stored as an image of the subject to be tracked;
The position of the AF frame based on the position of the face image detected by the face detection processing unit with respect to the photographed image obtained by the imaging unit when the AF frame includes a human face image First AF frame determining means for determining
Second AF frame determining means for determining the position of the AF frame based on the position of the image of the subject to be tracked detected by the pattern matching processing means with respect to the captured image obtained by the imaging means;
Either one of the first AF frame determination means and the second AF frame determination means is set as a priority AF frame determination means, and the AF frame is set at a position determined by the priority AF frame determination means. When the AF frame position cannot be determined by the priority AF frame determination means, the AF frame is changed to the position determined by the other AF frame determination means. Frame changing means,
An autofocus system equipped with an AF frame automatic tracking function, characterized by comprising   2. The autofocus system having an AF frame automatic tracking function according to claim 1, wherein the AF frame changing means uses the first AF frame determining means as the priority AF frame determining means.

Images