JP5426874B2 - Auto focus system - Google Patents

Description

  The present invention relates to an autofocus system, and more particularly, to a system for automatically tracking an AF frame by tracking a person in a captured image by pattern matching as a focusing target.

  Conventionally, in an autofocus (AF) system that automatically adjusts the focus of a camera, the camera must be instructed where to focus (where to focus). At this time, in a general camera or the like, the position to be focused is fixed at the center of the shooting range, and for example, the person in the center of the shooting range is focused. As an autofocus (AF) method, the contrast of the subject image is detected from the image signal captured by the image sensor, and the focus of the photographing lens is controlled so that the contrast becomes the highest, thereby achieving the best focus state. A contrast method that automatically focuses was used.

  The range of the subject to be AF in the shooting range of the camera or the range on the screen of the subject to be AF in the screen of the shot image when the shot image of the camera is reproduced is called an AF area. A frame indicating the outline of the AF area is called an AF frame.

By the way, when shooting a moving subject, it is inconvenient if the focus position is fixed as described above. Therefore, for example, when shooting a scene where the movement of a subject is intense, such as sports, with a TV camera or the like, a focus target (focus target) is used to detect and track the target subject to be focused from within the captured image. The image of the subject is memorized as a reference pattern, and the image that best matches the reference pattern image in the captured image of the newly captured frame is detected by pattern matching, and the position of the target subject is detected. (See, for example, Patent Document 1).
JP 2006-258944 A

  However, in the system that automatically tracks the AF frame as described above, when a person is tracked by pattern matching, there are a plurality of persons on the screen, and when the persons cross each other, the tracking is performed. There is a problem that it may become impossible.

  The present invention has been made in view of such circumstances, and in a system for automatically tracking an AF frame, when tracking a person by pattern matching, even if a plurality of persons intersect in the screen, the tracking target is surely An object of the present invention is to provide an autofocus system capable of tracking a camera.

In order to achieve the above object, the invention according to claim 1 is directed to an imaging unit that captures a subject image formed by an optical system, and a predetermined AF area among the captured images captured by the imaging unit. Auto focus means for adjusting the focus of the optical system so that the subject is in focus, tracking means for moving the AF area in accordance with the movement of the subject within the image range imaged by the imaging means, and a tracking target A reference pattern registration means for registering the first reference pattern and the second reference pattern, which are images of the image, and a match for detecting an image that best matches the reference pattern from the captured images captured by the imaging means means for calculating the image detection means, a correlation value indicating the degree of matching with respect to the reference pattern of the image detected by the matching image detection means, tracking And means for elephant determines whether moved, the, first, the same image is tracking target is registered as the first reference pattern and the second reference pattern, then it is imaged by the imaging means A captured image is captured, an image that best matches the second reference pattern is detected, a correlation value between the detected image and the second reference pattern is calculated and compared with a first threshold value, When the correlation value with the second reference pattern is larger than the first threshold value, it is determined whether or not the tracking target is moving. When the tracking target is moving, the matching image detection unit detects the tracking target. When the second reference pattern is updated with the image, the AF area is moved in accordance with the tracking target after movement, and then the next captured image is captured, and the tracking target is not moving, When the next captured image is captured as it is and the correlation value with the second reference pattern is equal to or less than the first threshold value, the best match with the first reference pattern among the captured images captured by the imaging unit A correlation value between the detected captured image and the first reference pattern is calculated and compared with a second threshold value, and the correlation value with the first reference pattern is calculated as the second threshold value. If larger, it is determined whether or not the tracking target is moving. If the tracking target is moving, the second reference pattern is updated with the image detected by the matching image detection means, and after the movement The AF area is moved according to the tracking target, and then the next captured image is captured. If the tracking target is not moved, the next captured image is captured as it is. Provide a system.
In order to achieve the above object, the invention according to claim 2 includes an imaging unit that captures a subject image formed by an optical system, and a predetermined AF area among the captured images captured by the imaging unit. Auto focus means for adjusting the focus of the optical system so that the subject is in focus, tracking means for moving the AF area in accordance with the movement of the subject within the image range imaged by the imaging means, and a tracking target The reference pattern registration means for registering the first reference pattern A, the second reference pattern B, and the third reference pattern C, which are the images of the image, and the reference pattern out of the captured images picked up by the image pickup means. Matching image detecting means for detecting the best matching image and correlation indicating the degree of matching of the image detected by the matching image detecting means with the reference pattern And means for determining whether or not the tracking target has moved. First, the same image as the tracking target is the first reference pattern A, the second reference pattern B, and the third reference image. The reference image is registered as a reference pattern C, and then a captured image captured by the imaging unit is captured, and an image that most closely matches the third reference pattern C is detected. The detected image and the third reference When the correlation value with the pattern C is calculated and compared with the first threshold value and the correlation value with the third reference pattern C is larger than the first threshold value, the correlation value with the pattern C is detected by the image detected by the matching image detection means. After the second reference pattern B is updated, it is determined whether or not the tracking target is moving. If the tracking target is moving, the third reference is determined based on the image detected by the matching image detection unit. Patter When the tracking area is updated and the AF area is moved in accordance with the tracking target after movement, the next captured image is captured. If the tracking target is not moved, the next captured image is captured as it is. When the correlation value with the third reference pattern C is not more than the first threshold value, the correlation value with the third reference pattern C is compared with the second threshold value, and the third reference pattern C and Is greater than the second threshold value, it is determined whether or not the tracking target is moving. If the tracking target is moving, the third reference is determined based on the image detected by the matching image detection means. After the pattern C is updated and the AF area is moved according to the tracking target after movement, the next captured image is captured, and when the tracking target is not moved, the next captured image is captured as it is. The above When the correlation value with the third reference pattern C is less than or equal to the second threshold value, an image that most closely matches the first reference pattern is detected from the captured images captured by the imaging unit and detected. When the correlation value between the captured image and the first reference pattern is calculated and compared with a third threshold value, and the correlation value with the first reference pattern is greater than the third threshold value, the matched image detection means After the second reference pattern B is updated with the image detected by the above, it is determined whether or not the tracking target is moving, and when the tracking target is moving, the image detected by the matching image detection means When the third reference pattern C is updated and the AF area is moved in accordance with the tracking target after movement, the next captured image is captured and the tracking target is not moved. When the next photographed image is captured and the correlation value with the first reference pattern A is equal to or smaller than the third threshold value, the second reference pattern B is the largest among the photographed images picked up by the image pickup means. A matching image is detected, a correlation value between the detected captured image and the second reference pattern B is calculated and compared with a fourth threshold value, and a correlation value with the second reference pattern B is If the threshold value is larger than 4, after the second reference pattern B is updated with the image detected by the matched image detecting means, it is determined whether or not the tracking target is moving, and the tracking target is moving. In this case, the third reference pattern C is updated with the image detected by the matching image detection means, and the AF area is moved according to the tracking target after movement, and then the next photographed image is captured. If the tracking target is not moving, to provide an autofocus system that it is characterized in that the following photographic image is captured.

  As a result, in the system that automatically tracks the AF frame, when a person is tracked by pattern matching, the tracking target can be reliably tracked even if a plurality of persons intersect in the screen.

  As described above, according to the present invention, when a person is tracked by pattern matching in a system for automatically tracking an AF frame, the tracking target is reliably tracked even if a plurality of persons intersect in the screen. Is possible.

  Hereinafter, an autofocus system according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the overall configuration of an embodiment of a video camera system to which an autofocus system according to the present invention is applied. This video camera system is, for example, an imaging system used for shooting with a broadcast television camera.

  As shown in FIG. 1, the video camera system 1 according to the present embodiment includes a television camera 10, an image processing unit 18, and the like.

  The television camera 10 includes a camera body 12 composed of an HD camera compatible with a high-definition television [HD (High Definition) TV] system, and a lens device 14 having a photographing lens (optical system) attached to a lens mount of the camera body 12. It consists of. Although not shown, the camera body 12 is supported on the platform so as to be movable in a turning direction (pan) and a vertical tilt direction (tilt).

  The camera body 12 is equipped with an image sensor (for example, a CCD), a required signal processing circuit, and the like, and an image formed by the photographing lens of the lens device 14 is subjected to signal processing after being photoelectrically converted by the image sensor. Necessary signal processing is performed by the circuit and output as an HDTV video signal (HDTV signal) from the video signal output terminal of the camera body 12 to the outside.

  In addition, the camera body 12 includes a viewfinder 13 so that an image currently captured by the television camera 10 is displayed on the viewfinder 13. Also, various information is displayed on the viewfinder 13, and for example, an AF frame that is a focus target range in autofocus described later is displayed superimposed on the captured video. Yes.

  The lens device 14 includes a photographing lens (zoom lens) 24 that is attached to the lens mount of the camera body 12. The subject 16 is imaged on the imaging surface of the imaging element of the camera body 12 by the photographing lens 24. Although not shown, the photographic lens 24 is provided with movable parts for adjusting photographing conditions such as a focus lens group, a zoom lens group, and a diaphragm as constituent parts thereof. Servo mechanism) is electrically driven. For example, the focus lens group and the zoom lens group move in the optical axis direction, the focus (subject distance) adjustment is performed by moving the focus lens group, and the focal length (zoom magnification) is adjusted by moving the zoom lens group. Adjustments are made.

  In a system related to autofocus (AF), it is sufficient that at least the focus lens group can be driven electrically, and the other movable parts may be driven only manually. Further, when the predetermined movable part is electrically driven in accordance with the operation of the operator, it is based on a control signal output in accordance with the operation of the operator from an operating means (not shown) such as an operating means of a controller connected to the lens device 14. Thus, the operation of the movable part is controlled, but details are omitted.

  The lens device 14 includes an AF unit 26 and a lens CPU (not shown). The lens CPU controls the entire lens device 14. The AF unit 26 is a processing unit for acquiring information necessary for performing AF focus control (automatic focus adjustment). Although not shown, the AF unit 26 includes an AF processing unit, an AF imaging circuit, and the like. Has been. The AF image pickup circuit is disposed in the lens device 14 to acquire an image signal for AF processing. An image pickup device such as an image pickup device CCD (referred to as an image pickup device for AF) or an output signal of the AF image pickup device is used in a predetermined format. And a processing circuit for outputting as a video signal. The video signal output from the AF imaging circuit is a luminance signal.

  On the image pickup surface of the AF image pickup device, subject light branched from the subject light incident on the image pickup device of the camera main body 12 is imaged by a half mirror or the like disposed on the optical path of the taking lens 24. . The shooting range and subject distance (distance of the subject in focus) with respect to the imaging area of the AF imaging element are configured to match the shooting range and subject distance with respect to the imaging area of the imaging element of the camera body 12. The subject image captured by the image sensor matches the subject image captured by the image sensor of the camera body 12. It should be noted that the shooting ranges of the two do not have to coincide completely, and for example, the shooting range of the AF image sensor may be a larger range including the shooting range of the image sensor of the camera body 12.

  The AF processing unit acquires a video signal from the AF imaging circuit, and calculates a focus evaluation value indicating the level of contrast of the subject image based on the video signal. For example, after extracting a high-frequency component signal of a video signal obtained from an AF image sensor with a high-pass filter, a signal in a range corresponding to the AF area targeted for AF is extracted from the high-frequency component signal. Accumulate one screen (one frame) at a time. The integrated value obtained for each screen in this way indicates the level of contrast of the subject image, and is given to the lens CPU as a focus evaluation value.

  The lens CPU obtains AF frame information (AF frame information) indicating the range (contour) of the AF area from the image processing unit 18 as will be described later, and the range within the AF frame specified by the AF frame information is obtained. The AF area is designated as the AF area. Then, the focus evaluation value obtained from the image (video signal) in the AF area is acquired from the AF processing unit.

  In this way, every time a video signal for one screen is acquired from the AF imaging circuit (each time the focus evaluation value is obtained by the AF processing unit), the focus evaluation value is acquired from the AF processing unit, and the acquired focus is obtained. The focus lens group is controlled so that the evaluation value is maximum (maximum), that is, the contrast of the subject image in the AF frame is maximum. For example, a hill-climbing method is generally known as a control method for a focus lens group based on a focus evaluation value, and the focus evaluation value starts to decrease by moving the focus lens group in a direction in which the focus evaluation value increases. Is detected, the focus lens group is set at that position. Thereby, the subject in the AF frame is automatically focused.

  Note that the above-described AF processing unit obtains a video signal from the AF image sensor mounted on the lens apparatus in order to calculate the focus evaluation value, but the video image captured by the image sensor of the camera body 12 It is good also as a structure which acquires a signal from the camera main body 12. FIG. Also, any AF means for automatically focusing on the subject within the AF frame may be used.

  In addition, the camera body 12 and the lens device 14, and the lens device 14 and an image processing unit 18 described later are connected to each other directly or via a cable or the like. As a result, the camera body 12 and the lens device 14 transmit and receive various kinds of information by serial communication via serial communication interfaces (SCI) 12a and 14a provided respectively. In addition, the lens device 14 and the image processing unit 18 transmit and receive various kinds of information to and from the serial communication interfaces 14a and 30a provided in the respective units by serial communication.

  Further, the video output connector of the camera body 12 and the video input connector of the image processing unit 18 are connected via a down converter 28 with a cable. As a result, the HDTV signal output from the video output connector of the camera body 12 is converted (down-converted) into a standard television [NTSC (National Television System Committee)] video signal (SDTV signal) by the down converter 28. Are input to the image processing unit 18.

  As will be described in detail later, the image processing unit 18 performs face authentication on the subject person in the captured subject image, and indicates that it is a subject (focus target) set to automatically track with autofocus. If it is certified, the lens device 14 is autofocus controlled through the AF unit 26.

  In addition, in order to perform face authentication and determine whether or not the subject currently being shot is an object to be automatically tracked with autofocus, it is not necessary to be a high-definition image using an HDTV signal. Therefore, as described above, the HDTV signal output from the camera body 12 is converted into the SDTV signal by the down converter 28.

  The image processing unit 18 is a device for designating the range (position, size, shape (aspect ratio)) of the AF frame when the AF unit 26 of the lens device 14 performs focus control by AF as described above. Thus, AF frame information for designating the range of the AF frame in the photographed image (photographing screen) of the television camera 10 is given from the image processing unit 18 to the lens device 14 by the serial communication. The AF unit 26 sets the range of the AF frame based on the AF frame information from the image processing unit 18 and executes the AF process as described above.

  The image processing unit 18 mainly includes a main board 30, a pattern matching processing calculation board 32, and a face authentication processing calculation board 34. The main board 30, the pattern matching processing calculation board 32, and the face authentication processing calculation board 34 are equipped with CPUs 38, 50, and 52, and individual calculation processing is performed for each board, and each of the CPUs 38, 50, 52 are connected by a bus or a control line so that data can be exchanged and arithmetic processing can be synchronized with each other. The image processing unit 18 includes a face authentication data card 74 that stores face authentication data for performing face authentication processing.

  The processing in the image processing unit 18 is performed on the main board 30 in an integrated manner. The main board 30 is equipped with an SCI 30a, a decoder (A / D converter) 36, a superimposer 42, a RAM 40, and the like, in addition to the CPU 38 that performs arithmetic processing.

  The SCI 30a is an interface circuit for performing serial communication with the SCI 14a of the lens device 14 as described above, and transmits the AF frame information and the like to the lens device 14.

  The decoder 36 is a circuit for converting a video signal (SDTV signal) of a captured video input to the image processing unit 18 from the down converter 28 into data that can be digitally processed by the image processing unit 18. A / D conversion processing for converting an analog SDTV signal into a video signal of digital data is performed.

  The RAM 40 is a memory that temporarily stores data used in the arithmetic processing of the CPU 38.

  On the other hand, the pattern matching processing calculation board 32 and the face authentication processing calculation board 34 are calculation boards for individually performing pattern matching and face detection / authentication processing. In addition to the CPUs 50 and 52 that perform calculation processing, image data VRAM 54, 56, etc. are temporarily stored.

  The image processing unit 18 is provided with an operation unit 20 integrally, or a part or all of the operation members of the operation unit 20 are provided in a separate device from the image processing unit 18 and are connected by a cable or the like. It is connected.

  Although details are omitted, the operation unit 20 includes a position operation member 60 (for example, a joystick or a trackball) for moving the position of the AF frame up and down and left and right manually by the user, and manually operating the size of the AF frame. A size operation member 62 (for example, a knob) for changing the shape of the AF frame, a shape operation member 64 (for example, the knob) for changing the shape of the AF frame by manual operation, a tracking start switch 68 for instructing the start of automatic tracking, automatic A tracking stop switch 70 for instructing the stop of tracking is provided, and the setting states of these operation members 60, 62, 64, 68, 70 are read by the CPU 38 of the main board 30 in the image processing unit 18. ing.

  The LCD 66 with a touch panel is configured so that settings such as a mode related to automatic tracking of the AF frame can be input by a touch operation. An image displayed on the LCD 66 by the CPU 38 of the image processing unit 18 is appropriately selected according to the setting content. It can be switched.

  Further, the image displayed on the LCD 66 with a touch panel is given via the superimposer 42 of the main board 30, and the superimposer 42 takes a video image taken by the TV camera 10 given from the decoder 36. The video signal and the image signal generated by the CPU 38 can be synthesized. As a result, similar to the viewfinder 13 set in the camera body 12, an image obtained by superimposing the captured image captured by the television camera 10 and the image of the AF frame currently set is displayed on the LCD 66 with a touch panel. The user can perform the same operation as that performed by the operation members 60, 62, 64, 68 and 70 by the touch operation on the screen.

  In addition to the LCD 66 with a touch panel, the operation unit 20 is provided with an external monitor 72 so that the same image as that displayed on the LCD 66 with a touch panel is displayed. The external monitor 72 allows a focus man other than the cameraman, for example, to see it and input an appropriate instruction using the touch panel function.

  In this embodiment, in a system for automatically tracking an AF frame, when a person is tracked by pattern matching, there are a plurality of persons on the screen, and even when the persons cross each other, tracking is not impossible. The tracking target can be reliably tracked.

  Next, the operation of the autofocus system according to the present invention will be described with reference to a flowchart.

  FIG. 2 is a flowchart showing the operation of one embodiment of the present invention.

  First, in step S100 of FIG. 2, when the photographer starts shooting, the position operation member 60 and the size operation member 62 of the operation unit 20 are viewed while viewing the image and the AF frame displayed on the screen of the viewfinder 13 or the LCD 66 with the touch panel. Then, the operation member such as the shape operation member 64 is operated, and the range of the AF frame is set so that the object to be tracked as the focus adjustment target falls within the range of the AF frame. As a result, the object is brought into focus by AF processing in the lens CPU of the lens device 14.

  When tracking starts in step S102, in step S104, the CPU 38 of the image processing unit 18 captures a captured image for one screen (one frame) from the video signal supplied from the AF unit 26 (AF processing unit) of the lens device 14. take in.

  In steps S106, S108, and S110, a buffer for registering a reference pattern to be used later for pattern matching is secured, and the same target image to be tracked obtained from the originally captured image is obtained. Stored as reference pattern A, reference pattern B, and reference pattern C. Here, the reference pattern B and the reference pattern C are updated as the latest image, and the reference pattern B is updated as the image closest to the latest image, depending on the image captured in the future. Pattern A is always stored as the first registered reference pattern.

  In step S112, the image processing unit 18 captures one screen of the captured image in the same manner as the previous time after a predetermined time has elapsed since the previous image capture.

  Next, in step S114, pattern matching processing is performed in the pattern matching processing calculation board 50 on the captured image that has been captured and the reference pattern C registered as the latest image, so that the similarity between the two images is expressed. A correlation value is calculated.

  In general, the correlation value (correlation coefficient) γxy of the two random variables X and Y is a ratio of the covariance μxy to the product σx · σy of the standard deviation σx and σy of each random variable. .

  That is, the correlation value γxy = μxy / σx · σy.

  Therefore, in this case, the correlation value c is defined as follows, for example. That is, it is assumed that the pixel value of the reference pattern image at the coordinates (x, y) is f (x, y), and the pixel value of the captured image compared with this is g (x, y). A pixel value obtained by shifting the captured image by m in the x direction and n in the y direction from the reference pattern image is defined as g (m + x, n + y), and f (x, y) and g (m + x, n + y) are defined as random variables. Considering this, assume that the correlation value between the captured image and the reference pattern image is c (m, n).

When the average values of the pixel values of the reference pattern image and the photographed image in a predetermined area are respectively f _av and g _av , the correlation value c (m, n) is defined by the following equation (1).

c (m, n)
= {Σ (| f (x, y) −f _av | × | g (m + x, n + y) −g _av |)
/ (Σ | f (x, y) −f _av | ² × Σ | g (m + x, n + y) −g _av | ² ) ^1/2 }
... (1)
Here, Σ in equation (1) represents the sum of x and y in a predetermined region.

  The correlation value takes a value between 0 and 1, and takes a value close to 1 when the correlation is high (that is, the more similar the two images are).

  Next, in step S116, it is determined whether or not the correlation value c just found is greater than 90% (that is, the correlation value c is greater than 0.9).

  As a result, when the correlation value c is greater than 90%, that is, when the captured image that has been captured is very similar to the reference pattern C, the reference pattern B is updated with the captured image in the next step S118. If this is the first update of the reference pattern B, the reference image B has been registered with the same first photographed image as that registered in the reference pattern A until then. Will be replaced.

  Next, in step S120, it is determined whether the tracking target has moved based on the correlation value c (m, n) calculated above. When the correlation value c (m, n) is high when m and n are not 0, the tracking target is from the position (x, y) so far to the position (m + x, n + y) on the image. It has moved.

  If it is determined that the tracking target is moving, the reference pattern C is updated with the latest image in the next step S122.

  If the tracking target is moving, the AF frame is updated in accordance with the tracking target after the movement in the next step S124.

  If the tracking target has not moved, the process returns to step S112 without updating the reference pattern C and the AF frame, and the same process is repeated.

  On the other hand, if it is determined in step S116 that the correlation value c (m, n) is not greater than 90%, then in step S126, the correlation value c (m, n) is now greater than 80%. Determine whether or not.

  If it is determined that the correlation value c (m, n) is greater than 80%, the process proceeds to step S120 without updating the reference pattern B, and it is determined whether or not the tracking target has moved. If there is, the reference pattern C is updated with the captured image in the next step S122.

  If it is determined in step S116 that the correlation value c (m, n) is not greater than 80%, the captured image is subjected to pattern matching with the reference pattern A in step S128, and the correlation value at this time is determined. Calculate

  In step S130, it is determined whether or not the correlation value between the captured image and the reference pattern A is greater than 80%.

  At this time, if the correlation value is greater than 80%, the reference pattern B is updated with the captured image in the next step S132. Thereafter, the process proceeds to step S120, and it is determined whether or not the photographed image has moved.

  If it is determined in step S130 that the correlation value is not greater than 80%, pattern matching is performed between the captured image and the reference pattern B in step 134, and the correlation value at this time is calculated.

  If the correlation value between the captured image and the reference pattern B is greater than 80%, the process proceeds to step S132, and the reference pattern B is updated with the captured image. In this way, the image before the correlation value is lowered is stored. Thereafter, the process proceeds to step S120, where it is determined whether the tracking target has moved.

  In step S136, if the correlation value between the captured image and the reference pattern B is not greater than 80%, it is determined that the captured image is no longer a tracking target, and if the correlation value decreases in this way, tracking is performed. , The process returns to step S112, the next image is captured, and the pattern matching with the reference pattern stored before the correlation value decreases is repeated. When the correlation value increases, tracking is resumed assuming that the tracking target is detected again from the screen.

  After all, the images immediately before the correlation value falls below a predetermined value are always registered in the reference patterns B and C, and in particular, the latest image having a certain degree of similarity with the first registered image is always registered in the reference pattern C. It will be registered.

  The description has been given with reference to the flowchart, and this will be described according to a specific example shown in FIG.

  First, as shown in FIG. 3A, it is assumed that the person P in the current shooting screen is being tracked, and an AF frame 90 is displayed on the face portion of the person P. This face image is registered as the first reference pattern. Thereafter, the reference pattern used for pattern matching is updated at a constant cycle.

  Note that the constant period here means that if one cycle of pattern matching is too fast, the correlation value will not decrease, and tracking will occur even if people cross, so it will only be updated once every several cycles. preferable.

  Next, as shown in FIG. 3B, it is assumed that a situation occurs in which the person P moves and intersects the person Q. At this time, an object other than the person P also enters the AF frame 90, and when the image in the AF frame 90 is pattern-matched with the first reference pattern, the correlation value is considerably lower than 1 (100%). turn into. When the person P further moves and hides behind the person Q, the correlation value becomes a value close to 0, and the tracking is stopped.

  Then, the reference pattern before the correlation value decreases is stored, and thereafter, pattern matching with the reference pattern stored before the correlation value decreases is repeated.

  Then, as shown in FIG. 3C, as a result of performing pattern matching with the reference pattern stored before the person P moves and reappears behind the person Q and the correlation value decreases, the correlation value is obtained. Is again raised, the AF frame 90 is set again for the face of the person P, and tracking is resumed.

  As described above, in the present embodiment, when the correlation value falls below a predetermined value, the tracking is stopped. Therefore, when a person crosses, the person who is different from the person who has been tracked so far It is possible to prevent tracking failure such as starting tracking.

  Next, another embodiment of the present invention will be described. In the embodiment described above, three reference patterns A, B, and C are used. However, in the embodiment described below, only two reference patterns A and C are used.

  FIG. 4 is a flowchart showing the operation of another embodiment of the present invention, which will be described below with reference to this flowchart.

  First, in step S200 of FIG. 4, the cameraman sets an AF frame. In step S202, tracking is started, and an image is captured in step S204. In step S206 and S208, the captured images are registered as a reference pattern A and a reference pattern C.

  Similar to the above-described embodiment, the reference pattern A is the first image indicating the tracking target, and is stored as it is without being updated. On the other hand, the reference pattern C is constantly updated so that the latest image is stored.

  In step S210, captured images are captured at regular time intervals. In step S212, pattern matching processing is performed on the captured image that has been captured and the reference pattern C, and a correlation value is calculated.

  In step S214, it is determined whether the calculated correlation value is greater than 90%.

  As a result, if it is determined that the correlation value is greater than 90%, it is determined in step S216 whether or not the tracking target has moved within the shooting screen.

  If it is determined that the tracking target has moved, the reference pattern C is updated with the current captured image in step S218. In this way, the image before the correlation value is lowered is stored as the reference pattern C. Further, in step S220, the AF frame is also updated in accordance with the tracking target moved within the shooting screen. Thereafter, the process returns to step S210, the next image is captured, and the same processing as above is repeated.

  If the tracking target is not moved, the process returns to step S210 without updating the reference pattern C and the AF frame, and the next process is repeated.

  On the other hand, if it is determined in step S214 that the correlation value is not greater than 90%, pattern matching is performed between the captured image and the first registered reference pattern A in step S222.

  This is because the determination in step S214 does not match the reference pattern C that has already been updated and is the latest one, determines that the tracking target has been lost, and compares the tracking target with the image at the beginning of tracking again. Is to rediscover and resume tracking.

  In step S224, it is determined whether or not the correlation value calculated by pattern matching the captured image and the reference pattern A is greater than 80%.

  If the correlation value is greater than 80%, the process proceeds to step S216, where it is determined whether the tracking target has moved within the shooting screen, and the same processing as before is performed.

  Further, when the correlation value is not larger than 80%, for example, when the correlation value is lowered due to, for example, another person crossing in front of the person being tracked, in such a case Stops tracking, returns to step S210, captures the next image, resumes tracking when the first reference pattern A is detected again, and repeats the above processing.

  In the embodiment of the present invention described above, there are at least two reference patterns, that is, a reference pattern that always holds the first registered image and a reference pattern that always stores the latest image. Perform pattern matching with, and track the tracking target. Then, a correlation value is calculated by pattern matching, and if this correlation value is larger than a certain value, the latest pattern is always stored in one reference pattern by updating the reference pattern with the image at that time. Further, when the correlation value becomes smaller than a certain value, tracking is stopped. Then, the next captured image is captured, pattern matching with the first reference pattern is performed, and tracking is started again when the first tracking target is detected.

  In this way, the correlation value becomes smaller than a certain value, and the image immediately before the tracking is stopped is stored in one reference pattern.

  As a result, even if a person crosses and loses sight of the tracking target, tracking is temporarily stopped and then tracking is resumed when the first tracking target is detected, thereby tracking what is not the tracking target. Such a tracking failure can be prevented.

  The autofocus system of the present invention has been described in detail above, but the present invention is not limited to the above examples, and various improvements and modifications may be made without departing from the scope of the present invention. Of course.

1 is a block diagram showing an overall configuration of an embodiment of a video camera system to which an autofocus system according to the present invention is applied. It is a flowchart which shows the effect | action of one Embodiment of this invention. (A)-(c) is explanatory drawing which shows the effect | action of this embodiment by a specific example. It is a flowchart which shows the effect | action of other embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Video camera system, 10 ... Television camera, 12 ... Camera body, 13 ... Viewfinder, 14 ... Lens apparatus, 16 ... Subject, 18 ... Image processing unit, 20 ... Operation part, 24 ... Shooting lens, 26 ... AF unit 28 ... down converter, 30 ... main board, 32 ... pattern matching processing calculation board, 34 ... face authentication processing calculation board, 66 ... LCD with touch panel, 72 ... external monitor

Claims (2)

Imaging means for imaging a subject image formed by an optical system;
Auto focus means for adjusting the focus of the optical system so that a subject in a predetermined AF area out of the captured image captured by the imaging means is in focus;
Tracking means for moving the AF area in accordance with movement of a subject within an image range imaged by the imaging means;
Reference pattern registration means for registering a first reference pattern and a second reference pattern which are images to be tracked ;
Matched image detection means for detecting an image that best matches the reference pattern from the captured images captured by the imaging means;
Means for calculating a correlation value indicating a degree of match of the image detected by the match image detection means with respect to the reference pattern;
Means for determining whether or not the tracking target has moved;
With
First, the same image to be tracked is registered as the first reference pattern and the second reference pattern,
Thereafter, the captured image captured by the imaging unit is captured, an image that best matches the second reference pattern is detected, and a correlation value between the detected image and the second reference pattern is calculated. If the correlation value with the second reference pattern is larger than the first threshold value compared with the first threshold value, it is determined whether the tracking target is moving, and when the tracking target is moving, The second reference pattern is updated with the image detected by the coincidence image detecting means, the AF area is moved according to the tracking target after the movement, and then the next captured image is captured, and the tracking target is If it has not moved, the next shot image is imported as is,
When the correlation value with the second reference pattern is equal to or less than a first threshold value, an image that best matches the first reference pattern is detected from the captured images captured by the imaging unit, and the detected captured image When the correlation value between the image and the first reference pattern is calculated and compared with a second threshold value, and the correlation value with the first reference pattern is greater than the second threshold value, the tracking target moves If the tracking target is moving, the second reference pattern is updated with the image detected by the matched image detecting means, and the AF area is adjusted to the tracking target after the movement. An autofocus system that is moved, and then captures the next captured image, and captures the next captured image as it is when the tracking target has not moved . Imaging means for imaging a subject image formed by an optical system;
Auto focus means for adjusting the focus of the optical system so that a subject in a predetermined AF area out of the captured image captured by the imaging means is in focus;
Tracking means for moving the AF area in accordance with movement of a subject within an image range imaged by the imaging means;
Reference pattern registration means for registering the first reference pattern A, the second reference pattern B, and the third reference pattern C, which are images to be tracked ;
Matched image detection means for detecting an image that best matches the reference pattern from the captured images captured by the imaging means;
Means for calculating a correlation value indicating a degree of match of the image detected by the match image detection means with respect to the reference pattern;
Means for determining whether or not the tracking target has moved;
With
First, the same image to be tracked is registered as the first reference pattern A, the second reference pattern B, and the third reference pattern C,
Thereafter, a captured image captured by the imaging unit is captured, an image that best matches the third reference pattern C is detected, and a correlation value between the detected image and the third reference pattern C is calculated. When the correlation value with the third reference pattern C is larger than the first threshold value, the second reference pattern B is determined by the image detected by the matched image detecting means. After the update, it is determined whether or not the tracking target is moving, and when the tracking target is moving, the third reference pattern C is updated with the image detected by the matching image detection means, After the AF area is moved according to the tracking target after movement, the next captured image is captured, and when the tracking target is not moving, the next captured image is captured as it is,
When the correlation value with the third reference pattern C is equal to or less than the first threshold value, the correlation value with the third reference pattern C is compared with the second threshold value, and the third reference pattern C is compared with the third reference pattern C. When the correlation value is larger than the second threshold value, it is determined whether or not the tracking target is moving. When the tracking target is moving, the third reference pattern is detected based on the image detected by the matching image detection unit. When C is updated and the AF area is moved in accordance with the tracking target after movement, the next captured image is captured, and when the tracking target is not moved, the next captured image is captured as it is,
When the correlation value with the third reference pattern C is less than or equal to the second threshold value, an image that most closely matches the first reference pattern is detected from the captured images captured by the imaging unit. If the correlation value between the captured image and the first reference pattern is calculated and compared with a third threshold value, and the correlation value with the first reference pattern is greater than the third threshold value, the matched image detection After the second reference pattern B is updated with the image detected by the means, it is determined whether or not the tracking target is moving. If the tracking target is moving, the matching image detection means detects the moving target. When the third reference pattern C is updated with the image and the AF area is moved in accordance with the moved tracking target, the next captured image is captured and the tracking target is not moved. Mom next photographic image is captured,
When the correlation value with the first reference pattern A is equal to or smaller than the third threshold value, an image that most closely matches the second reference pattern B is detected from the captured images captured by the imaging unit, and is detected. If the correlation value between the captured image and the second reference pattern B is calculated and compared with a fourth threshold value, and the correlation value with the second reference pattern B is greater than the fourth threshold value, After the second reference pattern B is updated with the image detected by the matching image detection means, it is determined whether or not the tracking target is moving. If the tracking target is moving, the matching image detection means The third reference pattern C is updated with the detected image, and after the AF area is moved in accordance with the tracking target after movement, the next captured image is captured and the tracking target is not moved. Place Autofocus system as characterized by the next captured image is captured.

Images