JP2017194787A - Image processing device and area tracking program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device and an area tracking program that improve tracking performance of a specific image imaged with a monocular camera.SOLUTION: To realize area tracking processing, the image processing device extracts a changing pixel area on a screen as a rectangular area, has two kinds of area tracking processing means for tracking this area, and continually performs area tracking gradually by these respective area tracking processing means. The image processing device comprises, as this two kinds of area tracking processing means, tracking processing means of an undetermined object and tracking processing means of a determined object. For the object, which is being tracked, migrated from the undetermined object to the determined object, the image processing device executes identification processing of the determined object to confirm that the determined object existing in a prediction area is the determined object being currently tracked.SELECTED DRAWING: Figure 9

Description

  The present invention relates to an image processing apparatus and a region tracking program that are suitable for application to tracking processing of a specific image captured by a monocular camera.

  As an image processing device that targets monitoring images captured with a monocular camera, a rectangular area including changed pixels is extracted from the current image and past image input on the screen at predetermined intervals, and the extracted rectangular area is processed. An image processing apparatus that tracks as a target object is known (Japanese Patent No. 4427052). This image processing apparatus determines whether or not the extracted area is a newly extracted area on the screen. When the extracted area is a newly extracted area, the extracted area is a target of the tracking process. As a new object, the registered object is an unconfirmed object and is tracked based on the information indicating the position and size of the area extracted this time and the information indicating the position and size of the previously extracted area. In addition, the position and size of the prediction area updated each time tracking is performed according to the prediction direction obtained by the least square approximation based on time-series position information when tracking an undefined object as a confirmed object continuously tracked for a predetermined time. A tracking processing function for tracking based on the information indicating the speed and the moving speed information is realized.

  Although this image processing device has already been put into practical use as a surveillance camera system, it has been recognized that there is a problem in tracking performance that misrecognizes the tracking target in a specific monitoring pattern with the passage of time when the tracking target is temporarily stopped. Therefore, there has been a demand for an image processing technique with improved tracking performance.

Japanese Patent Laid-Open No. 9-91439 JP 10-111946 A Japanese Patent Laid-Open No. 11-136664

  An object of the present invention is to provide an image processing apparatus and an area tracking program that solve the above-mentioned troublesome events with an economically advantageous configuration without causing a significant increase in processing load, and further improve tracking performance.

  In an embodiment of the present invention, a captured image input at a predetermined cycle is sequentially subjected to differential processing in the order of input, a rectangular region including changed pixels is extracted, and the extracted rectangular region is converted into a plurality of past rectangular regions. In the image processing apparatus that tracks based on the prediction area generated from the above, a plurality of local areas are set in the rectangular area, and a threshold value for pattern matching determination in pixel units at the pixel position that constitutes the rectangular area An external parameter setting means for setting a local area pattern extracting means for cutting out each image pattern of the local area from the rectangular area being tracked, and a local area for holding the image pattern as a template for matching evaluation Each time the pattern holding means and the local area pattern extraction means cut out the image pattern, Similarity determination means for comparing the image pattern held in the pattern holding means of the local area, performing matching evaluation based on the threshold value, and determining the presence or absence of correlation of the rectangular area being tracked, When the similarity determination unit determines that there is a correlation between the rectangular regions being tracked, the latest image pattern obtained by the local region pattern extraction unit cutting out the image pattern held in the local region pattern holding unit. There is provided an image processing apparatus comprising a template updating means for updating to the above.

  In addition, the embodiment of the present invention includes a region extraction processing unit that performs a differential process between a current image on the screen and a past image input at a predetermined period, and extracts a rectangular region including a changed pixel, and the region extraction processing unit Among the extracted areas, the newly extracted area is registered as a new object to be tracked, and the object registered by the new object registration process is defined as an undetermined object. Each time the area extraction processing means extracts an area, the unconfirmed object is tracked based on information indicating the position and size of the area extracted this time and information indicating the position and size of the area extracted last time. An indeterminate object tracking processing means and an indeterminate object tracked for a predetermined time by the indeterminate object tracking processing means. Each time the region extraction processing means extracts a region, the information indicating the position and size of the extracted region and the time-series position information at the time of tracking the unconfirmed object. Based on information indicating the position and size of the prediction region updated each time tracking is performed according to the prediction direction obtained by the least square approximation based on the information and the moving speed information, and a tracking processing unit for the determined object for tracking the determined object. In the image processing apparatus, a plurality of local regions are set in the rectangular region, and an external parameter setting unit that sets a threshold for pattern matching determination in a pixel unit at a pixel position constituting the rectangular region; A new confirmed object that has become a confirmed object from an undefined object is processed for the first time. As a target, each time the determined object is determined, a local region pattern extraction unit that extracts an image pattern of each local region from a rectangular region of the object, and a local region that holds the image pattern as a template for matching evaluation Each time the pattern holding unit of the area and the pattern extracting unit of the local area cut out the image pattern, the cut-out image pattern is collated with the image pattern held in the pattern holding unit of the local area, and the threshold value is set. The tracking object similarity determination means for determining the presence or absence of the correlation of the confirmed object being tracked and the tracking object similarity determination means determine whether the tracking of the confirmed object is correlated. When you identify the definitive object you are currently tracking There is provided an image processing apparatus comprising a template updating unit that updates an image pattern held in the local region pattern holding unit to a latest image pattern cut out by the local region pattern extracting unit.

  In the embodiment of the present invention, a captured image input at a predetermined period is sequentially subjected to differential processing in the order of input to extract a rectangular area including a changed pixel, and the extracted rectangular area is converted into a plurality of past rectangular shapes. An area tracking program for causing a computer to function as an image processing apparatus that tracks based on a prediction area generated from an area, wherein a plurality of local areas are set in the rectangular area, and the rectangular area is configured A function for setting an external parameter for setting a threshold for pattern matching determination in a pixel unit at a pixel position to be performed, a pattern extraction function for a local region that cuts out an image pattern of each of the local regions from a rectangular region being tracked, and the image A local area pattern holding function for holding a pattern as a template for matching evaluation, and the local area pattern extracting means includes: Each time the recorded image pattern is cut out, the cut out image pattern is collated with the image pattern held in the pattern holding unit in the local area, and the matching rectangle is evaluated based on the threshold value, and the rectangle being tracked A similarity determination function for determining the presence or absence of correlation in a region, and a template update function for updating the stored image pattern to the latest cut out image pattern when it is determined that there is correlation in the matching evaluation. An area tracking program for realizing the above is provided.

  Further, in the embodiment of the present invention, an imaging screen shot by a camera is input at a predetermined cycle, and a difference between the image on the screen input this time and the image on the screen input last time is subjected to differential processing, and changed pixels on the screen are detected. A region extraction function for extracting a rectangular region including the region extracted and a region extracted by the region extraction function is a newly extracted region by determining whether or not the region extracted by the region extraction function is a newly extracted region on the screen. When the object is registered by the new object registration processing function for registering the extracted area as a new object to be tracked and the new object registration processing function, Each time the area extraction function extracts the area, information indicating the position and size of the area extracted this time and the position and size of the previously extracted area An unconfirmed object tracking processing function that tracks the unconfirmed object based on the information indicated, and an unconfirmed object that the unconfirmed object tracking processing function has continuously tracked for a predetermined period of time Each time the region extraction processing function extracts a region, according to the prediction direction obtained by the least square approximation based on the information indicating the position and size of the extracted region and the time-series position information at the time of tracking the uncertain object The computer is caused to function as an image processing apparatus having a confirmed object tracking processing function for tracking the confirmed object based on the information indicating the position and size of the prediction region updated each time tracking is performed and the moving speed information. An area tracking program for a plurality of local areas in the rectangular area A function for setting an external parameter that sets a threshold for pattern matching determination in pixel units at the pixel position that constitutes the rectangular area, and a new confirmed object that has become a confirmed object from the unconfirmed object for the first time Each time the determined object is determined as a processing target, a local region pattern extraction function for cutting out each local region image pattern from the rectangular region of the object, and the image pattern as a template for matching evaluation A local area pattern holding function, and each time the image pattern is cut out, the cut out image pattern is compared with the held image pattern, and matching evaluation is performed based on the threshold value, and tracking is performed. A tracking object that determines the presence or absence of correlation between certain objects When it is determined that there is a correlation in the similarity determination function of the object and the matching evaluation, the confirmed object currently being tracked is identified, and the held image pattern is updated to the latest cut out image pattern. Provided is an area tracking program for causing a computer to implement a template update function for performing matching evaluation.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. 5 is a flowchart showing a processing procedure of the image processing apparatus according to the embodiment. Operation | movement explanatory drawing for demonstrating the processing operation | movement of the undetermined object tracking part which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the processing operation | movement of the undetermined object tracking part which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the processing operation of the definite object tracking part which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the processing operation of the definite object tracking part which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the processing operation of the definite object tracking part which concerns on the said embodiment. The functional block diagram which shows the component of the principal part which implement | achieves the identification processing function of the fixed object which concerns on the said embodiment. The flowchart which shows the tracking process procedure of the fixed object which concerns on the said embodiment. The flowchart which shows the similarity determination process procedure by the pattern matching implemented in the tracking process of the said definite object. The figure which shows the relationship between the parameter table for correlation determination for demonstrating the similarity determination processing operation | movement by the said pattern matching, and the presence position (for example, a foot position of an object when it is set as a rectangular lower end center position). The figure which shows the example of a setting of the local area | region where the said pattern matching is implemented. The figure which shows the example of a setting of the local area | region where the said pattern matching is implemented. The figure which shows the example of a setting of the local area | region where the said pattern matching is implemented.

  Embodiments of the present invention will be described below with reference to the drawings.

  In this embodiment, in the image processing of a monocular camera (one camera), every time an image is captured, a change pixel area in the image is extracted as a rectangular area from the difference between the past image and the current image, and is time-sequentially. The current area is associated from the obtained extracted area information, and the movement of the area is tracked. In order to realize this area tracking processing, the change pixel area on the screen is extracted as a rectangular area, and has two types of area tracking processing means for tracking this area. Continuous area tracking enables efficient area tracking with reduced processing load and increased processing speed. As these two types of area tracking processing means, an unconfirmed object tracking processing means and a confirmed object tracking processing means are provided. The indeterminate object tracking means temporarily generates a prediction area obtained by enlarging the previously extracted rectangular area based on the area information of the area extracted in time series, and the rectangular area extracted this time is included in the prediction area. When the two intersect, the object is traced with the rectangular area extracted this time as an undetermined object. The deterministic object tracking means obtains the prediction generated by the least square (square) approximation based on the region information of the extracted region and the time-series position information associated with the tracking of the undetermined object, and is generated according to the prediction direction updated at each tracking. The confirmed object is tracked based on the area information and the moving speed information.

  A tracking object confirmation process for confirming that the confirmed object existing in the prediction area is the confirmed object currently being tracked is performed for the object being tracked that has been transferred from the unconfirmed object to the confirmed object. In other words, a confirmation process for identifying a confirmed object to be tracked is performed. The identification process of the confirmed object currently being tracked is performed by a correlation check by pattern matching.

  As shown in FIG. 8, the fixed object identification processing mechanism according to this embodiment sets a plurality of local areas in the rectangular area, and performs pattern matching on a pixel basis at the pixel positions constituting the rectangular area. An external parameter setting means Pa for setting a threshold for determination, and a local area pattern extracting means Pb for cutting out the image pattern of each of the local areas from a rectangular area of a new confirmed object that has become a confirmed object from the undefined object. Each time the local object pattern holding means Pc that holds the image pattern as a template for matching evaluation and the determined object is determined in a state where the template is held in the local area pattern holding means Pc, the object From the rectangular area of A local region pattern extracting unit Pd for cutting out a pattern, and each time the local region pattern extracting unit Pd cuts out the image pattern, a cut image pattern and an image pattern held in the local region pattern holding unit Pc The tracking object similarity determination means Pe for determining the presence / absence of correlation of the tracked confirmed object and the tracking object similarity determination means Pe When the determined object is determined to have a correlation, the latest object obtained by identifying the currently tracked confirmed object and cutting out the image pattern held in the local area pattern holding means Pc by the local area pattern extracting means Pd. Template updating means Pf for updating the image pattern. The local area pattern extraction means Pb is provided as an additional function section in an undefined object tracking section 157b described later, and the local area pattern extraction means Pd, similarity determination means Pe, and template update means Pf are determined object tracking sections described later. 157a is provided as an additional function unit.

  In this way, the correlation check by pattern matching is not performed on the image area surrounded by the rectangle of the changed pixel area, but is an image pattern cut out from a plurality of local areas set in the image area surrounded by the rectangle. This is performed by matching evaluation based on the correlation check.

  The matching evaluation based on the correlation check of the image pattern in the local area can greatly reduce the processing load for the matching evaluation as compared with the case where the entire image area surrounded by the rectangle is subjected to the correlation check. This makes it possible to carry out identification object identification processing without affecting the real-time processing function in an existing image processing system, and it is possible to construct an image processing system with improved tracking performance.

  Details of the fixed object identification processing mechanism shown in FIG. 8 will be described later with reference to FIGS.

  The configuration of the image processing apparatus according to the above embodiment is shown in FIG. In this embodiment, image data for one frame captured from the camera is simply referred to as an image, an image on a screen, or an image for one screen.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention includes a camera 11, a capture unit 12, an image processing storage unit 13, an image processing unit 15, and a display unit 16. Is done.

  The camera 11 includes a lens unit and an image pickup device (for example, a CCD solid-state image pickup device or a CMOS image sensor) provided at an image forming position of the lens unit, so that a subject (animal body) that moves outdoors or indoors can be used. A captured image for one screen is output to a target in a predetermined pixel unit (for example, one frame 320 × 240 pixels = QVGA).

  The capture unit 12 has a processing function of capturing an image in frame units captured by the camera 11 as a processing target image (input image) of the image processing unit 15 and holding it in the image buffer 14 in the image processing storage unit 13. The input image on the screen captured in the image buffer 14 is an original image here, but may be an edge image.

  The image processing storage unit 13 stores various data including the input image captured by the capture unit 12 and each image being processed, which is used for the processing of the image processing unit 15. In this image processing storage unit 13, under the control of the image processing unit 15, among the frame-unit images captured by the capture unit 12, an image for one screen captured this time (current image) and one previously captured image are stored. An area constituting the image buffer 14 that holds images for the screen (past images) as processing target images is secured, and an image area used for the processing of the image processing unit 15 is secured. Further, the image processing storage unit 13 stores various parameters and control data used for the processing of the image processing unit 15, various kinds of information generated or obtained in registration and tracking processing of the confirmed and unconfirmed objects. An information storage area is also secured.

  In this storage area, in addition to the work area including the image buffer 14 described above, various areas for performing a process for identifying a confirmed object, which will be described later, are secured. As shown in FIG. 11, in the identification process of the definite object, as shown in FIG. 11, a threshold value (correlation coefficient) for pattern matching determination of a pixel configuration (320 × 240 pixels) corresponding to one frame of the input image is set. A plurality of sets of pattern matching with different calculation methods for performing matching evaluation with reference to a parameter table (CC-TBL) and a threshold value (correlation coefficient) set in the parameter table for correlation determination (CC-TBL) The evaluation formula (described later) is prepared in advance. Further, in the storage area, the tracking object exists at a plurality of locations in the rectangular area (AS) extracted by the area extraction processing function of the image processing unit 15 to be described later (for example, when the position is the center position of the lower end of the rectangle). , The position coordinates of the object's foot position (PF), and the position coordinates for setting the local area (A1 to A3) to be a small image cutout window of the tracking object set based on the existence position (PF) Parameter areas to be used, buffer areas for holding small screen image patterns (P1 to P3) cut out from the rectangular area images based on the local areas (A1 to A3), and matching evaluation results Various work areas, such as a flag area for holding (with / without correlation), used for the identification object identification process are secured. Here, a buffer that holds an image pattern (template) cut out from a local region is referred to as a pattern model holding unit. A flag area that holds the result of matching evaluation is referred to as a similarity determination flag.

  The image processing unit 15 generates a difference binarized image obtained by binarizing the past image and the current image held in the image buffer 14 and removing noise included in the difference binarized image. In this difference binarized image, a rectangular region including a change pixel is extracted, and a series of image processing for tracking the region is performed on the rectangular region as a processing target. The preprocessing unit 151, the difference processing unit 152, a binarization processing unit 153, a noise removal filter 154, a regionization processing unit 156, a region tracking processing unit 157, and the like.

  Among these components of the image processing unit 15, the preprocessing unit 151, the difference processing unit 152, the binarization processing unit 153, the noise removal filter 154, and the regionalization processing unit 156 are captured from the camera 11 at a predetermined cycle. Realizes an area extraction processing function that extracts a rectangular area including changed pixels by performing a differential process on the current image on the screen (the image on the screen captured this time) and the past image (the image on the screen captured last time). To do. Further, among the components of the image processing unit 15 described above, the preprocessing unit 151, the difference processing unit 152, and the binarization processing unit 153 perform differential processing on the current image captured this time and the past image captured last time, A difference binarized image generating unit that generates a binarized difference binarized image is realized.

  Among the components of the image processing unit 15 described above, the preprocessing unit 151 performs the capture unit 12 when comparing the past image stored in the image buffer 13 with the current image (difference processing for extracting the changed pixel region). Pre-processing (smoothing processing, edge processing, etc.) determined in advance is performed on the input image for one screen captured by the. The difference processing unit 152 compares the past image and the current image that have passed through the preprocessing unit 151 in pixel units, and extracts a difference region.

  The binarization processing unit 153 binarizes the image subjected to the difference processing by the difference processing unit 152 according to a preset threshold value stored in the image storage unit 13 to generate a difference binarized image. The difference binarized image is stored in an area (difference binarized image area) secured in advance in the image processing storage unit 13.

  The noise removal filter 154 includes a diffusion filter 154a and a contraction filter 154b, and removes noise (sesame salt-like change pixels) unnecessary for extraction of a moving body region from the difference binary image. The diffusion filter 154a that constitutes a component of the noise removal filter 154 sequentially sets a target pixel for each pixel except the outermost periphery of the difference binarized image, and in a diagonal pixel adjacent to the set target pixel. When a change pixel exists in (four pixels), a diffusion filtering process using the target pixel as the change pixel is performed. The contraction filter 154b performs contraction filtering processing on the image processed by the diffusion filter 154a based on the surrounding 8 pixels or 4 pixels adjacent to the target pixel. The difference binarized image for each frame (screen) from which noise has been removed by the noise removal filter 154 is stored in the image area of the image processing storage unit 13.

  The area processing unit 156 performs a process for converting the difference binarized image from which noise has been removed by the noise removal filter 154 into a rectangular area on the screen. In this area conversion process, the change pixel area on the screen extracted as the difference binarized image is surrounded by a rectangle, and this rectangle area is extracted from the screen, whereby the difference binarized image is Region into regions. The region processing unit 156 performs region tracking using the upper left (x0, y0) and lower right (x1, y1) coordinate data indicating the position and size on the screen indicating the rectangular region as rectangular region information. The data is sent to the processing unit 157.

  The region tracking processing unit 157 performs processing for tracking the rectangular region extracted by the regioning processing unit 156 based on the rectangular region information received from the regioning processing unit 156. The area tracking processing unit 157 replaces the difference binarized image extracted by comparing images (animal body images) with motion on the screen every time the area processing unit 156 extracts a rectangular area. The rectangular area extracted by the processing unit 156 is tracked as a tracking target object. The area tracking processing unit 157 includes a confirmed object tracking unit 157a, an unconfirmed object tracking unit 157b, a new object registration unit 157c, and an unnecessary object removing unit 157d.

  The unconfirmed object tracking unit 157b tracks an object (unconfirmed object) for an area on the screen excluding the object (confirmed object) tracked by the confirmed object tracking unit 157a.

  The new object registration unit 157c performs a new object registration process for the area on the screen excluding the object tracked by the confirmed object tracking unit 157a and the object tracked by the indeterminate object tracking unit 157b.

  The unconfirmed object tracking unit 157b has a processing function for generating and storing predetermined information used by the confirmed object tracking unit 157a for region prediction for each region being tracked. In addition, the confirmed object tracking unit 157a uses the region information generated by the unconfirmed object tracking unit 157b each time region tracking is performed (confirms the tracking object), continues tracking, and is used for tracking. It has a processing function of updating predetermined information used for region prediction using region information.

  The confirmed object tracking unit 157a uses the unconfirmed object that has been continuously tracked (successfully tracked) by the unconfirmed object tracking unit 157b as a confirmed object, and continuously tracks the object. Each time the segmentation processing unit 156 extracts a rectangular region, the information on the rectangular region extracted by the segmentation processing unit 156 and the time-series position information generated by the unconfirmed object tracking unit 157b when tracking the unconfirmed object are included. The object (determined object) is tracked based on the prediction position and the prediction area information according to the prediction direction obtained by the least square approximation. The confirmed object tracking unit 157a receives information acquired by the unconfirmed object tracking unit 157b in the object tracking process over a certain period (the area width, the average value and the standard deviation of the height, and the mutual mutual screens described later). The validity of the area (neighboring area) used for tracking is checked using the average value and standard deviation of the speed of the rectangular area moving on the screen obtained by the association (steps S25 to S27 in FIG. 9). reference). If the area is inappropriate, tracking is continued using the prediction information. Further, the position of the current area is predicted using the predicted moving direction information obtained by the least square (square) approximation formula based on the position and speed information of the area acquired by the tracking process of the uncertain object, and the vicinity of the position Correspondence with the region that appeared in is performed. The tracking process of the definite object ends with the number of continuous tracking (for example, 5 times) or the total number of tracking (for example, 300 times) based on the prediction information according to the set parameter value. Further, the confirmed object tracking unit 157a updates the area information including the object information, the moving speed information, and the prediction direction generated by the unconfirmed object tracking unit 157b in the object tracking process. Specific examples of these pieces of information will be described later.

  The unconfirmed object tracking unit 157b tracks the new object registered by the new object registration unit 157c as an unconfirmed object, and whenever the regionizing processing unit 156 extracts a rectangular region, the previously extracted rectangular region ( A prediction region with an enlarged region width and height) is temporarily generated, and when the rectangular region extracted this time intersects with the prediction region, the currently extracted rectangular region is regarded as an undetermined object and object tracking is performed. I do.

  The unconfirmed object tracking unit 157b enlarges the width (x direction) and height (y direction) of the previously extracted rectangular area each time the area processing unit 156 extracts the rectangular area. When the rectangular area extracted this time intersects with this prediction area as the prediction area, the object is traced with the rectangular area extracted this time as an undetermined object. In this embodiment, the tracking process is simplified and speeded up by generating a prediction area in which the width and height of the area are expanded in accordance with a preset parameter value (ew / eh). In each case, it is possible to set the width and height of the prediction region based on the information of the region that changes in time series, and to make the width and height that the prediction region expands variable.

  Further, the unconfirmed object tracking unit 157b is information necessary for the object tracking unit 157a to continuously track the object based on the information of the region that changes in time series in the object tracking process. Is generated. Here, in the object tracking process, predetermined information including area information including object information and moving speed information is generated based on information of areas changing in time series. Object information includes rectangle and area information that identifies the area, and area cutout image (not required for area tracking), and area information includes area width information (area width, area width square value) , Region height information (region height, region height square value), region width statistical information (average value, standard deviation, variation rate, sum, square sum (average value and variation rate are used when generating forecast information)), Region height statistical information (average value, standard deviation, variation rate, sum, sum of squares, etc. (average value and variation rate are used when generating prediction information)) and the like are included. Moving speed information includes moving speed information (moving speed, moving speed square value), moving speed statistical information (average value, standard deviation, fluctuation rate, sum, square sum (average value and fluctuation rate are used when generating forecast information) ) Etc. are included.

  The confirmed object tracking unit 157a uses the information generated by the tracking process of the unconfirmed object tracking unit 157b to perform the tracking process of the confirmed object while updating the information.

  In the above-described confirmed object tracking unit 157a and unconfirmed object tracking unit 157b, a confirmed object identification process for identifying a confirmed object being tracked is provided as an additional function.

  An outline of this additional function is shown by functional blocks in FIG.

  The unconfirmed object tracking unit 157b identifies the confirmed object when the unconfirmed object that has been continuously tracked (successfully tracked) a plurality of times is continuously tracked as a confirmed object by the confirmed object tracking unit 157a. In order to carry out the processing, the image pattern (P1 to P3) of the local area (A1 to A3) is cut out from the rectangular area of the object, and the cut image pattern (P1 to P1) is determined. P3) is provided with a pattern extraction processing function of a local region that is held in a pattern model holding unit provided in the image processing storage unit 13 as an initial pattern model (template) to be subjected to matching evaluation.

  In addition, since the confirmed object tracking unit 157a performs the identification process of the confirmed object, every time the determined object is determined, the image pattern (P1 to P3) of the local area (A1 to A3) from the rectangular area of the new confirmed object is determined. ), And the image pattern (P1 to P3) of the newly confirmed object thus cut out is collated with the image pattern (P1 to P3) of the already confirmed object held as a template in the pattern model holding unit. Matching evaluation is performed based on the correlation coefficient of the tracking object existing position set in the parameter table (CC-TBL) (for example, the foot position of the object in the case of the center position of the lower end of the rectangle) (PF). , Similarities of tracking objects that determine the presence / absence of correlation of the tracking object When the fixed processing function and the similarity determination process determine that there is a correlation between the tracked confirmed objects, the currently tracked confirmed object is identified (the latest confirmed object is the currently tracked confirmed object) And update the image pattern (P1 to P3) held in the pattern model holding unit to the latest image pattern (P1 to P3) cut out from the newly confirmed object (update the template image), An image pattern update processing function is provided in preparation for the next matching evaluation.

  The confirmed object identifying process for identifying the confirmed object being tracked will be described later with reference to FIGS.

  The new object registration unit 157c recognizes as a new object a rectangular area newly appearing in an area on the screen excluding the tracking area of the object being tracked by the confirmed object tracking part 157a and the unconfirmed object tracking part 157b. And register it. In this new object registration process, object information indicating the position (coordinates) and area of a rectangular area is generated based on the area information received from the area processing unit 156, and this object information is stored in the image processing storage unit 13. Are stored in a predetermined storage area.

  The unnecessary object removing unit 157d performs a process of deleting unnecessary objects (determined objects and indeterminate objects). The unnecessary object removing unit 157d deletes the unconfirmed object or the confirmed object, for example, when the object being tracked as the unconfirmed object or the confirmed object ends (deletes the unnecessary object from the image processing storage unit 13). ).

  The display unit 16 displays and outputs the moving object region image-processed by the image processing unit 15.

  Here, the object tracking processing operation in the image processing unit 15 will be described with reference to FIGS.

  FIG. 2 shows the object tracking processing procedure in the image processing unit 15 described above.

  In the image processing unit 15, the difference processing unit 152 compares the past image and the current image held in the image buffer 14 in pixel units, and extracts the changed pixel unit region as a difference region (step S11). The image subjected to the difference processing by the difference processing unit 152 is binarized by the binarization processing unit 153, and is segmented into a rectangular area on the screen by the area processing unit 156 (step S12).

  When the area information processing unit 156 inputs the rectangular area information that has been converted into an area by the area processing unit 156, the area tracking processing unit 157 performs tracking processing on the input rectangular area information as a tracking target object.

  At the time of initial operation, there are no objects (undetermined objects and confirmed objects) to be tracked, so the rectangular area input from the area processing unit 156 is registered as a new object in the new object registration unit 157c. (Step S15).

  When a new object is registered in the new object registration unit 157c, the region processing unit 156 extracts a rectangular region again. In the next processing cycle and thereafter, the unconfirmed object tracking unit 157b is updated by the new object registration unit 157c. The tracking process is started with the registered new object as an undetermined object (step S14).

  The tracking of the unconfirmed object in the unconfirmed object tracking unit 157b is performed by using the position and area information of the object extracted last time for the object entered in the registration process of the new object registration unit 157c. The position of the appearance area (hereinafter referred to as the prediction area) is predicted as follows.

  As shown in FIG. 3, each time the uncertain object tracking unit 157 b extracts a rectangular region, the regionizing processing unit 156 generates a prediction region 302 obtained by enlarging the previously extracted rectangular region 301. In addition, when the rectangular area 303 extracted this time intersects, the object is traced with the rectangular area 303 extracted this time as an undetermined object.

  Each time the region processing unit 156 extracts a rectangular region, the unconfirmed object tracking unit 157b performs the position coordinates (x0, y0), (x1, y1) of the region on the previously extracted rectangular region 301, Also, the center coordinates (center of gravity position) cp (mid X, mid Y) obtained from the width and height information of the area generated based on the coordinate values and the set parameter values (ex / eh) are also obtained. In addition, a prediction region 302 is generated in which the region width is expanded by ex pixels and the region height is expanded by eh pixels in both directions, and when the rectangular region 303 extracted this time intersects with the prediction region 302, The extracted rectangular area 303 is set as an undetermined object, and object tracking is continued. Note that the prediction area 302 is a temporary area that disappears with the end of the current tracking process.

  At this time, when no rectangular area extracted this time intersects the prediction area, the unnecessary object removing unit 157d deletes the unconfirmed object as an unnecessary object (step S16).

  Further, when the rectangular area extracted by the area processing unit 156 is a tracking target area, the unconfirmed object tracking section 157b determines the position coordinates (x0, y0), (x1, y1) of the area and the position coordinates. Each information such as the width (x1-x0) and height (y1-y0), area, and center coordinates cp (mid X, mid Y) of the generated area is determined in advance in the image processing storage unit 13. Store in the storage area.

  By the tracking process of the above-mentioned unconfirmed object tracking unit 157b, a certain unconfirmed object is continuously tracked a plurality of times, and the number of tracking processes becomes the number of tracking according to a preset parameter value (for example, five times). At this time, it is determined whether or not this unconfirmed object is to be a confirmed object. This determination process may be performed by either the confirmed object tracking unit 157a or the unconfirmed object tracking unit 157b, but here, it is assumed that the unconfirmed object tracking unit 157b performs.

  When the unconfirmed object is continuously tracked for a predetermined number of times (here, 5 times), the unconfirmed object tracking unit 157b is based on the information of each region generated and collected when the unconfirmed object is continuously tracked. , Area width, area height, and moving speed standard deviations are obtained, and the standard deviations are set in advance by parameters according to the allowable standard deviation (s) shown in FIG. The allowable standard deviation is set with different parameters). Here, when each standard deviation of the region width, region height, and moving speed generated based on the information of the unconfirmed object is within the allowable standard deviation (s) set by the parameters, the unconfirmed Let the object be a definite object. When at least one standard deviation among the standard deviations of the area width, area height, and moving speed generated based on the information of the undefined object is larger than the allowable standard deviation (s) set by the parameter, The tracking of this undetermined object is terminated because the variation of the appearance area is large. The undetermined object is deleted by the unnecessary object removing unit 157d.

  In this unconfirmed object tracking process, images (P1 to P3) of the local regions (A1 to A3) are cut out from the rectangular regions of the object, and the cut image patterns (P1 to P1) are determined. The pattern extraction process of the local area | region hold | maintained in the pattern model holding | maintenance part provided in the image processing memory | storage part 13 is implemented by making P3) into the pattern model holding | maintenance part 13 provided as the first pattern model (template) used as the object of matching evaluation.

  With this determination, the confirmed object tracking unit 157a calculates a prediction direction, a region width, a region height, and a moving rate variation rate for the object determined to be a confirmed object, and uses these information to perform region prediction. Then, the tracking process is continuously performed for the object determined as the confirmed object (step S13).

  Whenever the region processing unit 156 extracts a rectangular region, the confirmed object tracking unit 157a performs the unconfirmed object tracking unit 157b at the time of tracking the uncertain object with respect to the rectangular region extracted by the region processing unit 156. A prediction area is generated at a prediction position according to the prediction direction obtained by the least square approximation based on the generated time-series position information, and tracking of the definite object is started using the prediction area.

  As shown in FIG. 5, the tracking process of the fixed object includes the position and area of the previously extracted rectangular area (past area) 501, the barycentric position cp (mid X, mid Y) of the area, and the prediction direction. A prediction area 502 is generated based on the predicted movement direction Pd (angle θb) obtained from (d) and the movement speed, and the rectangular area (current area) 503 extracted this time intersects with this prediction area 502. At this time, the rectangular area 503 extracted this time is set as a confirmed object, and the object tracking is continued. In this case, as shown in FIG. 6, the predicted moving direction Pd (angle θb) is calculated by obtaining a coefficient A and a coefficient B by the least square method from the position statistical information and calculating a correlation coefficient. Then, using the current position information (x) and the oldest position information (x), when the correlation coefficient of the approximate line is high, Y on the approximate line with respect to the latest and oldest position information x is obtained, The predicted movement direction Pd (angle θa) is obtained from the position information of When the correlation is low, the predicted moving direction (angle θb) is obtained from the latest and oldest coordinate information y and used.

  In the tracking process of the definite object, when a plurality of regions (neighboring regions) intersect with the prediction region 502, the plurality of neighboring regions are set as candidate regions, respectively, and from the candidate region having the largest intersection area, A validity judgment is performed, and a candidate area with high validity is set as a tracking target area (a confirmed object to be tracked). In this case, the validity is determined by using the area width, area height, moving speed, standard deviation of each statistical information × 3 (3σ) in FIG. The validity is judged, and when the difference is larger than (3σ), it is judged not valid. When the validity of at least two of the area width, area height, and moving speed is confirmed, the candidate area is set as a tracking target area (determined object to be tracked). When the validity is not confirmed, the same determination process is performed on the next candidate area. When the tracking target region is determined, the region information is added, and the region width statistical information, the region height statistical information, the moving speed statistical information, and the prediction direction are updated. If the tracking target area cannot be determined, the statistical information and the prediction direction are updated using the prediction area as the current area. In the tracking process of the definite object, when a state where no region intersects the predicted region continuously occurs and the number of continuous tracking by the prediction information according to the set parameter value (for example, five times) is reached, Alternatively, the process ends when the total number of tracking times (for example, 300 times) is reached.

  In the tracking process of the confirmed object, it is determined that the object is a confirmed object to be tracked, and the validity is confirmed for the tracking target region (neighboring region), that is, for the confirmed object (candidate) determined to be valid. Thus, a process for identifying a confirmed object for identifying a confirmed object being tracked is performed. FIG. 9 shows a tracking process procedure of the confirmed object including this identification process. In the figure, the procedure for tracking a confirmed object described above is shown in steps S21 to S27, and the procedure for identifying a confirmed object for identifying a confirmed object is shown in steps S30 (S31 to S34). Details of step S31 shown in FIG. 9 are shown in FIG.

  By repeatedly executing the new object registration process (step S15), the unconfirmed object tracking process (step S13), and the confirmed object tracking process (step S14) as described above, It is possible to efficiently execute the tracking process of the change area. Further, by providing each processing function of the area tracking processing unit 157 as a program package, a simple and simple combination of an unconfirmed object tracking processing function and a confirmed object tracking processing function in an existing image processing system. An image tracking processing system capable of high-speed processing can be easily constructed.

  Next, with reference to FIG. 9 to FIG. 14, the operation of identifying a confirmed object for identifying the confirmed object being tracked will be described. FIG. 9 shows a tracking process procedure for a confirmed object, FIG. 10 shows a similarity determination processing procedure based on pattern matching performed in the tracking process for the confirmed object shown in FIG. 9, and the similarity determination processing operation based on this pattern matching will be described. FIG. 11 shows the relationship between the correlation determination parameter table (CC-TBL) and the tracking object location (PF), and an example of setting the local areas (A1 to A3) in which this pattern matching is performed is shown in FIG. Thru | or FIG. Here, as an example, the position (PF) of the tracking object is the foot position (rectangular bottom center position).

In addition, as a matching evaluation expression used for the identification process of the deterministic object for identifying the deterministic object being tracked, here, the SSD (the sum of squares of the luminance value difference) represented by the expression (1) and the expression (2) SAD (absolute value sum of luminance value differences) shown, NCC (Normalized Cross-Correlation) shown in Expression (3), and ZNCC (Zero-means Normalized Cross shown in Expression (4) -Correlation Zero average normalized cross correlation) is prepared, and one evaluation formula is selected by external parameters (Im and Tm in the formula are average values of the input image and the template image, respectively) Represents). The SSD shown in the equation (1) and the SAD shown in the equation (2) are more similar as the values are smaller. The NCC shown in the equation (3) and the equations (4) and (5) are shown. Since each ZNCC is closer to 1 (upper limit), the similarity becomes higher. Therefore, the selected evaluation is included in the correlation determination parameter table (CC-TBL) referred to in the confirmation process of the confirmed object for identifying the confirmed object. A threshold value (correlation coefficient) having a value specific to the equation is set.

  When performing the process of identifying a deterministic object that identifies a deterministic object being tracked, a matching evaluation formula for identifying the deterministic object being tracked is selected at the initial setting stage, and a pattern that conforms to this evaluation formula A correlation coefficient serving as a threshold for matching determination is set in the correlation determination parameter table (CC-TBL). In addition, for the rectangular area (AS) extracted when tracking the object, the foot position (PF) of the tracking object and the local area (A1 to A3) with the foot position (PF) as a reference position are external parameters. Set as

  In the process of identifying a confirmed object for identifying a confirmed object being tracked, there is an object being tracked that becomes a confirmed object from the unconfirmed object in the tracking process of the undefined object described above (the tracking process in step S14 shown in FIG. 2). Then, the images (P1 to P3) of the local areas (A1 to A3) are cut out from the rectangular area (AS) of the object, and the cut out image patterns (P1 to P3) are the first pattern models to be subjected to matching evaluation. A pattern extraction process of a local area held in a pattern model holding unit provided in the image processing storage unit 13 as a (template) is performed.

  As already described in the process of step S13 shown in FIG. 2, the confirmed object tracking unit 157a sets the unconfirmed object that the unconfirmed object tracking unit 157b has successfully tracked for a plurality of times as a confirmed object, and continues. To track the object. Here, every time the region processing unit 156 extracts a rectangular region, information on the rectangular region extracted by the region processing unit 156 and the time series generated by the unconfirmed object tracking unit 157b during tracking of the unconfirmed object The confirmed object is tracked based on the predicted position and the prediction area information according to the prediction direction obtained by the least square approximation based on the position information. In this tracking process, the screen obtained by the uncertain object tracking unit 157b obtained by the tracking processing of the object over a certain period and obtained by associating the area width, height average value and standard deviation, and successive screens with each other The validity of the area (neighboring area) used for tracking is checked using information such as the average value and standard deviation of the speed of the rectangular area moving above (steps S21 to S27 in FIG. 9). If the area is inappropriate, tracking is continued using the prediction information. If it is determined that the area is valid (step S27 in FIG. 9), the process of identifying the confirmed object currently being tracked is performed for the confirmed object determined to be valid (step S30 in FIG. 9). This identification process is carried out with the image set (P1 to P3) of the three areas (A1 to A3) set in the rectangular area (AS) as a determination target.

  This identification process cuts out the images (P1 to P3) of the local areas (A1 to A3) from the rectangular area (AS) of the object each time the validity of the confirmed object is determined. The tracking set in the correlation determination parameter table (CC-TBL) by comparing the image patterns (P1 to P3) with the image patterns (P1 to P3) of the confirmed objects held as templates in the pattern model holding unit. Matching evaluation is performed based on the correlation coefficient of the object's foot position (PF) to determine the presence or absence of correlation of the confirmed object being tracked, and the confirmed object determined to be valid is currently being tracked Similarity determination is made as to whether or not the object is present (step S31 in FIG. 9).

  If it is determined in this similarity determination process that the confirmed object being tracked has a correlation (Yes in step S32 in FIG. 9), a flag having a correlation is set in the similarity determination flag provided in the image processing storage unit 13, and the currently tracked object is being tracked. Identifying a confirmed object (confirming that the latest confirmed object is a confirmed object currently being tracked), and using the new confirmed object, image patterns (P1 to P3) held as templates in the pattern model holding unit The latest cut out image pattern (P1 to P3) is updated (the template is updated) to prepare for the next matching evaluation (step S33 in FIG. 9).

  Further, when it is determined that there is no validity in the determination of validity (No in step S27 in FIG. 9), or in the case where it is determined that there is no correlation in the determination of the presence or absence of correlation (No in step S32 in FIG. 9), Similarly to the case where it is determined that the region does not exist, that is, the region does not exist in the vicinity of the predicted position (No in Step S25 in FIG. 9), the confirmed object that is the target of the current determination is treated as a prediction that is excluded from the tracked object, Preparation for the next matching evaluation is performed without updating the template (step S34 in FIG. 9).

  FIG. 10 shows a processing procedure in the similarity determination (step S31 in FIG. 9) of whether or not the determined object determined to be valid is a determined object currently being tracked.

  In this similarity determination process, after the similarity determination flag provided in the image processing storage unit 13 is initially set to be correlated (step S311 in FIG. 10), the correlation determination parameter table (CC-TBL) is referred to to determine the validity. The correlation coefficient of the foot position (PF) of the rectangular area (AS) of the confirmed object determined to have the property is acquired (step S312 in FIG. 10).

  Here, when the correlation function of the foot position (PF) is off indicating that it is not used, it is determined that there is a correlation without performing the correlation presence / absence determination process, and the next similarity determination (matching evaluation determination) is performed. (Yes in step S313 in FIG. 10).

When the correlation function of the foot position (PF) acquired from the correlation determination parameter table (CC-TBL) is a decimal number of 1 or less,
The images (P1 to P3) of the local areas (A1 to A3) are acquired from the rectangular area (AS) of the determined object determined to be valid as the object pattern model of the currently determined area (step S314 in FIG. 10). The image pattern (P1 to P3) of the newly confirmed object is collated with the image pattern (P1 to P3) of the already confirmed object held as a template (existing object pattern model) in the pattern model holding unit, for correlation determination Matching evaluation is performed based on the correlation coefficient of the foot position (PF) set in the parameter table (CC-TBL) (steps S315 to S316 in FIG. 10).

  If it is determined that there is a correlation, the current similarity determination (matching evaluation determination) is terminated with the correlation determination flag set in the similarity determination flag (Yes in step S316 in FIG. 10).

  If it is determined that there is no correlation (No in step S316 in FIG. 10), a non-correlation flag is set in the similarity determination flag and the process proceeds to the next similarity determination (matching evaluation determination) (step S317 in FIG. 10).

  As described above, since the confirmation process for identifying the confirmed object being tracked is performed every time the tracking process for the confirmed object is performed, it is possible to monitor the movement of the moving object region with high reliability with improved tracking performance. It becomes possible.

  Specific examples in the case where local areas (A1 to A3) are set in a rectangular area (AS) of a definite object that surrounds a moving object and is a target to be tracked are shown in FIGS. FIG. 12 shows an example of setting the local area (A1 to A3) when the moving object to be tracked is a person. Example of setting the local area (A1 to A3) when the moving object to be tracked is a vehicle Is shown in FIG. 13 and FIG. FIG. 13 shows an example in which a vehicle is photographed from diagonally above the road, and FIG. 14 shows an example in which the vehicle is photographed from diagonally above the road. Here, in any case, an example is shown in which local regions are set at three locations indicated by reference signs A1, A2, and A3 in a rectangular region (AS). Each of the local areas (A1 to A3) is composed of a dot matrix of 8 pixels × 8 pixels. When the height size of the area (AS) is H and the width is W, the local area (AS) The half position of the base is the foot position (PF) of the tracking object, the foot position (PF) is the lower end in the vertical direction, and the coefficients (P1, A rectangular pixel set (block image) of 8 pixels × 8 pixels multiplied by P2, P3 = external parameters) is set. Here, in any case, three regions of P1 = 11/12, P2 = 3/4, and P3 = 1/2 are given as default values with a height position of 1 / 2H or more as a lower limit, and each is a template. Used.

By the way, if these three regions are the person shown in FIG. 12, an approximate value of the human body size is obtained from the human body measurement using the height of the human body, and there is little rapid change except for the lower body where the rapid change is severe. The head, chest and waist are used as templates. In the case of the vehicle shown in FIGS. 13 and 14 as well, P1 = 11/12, P2 = 3 / 4, 3 areas of P3 = 1/2 are given as default values. By setting the three regions in this region (AS) as local regions (A1 to A3) as template extraction regions, the most effective tracking performance was obtained. Image with improved tracking performance with a slight increase in processing load that does not affect the real-time processing function due to the identification processing of the confirmed object by the pinpoint pattern matching evaluation by this local region (A1 to A3) A processing system can be constructed.

  In the above-described embodiment, three local regions (A1 to A3) are set in the rectangular region (AS) of the confirmed object. However, the present invention is not limited to this, and the processing performance and processing capability of the system are not limited. In consideration of this, it is possible to set a local region of any part of two or more parts. Further, in the above-described embodiment, as a matching evaluation expression used for the confirmation process of the confirmed object for identifying the confirmed object being tracked, here, the SSD (the sum of squares of the luminance value difference) represented by the expression (1), SAD (sum of absolute values of differences in luminance values) shown in equation (2), NCC (normalized cross-correlation) shown in equation (3), and ZNCC (in equation (4)) Although an example of preparing four types of evaluation formulas (Zero-means Normalized Cross-Correlation) has been shown, the present invention is not limited to this. For example, even in functional design using only one type of evaluation formula Good. In short, various design changes and processing procedure changes can be made without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 11 ... Camera, 12 ... Capture part, 13 ... Image processing memory | storage part, 15 ... Image processing part, 16 ... Display part, 151 ... Pre-processing part, 152 ... Difference processing part, 153 ... Binarization processing part, 154 ... Noise Removal filter, 156... Regionalization processing unit, 157... Region tracking processing unit, 157 a... Confirmed object tracking unit, 157 b... Unconfirmed object tracking unit, 157 c. Correlation determination parameter table, AS... Rectangular area, PF... Tracking object existence position (for example, the position of the foot of the object when it is the center position of the lower end of the rectangle), A1 to A3.

The outline of this additional function is shown in functional blocks in FIG.

Above, registration of a new object and (step S15), and the tracking process undetermined object (step S 14), by repeatedly executing the tracking processing of the deterministic object (step S 13), captured by the camera 11 The tracking process of the change area on the screen can be executed efficiently. Further, by providing each processing function of the area tracking processing unit 157 as a program package, a simple and simple combination of an unconfirmed object tracking processing function and a confirmed object tracking processing function in an existing image processing system. An image tracking processing system capable of high-speed processing can be easily constructed.

Claims (11)

The captured image input at a predetermined cycle is sequentially subjected to differential processing in the order of input, a rectangular area including the changed pixel is extracted, and the extracted rectangular area is generated from a plurality of past rectangular areas. In an image processing apparatus that tracks
A plurality of local regions in the rectangular region, and an external parameter setting means for setting a threshold value for pattern matching determination in pixel units at pixel positions constituting the rectangular region;
Pattern extraction means for local regions that cut out the image pattern of each of the local regions from the rectangular region being tracked;
A local area pattern holding means for holding the image pattern as a template for matching evaluation;
Each time the local area pattern extraction means cuts out the image pattern, the cut image pattern is compared with the image pattern held in the local area pattern holding means, and matching evaluation is performed based on the threshold value. Similarity determination means for determining the presence or absence of correlation of the rectangular region being tracked,
When the similarity determination unit determines that there is a correlation between the rectangular regions being tracked, the latest image pattern obtained by the local region pattern extraction unit cutting out the image pattern held in the local region pattern holding unit. A template updating means for updating to
An image processing apparatus comprising: A difference extraction process is performed between the current image and the past image on the screen input at a predetermined cycle to extract a rectangular area including the changed pixel, and a new one of the areas extracted by the area extraction processing means A new object registration processing means for registering the extracted area as a new object to be tracked and an object registered by the new object registration processing means as an undetermined object. An unconfirmed object tracking processing means for tracking the unconfirmed object based on information indicating the position and size of the region extracted this time and information indicating the position and size of the region extracted last time, The unconfirmed object tracked for a predetermined time by the unconfirmed object tracking processing means Each time the area extraction processing means extracts an area as a confirmed object, information indicating the position and size of the extracted area, and least square approximation based on time-series position information at the time of tracking the undefined object An image processing apparatus comprising: a confirmed object tracking processing unit that tracks the confirmed object based on information indicating the position and size of a prediction area that is updated each time tracking is performed according to a prediction direction obtained by the above and movement speed information. ,
A plurality of local regions in the rectangular region, and an external parameter setting means for setting a threshold value for pattern matching determination in pixel units at pixel positions constituting the rectangular region;
Extracting a local area pattern that cuts out an image pattern of each local area from a rectangular area of the object each time the deterministic object is determined, with a new definite object that has become a definite object from the undetermined object as an initial processing target Means,
A local area pattern holding means for holding the image pattern as a template for matching evaluation;
Each time the local area pattern extraction means cuts out the image pattern, the cut image pattern is compared with the image pattern held in the local area pattern holding means, and matching evaluation is performed based on the threshold value. Tracking object similarity determination means for determining the presence or absence of correlation of the confirmed object being tracked,
When the similarity determination unit of the tracking object determines that there is a correlation between the confirmed objects being tracked, the confirmed object currently being tracked is identified, and the image pattern held in the pattern holding unit of the local region is Template update means for updating to the latest image pattern extracted by the pattern extraction means of the area;
An image processing apparatus comprising: In the local area, the position in the rectangular area is set with respect to the existing position, with the predetermined position of the base in the rectangular area being the existing position of the tracking object surrounded by the rectangle,
The tracking object similarity determination unit includes a correlation determination table in which a correlation coefficient for pattern matching evaluation serving as the threshold is set in units of pixels of the input screen, and the correlation determination table at the existing position The image processing apparatus according to claim 1, wherein similarity determination of the tracking object is performed using the above correlation coefficient.   The local area is a rectangular area for a partial area having a low rate of change based on a distribution of changed pixels in a rectangular area extracted from an image obtained by photographing an object model assuming a moving object to be tracked. The image processing apparatus according to claim 1, wherein the image processing apparatus is set as a small image cutout window of the tracking object at a plurality of locations.   A plurality of local regions are defined in the rectangular region based on a height H from the existence position, and each image in the region is a target of the matching evaluation as an object model pattern. The image processing apparatus according to claim 3 or 4.   The tracking object similarity determination means includes a plurality of sets of pattern matching evaluation formulas having different calculation methods, and the external parameter includes any one of the plurality of sets of pattern matching evaluation formulas. The image processing apparatus according to claim 1, further comprising a parameter specified as an evaluation formula for pattern matching performed by the tracking object similarity determination unit.   The image processing apparatus according to claim 6, wherein the correlation coefficient is set in the correlation determination table based on an evaluation formula selected from a plurality of sets of pattern matching evaluation formulas. The captured image input at a predetermined cycle is sequentially subjected to differential processing in the order of input, a rectangular area including the changed pixel is extracted, and the extracted rectangular area is generated from a plurality of past rectangular areas. An area tracking program for causing a computer to function as an image processing apparatus for tracking
A setting function of an external parameter that sets a plurality of local areas in the rectangular area, and sets a threshold value for pattern matching determination in pixel units at pixel positions constituting the rectangular area;
A pattern extraction function of a local area that cuts out an image pattern of each of the local areas from the rectangular area being tracked;
A local area pattern holding function for holding the image pattern as a template for matching evaluation;
Each time the local area pattern extraction means cuts out the image pattern, the cut image pattern is compared with the image pattern held in the local area pattern holding means, and matching evaluation is performed based on the threshold value. A similarity determination function for determining the presence or absence of correlation of the rectangular region being tracked,
An area tracking program for causing a computer to realize a template update function for updating the held image pattern to the cut-out latest image pattern when it is determined that there is a correlation in the matching evaluation. An area in which an imaged screen shot with a camera is input at a predetermined cycle, and a rectangular area including a changed pixel on the screen is extracted by performing a difference process on the image on the screen input this time and the image on the screen input last time. It is determined whether the area extracted by the extraction function and the area extraction function is a newly extracted area on the screen, and when the area is a newly extracted area, For a new object registration processing function to be registered as a new object to be tracked and an object registered by the new object registration processing function, the region extraction function extracts the region as an unconfirmed object. Each time, the information indicating the position and size of the area extracted this time and the information indicating the position and size of the area extracted last time are used. With respect to an indeterminate object tracking process function that tracks a deterministic object and an undetermined object that the undetermined object tracking process function has continuously tracked for a predetermined time, the area extraction process function sets the area as the definite object. A prediction area that is updated each time tracking is performed according to a prediction direction obtained by least square approximation based on information indicating the position and size of the extracted area and time-series position information at the time of tracking the undefined object. An area tracking program for causing a computer to function as an image processing apparatus having a tracking process function of a confirmed object that tracks the confirmed object based on information indicating position and size and movement speed information,
A setting function of an external parameter that sets a plurality of local areas in the rectangular area, and sets a threshold value for pattern matching determination in pixel units at pixel positions constituting the rectangular area;
Extracting a local area pattern that cuts out an image pattern of each local area from a rectangular area of the object each time the deterministic object is determined, with a new definite object that has become a definite object from the undetermined object as an initial processing target Function and
A local area pattern holding function for holding the image pattern as a template for matching evaluation;
Each time the image pattern is cut out, the cut-out image pattern and the held image pattern are collated, and matching evaluation is performed based on the threshold value to determine the presence or absence of correlation of the confirmed object being tracked. A tracking object similarity determination function,
When it is determined that there is a correlation in the matching evaluation, a template that is currently being tracked is identified and the held image pattern is updated to the cut-out latest image pattern to prepare for the next matching evaluation An area tracking program that allows a computer to implement an update function. In the local area, the position in the rectangular area is set with respect to the existing position, with the predetermined position of the base in the rectangular area being the existing position of the tracking object surrounded by the rectangle,
The tracking object similarity determination function includes a correlation determination table in which a correlation coefficient for pattern matching evaluation is set in units of pixels of the input screen, and a phase on the correlation determination table at the existing position. 10. The area tracking program according to claim 8, wherein similarity determination of the tracking object is performed using a relation number.   The similarity determination function of the tracking object includes a plurality of sets of pattern matching evaluation expressions having different calculation methods, and the correlation determination table stores the phase matching expressions based on the evaluation expressions selected from these evaluation expressions. 11. The area tracking program according to claim 10, wherein a relation number is set.

Images