JP2011198244A - Object recognition system, monitoring system using the same, and watching system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a recognition system which can stably and precisely obtain background in any situation, and does not fail in detecting even if influenced by illumination fluctuation, shielding, etc., and to provide a monitoring system and a watching system.SOLUTION: The recognition system functions fundamentally based on majority logic. The recognition system obtains a background parameter which reflects background by an observation means, and a presence position parameter reflecting an object's presence position, determines whether the background parameter exists within the predetermined limit with a predetermined threshold or more, if the background parameter exists, sets the background parameter as a parameter which shows a background candidate area, determines whether the presence position parameter exists within the predetermined limits with the predetermined threshold or more, and if the presence position parameter exists, sets the presence position parameter as a parameter which shows the object candidate area.

Description

  To ensure security, such as the discovery and tracking of suspicious persons (suspicious persons) such as terrorists, tails, pickles, fights, etc. in public places such as subways, stations, airports, etc. Detecting and recognizing human postures / motions / movements and the absence of an owner to detect abnormalities as an indispensable technology for surveillance systems or systems for monitoring elderly people, children, sick people, etc. in buildings such as homes and schools The present invention relates to a monitoring system and a monitoring system using sensor information processing technology.

  Most conventional surveillance systems use fixedly installed surveillance cameras. On the other hand, with regard to the monitoring system for the elderly and children using portable sensors, there are a few cases where children have a GPS-equipped mobile phone, and there are no others. Most conventional surveillance technologies use only one media type (mainly a video camera), and even if it can cover a wide area, the detection of suspicious persons, suspicious objects, and abnormal situations can be detected by human eyes. I had to rely on it. The present invention reinforces these drawbacks and aims to realize a highly reliable automatic monitoring system and watching system.

  Other than surveillance cameras, suspicious persons and suspicious objects are typically detected by X-ray inspection at airports, visual inspection by officials, thermography inspection to detect fever, facial photographs and fingerprint verification. In past research, automatic detection of stationary objects and people with specific movements using image processing algorithms is limited to motion recognition (walking, running, falling, etc.) in a relatively stable and less complex environment. Therefore, it has not reached a level where a practical 24-hour automatic monitoring system and a monitoring system can be realized.

Teddy Ko, “A survey on behavior analysis in video surveillance for homeland security applications,” 37th IEEE Applied Imagery Pattern Recognition Workshop, Washington, DC, USA, pp. 1-8, Oct. 15-17, 2008.

  However, the conventional monitoring system and watching system have the following problems. In other words, the background difference is often used to detect the target stationary object, but an accurate background is necessary for this purpose. In the method using the moving average over time, there is a problem that an error becomes large if there is an influence of a shield, a shadow, a reflection by a moving object, or a change in illumination conditions.

In addition, detection failure (undetected and false detection) often occurs due to lighting fluctuations or shielding by moving objects around the target, and this is caused by missing observation values or noise. The problem is that the target object cannot be detected even if it is present, or it is erroneously detected even if it does not exist. There is a problem that post-processing becomes difficult because the target object is detected as one region.

Furthermore, there is a problem that the accuracy of the position of the object obtained from the background difference is lowered due to the influence of noise or the like.

  The present invention is an object recognition system proposed to solve such a problem, and includes an observation means for observing a monitoring area and / or a monitoring area within a certain period of time, and observation by the observation means. The background determination means for determining whether or not the obtained background parameter reflecting the background exists within a predetermined range at a predetermined threshold value or more, and the background determination means exists at a predetermined threshold value or more. The background parameter within the predetermined range is set as a parameter indicating the background candidate region, the background extraction means for extracting the background, and the object obtained by the observation of the observation means Object determination means for determining whether or not the presence position parameter of the object reflecting the presence position is within a predetermined range at a predetermined threshold value or more, and the object determination means is a predetermined threshold value If it is determined that the target exists, the target parameter corresponding to the target position parameter within the predetermined range is set as a parameter indicating the target candidate area, and the target is extracted. And an object extraction means.

In order to determine “greater than or equal to a predetermined threshold”, it is only necessary to indicate that a large number of the parameters exist, and “a predetermined ratio or more”, “a predetermined number or more”, or “a predetermined cumulative value or more”. Alternatively, the determination may be made by using any criterion of “within a certain range where there are many parameters”. Typical parameters that reflect the contents and attributes of the background and / or object include background parameters (for example, pixel values) and object parameters (for example, pixel values) as parameters that reflect the background and the object itself. There are an object location parameter (for example, a region centroid) as a parameter reflecting the presence of the object, a movement of the centroid of the object as a parameter reflecting the movement of the object, and the like.

Further, the background determination unit removes a portion determined not to exist at a predetermined threshold or more from the background candidate region, and the background parameter within the predetermined range set by the background extraction unit is determined as the background candidate region. It may be provided with a background estimation means that is set as a parameter to be shown and determines a background parameter from the background candidate region.

Further, when it is determined that the object determination unit exists at a predetermined threshold value or more, a parameter corresponding to the object presence position within the predetermined range set by the object extraction unit is used as the object determination unit. There may be provided an object extracting means for setting an object parameter corresponding to the obtained area, which is set as a parameter indicating a candidate area of the object of the means.

According to the present invention, it is possible to stably obtain a highly accurate background in any situation by adaptively extracting and updating the background. In addition, even if affected by fluctuations in lighting or shielding by moving objects around the target, the target is present due to such missing observation values or noise without failing in detection. However, it is possible to solve the problem that it cannot be detected or erroneously detected.

When the object location parameter at different time points are overlapped, the overlap parameter determination means for determining whether or not the overlapped parameter is equal to or greater than a predetermined threshold, and the overlap parameter determination means is equal to or greater than the predetermined threshold If it is determined, an object area extracting unit that extracts an area corresponding to a parameter equal to or greater than the threshold as an object area may be provided.

  Whether the correlation between the region parameters of the frame within a certain time from the appearance of the object to the present in the object region extracted as a stationary object by the object extraction means is greater than or equal to the first threshold value, Data of the object when it is determined that the determination unit determines whether the threshold value is equal to or lower than the first threshold value and equal to or lower than the second threshold value, and the determination unit is equal to or higher than the first threshold value. If the target is determined to be temporarily disturbed and determined to be equal to or greater than the second threshold and equal to or greater than the first threshold, the target is determined to be a stationary object, and the target is determined to be equal to or smaller than the second threshold. The object may include an object determination unit that determines that the object is a moving body with a small movement.

If it is possible to know whether the detected target part is completely stationary or slightly moving, it is useful for identifying whether it is a person or an object. Here, the degree of correlation has been described as having a value between 0 and 1 assuming a normalized correlation coefficient, but it is necessary to reverse the magnitude when using a distance or the like.

  The object detection system of the present invention is used for a surveillance system for ensuring security such as discovery and tracking of a suspicious person and discovery of a suspicious object, a monitoring system for elderly persons, children, sick persons, etc. in buildings such as homes and schools. be able to.

  According to the present invention, a highly accurate background can be stably obtained under any circumstances. In addition, even if affected by fluctuations in lighting or shielding by moving objects around the target, the target is present due to such missing observation values or noise without failing in detection. However, it is possible to solve the problem that it cannot be detected or erroneously detected.

Configuration diagram of the entire system including processing means in the present embodiment Process flow diagram of the entire system in this embodiment The figure which shows how to obtain | require the background in this embodiment Diagram showing parameters related to bounding box The figure which shows the detection method of the target stationary object The figure which shows the detection method of the stationary object using the bounding box The figure which shows the detection method of the object which moves constant Processing block diagram showing the process of detecting a suspicious object with no owner The figure which shows another detection method of the target stationary object The figure which shows the method of identifying the state of the detected stationary object The figure which shows the area segmentation method for the minute motion detection of the stationary object

  Hereinafter, embodiments of the present invention will be described based on the drawings and mathematical expressions. FIG. 1 is a configuration diagram of the entire system including processing means in the present embodiment. The system 100 includes one or more sensors 101 and 102 for observing a monitoring area and / or a monitoring area within a certain time, and a control unit 103 that analyzes and integrates data received from the sensors 101 and 102. The sensors 101 and 102 have a function as observation means for observing the presence of a background and an object within a certain time, and are configured by a fixed sensor 101, a portable sensor 102, and the like.

The following sensors may be used as observation means for observing scenes to be monitored and watched over within a certain period of time. The types of sensors are “0 dimension: position measurement GPS, temperature, humidity”, “1 dimension: accelerometer (3 axes), angular accelerometer (3 axes), inclinometer (3 directions, can be included in the angular accelerometer. ), Microphone (acoustic) ”,“ 2D: normal visible light camera, near infrared camera, far infrared camera, thermography ”,“ 3D: 3D data (range sensor, 3D position sensor, 3D motion sensor, 3D ranging sensor) ”and the like. Here, for the purpose of original use and intuitive intelligibility, explanation will be given mainly using images, but the same applies to the case of recognizing an object from observation data such as other media (such as three-dimensional data). Applicable to. Also, not only zero-order approximation but also linear approximation or higher-order approximation can be used to cope with various fluctuations, and the least mean square error (LMS) or maximum is used to minimize the error. A technique for obtaining an optimum value by minimizing an error may be properly used as necessary.

The control means 103 is a background determination means, background extraction means, background estimation means for extracting a background based on images acquired from the sensors 101, 102, an object determination means for extracting an object, an object Extraction means and object presence position estimation means are provided. Specifically, these means include, for example, a personal computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), and the like. By executing a program stored in a ROM, HDD, or the like, functions as each means are fulfilled.

Before describing each means, first, the flow of the entire system in the object recognition system (or monitoring system, watching system) of this embodiment will be briefly described. FIG. 2 is a processing flow diagram of the entire system in the present embodiment. First, the background as the initial value is obtained before the observation. Here, the background means a thing that is stationary for a certain period after the observation starts before the start of observation, regardless of the person or object, and the initial background is the background at the start of observation, the short-time background. A long-time background is distinguished by the length of stationary time. In other words, even if it appears from the middle, it may be a thing that has been in the scene for a certain period of time and has been stationary, but the short-time background and the long-time background are things that remain stationary within the scene. Depending on whether you want to detect as a stationary object.

As shown in FIG. 2, the data from the sensor 200 is received (S201), and the background is extracted and updated (S202). First, an object region is extracted based on a background difference, an inter-frame difference or the like (S203), finely adjusted in position and size (S204), segmented (S205), and region features are extracted (S206). . Position estimation in a three-dimensional real space is performed (S207), and the posture and movement of a target stationary object or a moving object that moves in a certain manner are detected, and a suspicious object is detected (S208 to S213). Specifically, it is determined whether it is an object or a person (S208). If it is an object, it is determined as a suspicious object (S209), and tracking of the person who brought this suspicious object is started (S210). If it is a person, an abnormal situation is detected from the recognition result of posture / motion (S211 and S212), and if an abnormality is found, a notification or the like is taken (S213). A series of processing is completed by these procedures, but the above processing is repeated during monitoring and watching time.

Next, how to obtain the background in this embodiment will be described with reference to FIG. FIG. 3 is a diagram showing how to obtain the background in the present embodiment. In FIG. 3, pixel values are selected as background parameters reflecting the background, but three-dimensional data or the like can also be selected. An area where the pixel value within a certain time is outside the specified range is detected as a temporary change due to shielding, shadows, reflections, or changes in lighting conditions, and the data is temporarily or partially missing Are updated by using the value of the part removed (pixel value or the like), or the illumination conditions that change at a constant rate are updated by approximating the change. Specifically, first, the background determination unit determines whether or not the pixel value reflecting the background exists within a predetermined range at a predetermined threshold or more. For example, the temporal change of each pixel is obtained, and it is determined whether or not the pixel value exists within a certain range. When it is determined that the pixel exists, the background extraction unit calculates the background value of the pixel at the position using the pixel value.

In contrast, when the data is temporarily disturbed (for example, occlusion, shadows, shadows, reflections, fluctuations in illumination conditions, etc. by the moving object), the background determination means has a time zone in which the pixel value does not exist above a predetermined threshold May be determined. In such a case, the background estimation means selects an optimal value using any of the following means from among the pixel values existing within a predetermined range at a predetermined threshold or more, and the position, It can also be the background value of the pixel in the time zone. Here, in order to determine an optimum value, a weighted moving average value (which may be a simple average value), a median value, a peak value of a histogram, a value having a high appearance frequency, or the like may be used.

In this way, it is possible to cope with background fluctuations by detecting what temporarily disturbs the data, removing it, and using linear approximation or the like of pixel values for a certain period of time. This method can also be applied to media other than images, such as sensor data and three-dimensional data. It can be determined by controlling the time interval according to the necessity how long the object is stationary in the scene. When using background subtraction, it is premised that an accurate background is obtained, and in order for the background update to work effectively, the moving object region is extracted with high accuracy, illumination fluctuations, etc. are not so severe, It is desirable that the influence of the above is small. In addition, the time when the object appears on the scene and starts to stop and the time when the object ends are also known.

Select from the largest eigenvalues of the correlation matrix of the region parameter of each frame until the cumulative contribution rate reaches a certain value or higher, and shield the region extracted by projection / backprojection using the corresponding eigenvector, The remaining area may be obtained as a background candidate area by considering it as a moving object or illumination variation and excluding it from the background candidate area. When obtaining the correlation matrix, if each frame is not one-dimensional data but two-dimensional or three-dimensional, a technique of making it one-dimensional by rearranging the array may be used.

Next, a procedure for recognizing an object will be described. An object is a stationary object or an object that moves in a certain manner and is a target of detection and recognition, and includes people and objects. Here, the target object position parameter is a parameter that reflects the target position of the target object. The target object silhouette, the center of gravity of the bounding box, the maximum and minimum values of the area, the absolute value of the background difference, and the like will be described later. Includes motion measures. The motion measure is used in correspondence with the position of the object. Taking the target image as an example, the position and size of the target object can be known by obtaining the area on the image corresponding to the position parameter. The background obtained by the method described above may be used for parameter detection. The object parameter is a parameter that reflects the object, and the background parameter is a parameter that reflects the background. Specific examples include pixel values and 3D position data. Both are collectively referred to as region parameters.

As shown in FIG. 4, the bounding box is ideally a minimum rectangle or a rectangular parallelepiped region surrounding a certain object, but the minimum one cannot always be extracted. In addition, if it is a parameter representing the position of the candidate object region instead of the center of gravity used as the existence position parameter, it can be replaced with other values, for example, the position of the upper left corner of the bounding box, the top point, It is also possible to select the lowest point. The silhouette is a black and white representation of an object (person, object) region extracted by background difference or the like. Three-dimensional data may be a generic term for distance image data in a three-dimensional space, and acquired from a range sensor or a three-dimensional ranging sensor that can obtain three-dimensional data by measuring the distance to an object. Can do.

FIG. 5 is a diagram illustrating a method for detecting a target stationary object. In this figure, the target object position estimation means estimates the target position of the target object using the x coordinate of the center of gravity of the bounding box as the target object position parameter. After that, the object area extraction means extracts the common part of the bounding box having the estimated parameters or the part having the overlap more than a certain threshold as the object area. FIG. 6 is a diagram illustrating a method for detecting a stationary object using a bounding box. Furthermore, it shows that the time for determining as a stationary object can be controlled. In addition, in order to detect a moving object that moves constantly, any object that reflects its movement and position may be used. For example, the periodicity of movement of an object, aspect ratio, moment ratio, speed, acceleration, angle Acceleration, inclination angle, movement efficiency (details will be described later), and the like can also be used. FIG. 7 shows a method for detecting a moving object that moves in a certain manner in this embodiment. This figure shows that it is possible to detect that the subject suddenly moved differently from the difference in the motion parameters. Hereinafter, the object detection procedure (step 1 to step 5) in this embodiment will be described in detail.

The object extraction means calculates the value of the parameter reflecting the presence (position, movement) of the object by observing the process of (Step 1) within a certain period of time (T).

Since it is assumed that the observed value may be missing in the process of (Step 2), the obtained parameters do not always exist continuously, and the values may vary. Therefore, a certain vertical width ± δ is provided, and variation within this range is allowed. Specifically, the object determining means determines whether parameter values equal to or greater than a certain threshold are concentrated within a range of ± δ. If it is concentrated (≧ kT, 0 <k <1), it is determined as a candidate area of the target stationary object. The real coefficient k is determined by how much omission is allowed. The threshold value δ is determined based on the average value of the observation parameter fluctuation of the object, but it can be set larger with a little margin (about 10% to 30%) in order to increase the recall rate. ) In general, the precision rate is lowered, so it is determined according to the requirements of the time. Moreover, you may employ | adopt a median.

Regarding the processing of (Step 3), the place where the object is determined to be blocked in Step 3 (that is, when it is determined that the object determination means is not concentrated) is determined that the object update means is not concentrated. The error can be reduced by removing the parameter from the calculation and recalculating based on the parameter determined in step 2. By doing so, the position estimation accuracy is improved, a more accurate candidate region can be detected, and the accuracy of the parameter reflecting the position is also improved. This process (step 3) is an option (selection), and may be skipped if the parameters are obtained stably and accurately in the first process.

If the target is detected after repeating the above steps as necessary for the processing of (Step 4), it can be identified as a person or an object using a shape measure or a motion measure. Then, it becomes possible to detect a suspicious object / suspicious person by posture / motion recognition or the like. Here, the shape measure is a mono sashi that measures the shape of a thing. There are histograms, HOG, etc. In addition, HOG (Histgrams of Oriented Gradients) features are gradient-based features and Fourier descriptors (Fourier
Descriptor is a feature amount representing the shape of the boundary curve of the object. The motion measure is a monosashi that makes a difference in motion, and includes speed, acceleration, angular acceleration, tilt angle, movement efficiency (for example, SR, DR), and the like. Here, a stationary object other than a person and having no owner is regarded as a suspicious object.

When a suspicious object is detected in the process of (Step 5), if someone has carried it, the position parameter between the person and the suspicious object is the common part somewhere within the observation time. You should be able to track the person who brought the suspicious object from that point. If you go back in time, you can also get information on where the person came from. If suspicious behavior is detected, the person can be traced.

By taking the above steps, intermittent data is added even if the target object is a stationary object, but there is insufficient continuous time to determine that it is a stationary object. Detection is possible by (> threshold). Regardless of the person or object, the target stationary object does not exist in the observation scene at a certain time, for example, when starting the first observation, and then exists in the scene from the middle and keeps still. It points to what is. For example, it refers to something that someone has left behind, deliberately left behind, or a person who walks and sits in a chair. Those that existed from the beginning of the observation are not included. In addition, even if it appears from the middle, it may be present in the scene for a certain period of time and remain stationary.

FIG. 5 exemplarily shows the processing when the center of gravity (x coordinate) of the bounding box is used as a parameter. Here, the description will be made using an image for intuitive understanding, but the present invention can be similarly applied to cases where observation data of other parameters (such as three-dimensional data and an acceleration sensor) is missing. In order to minimize the error, generally, a method of obtaining an optimum value by performing linear approximation and minimizing the minimum mean square error or the maximum error is used in many cases, but the zero order is used to cope with various fluctuations. Not only approximation but also higher order approximation may be used. In addition, BB ₂₁ , BB ₂₂ , and BB ₃₁ are included in BB ₁ (including motion prediction), or there is a certain amount of common parts included, and BB ₂₁ and BB ₃₁ are in a stationary state for a certain amount. It is found that the object is a stationary object, and after another person is shielded at time t ₄ , it appears again as a single unit at time t ₅ as BB ₅ . After this, it is necessary to determine whether the object is a person or an object, and if it is an object, it is determined to be a suspicious object. Since the person BB ₃₂ who went back from time t _{1 to} t ₃ and carried away the suspicious object is identified as a suspicious person, tracking may be started. Here, BB represents a bounding box, and (G _x , G _y ) represents its center of gravity. Taking an image as an example, it is possible to know the position and size of an object by obtaining an area on the image corresponding to a parameter.

Hereinafter, a stationary object detection procedure using a bounding box will be described with reference to FIG. First, find the common set. Specifically, first, the ID number of the bounding box BB is voted for each pixel within a predetermined time. Subsequently, for each ID number, the number of votes V1 (ID, x, y; t0, Fl) is counted within the pixel during the long-time frame number Fl until time t0. Then, the number of votes V2 (x, y; t0, F) is counted within the pixel during the short-time frame number Fs up to time t0 regardless of the ID number. When the ID number can be correctly specified, V1 (ID, x, y; t0, F) can be used, and when the ID number cannot be specified, V2 (x, y; t0, F) can be used.

Next, a pixel having a large number of votes is determined as a still object candidate region from the obtained totaling result. For example, let V1 (ID, x, y; t0, Fl)> k1 × Fl (k1 = 0.1) be a candidate region for a long-time stationary object, and V2 (x, y; t0, Fs)> k2 A region of × Fs (k2 = 0.7) is set as a candidate region for a short-time stationary object. By performing such processing from the common set, a target stationary object can be detected.

The following equation shows the definition of a moment that is commonly used as a commonly used shape feature, but if it is a parameter that can express the dispersion around a point or an axis, it can be used instead of the moment in the text. it can. For example, histogram dispersion along a certain axis can be used.

The moment around the center of gravity is expressed as

The vertical and horizontal moments passing through the center of gravity are expressed as follows:

Here, all moment parameters can be used independently, but the ratio ((4), (8)) is more effective. In the above two types of moment parameters, the moment calculation of (2), (3), (6), (7) is divided into upper and lower here, but when using for human detection, the ratio of height is set The ratio is divided into three parts (head, torso, leg) (can be divided into two parts around the waist, or divided into some meaningful parts) and calculated for each range (for each interval) Height: k ₁ * H to k ₂ * H, 0 ≦ k ₁ <k ₂ ≦ 1). In this way, even if there is partial shielding, the visible portion can be used and a certain degree of judgment can be made, so that a robust system that is strong against shielding can be constructed.

Next, what is used as a motion measure is listed.

The above is a two-dimensional story, but it can be similarly defined in a three-dimensional space. Here, the movement efficiency is a parameter that indicates how efficiently the movement is primarily performed, and takes a value in the range of 0 to 1. Specifically, the ratio of the travel distance to the displacement distance ( (Hereinafter referred to as SR), the difference between the position before the T frame and the current position, and the ratio of the total distance moved between them (hereinafter referred to as DR) can also be used. It is also possible to extract the meaningless movements from unnecessary movements and irregularities of the limbs, useless movements such as bending the waist, shaking the head and neck, and movements of the upper and lower bodies, and using the ratio. SR and DR may be defined using the reciprocals of equations (8) and (9), but here take values in the range of 0 to 1 (0 ≦ SR ≦ 1, 0 ≦ DR ≦ 1). Define as above. In general, a frame means an image, but it can also be applied to other multimedia contents such as three-dimensional data. Content is a generic term for data obtained from sensors, and when content changes with time, data at each time is called a frame. For example, taking a moving image as an example, a plurality of frames (still images) are collected to form one moving image. In general, a frame is often used for an image, but here, the same term is used for other contents.

In the present embodiment, the method for detecting a stationary object is characterized in that it does not require calculation of a parameter reflecting the position of the target object, extraction of a bounding box, or the like, and can directly detect a candidate area of a target stationary object, Follow the steps below. First, the result of the absolute value of the background difference or the like is used as it is, or binarized to obtain a candidate area of the object as a silhouette. To do. The overlap determination means determines the degree of overlap from the absolute value of the background difference or the binarized image of the absolute value or the cumulative value of the silhouette value for each pixel, and if the value is equal to or greater than a predetermined threshold, To do. At this time, normalization such as division by the number of frames may be performed.

FIG. 8 shows the detection result of the suspicious object absent from the owner on the station platform using the absolute value of the background difference as the presence position parameter of the object. In this figure, the overlap parameter judgment means is either (1) the accumulated value of the shade value of the silhouette area, or (2) the number of overlapping frames after binarizing the silhouette (the number of frames having the value 1). Is used to determine the degree of overlap. Next, the object area means extracts the object area if the value is equal to or greater than a predetermined threshold value. FIG. 7B shows the result of using silhouettes in the enlarged view of the intermediate frame in FIG. Originally, there is no data in the background, but it is displayed in a semi-transparent form for easy understanding. FIG. 3C shows that the stationary time of the object to be detected can be controlled by setting the threshold value. In general, luggage is smaller than people, and the shape is often rectangular and has a large aspect ratio. Although it can be determined that a person with a size below a certain level when converted to a size in a three-dimensional space is not a person, a golf bag or a suitcase may be mistakenly recognized as a person. If it is necessary to distinguish between details, shape measures and other methods may be used together. FIG. 4D shows this. FIG. 9 shows a general method for detecting a stationary object that does not require extraction of a bounding box or the like. In FIG. 9, when the absolute value of the background difference is used as the presence position parameter of the object, there are a case where the value is used as it is and a case where the value (silhouette) after binarization is used. After that, a certain threshold value is set, region extraction is performed, and region parameters are obtained.

The processing after the target stationary object is extracted is the same as Step 4 and Step 5 described above. Although the background difference is used to extract the candidate area, it is assumed that the sensor is fixedly installed, so it is considered that there will be no significant environmental change, but the image of the image due to fluctuations in lighting conditions etc. In order to cope with changes in brightness or the like, it is better to update the background according to the above-described procedure. A similar method can be applied to three-dimensional data, but in that case, voxels may be used instead of pixels. Here, a voxel is a unit constituting a three-dimensional digital image in which thickness information is added to a unit pixel constituting the two-dimensional digital image, and is expressed as a particle having a thickness.

FIG. 10 shows a temporary effect of the detected stationary object that is more completely stationary or slightly moving, or the influence of shielding, shadow, reflection, or illumination variation by the moving object in this embodiment. It is the figure which showed the principle which identifies these. Here, a temporal cross-correlation coefficient of the object parameter in the object candidate region is used as the degree of correlation, and an appropriate threshold value is set. If the cross-correlation coefficient is Th ₁ to 1, the object, Th ₂ to If the Th ₁ portion is present, it indicates that the person is determined to be a person, and if it is 0 to Th ₂ , it is determined that the data is temporarily disturbed, for example, a portion of shielding, shadow, reflection, or illumination variation. After the object is detected in this manner, the object is generally completely stationary, and the person can distinguish the person from the object by using subtle movement (a little locally). In this figure, it is determined which range of the threshold value the temporal correlation degree of the object parameter in the object region extracted by the object extracting means is, and then the object determining means is small and the completely stationary object. It shows how to distinguish people with movements (generally people are not completely stationary).

In addition, the object candidate area extracted to obtain a wide range of motion may be used with a slight enlargement, and a temporal cross-correlation coefficient is calculated within the area, and “substantially no change, that is, cross-correlation” If the number stays close to 1, it is a completely stationary object, and if it changes occasionally or moves within a few times per second, the human possibility is high. However, if there is a very large change and it returns to its original state later, it is a temporary or partial effect, for example, it is considered to be an effect of shielding, shadows, reflections, or lighting fluctuations. If not, the object is considered to have moved. As a region for calculating the cross-correlation coefficient, a bounding box or the like may be used instead of the silhouette although accuracy is reduced, or an edge image or the like subjected to differentiation processing may be used as target data instead of the original image itself. .

In addition, when the object candidate area is large and part of it moves small, the cross-correlation coefficient is close to 1, which may make detection difficult. In that case, the object candidate area is further divided into smaller areas. Thus, detection is facilitated by calculating the cross-correlation coefficient for each region. As a dividing method, it is preferable to divide the area so that the areas are as uniform as possible. For this purpose, for example, a vertical histogram is used to determine the dividing position so that the areas of the upper and lower areas are equal, and further, the horizontal histogram is used for each area divided vertically so that the areas of the left and right areas are equal. By determining the position, it can be divided into four regions having the same area. You may divide | segment by changing the order of the division | segmentation of up and down, right and left. Moreover, when the boundary part of a division area moves, you may use an overlapping area | region. This is shown in FIG.

Further, other than the cross-correlation coefficient, for example, a mean square error may be used as long as it reflects a difference between frames, that is, a change (motion). This method is characterized by being easily and stably discernible. However, the person who does not move by the principle or the person who does not understand the movement backwards cannot distinguish the object from the person, and the object that moves slightly cannot be distinguished from the person. You may use together with the method of. The method for obtaining the reference image is to select the above-mentioned optimum pixel value in the silhouette area, that is, a weighted moving average value (which can be a simple average value), a median value, a peak value of a histogram, a value having a high appearance frequency, etc. Can be used. Also, the degree of motion can be estimated from the variance in the silhouette region within a certain time, and normalization may be performed to absorb differences in brightness and contrast.

In order to obtain an image serving as a reference for calculating the degree of correlation, a means for obtaining a background or an object from the above-described changing environment may be used. In addition, when using the standard deviation of the error in the silhouette or the bounding box area, a value divided by the average value of the in-area luminance values may be used in order to make it less susceptible to illumination. In the above description, the degree of correlation is obtained for each frame. However, a difference in the time series, that is, a change (motion) reflecting each pixel, for example, variance may be used. The state that changes greatly among “completely stationary, slightly moving, and greatly changing” can also be detected using the brightness of the object region. In addition, when the target area is small, the calculation of correlation may be inaccurate, but when the size on the image is converted to real space, it is extremely small compared to the standard size of a person Since it cannot be a person, it may be judged as a thing.

Regarding the selection of parameters, detection and recognition accuracy generally increases when converted to motion in a three-dimensional space rather than using motion on an image as it is. If a parameter reflecting a certain movement can be acquired at a short time interval, detection is possible even if shielding frequently occurs. If the shielding does not occur so frequently, the time interval for parameter acquisition can be increased.

As described above, according to the present invention, shielding is frequently generated in a crowded place and the like, and the observation value may be lost due to various causes such as the influence of illumination variation. Even so, it is possible to track the suspicious person who found the suspicious object and left it. At the ticket gates and airports of the station, people who are hungry, drunk, people who crawl around, people who pass through the ticket gates for unpaid rides, people who jump up, from the set line It can be used in a surveillance system for ensuring security, such as detection of a person who moves suspiciously moving away, a suspicious object absent from the owner, and tracking of the suspicious person. In addition, it is useful for pedestrian detection using in-vehicle far-infrared camera images, so it is useful for preventing serious accidents.

In homes, public buildings, school routes, etc., detect whether there are abnormalities such as sudden fall, cramping, or illness of sick people, children, elderly people, etc. It can be used to build a monitoring system for reporting.

100 network 101 fixed sensor 102 portable sensor 103 control means

Claims (11)

Observation means for observing the monitoring area and / or the monitoring area within a certain period of time;
Parameter determination means for determining whether a parameter reflecting the background and / or the contents or attributes of the object obtained by the observation of the observation means exists within a predetermined range at a predetermined threshold value or more;
When the parameter determining means determines that the parameter is present at a predetermined threshold or more, the parameters and the attributes of the background and / or object are extracted by setting the parameters within the predetermined range as the parameters to be obtained. An object recognition system having feature extraction means. A background determination unit that determines whether a background parameter that reflects the background obtained by observation of the observation unit exists within a predetermined range at a predetermined threshold value or more;
A background extraction unit for extracting a background by setting a background parameter within the predetermined range as a parameter indicating a background candidate area when it is determined that the background determination unit exists above a predetermined threshold; ,
The object recognition system according to claim 1, comprising: Object existence position determination for determining whether or not the object existence position parameter reflecting the object existence position obtained by observation of the observation means is present within a predetermined range at a predetermined threshold or more. Means,
When the object presence position determination unit determines that the object exists at a predetermined threshold value or more, the object candidate region is extracted from the existence position parameter within the predetermined range, and the object is extracted from the object parameter. Object extraction means;
The object recognition system according to claim 1, comprising: An object determination means for determining whether or not an object parameter reflecting the object obtained by observation of the observation means exists within a predetermined range at a predetermined threshold value or more;
When the object determination means determines that the object exists at a predetermined threshold value or more, the object is extracted by setting the object parameter within the predetermined range as a parameter indicating the object candidate area. Object extraction means;
The object recognition system according to claim 1, comprising: Whether or not the minute motion parameter reflecting the minute motion obtained by the observation of the observation means exists within a predetermined range at a predetermined threshold value or more and / or exists at a predetermined threshold value or less. A minute movement determining means for determining;
A minute motion detecting means capable of determining whether the object is a complete stationary object or a minute motion according to the existence range of the parameter determined by the minute motion determining means;
5. The object recognition system according to claim 1, comprising: A portion indicating that the background determination means does not exist above a predetermined threshold is excluded from the background candidate area, and a background parameter within a predetermined range set by the background extraction means is used as a parameter indicating a background candidate area. Background estimation means for determining the background parameter from the background candidate region,
The object recognition system according to claim 1 or 2, further comprising: A portion determined by the object determining unit not to exist at a predetermined threshold or more is removed from the object candidate region, and an object presence position parameter within a predetermined range set by the object extracting unit An object existence position estimating means for setting as a parameter indicating an object candidate area;
4. The object recognition system according to claim 1 or 3, wherein the object extraction means extracts an object region based on the parameter set by the object existence position estimation means and determines an existence position parameter. The part determined that the object determining means does not exist at a predetermined threshold or more is excluded from the object candidate area, and the object parameter within the predetermined range set by the object extracting means is The object recognition system according to claim 1, further comprising object estimation means that is set as a parameter indicating a candidate area and determines an object parameter from the object candidate area. Determination means for determining high, medium and low according to the existence range from the temporal correlation degree of the object parameter in the object region extracted as the stationary object by the object extraction means;
If it is determined that the determination means is high, the object is determined to be a stationary object. If it is determined to be medium, the object is determined to be a moving object with a small movement, and if it is determined to be low, the object Object judging means for judging that the data of
An object recognition system according to any one of claims 1 to 8. A monitoring system using the object recognition system according to claim 1. A monitoring system using the object recognition system according to claim 1.