JPH06266840A - Status detector for moving object - Google Patents

Abstract

PURPOSE:To accurately recognize the status of the whole movement of a moving object (area) or an object (area) executing unique movement by recognizing the moving status of a moving object based upon the distribution pattern of feature values of the object. CONSTITUTION:A picture processing part 100 inputs a picture photographed by a camera 101 and stores it in a picture memory 104. An inter-picture operation circuit 105, a binarizing circuit 106, a labeling circuit 107, and a feature value extracting circuit 108 respectively execute inter-picture operation, binarization, labeling and feature value extracting processing by using the data stored in the memory 104. A CPU 112 generates the distribution pattern of feature values relating to the position, moving distance, moving speed, and moving direction of each object, the variation of the moving direction, a direction difference between a moving vector and an initial vector, etc., based upon the processed results of the processing part 100. The dynamic status of the moving object is detected from the distribution pattern and the dynamic status of the whole moving object or an object or an area moving differently from others is displayed on a monitor 114.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for recognizing the state of a moving object, that is, the condition of the whole movement or an object or area having a unique movement.

[0002]

2. Description of the Related Art Heretofore, there has been known a method in which an average value of moving directions of a linearly moving object is used as a reference direction in order to grasp a region having a unique motion, and a unique region is obtained from a deviation from the reference direction. There is.

[0003]

In the above-mentioned prior art, since the reference direction is obtained by using the average value of all objects, there is a problem that the reference direction cannot be obtained correctly due to the influence of disturbance.

An object of the present invention is to provide a moving object state detecting apparatus capable of accurately recognizing a state of motion of an entire moving object (region) and an object (region) having a peculiar motion without being affected by disturbance. In offer.

[0005]

In order to achieve the above object, the present invention provides the position of each object from the input image information,
Generates a distribution pattern of movement-related features such as movement distance, movement speed, movement direction, movement direction change amount, direction difference between movement vector and initial vector, and recognizes the dynamic state of a moving object from this distribution pattern. It was done like this.

[0006]

According to the state detecting apparatus for a moving object of the present invention, since the dynamic state of the entire moving object or an object or a region which moves differently from other objects is obtained as a distribution pattern of the characteristic amount, the disturbance It can be recognized automatically with high accuracy without being affected.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to FIG.

A moving object state detecting device 9 according to the present embodiment.
0 is an image processing unit 10 that extracts the feature amount of the object from the image captured by the camera 101 that captures the image of the target object.
0, control of the entire apparatus and CPU 112 that detects the state of a moving object from the processing result of the image processing unit 100, memory 113 that stores measurement results, etc., monitor 114 that displays images and various information, communication with monitoring center 120 It has a communication device 115 for performing.

The image processing unit 100 includes an A / D converter 10
2, image memory 104, inter-image operation circuit 105, binarization circuit 106, labeling circuit 107, feature amount extraction circuit 1
08, the D / A converter 110 is provided.

The image memory 104 is, for example, 256 × 25.
A gray-scale memory of 6 pixels is provided for k sheets G1 to Gk, and j binary image memories for storing binary images are provided as needed for j sheets B1 to Bj.

The operation will be described below.

On the basis of a command from the CPU 112, the image processing section 100 takes in an image signal photographed by the camera 101, and the A / D converter 102, for example, 1
It is converted into density data of 28 gradations and stored in the image memory 104.

Further, the image processing unit 100 includes a CPU 1
Based on the command of 12, the data in the image memory 104 is used to perform inter-image calculation, binarization, labeling, feature amount extraction, and other processes, respectively, inter-image arithmetic circuit 105, binarization circuit 106, and labeling circuit 107. Then, it is processed by the characteristic amount extraction circuit 108 and the like, and the processing result and the like are converted into a video signal by the D / A converter 110 as necessary and displayed on the monitor 114. Subsequently, the CPU 112 detects a peculiar object (region) described later and determines the presence or absence of the peculiar object. This is notified to the observer of the center 120.

FIG. 15 shows another system configuration example. The moving object state detection device 90 ′ includes a camera 101
a, 101b, 101c, 101d, and the like, images are captured by the image processing unit 100 ′ from the plurality of cameras, and the image processing unit 1
At 00 ', the input images are selected one by one by the camera switch 116 and processed. The result of processing a plurality of images is the CPU 11
2, the CPU 112 sequentially recognizes the state of the moving object and comprehensively determines the relationship between the moving states of a plurality of images or a plurality of points, or transmits a sequential processing result to the monitoring center 120, The monitoring center 120 comprehensively determines the relationship between a plurality of images or movement states of a plurality of points. In this configuration example, the case where there are four cameras is shown, but the number is not limited at all.

FIG. 16 shows another system configuration example. The moving object state detection device 90 ″ includes a camera 101.
a, 101b, 101c, 101d, and the like, images are captured by the image processing unit 100 ″ from a plurality of cameras, and the image processing unit 1
In 00 ″, the input images are selected one by one by the camera switching unit 116, after A / D conversion, the screens are combined by the screen combining unit 103 and stored in the large image memory 104 ′.
The large image memory 104 'is processed. The processing result of the image is sent to the CPU 112, the state of the moving object is recognized by the CPU 112, and the relationship between the moving states of a plurality of points is comprehensively determined, or the processing result is transmitted to the monitoring center 120, and the monitoring center 120. Comprehensively determine the relationship of movement status at multiple points. In this configuration example, the case where there are four cameras is shown, but the number is not limited at all.

Furthermore, another example of system configuration is shown in FIG.
Shown in. Moving object state detection device 90a, 90b, 90
Each of c and 90d captures an image using the camera 101 and processes the image in the image processing unit 100. The processing result of the image is sent to the CPU 112, the CPU 112 recognizes the state of the moving object, transmits the processing result to the monitoring center 120, and the monitoring center 120 comprehensively determines the relationship between the plurality of images or the moving state of the plurality of points. . In this configuration example, the case where the number of state detection devices is four is shown, but the number is not limited at all.

Next, the process of detecting the condition of the whole motion and the object (region) having a peculiar motion, which is the central part of the present invention, will be described with reference to FIG.

First, in the input image capturing section 11, the image processing section 100 stores the image captured from the camera 101 in the image memory 104. The differential image extraction 12 obtains a differential image s (t) between the background image previously stored in the image memory 104 and the input image. Binarization processing 13
Binarizes the difference image s (t) with a predetermined threshold, and
A value image b (t) is created. Then, each object is labeled by the labeling 14, and the barycentric coordinates of each object are obtained by the barycentric calculation 15. Next, the association 16 with the object in the previous input image is performed. Various methods such as template matching, correlation method, and feature tracking method have been reported as the matching method. Here, as one of the feature tracking methods, the one having the smallest distance between the centers of gravity of the objects is simply described. It is decided to correspond. If it is possible to make correspondence, feature amount calculation 17
Then, the feature amount of each object is obtained. The feature amount includes start point coordinates, end point coordinates, movement distance, movement distance change amount, movement direction, movement direction change amount, and the like. About these,
Will be explained. In FIG. 3, the time t-1 of the object A,
The object positions at t and t + 1 are At-1, At, At +
Expressed as 1. Assuming that the movement vector of the object A is MV, the movement distances in the x direction and the y direction are dx, dy, the distance L moved by the object A from time t−1 to time t is

[0019]

[Equation 1]

Then, the moving direction is

[0021]

[Equation 2]

The amount of change in the moving direction is

[0023]

[Equation 3]

Then, the moving speed is

[0025]

[Equation 4]

Each is represented by In addition, here, the moving direction is assumed to measure the angle clockwise.

Further, in a case where a region that partially moves differently in the entire region, or when there are a plurality of objects, an object that moves differently from other objects is defined as a peculiar region. For example, if a person crosses a certain flow of people,
That person has made a unique move. In FIG. 4, among the objects 25a to 25j, the objects 25c and 25j are peculiar regions.

FIG. 5 shows an example of a singular region for a linear constant flow. Arrows 26b, 26e, and 26f indicate the directions of movements that are unique regions. Here, the direction of the entire flow is called the reference direction. The reference direction is obtained using the frequency distribution shown in FIG. That is, the frequency distribution is obtained with the horizontal direction being the moving direction θ obtained for each object in Equation 2. This is the generation 18 of the distribution pattern in FIG. At this time, θ is obtained in units of α degrees (for example, every 10 degrees). Then, in the recognition 19 of the distribution pattern, a direction having the highest frequency is obtained, and this is used as a reference direction, and an object having a difference of ± β degrees (for example, ± 10 degrees) or more from this reference direction is extracted as a unique region.
As another method of obtaining the reference direction, a direction θ having a certain frequency γ or more can be obtained, and the obtained plurality of directions θ can be used as the reference directions. Furthermore, the reference direction may be an average value of the plurality of the directions θ in which the frequency is a certain value γ or more.

In the following, each process of generating a distribution pattern 18 and recognizing a distribution pattern 19 in the case of circular motion, radial diffusion, and radial contraction will be described.

FIG. 7 shows an example of a singular region for circular motion. Arrows 27c and 27d indicate the direction of movement that becomes the peculiar region. This is an object or region that does not enter the flow of a circle among those that make a circular motion. On the other hand, the amount of change in the moving direction of each object is calculated by Equation 3, and the frequency distribution with the amount of change Δθ in the moving direction as the horizontal axis is calculated in the same manner as in the case of the moving direction described above, and the maximum frequency is calculated. The amount of change to take, or the amount of change to take a certain frequency γ or more, or the frequency has a certain value γ
The average value of a plurality of the above-described change amounts can be set as the reference direction change amount. An object having a difference between the reference direction change amount and the direction change amount of ± β degrees (for example, ± 10 degrees) or more can be extracted as a unique region.

FIG. 8 shows an example of a peculiar region in the case of radial diffusion. This corresponds to the flow of people when the area near the center becomes dangerous, and the flow of people at each point of the sidewalk at the scramble intersection. Arrows 28a and 28b indicate the direction of movement that is the unique region. In such a situation, x of the starting point coordinates (xis, yis) of the movement vector MV _i of each object
Then, the frequency distribution with the abscissa as the y coordinate and the y coordinate is obtained. At this time, the coordinate value is limited to α units (for example, every 20 units).
Then, the coordinate value having the maximum frequency is set as the reference position. Further, similarly to the case of the moving direction described above, an average value of a plurality of coordinate values having a certain frequency or more can be set as the reference position. The distance from the reference position to the starting point coordinates of each object is obtained, and an object having a distance β or more can be extracted as a unique region.

As another method, each initial vector from the reference position to the starting point coordinate of each object is obtained, and the direction difference (Δθ _0i = θ) between the moving direction θ _i of the moving vector and the direction θ _0i of the initial vector is obtained. _i −θ _0i ) is obtained, and an object whose direction difference Δθ _0i is ± β degrees (for example, ± 5 degrees) or more can be extracted as a unique region. Alternatively, the frequency distribution of the direction difference Δθ _0i is obtained, and the direction difference having the maximum frequency, or the direction difference having a certain frequency γ or more, or the frequency having a certain value γ or more, as in the case of the moving direction described above. The average value of the plurality of directional differences can be used as the reference directional difference. An object having a difference in direction difference of ± β degrees (for example, ± 5 degrees) or more from this reference direction difference can be extracted as a unique region.

FIG. 9 shows an example of a peculiar region in the case of radially contracting. Arrows 29a and 29b indicate the direction of movement that is the unique region. This corresponds to the case where when an accident, incident, accident, etc. occurs, most of them gather at that point, but some indifferent persons or people with other purposes move away from it. . In such a situation, contrary to FIG. 8, the end point coordinates (x _i of the movement vector MV _i of each object are
The frequency distribution is obtained with the x and y coordinates of (d, yid) as the abscissa, and the reference position is obtained in the same manner as in the case of the radial diffusion described above. The distance from the reference position to the end point coordinates of each object is obtained, and an object having this distance of a certain value β or more can be extracted as a peculiar region.

As another method, the direction difference (Δθ _0i ) is obtained in the same manner as in the case of the radial diffusion described above, and the direction difference (Δθ ′ _0i = Δθ _0i −180) is further obtained, and the direction difference Δθ ′ is obtained.
An object whose _0i is ± β degrees (for example, ± 5 degrees) or more can be extracted as a unique region. Alternatively, the frequency distribution of the direction difference Δθ ′ _0i is obtained, and the direction difference having the maximum frequency or the direction difference having a certain frequency γ or more, or the frequency having a certain value γ is obtained as in the case of the moving direction described above. An average value of a plurality of the above-mentioned direction differences can be used as the reference direction difference. An object having a difference in direction difference of ± β degrees (for example, ± 5 degrees) or more from this reference direction difference can be extracted as a unique region.

Further, for the moving distance L or the moving speed V, the frequency distribution of the moving distance L or the moving speed V is obtained in the same manner as described above, and the reference moving distance or the reference moving speed is obtained from them. Then, an object having a difference between the reference movement distance or the reference movement speed and the movement distance or the movement speed of ± β (for example, ± 10) or more can be extracted as the unique region. It is possible to use the values at one time for the moving direction, direction change amount, direction difference between the initial vector and the moving vector, etc. described above, but in order to improve the accuracy, the average of the measured values of each object at multiple times is used. Using the value of
It is conceivable that the direction change amount, the direction difference between the initial vector and the movement vector, and the like are obtained, and the values are used to extract the peculiar region.

Further, as the processing of the distribution pattern recognition 19, a method using pattern matching or a neural network will be described.

FIG. 10 (a) shows the movement of an object moving on a certain road 50, and the distribution pattern of the movement vector is shown in FIG. 10 (b). 10B, the horizontal axis represents the position, and the vertical axis represents the moving distance of the object, the number of objects (frequency of appearance), the cumulative value of the moving distance of each object, or the like. When FIG. 10B shows a distribution pattern of measured values at a plurality of monitoring points, the horizontal axis in the figure represents each point.
It is possible to determine the relationship between the current road condition and the moving condition at each point by determining which of the several reference distribution patterns prepared in advance by the system as shown in FIG. it can. For example, a normal distribution pattern on a certain road is shown in FIG.
If (a) is used and the current distribution pattern is (b), there must be some resistance (for example, there is an obstacle or a warp) near the center of the road. I understand. The input distribution pattern can be determined by pattern matching between the input distribution pattern and the reference distribution pattern or a neural network described later. For example, in FIG.
When applied to 0, FIG. 11A shows that there is resistance on both sides of the road, the speed is slower than the center, the amount of flowing objects is small, and so on. On the contrary, FIG. 11B shows that there is resistance at the center of the road, the speed is slower than both sides, and the amount of flowing objects is small. FIG. 11C shows that there is resistance on the right side of the road, the speed is slower than on the left side, and the amount of flowing objects is small. On the contrary, FIG. 11D shows that there is resistance on the left side of the road, the speed is slower than the right side, the amount of flowing objects is small, and the like. FIG. 11E shows that there is no particular resistance on the road, the speed is the same as a whole, the amount of moving objects is the same, and the like. The above shows an example of a one-way flow, but a bidirectional flow is also shown in FIG.
It can be similarly determined using the reference distribution pattern shown in FIG.
For more complicated distribution patterns, a neural network can be used to determine what proportion of the reference distribution patterns the input distribution patterns are combined with. That is, the number of neurons in the input layer of the neural network is set to be the same as the number of inputs in the distribution pattern, and the number of neurons in the output layer is set to be the same as the number of reference distribution patterns, and pattern classification is performed. That is, by inputting the moving distance of each object or the number of objects or the cumulative value of the moving distance of each object, which is a constituent element of the distribution pattern, to each neuron of the input layer of the neural network, each reference distribution is output from the output layer. The composition ratio of the pattern is output. By this, the state of the whole movement (group flow),
Alternatively, it is possible to detect an object or a region that makes a peculiar motion in the entire motion (group flow).

Although the case of linear movement is described above, FIG. 13 shows an example of a distribution pattern in the case of radial diffusion movement, and FIG. 14 shows an example of a distribution pattern in the case of radial contraction. Indicates. 13 (b) and 14 (b), the horizontal axis represents the moving direction (θ) of each object viewed clockwise when the x-axis direction is 0 degree.

The case of the two-dimensional distribution pattern has been described above. As the distribution pattern, the position is defined by parameters of the x-axis, the moving distance, the moving speed, the moving direction, the change amount of the moving direction, the moving vector and the initial value. The same idea can be applied to a three-dimensional distribution pattern in which one of the direction differences from the vector is the y-axis parameter and the frequency is the z-axis parameter.

After the distribution pattern is recognized by the method described above, the presence / absence determination 20 of the abnormal state and the peculiar object (region) is performed. And abnormal state or peculiar object (area)
If there is, the fact is displayed to the monitor or the like, the current state is displayed on the monitor, and the abnormal condition and the peculiar object (area) are notified 21 by leaving the judgment to the monitor. When the appropriate treatment is completed, the supervisor again instructs the start of the treatment to repeat the same treatment as above. On the other hand, if there is no abnormal state or a peculiar object (region), the process returns to the first process, a new image is input, and the same process as above is repeated.

[0041]

According to the present invention, it is possible to recognize the state of the whole movement (group current), the state different from other things, or the moving object (region), so that the inside of a station, an underground mall, a shopping district, etc. The effect of automatically monitoring and recognizing the state of the whole movement (group flow) in and the object (human being, etc.) having a peculiar movement.

Further, by processing the images input from a plurality of cameras, it is possible to grasp the situation spreading over a wide area, and it is possible to grasp the monitoring that was judged one by one in relation to the whole. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a moving object state detection device.

FIG. 2 is a flowchart showing a flow of a unique object detection process which is an embodiment of the present invention.

FIG. 3 is an explanatory diagram of feature quantities.

FIG. 4 is an explanatory diagram of a unique region.

FIG. 5 is an explanatory diagram of a peculiar region for linear motion.

FIG. 6 is an explanatory diagram of a frequency distribution in a moving direction.

FIG. 7 is an explanatory diagram of a singular region for circular motion.

FIG. 8 is an explanatory diagram of a peculiar region in the case of radial diffusion.

FIG. 9 is an explanatory diagram of a peculiar region when radially contracting.

FIG. 10 is an explanatory diagram of a distribution pattern of linear motion.

FIG. 11 is an explanatory diagram of a reference distribution pattern in one direction.

FIG. 12 is an explanatory diagram of a bidirectional reference distribution pattern.

FIG. 13 is an explanatory diagram of a distribution pattern in the case of radially spreading.

FIG. 14 is an explanatory diagram of a distribution pattern when radially contracting.

FIG. 15 is a block diagram showing another configuration of a moving object state detection device.

FIG. 16 is a block diagram showing another configuration of a moving object state detection device.

FIG. 17 is a block diagram showing another configuration of a moving object state detection device.

[Explanation of symbols]

100 ... Image processing unit, 101 ... Camera, 112 ... CP
U, 114 ... Monitor, 115 ... Communication device.

Claims (12)

[Claims] 1. An image input means for capturing an image of a target, an image processing means for performing various image processing on the image captured by the image input means, and a data processing means for performing various data processing, In tracking a moving object, motion information of the moving object is extracted from the image input by the image input means, and the position of each object is extracted from the extracted information.
One of the moving distance, the moving speed, the moving direction, the change amount of the moving direction, and the direction difference between the moving vector and the initial vector is used as a parameter on the horizontal axis, and at least the object frequency, the cumulative moving distance, and the cumulative A distribution pattern generation means for generating a two-dimensional distribution pattern having one of the movement speeds as a vertical axis parameter, and a movement state of the moving object is recognized based on the two-dimensional distribution pattern from the distribution pattern generation means. A device for detecting a state of a moving object, which comprises: a means for displaying the recognition result or the image; and a communication means for communicating the recognition result or the image. 2. An image input means for capturing an image of an object, an image processing means for performing various image processing on the image captured by the image input means, and a data processing means for performing various data processing, In tracking a moving object, motion information of the moving object is extracted from the image input by the image input means, the position of each object is used as a uniaxial parameter from the extracted information, and the moving distance, moving speed, moving direction, A distribution pattern that generates a three-dimensional distribution pattern in which one of the amount of change in the movement direction and the direction difference between the movement vector and the initial vector is used as another uniaxial parameter, and the frequency is used as another uniaxial parameter. Generating means, means for recognizing the moving state of the moving object based on the three-dimensional distribution pattern from the distribution pattern generating means, recognition result or image. Display means for displaying the recognition result or state detecting apparatus of a moving object, characterized by comprising a communication means for communicating the image. 3. The moving object state detecting apparatus according to claim 1 or 2, wherein the input images from a plurality of image input means are processed, and the state of the moving object spread over a wide area is grasped from the overall relation of the processing results. A characteristic detecting device for a moving object. 4. The moving object state detecting device according to claim 1, wherein the input images from a plurality of image input means are combined, and the combined images are processed to grasp the state of the moving object spread over a wide area. A state detecting device for a moving object characterized by. 5. The moving object state detecting apparatus according to claim 1, wherein the feature quantity having the maximum frequency is obtained from the frequency distribution of the feature quantity of each object, and the reference quantity is obtained as the reference quantity of the feature quantity of each object. A device for detecting the state of a moving object, which is characterized by recognizing an object or a region having a peculiar motion in a group flow, which is the whole motion. 6. The moving object state detecting apparatus according to claim 1, wherein a characteristic amount having a frequency of a certain value or more is obtained from the frequency distribution of the characteristic amounts of each object, and the obtained plurality of characteristic amounts are used as references. A state detecting device for a moving object, characterized by recognizing an object or a region having a unique motion in a group flow, which is an overall motion, based on a difference from the reference amount. 7. The moving object state detecting apparatus according to claim 1 or 2, wherein a characteristic value having a frequency equal to or higher than a certain value is obtained from a frequency distribution of the characteristic amounts of the respective objects, and an average value of the obtained plurality of characteristic values is obtained. Is used as a reference amount, and a state detection device for a moving object is characterized by recognizing an object or a region having a peculiar movement in a group flow, which is an overall movement, from the difference from the reference amount. 8. The moving object state detecting apparatus according to claim 1 or 2, wherein the distribution pattern of the feature amount of each object is obtained, and the entire motion state is recognized by matching with a reference distribution pattern prepared in advance. A state detecting device for a moving object characterized by. 9. The moving object state detecting apparatus according to claim 1 or 2, wherein a distribution pattern of feature quantities of each object is obtained, the distribution pattern is input to a neural network prepared in advance, and the output value of the neural network is used. , A moving object state detection device characterized by recognizing the state of the entire movement. 10. The moving object state detecting apparatus according to claim 1, wherein a moving vector is obtained from a moving distance and a moving direction of a feature amount of each object, and a reference position is obtained from a starting point coordinate or an ending point coordinate of each object. Means, means for obtaining an initial vector from the reference position to the starting point coordinates of each object, means for obtaining the difference between the direction of the initial vector and the direction of the movement vector, and using the direction difference extracted by the means, A moving object state detecting device characterized by recognizing a moving state of an object. 11. The state detection device according to claim 10, wherein a plurality of monitoring points are provided, at least one image input means is installed at the monitoring points, the input information of the input means is processed, and the processing result is monitored. A state detecting device for a moving object, which monitors and grasps the state of the moving object by transmitting the information to a center and performing integrated processing at the monitoring center. 12. The state detecting device according to claim 10, wherein a plurality of monitoring points are provided, at least one image input means is installed at the monitoring points, and input information of the input means is combined and processed. A state detecting device for a moving object, characterized by monitoring and grasping the state of the moving object.