JP2006268678A - Device and method for detecting stopping or low-speed vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for detecting a stopping or low-speed vehicle, capable of recognizing a vehicle in a stopped state in a relatively short time with high reliability even in a case such that an intended vehicle cannot be captured as a continuous image because it is interrupted by another vehicle. <P>SOLUTION: A vehicle recognition part 30 detects vehicle candidate areas from an image of a subject including vehicles on a road, which is input from an image input part 21, and recognizes characteristic points of the vehicles. A stopping or low-speed event recognition part 34 stores information showing stopping vehicle candidates in a stopping holding information storage part 33 when it loses a vehicle after detecting its stopping state or low-speed state. The stopping or low-speed event recognition part 34 determines, based on the detection state of the vehicle recognition part 30 and the information held in the stopping holding information storage part 33, a stopping vehicle or low-speed vehicle. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a stopped low-speed vehicle detection apparatus and method used to detect an abnormal vehicle that has stopped or entered a low-speed state due to a failure or the like, based on an image obtained by photographing a subject on a road.

  Conventionally, for example, a surveillance camera is installed on a highway or in the vicinity of a general road to photograph a vehicle traveling on the road, and an image of the vehicle obtained by the photographing is displayed on, for example, a video display device in a control room. The configured monitoring system is widely adopted to manage the road traffic condition.

  In this type of general monitoring system, the observer monitors the displayed video by visually confirming it. For example, in the technique of Patent Document 1, the video from the monitoring camera is analyzed. Thus, it is proposed that the sudden event is automatically detected, the occurrence of the sudden event is displayed, and the input video is switched to the on-site video of the sudden event occurrence point.

  Various techniques for detecting a vehicle from an image of the vehicle on the road have been proposed. For example, the technique disclosed in Patent Document 2 proposes that the vehicle type at night is determined with high accuracy by extracting and determining the area of the headlight of the vehicle from the image.

  Image processing technology for extracting the vehicle body part from the photographed image on the road includes a background difference method for extracting the feature of the vehicle by taking the difference between the background image of the road and the current image, and the image of the previous frame A frame difference method for extracting vehicle features by taking a difference between images from the current frame is conceivable.

JP-A-5-250595 JP 11-353580 A

  As described above, by detecting the vehicle from the captured image on the road and tracking the detected vehicle in the captured image, calculating the speed of the vehicle, detecting the vehicle stopped due to failure, etc. It is possible to grasp the traffic situation of the target area on the road. However, since the road conditions change variously, it is not always possible to photograph the target vehicle under conditions where detection is easy.

  For example, even if the device detects that the speed of a specific vehicle has dropped abnormally or has stopped, there may actually be various detection errors, so that the detection state continued for at least a predetermined time. If it is not confirmed, a reliable detection result cannot be obtained. However, when a plurality of lanes are simultaneously captured by one monitoring camera, there is a possibility that other vehicles pass one after another between the stopped target vehicle and the monitoring camera. In this case, since the target vehicle cannot be captured as a continuous image, it cannot be identified whether or not the target vehicle is a surely stopped vehicle.

  Even if the target vehicle cannot be captured as a continuous image due to being blocked by another vehicle, the present invention can recognize a stopped or low-speed vehicle with high reliability within a relatively short time. It is an object of the present invention to provide a stop low-speed vehicle detection device and method.

  The first stop low-speed vehicle detection device of the present invention includes an image input means for inputting a photographed image obtained by photographing a subject including a vehicle on a road, and a vehicle candidate area corresponding to the vehicle in the photographed image. Vehicle recognition means for recognizing at least a feature point of the vehicle based on information on the vehicle candidate area, and information on the vehicle candidate area detected by the vehicle recognition means or information on the feature point Vehicle stop detection means for detecting a stop state or a low speed state, stop information storage means for storing information indicating a stop vehicle candidate when the vehicle stop detection means detects a stop state or a low speed state of the vehicle, and the vehicle Based on the detection state of the stop detection means and the information held in the stop information storage means, the stop low speed event recognition that recognizes the stop vehicle or the low speed vehicle and outputs a signal as the stop low speed event. It includes a stage, a.

  According to this configuration, after the target vehicle in the stopped state or the low speed state is detected as a candidate by the vehicle stop detection means, the video of the target vehicle is captured only intermittently due to the passage of another vehicle (interfering vehicle) or the like. Even if it is not done, the information about the target vehicle is stored in the stop information storage means, so when the video of the target vehicle appears again after passing through the disturbing vehicle, the information on the currently captured target vehicle and By using the past target vehicle information stored in the stop information storage means, the stop low speed event recognition means can recognize with high reliability that the target vehicle is in a stop state or a low speed state. Become.

  The second stop low-speed vehicle detection device of the present invention is the first stop low-speed vehicle detection device, wherein the stop information storage means is a target vehicle corresponding to a vehicle detected by the vehicle recognition means in the past. Only when it is not detected by the recognizing means and the target vehicle is detected to be in a stopped state or a low speed state by the vehicle stop detecting means, the corresponding information as a stopped vehicle candidate is stored.

  With this configuration, information on vehicles that do not need to be saved is not registered, such as when the target vehicle is continuously photographed or when the detected vehicle is unlikely to be in a low-speed state, so stop information is stored. The storage area of the means can be used more effectively.

  A third stop low-speed vehicle detection device according to the present invention is the first or second stop low-speed vehicle detection device, wherein the stop information storage means includes at least a vehicle identification number as information representing the stop vehicle candidate. The information indicating the position of the detected feature point of the vehicle and the length of time when the stop state is detected is stored.

  With this configuration, by comparing the position of the feature point related to the past vehicle stored in the stop information storage means and the position of the target vehicle in the latest image, it is possible to identify whether or not they are in a correspondence relationship. Is possible. Further, by using the time stored in the stop information storage means, it is possible to calculate the cumulative value related to the time when the stop state or the low speed state is detected.

  A fourth stop low-speed vehicle detection device according to the present invention is the third stop low-speed vehicle detection device, wherein the vehicle recognition unit detects a first vehicle in which vehicle information is stored in the stop information storage unit. When the vehicle stop detection means detects the stop state or the low speed state of the second vehicle as a new stop vehicle candidate after being stopped, the vehicle stop detection means is held in the stop information storage means The distance between the position of the feature point of the first vehicle and the position of the feature point of the second vehicle is compared. If the distance is less than or equal to a predetermined distance, the first vehicle and the second vehicle are The stop time of the vehicle is assumed to be the same as the sum of the stop time for the first vehicle and the stop time for the second vehicle.

  With this configuration, the distance between the position of the feature point related to the past vehicle stored in the stop information storage means and the position of the target vehicle in the latest image is checked, and when the distance is close, the two vehicles are associated with each other. Even when the target vehicle appears intermittently in the captured image, the situation of the same target vehicle can be monitored. In addition, since the stop time is grasped using the result of adding the stop times of the vehicles associated with each other, the time when the stop state or the low speed state is detected even when the video of the target vehicle is repeatedly interrupted It is possible to know the cumulative value of and to recognize with high reliability.

  A fifth stop low-speed vehicle detection device according to the present invention is the third stop low-speed vehicle detection device, wherein the stop information storage means further includes image information of the vehicle candidate region as information representing the stop vehicle candidate. Save.

  With this configuration, it is possible to obtain image information of the target vehicle detected in the past by referring to the stop information storage unit.

  A sixth stop low-speed vehicle detection device according to the present invention is the fifth stop low-speed vehicle detection device, wherein the vehicle recognition unit detects a first vehicle in which vehicle information is stored in the stop information storage unit. When the vehicle stop detection means detects the stop state or the low speed state of the second vehicle as a new stop vehicle candidate after being stopped, the vehicle stop detection means is held in the stop information storage means The distance between the position of the feature point of the first vehicle and the position of the feature point of the second vehicle is compared, and the image information of the vehicle candidate area related to the first vehicle and the vehicle candidate area related to the second vehicle When the distance is equal to or less than a predetermined value and the similarity is high, the first vehicle and the second vehicle are regarded as the same, and the vehicle is stopped. Time to the first A time obtained by adding the stop time for the second vehicle and stop time for both.

  With this configuration, whether or not the first vehicle and the second vehicle match is determined based not only on the difference in distance but also on the similarity of the image information. Therefore, the first vehicle and the second vehicle It is difficult for errors to be associated with each other.

  A seventh stop low-speed vehicle detection device according to the present invention is the sixth stop low-speed vehicle detection device, wherein the vehicle stop detection means includes image information of a vehicle candidate area relating to the first vehicle and a second vehicle. The similarity is calculated based on the average value of the brightness difference of each pixel with respect to the image information of the vehicle candidate area regarding.

  With this configuration, since the similarity of the image information between the first vehicle and the second vehicle is identified by the average value of the brightness difference of each pixel of the image information, a highly reliable result can be obtained with a relatively simple process. can get.

  An eighth stop low-speed vehicle detection device according to the present invention is the sixth stop low-speed vehicle detection device, wherein the vehicle stop detection means includes image information of a vehicle candidate area related to the first vehicle and a second vehicle. The similarity is calculated based on a correlation coefficient with the image information of the vehicle candidate area relating to the vehicle.

  With this configuration, since the similarity of the image information between the first vehicle and the second vehicle is identified by the correlation coefficient between the image information, a highly reliable result can be obtained.

  A ninth stop low-speed vehicle detection device according to the present invention is the sixth stop low-speed vehicle detection device, wherein the vehicle stop detection means includes differential data of image information of a vehicle candidate area related to the first vehicle and The similarity is calculated based on the average value of the differential value of each pixel with the differential data of the image information of the vehicle candidate area for the second vehicle.

  With this configuration, the similarity between the image information of the first vehicle and the second vehicle is identified by the average value of the differential values of the pixels of the differential data of the image information, so that a highly reliable result is obtained. It is done.

  A tenth stop low-speed vehicle detection device according to the present invention is the sixth stop low-speed vehicle detection device, wherein the vehicle stop detection means includes differential data of image information of a vehicle candidate area related to the first vehicle and The similarity is calculated based on a correlation coefficient with the differential data of the image information of the vehicle candidate area relating to the second vehicle.

  With this configuration, since the similarity of the image information between the first vehicle and the second vehicle is identified by the correlation coefficient with the differential data of the image information, a highly reliable result can be obtained.

  An eleventh stop low-speed vehicle detection device according to the present invention is any one of the first to tenth stop low-speed vehicle detection devices, and matches the first vehicle information held in the stop information storage means. Alternatively, when a second vehicle having similar vehicle information is not detected for a predetermined time, the stop information storage means discards the information on the first vehicle.

  With this configuration, even when the target vehicle to be detected leaves the video when it is not photographed due to the passage of another vehicle (interfering vehicle), etc. Since the information is discarded, the storage area on the stop information storage means can be used effectively, and the occurrence of useless processing for information that is not used can be suppressed.

  According to a first stop low-speed vehicle detection method of the present invention, an image input step of inputting a photographed image obtained by photographing a subject including a vehicle on a road, and a vehicle candidate area corresponding to the vehicle in the photographed image are obtained. A vehicle recognition step for recognizing at least a feature point of the vehicle based on the information on the vehicle candidate region, and information on the vehicle candidate region detected in the vehicle recognition step or information on the feature point A vehicle stop detection step for detecting a stop state or a low speed state; a stop information storage step for storing information representing a stop vehicle candidate when the vehicle stop state or low speed state is detected in the vehicle stop detection step; Based on the detection state in the stop detection step and the stored stop vehicle candidate information, the stop vehicle or low speed vehicle is recognized and the stop low speed event is detected. Having a stop slow event recognition step of outputting a signal as

  According to this method, after the target vehicle in the stopped state or the low speed state is detected as a candidate by the vehicle stop detection means, the video of the target vehicle is only intermittently captured due to the passage of another vehicle (interfering vehicle) or the like. Even if it is not done, the information about the target vehicle is stored in the stop information storage means, so when the video of the target vehicle appears again after passing through the disturbing vehicle, the information on the currently captured target vehicle and By using the past target vehicle information stored in the stop information storage means, the stop low speed event recognition means can recognize with high reliability that the target vehicle is in a stop state or a low speed state. Become.

  The second stop low-speed vehicle detection method of the present invention is the first stop low-speed vehicle detection method, and in the stop information storage step, a target vehicle corresponding to a vehicle detected in the past by the vehicle recognition procedure is a vehicle. Only when the target vehicle is not detected in the recognition procedure and the target vehicle is detected to be in the stop state or the low speed state in the vehicle stop detection step, the corresponding information as the stop vehicle candidate is stored.

  This method does not register information on vehicles that do not need to be saved, such as when the target vehicle is being shot continuously, or when it is unlikely that the detected vehicle is in a low-speed state. The storage area of the means can be used more effectively.

  A third stop low-speed vehicle detection method of the present invention is the first or second stop low-speed vehicle detection method described above, and in the stop information storage step, at least a vehicle identification number is used as information representing the stop vehicle candidate. The information indicating the position of the detected feature point of the vehicle and the length of time when the stop state is detected is stored.

  By this method, by comparing the position of the feature point related to the past vehicle stored in the stop information storage means and the position of the target vehicle in the latest image, it is identified whether or not they are in a correspondence relationship. Is possible. Further, by using the time stored in the stop information storage means, it is possible to calculate the cumulative value related to the time when the stop state or the low speed state is detected.

  The fourth stop low-speed vehicle detection method of the present invention is the third stop low-speed vehicle detection method described above, after the first vehicle in which the vehicle information is stored in the stop information storage step is no longer detected. When a stopped state or low-speed state of the second vehicle is detected as a new stopped vehicle candidate, the position of the stored feature point of the first vehicle and the position of the feature point of the second vehicle The step of comparing the distance and the first vehicle and the second vehicle are regarded as the same when the compared distance is equal to or smaller than a predetermined distance, and the stop time for the first vehicle is defined as the stop time of the vehicle. And a step of adding a stop time related to the second vehicle.

  By this method, the distance between the position of the feature point relating to the past vehicle stored in the stop information storage means and the position of the target vehicle in the latest image is checked, and when the distance is close, the two vehicles are associated with each other. Even when the target vehicle appears intermittently in the captured image, the situation of the same target vehicle can be monitored. In addition, since the stop time is grasped using the result of adding the stop times of the vehicles associated with each other, the time when the stop state or the low speed state is detected even when the video of the target vehicle is repeatedly interrupted It is possible to know the cumulative value of and to recognize with high reliability.

  The fifth stop low-speed vehicle detection method of the present invention is the third stop low-speed vehicle detection method, wherein in the stop information storage step, image information of the vehicle candidate area is further used as information indicating the stop vehicle candidate. Save.

  By this method, it is possible to obtain image information of the target vehicle detected in the past by referring to the stop information storage means.

  The sixth stop low-speed vehicle detection method of the present invention is the fifth stop low-speed vehicle detection method, wherein after the first vehicle in which the vehicle information is stored in the stop information storage step is not detected. When a stopped state or low-speed state of the second vehicle is detected as a new stopped vehicle candidate, the position of the stored feature point of the first vehicle and the position of the feature point of the second vehicle A step of comparing a distance, a similarity calculating step of calculating a similarity between the image information of the vehicle candidate area related to the first vehicle and the image information of the vehicle candidate area related to the second vehicle, and the compared distance If it is equal to or less than a predetermined value and the similarity is high, the first vehicle and the second vehicle are regarded as the same, and the stop time for the first vehicle and the second vehicle are set as the stop time of the vehicle. Related to the vehicle Further comprising a step of adding the stop time that the.

  With this method, whether or not the first vehicle and the second vehicle match is determined based not only on the difference in distance but also on the similarity of the image information. Therefore, the first vehicle and the second vehicle It is difficult for errors to be associated with each other.

  The seventh stop low-speed vehicle detection method of the present invention is the sixth stop low-speed vehicle detection method, wherein in the similarity calculation step, the image information of the vehicle candidate area related to the first vehicle and the second vehicle The similarity is calculated based on the average value of the brightness difference of each pixel with respect to the image information of the vehicle candidate area regarding.

  By this method, since the similarity of the image information between the first vehicle and the second vehicle is identified by the average value of the brightness difference of each pixel of the image information, a highly reliable result can be obtained with a relatively simple process. can get.

  An eighth stop low-speed vehicle detection method according to the present invention is the sixth stop low-speed vehicle detection method, wherein in the similarity calculation step, image information of a vehicle candidate area relating to the first vehicle and a second vehicle The similarity is calculated based on a correlation coefficient with the image information of the vehicle candidate area relating to the vehicle.

  By this method, the similarity of the image information between the first vehicle and the second vehicle is identified by the correlation coefficient between the image information, so that a highly reliable result can be obtained.

  A ninth stop low-speed vehicle detection method of the present invention is the sixth stop low-speed vehicle detection method, wherein, in the similarity calculation step, differential data of image information of a vehicle candidate area relating to the first vehicle and The similarity is calculated based on the average value of the differential value of each pixel with the differential data of the image information of the vehicle candidate area for the second vehicle.

  According to this method, the similarity between the image information of the first vehicle and the second vehicle is identified by the average value of the differential values of the pixels of the differential data of the image information, so that a highly reliable result is obtained. It is done.

  The tenth stop low-speed vehicle detection method of the present invention is the sixth stop low-speed vehicle detection method, wherein in the similarity calculation step, the differential data of the image information of the vehicle candidate area relating to the first vehicle and the first The similarity is calculated based on a correlation coefficient with the differential data of the image information of the vehicle candidate area relating to the second vehicle.

  By this method, the similarity between the image information of the first vehicle and the second vehicle is identified by the correlation coefficient with the differential data of the image information, so that a highly reliable result can be obtained.

  An eleventh stop low-speed vehicle detection method according to the present invention is any one of the first to tenth stop low-speed vehicle detection methods, and is consistent with the first vehicle information stored in the stop information storage procedure or When a second vehicle having similar vehicle information is not detected for a predetermined time, the method further includes a step of discarding the stored information on the first vehicle.

  By this method, even if the target vehicle to be detected leaves the video when it is not photographed due to the passage of another vehicle (interfering vehicle), etc. Since the information is discarded, the storage area on the stop information storage means can be used effectively, and the occurrence of useless processing for information that is not used can be suppressed.

  According to the present invention, even when the target vehicle cannot be captured as a continuous image because of being blocked by another vehicle, it is possible to recognize a vehicle in a stopped state or a low speed state with high reliability.

  Embodiments of the present invention will be described below with reference to FIGS.

  For example, on a road where there is no traffic light such as an expressway, all vehicles usually continue to travel at a speed higher than a predetermined speed. In such an environment, a vehicle stopped on the road or a vehicle whose speed is abnormally lowered means that it is a broken vehicle or an accident. Therefore, in such a system for monitoring a road, it is desirable that a stopped vehicle or an abnormally lowered vehicle can be recognized quickly and without malfunction.

  Therefore, in this embodiment, it is assumed that the present invention is applied to a monitoring system configured as shown in FIG. FIG. 1 is a block diagram illustrating a configuration example of a main part of a monitoring system according to an embodiment of the present invention. As shown in FIG. 1, in order to photograph a vehicle traveling on a road 11 as a subject to be monitored, a plurality of surveillance cameras 12 fixed at predetermined positions on the side of the road 11 are provided.

  Moreover, each monitoring camera 12 is arrange | positioned toward the advancing direction of the vehicle 13, and each vehicle 13 is image | photographed from back. Moreover, as a place where the surveillance camera 12 is actually installed, for example, in a tunnel is assumed. By photographing the vehicle from behind, halation, smear, and the like due to the vehicle headlight can be prevented.

  The monitoring system shown in FIG. 1 includes an image processing unit 14 and an event determination unit 15. The image processing unit 14 repeatedly inputs images obtained by photographing by the monitoring cameras 12 as monitoring images continuously or periodically, and performs necessary image processing. Specifically, vehicle detection, vehicle speed detection, fallen object detection, and the like are performed. That is, a vehicle is extracted based on an image taken from the rear of the vehicle, and a vehicle width, a vehicle length, and a vehicle tail position are detected. Then, the detected vehicle is tracked, the vehicle speed is detected based on the change in position, and candidates for a low-speed vehicle or a stopped vehicle are detected. Based on the processing result of the image processing unit 14, the event determination unit 15 determines an event on the target road regarding detection of a sudden event such as a traffic jam or a vehicle failure or accident.

  Next, an outline of processing in the image processing unit 14 and the event determination unit 15 will be described. FIG. 2 is a flowchart showing a specific example related to the processing of the image processing unit and the event determination unit according to the embodiment of the present invention.

  In step S1, the image processing unit 14 first creates background image data and background differential image data as initial data. Here, after the power is turned on or reset, for example, 300 times of captured image data (images for 30 seconds) captured from the monitoring camera 12 every 0.1 seconds are averaged between frames to obtain a background image. Also, a background differential image is created by performing differential processing of the background image data. Furthermore, since the brightness of the lighting on the road is different between night and day, the night mode or the day mode is determined based on the background image, and the mode corresponding to the current time is set.

  In the next step S2, image data to be used for image analysis is created from the captured image data captured from the monitoring camera 12. Specifically, the video signal of the NTSC signal input from the surveillance camera 12 is converted to 320 by performing AD conversion (analog-digital conversion) for each frame with 240 lines in the vertical direction, 320 pixels in the horizontal direction, and 256 luminance levels. Image data of pixel × 240 pixels is created. At this time, only an even field is AD-converted to create one image data every 1/30 seconds. In the subsequent image analysis, three image data of the current image, the image before ΔT, and the image before 2ΔT (here, ΔT = 100 ms) are used.

  In step S3, image preprocessing such as feature extraction calculation and binarization processing is performed on the created image data to create a feature image from which the features of the vehicle are extracted. In the feature extraction calculation, feature extraction is performed by using feature extraction methods such as a background difference method, a differential background difference method, a frame difference method, and a differential frame difference method to create a difference image, and a feature image is created. Here, in order to increase the processing speed of the post-processing, for example, a feature image is created by performing processing such as feature extraction, and then the image data of 320 pixels × 240 pixels is compressed to 80 pixels × 60 pixels. Create an extracted compressed image. Further, a light extraction image of 320 pixels × 240 pixels is created from the image data created for image analysis.

  In the next step S4, image analysis is performed using the above-described feature extraction compressed image and light extraction image to extract vehicles and the like. At this time, the vehicle body analysis is performed using the feature extraction compressed image, and the vehicle rear, the vehicle length, and the vehicle width are determined. Further, light (tail light image) analysis is performed using the light extraction image, and the tail position of the vehicle is determined by detecting the tail light of the vehicle. For example, the vehicle position is determined from the detection result of the rectangular vehicle body (detection of the vehicle candidate area) by the vehicle body analysis and the result of the vehicle tail detection by the light analysis. Further, the falling object analysis is performed using the feature extraction compressed image, and the position and the vertical and horizontal dimensions of the falling object are determined.

  Thereafter, the detected vehicle position data is added or newly created to the travel locus data, and the vehicle tracking is performed by adding the detected vehicle position data to the travel locus data for each cycle of this processing (step S5). Moreover, the moving speed of each detected vehicle is calculated.

  Further, the speed of the vehicle group in the captured image is calculated by a method different from the vehicle tracking (step S6). Here, the entire vehicle is regarded as one lump car group from the movement amount of the luminance value of the entire image, and the moving speed of this car group is calculated. This vehicle group speed can be used for determining a traffic jam at the time of a traffic jam.

  Next, the event determination unit 15 determines whether or not an event such as “stop”, “low speed”, “congestion”, “falling object” has occurred based on the results of the vehicle tracking, the vehicle group speed calculation, the falling object detection, etc. (Step S7).

  Then, the image processing unit 14 updates the background image data and the background differential image data to the latest background image for image processing in the next processing cycle (step S8). Further, the night mode or the day mode is determined based on the new background image, and the mode is updated to the current time point. Thereafter, the process returns to step S2, and the processing cycle of steps S2 to S8 is repeated at predetermined intervals (here, 100 ms).

  Next, the stop low-speed vehicle detection process that is a characteristic element of the embodiment of the present invention will be described in detail. FIG. 3 is a block diagram illustrating a functional configuration for performing vehicle recognition and stop low-speed vehicle detection in the image processing unit and the event determination unit according to the embodiment of the present invention.

  The image processing unit 14 includes an image input unit 21, a background image creation unit 22, a feature image creation unit 23, a feature code creation unit 24, an image storage unit 25, a feature code storage unit 29, and a vehicle recognition unit 30. The event determination unit 15 includes a stop holding information storage unit 33 and a stop low speed event recognition unit 34.

  The image input unit 21 inputs image data obtained by photographing with the monitoring camera 12. The background image creation unit 22 creates a background image from the input image data. The feature image creating unit 23 creates a feature image by performing processing such as background difference, differential background difference, frame difference, and differential frame difference using the input image and the background image. The feature code creation unit 24 creates a feature code indicating the feature of each pixel for calculating the vehicle candidate region based on the feature image. The image storage unit 25 stores image data, and the feature code storage unit 29 stores feature codes. The vehicle recognition unit 30 performs vehicle recognition processing.

  The image storage unit 25 includes a current image storage unit 26 that stores a current image, a background image storage unit 27 that stores a background image, and a feature image storage unit 28 that stores a feature image. Further, the vehicle recognition unit 30 includes a vehicle head / tail detection unit 31 and a vehicle tracking unit 32. The vehicle head and tail detection unit 31 detects the positions of the vehicle head and the vehicle tail, which are feature points of the vehicle, based on the current image and the calculated feature code. The vehicle tracking unit 32 tracks the vehicle using the vehicle head and the vehicle tail detected by the vehicle head / vehicle tail detection unit 31. That is, it grasps how the vehicle to be tracked is moving, and detects the speed and the like from the change in position.

  The image storage unit 25, the feature code storage unit 29, and the stop holding information storage unit 33 may be configured by a memory or the like. The background image creation unit 22, the feature image creation unit 23, the feature code creation unit 24, the vehicle recognition unit 30 and the stop low-speed event recognition unit 34 may be configured by a processor or the like that performs each process.

  The current image storage unit 26 stores the current image data input from the image input unit 21. As input image data, a grayscale image of 256 gradations is used. The background image creation unit 22 creates a background image of only the background using past current image data and stores the background image in the background image storage unit 27. The feature image creation unit 23 uses the current image data stored in the current image storage unit 26 and the background image data stored in the background image storage unit 27 to use a background difference, a differential background difference, a frame difference, and a differential frame difference. The feature image for detecting the vehicle in the photographed image is created and stored in the feature image storage unit 28.

  The feature code creating unit 24 creates a feature code indicating the feature of each pixel for calculating the vehicle candidate area based on the feature image, and stores it in the feature code storage unit 29. The feature code is obtained by assigning codes such as 0, 1, 2,..., 9 to each pixel in the feature image. For example, 0 is the part that is the same as the background and no data is obtained by differential processing, 1 is the discarded data that seems to be noise, etc., 2 is the effective data valid as a vehicle, 2 is the part that is smaller than the vehicle width, Is data that makes it possible to determine the position of the vehicle, such that 3 is a candidate vehicle part, 8 is a part recognized as a vehicle body, 9 is a part recognized as a car tail, and so on.

  The vehicle head / tail detection unit 31 calculates, for example, a rectangular region as a vehicle candidate using the current image data stored in the current image storage unit 26 and the feature code stored in the feature code storage unit 29, Recognize the head and tail. Normally, since the vehicle is traveling, the position of the vehicle head and the vehicle tail recognized by the vehicle head and vehicle tail detection unit 31 changes in time series as the image captured by each monitoring camera 12 changes. Therefore, the vehicle tracking unit 32 grasps the position of the vehicle head and the vehicle tail of each vehicle that changes in time series and the change thereof, traces the position of the detected vehicle, calculates a travel locus, and detects the moving speed of the detected vehicle. Is calculated. Note that the vehicle head / vehicle tail detection unit 31 may be configured to detect at least one of the vehicle head and the vehicle tail according to the direction in which the road is photographed.

  By the way, it is very important to recognize a stopped vehicle, that is, a vehicle whose speed is abnormally lowered or a vehicle that has stopped completely as a special event. However, for example, if a video disturbance, recognition error, temporary sudden deceleration of the target vehicle, etc. are erroneously recognized as the occurrence of a stopped vehicle, a serious problem arises in terms of system reliability.

  In order to increase the reliability regarding detection of a stopped vehicle or a low-speed vehicle, it is effective to identify whether or not the same detection state continues for a long time. For example, if a low-speed state is detected continuously for 5 seconds, there is no possibility of image disturbance, recognition error, or temporary sudden deceleration of the target vehicle. You may recognize as a vehicle or a stop vehicle.

  However, if processing is performed so that the final determination is made after confirming that the same state continues for a predetermined time, there is a possibility that the stopped vehicle to be detected cannot be recognized in the actual use environment.

  FIG. 7 is a front view showing a specific example of a subject photographed by the surveillance camera. In the example shown in FIG. 7, it is assumed that a video including vehicles traveling in two lanes 111 and 112 with the same monitoring camera is taken with a slight shift in time, a few seconds after the state of FIG. This represents a case where the state changes to the state of FIG. 7B and then changes to the state of FIG. 7C after a few seconds.

  In this case, the vehicle stopped on the lane 112 in the image of FIG. 7A appears in the rectangular vehicle candidate area 101, but when a little time passes, the lane 111 is enlarged as shown in FIG. 7B. Therefore, the image of the target vehicle that has stopped on the lane 112 is blocked from the vehicle candidate area 101.

  Therefore, even when a candidate for a stopped vehicle is detected based on the image of FIG. 7A, the image changes to the image shown in FIG. 7B before 5 seconds elapse, and the candidate vehicle is lost. I can't finally determine if the vehicle is really stopped. If the obstructing vehicle 102 in the image shown in FIG. 7B passes, the image of FIG. 7C is obtained, and the stopped vehicle can be captured again in the vehicle candidate area 101. In such a state where the vehicle is traveling, the state shown in FIG. 7B is obtained again in a short cycle, so that the candidate vehicle is lost again before 5 seconds elapse. Therefore, it will not be possible to recognize the stopped vehicle until long, or it will take a long time to recognize the stopped vehicle.

  Therefore, in the present embodiment, information related to a candidate vehicle in a stopped state or a low speed state that has been detected and lost in the past is temporarily stored in the stop holding information storage unit 33 under the control of the stopped low speed event recognition unit 34. The stop low-speed event recognition unit 34 uses information stored in the stop holding information storage unit 33 when finally determining whether the candidate vehicle in the stop state or the low-speed state is a stop vehicle or a low-speed vehicle.

  Below, the stop low-speed vehicle detection process in the stop holding information storage part 33 and the stop low speed event recognition part 34 is demonstrated in detail. Here, two processing examples of the first embodiment and the second embodiment are shown.

(First embodiment)
4-6 is a figure which shows the operation | movement of the concrete stop low speed event recognition part 34, and the operation | movement shown in FIG.4, FIG.5 and FIG.6 is processed in parallel. Actually, for example, the respective processes may be periodically repeated in the order of FIGS.

  FIG. 4 is a flowchart showing a first process executed for the main operation of the stopped low-speed vehicle detection device according to the first embodiment. The stop low-speed event recognition unit 34 proceeds to the next step S12 when detecting that the tracking by the vehicle tracking unit 32 is completed in step S11. In step S12, at the time of the tracking process of the vehicle tracking unit 32, for example, it is identified whether there is a vehicle that has been lost due to a change in the situation as shown in FIG.

  When there is a candidate vehicle that has been lost, the process proceeds from step S12 to step S13, and it is identified whether or not the stop time has been measured for the corresponding vehicle. This stop time is measured when the vehicle speed related to the corresponding vehicle is equal to or lower than a predetermined value, and is measured in step S27 of FIG. 5 described later. If the stop time has been measured, the process proceeds from step S13 to S14.

  In step S14, the information of the corresponding vehicle (stopped vehicle candidate that has been lost) is stored in the stop holding information storage unit 33 as stop holding information. The stop holding information includes information representing a vehicle identification number for distinguishing a plurality of vehicles, a tail position of the corresponding vehicle, and a stop time of the corresponding vehicle.

  In other words, when the vehicle disappears at the time of tracking and the vehicle is below a predetermined speed, the vehicle information is stored in the stop holding information storage unit 33.

  FIG. 5 is a flowchart showing a second process executed for the main operation of the stopped low-speed vehicle detection device according to the first embodiment. When the stop low-speed event recognition unit 34 detects in step S21 that the tracking in the vehicle tracking unit 32 is completed, the process proceeds to the next step S22. In step S22, the vehicle speed of the candidate vehicle found by the tracking of the vehicle tracking unit 32 is compared with a vehicle stop determination speed Vth that is a predetermined threshold value to identify whether the candidate vehicle is almost stopped. When the condition of (vehicle speed <Vth) is satisfied, the vehicle is regarded as a candidate for a stopped vehicle, and the process proceeds from step S22 to S23. Step S22 is performed for all detected vehicles. If no vehicle is detected, the process ends.

  In step S23, it is checked whether or not the above-described stop holding information exists on the stop holding information storage unit 33. If stop holding information exists, the process proceeds to step S24, and if not, the process proceeds to step S27.

  In step S24, it is determined whether or not the vehicle detected in the past as the stop vehicle candidate and held in the stop hold information storage unit 33 as stop hold information is the same as the stop vehicle candidate detected by the latest tracking. Identify. In this example, the distance between the vehicle rear positions of each vehicle is compared with a predetermined threshold value to identify whether or not both are the same vehicle. That is, when the distance is relatively close, both are regarded as the same vehicle, and the process proceeds from step S24 to S25. When the distance is long, the process proceeds from step S24 to S27.

  In step S25, the candidate of the stopped vehicle detected by the latest tracking is associated with the stop holding information on the stop holding information storage unit 33 as that of the same vehicle. Specifically, a registered vehicle identification number (a vehicle identification number included in the stop holding information) is assigned as the vehicle identification number of the currently detected vehicle, and as the currently detected vehicle stop time, The registration stop time (stop time included in the stop holding information) is added to the stop time.

  In step S26, the stop holding information used in step S25 is deleted from the stop holding information storage unit 33 and discarded.

  In step S27, measurement of the stop time relating to the candidate stop vehicle currently being monitored is started. In addition, when progressing to step S27 from step S26, the time value measured previously as an initial value of stop time will exist from the beginning.

  In step S28, it is identified whether or not the stop time measured in step S27 satisfies a predetermined condition as a stop event. In this example, the stop time measured in step S27 (accumulated value when measurement is interrupted in the middle) is compared with a threshold value (for example, 5 seconds), and if the stop time exceeds the threshold value, it is considered that a stop event has occurred. In this case, in the next step S29, the stop low-speed event recognition unit 34 outputs special information indicating “occurrence of stop event” to an external device.

  FIG. 6 is a flowchart showing a third process executed for the main operation of the stopped low-speed vehicle detection device according to the first embodiment. The stop low speed event recognition unit 34 identifies whether stop hold information is held in the stop hold information storage unit 33 in step S31. If valid stop holding information is held, the process proceeds from step S31 to S32. In step S32, the value of the stop holding time counter Ch is counted up. This stop holding time counter Ch is used to measure the length of time that the same stop holding information is held in the stop holding information storage unit 33.

  In the next step S33, the value of the stop holding time counter Ch is compared with a predetermined threshold, that is, the stop holding time upper limit value T. When the condition of (T <Ch) is satisfied, the process proceeds from step S33 to S34. In step S34, the stop holding information on the stop holding information storage unit 33 is discarded. In this case, the value of the stop holding time counter Ch is cleared.

  That is, after a stop vehicle to be detected is detected as a candidate, for example, when a change as shown in FIG. 7B occurs and the candidate vehicle is temporarily lost, before a long time elapses. The state changes to that shown in FIG. 7C, and candidate vehicles are detected again. Therefore, the stop holding information on the corresponding stop holding information storage unit 33 is referred to in step S25 of FIG. 5, and the stop holding information is discarded in step S26. That is, the valid stop holding information is not held in the stop holding information storage unit 33 for a long time, so that the invalid stop holding information can be eliminated by executing step S34 of FIG.

  According to the first embodiment of the present invention as described above, even when the video of the target low-speed vehicle is only intermittently captured due to the influence of the passage of another vehicle (interfering vehicle) or the like. It becomes possible to recognize with high reliability.

(Second Embodiment)
A second embodiment of the present invention will be described with reference to FIG. FIG. 8 is a flowchart showing processing executed for the main operation of the stopped low-speed vehicle detection device according to the second embodiment.

  The operation of the second embodiment is a modification of the first embodiment. Processing different from that of the first embodiment will be described below. In FIG. 8, the steps corresponding to those in FIG.

  In this embodiment, when identifying whether the vehicle corresponding to the stop holding information on the stop holding information storage unit 33 and the vehicle currently being tracked are in a correspondence relationship, not only the distance between the vehicle positions, The similarity of the image of the attention area is also used. Therefore, in this embodiment, the stop low-speed event recognition unit 34, in the above-described step S14 (see FIG. 4), the rectangular area (for example, the vehicle candidate in FIG. 7A) in the image that is focused on as the stopped vehicle candidate. Information corresponding to the image data in the area 101) is also stored in the stop holding information storage unit 33.

  In step S24 of FIG. 8, the stop holding information exists on the stop holding information storage unit 33, and the distance between the vehicle position in the stop holding information and the candidate for the stopped vehicle detected by the latest tracking is relatively short. In this case, the process proceeds from step S24 to S41 in FIG.

  In step S41, the similarity between the image information of the rectangular area included in the stop holding information held in the stop holding information storage unit 33 and the image information of the rectangular area corresponding to the vehicle currently focused on in the latest image. Is calculated. In step S42, the calculated similarity is compared with a predetermined threshold value to identify whether the two images match. If the two images match, the process proceeds from step S42 to S25, and the same processing as in the first embodiment is executed.

  Note that, for example, various methods as shown below can be considered as the similarity calculation processing in step S41.

(1) A brightness difference is calculated for each pixel of the entire rectangular area, and an average value in the entire rectangular area is calculated as a similarity. When the sizes of the two rectangular areas are different, the small area is scanned within the large area, for example, from the upper left to the lower right, and the minimum value of the calculated average value of the brightness differences is used as the similarity.
(2) A correlation coefficient between two pieces of image information is calculated, and this correlation coefficient is used as a similarity. When the sizes of the two rectangular areas are different, scanning is performed in the same manner as in (1), and the maximum value of the correlation coefficient is set as the similarity.
(3) For the differential data of each pixel information, a differential value difference is calculated for each pixel, and an average value in the entire rectangular area is calculated as the similarity.
(4) A correlation coefficient between images is calculated for the differential data of each image information, and this correlation coefficient is used as the similarity.

  According to the second embodiment of the present invention, whether or not the vehicle stored in the stop information storage unit and the target vehicle in the latest image are the same vehicle is determined not only by the difference in distance but also by image information. Therefore, it is possible to recognize a stopped low-speed vehicle with high reliability.

  Even if the target vehicle cannot be captured as a continuous image because it is blocked by other vehicles, the present invention has an effect of being able to recognize a vehicle in a stopped state or a low-speed state with high reliability. This is useful for a low-speed vehicle detection device and method.

The block diagram which shows the structural example of the principal part of the monitoring system of embodiment of this invention The flowchart which shows the specific example regarding the process of the image process part which concerns on embodiment of this invention, and an event determination part The block diagram which shows the function structure which performs vehicle recognition and stop low-speed vehicle detection in the image process part and event determination part which concern on embodiment of this invention The flowchart which shows the 1st process performed for the main operation | movement of the stop low-speed vehicle detection apparatus which concerns on 1st Embodiment. The flowchart which shows the 2nd process performed for main operation | movement of the stop low-speed vehicle detection apparatus which concerns on 1st Embodiment. The flowchart which shows the 3rd process performed for the main operation | movement of the stop low-speed vehicle detection apparatus which concerns on 1st Embodiment. Front view showing a specific example of a subject photographed by a surveillance camera The flowchart which shows the process performed for the main operation | movement of the stop low-speed vehicle detection apparatus which concerns on 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Road 12 Monitoring camera 13 Vehicle 14 Image processing part 15 Event determination part 21 Image input part 22 Background image creation part 23 Feature image creation part 24 Feature code creation part 25 Image storage part 26 Current image storage part 27 Background image storage part 28 Feature Image storage unit 29 Feature code storage unit 30 Vehicle recognition unit 31 Car head and tail detection unit 32 Vehicle tracking unit 33 Stop holding information storage unit 34 Stop low-speed event recognition unit 101 Vehicle candidate area 102 Interfering vehicle 111, 112 Lane

Claims (22)

Image input means for inputting a photographed image obtained by photographing a subject including a vehicle on the road;
Vehicle recognition means for detecting a vehicle candidate area corresponding to the vehicle in the captured image and recognizing at least a feature point of the vehicle based on information of the vehicle candidate area;
Vehicle stop detection means for detecting a stop state or a low speed state of the vehicle based on information on the vehicle candidate area detected by the vehicle recognition means or information on the feature points;
When the vehicle stop detection means detects a stop state or a low speed state of the vehicle, stop information storage means for storing information representing a stopped vehicle candidate;
Based on the detection state of the vehicle stop detection means and the information held in the stop information storage means, a stop low speed event recognition means for recognizing a stop vehicle or a low speed vehicle and outputting a signal as a stop low speed event;
A low-speed vehicle detection device comprising: The stop low-speed vehicle detection device according to claim 1,
In the stop information storage means, a target vehicle corresponding to a vehicle detected by the vehicle recognition means in the past is no longer detected by the vehicle recognition means, and the stop state or low speed state of the target vehicle is detected by the vehicle stop detection means. Stopped low-speed vehicle detection device that stores relevant information as a stopped vehicle candidate only when The stop low-speed vehicle detection device according to claim 1 or 2,
The stop information storage means stores information indicating at least a vehicle identification number, a position of a detected feature point of the vehicle, and a length of time when a stop state is detected as information indicating the stopped vehicle candidate. Low speed vehicle detection device. The stop low-speed vehicle detection device according to claim 3,
After the first vehicle in which the vehicle information is stored in the stop information storage unit is no longer detected by the vehicle recognition unit, the vehicle stop detection unit sets the second vehicle stop state or low speed as a new stop vehicle candidate. When the state is detected, the vehicle stop detection means compares the distance between the position of the feature point of the first vehicle held in the stop information storage means and the position of the feature point of the second vehicle. If the distance is equal to or smaller than the predetermined distance, the first vehicle and the second vehicle are regarded as the same, and the stop time of the vehicle is determined as the stop time related to the first vehicle and the second vehicle. A stop low-speed vehicle detection device having a time obtained by adding a stop time. The stop low-speed vehicle detection device according to claim 3,
The stop information storage means is a stop low-speed vehicle detection device that further stores image information of the vehicle candidate area as information indicating the stopped vehicle candidate. The stop low-speed vehicle detection device according to claim 5,
After the first vehicle in which the vehicle information is stored in the stop information storage unit is no longer detected by the vehicle recognition unit, the vehicle stop detection unit sets the second vehicle stop state or low speed as a new stop vehicle candidate. When the state is detected, the vehicle stop detection means compares the distance between the position of the feature point of the first vehicle and the position of the feature point of the second vehicle held in the stop information storage means. In addition, the similarity between the image information of the vehicle candidate area relating to the first vehicle and the image information of the vehicle candidate area relating to the second vehicle is calculated, and the distance is equal to or less than a predetermined value and the similarity is high. In this case, the first vehicle and the second vehicle are regarded as the same, and the stop time of the vehicle is the sum of the stop time related to the first vehicle and the stop time related to the second vehicle. Stop to Speed vehicle detection device. The stop low-speed vehicle detection device according to claim 6,
The vehicle stop detection means calculates the similarity based on the average value of the brightness differences of the pixels of the image information of the vehicle candidate area relating to the first vehicle and the image information of the vehicle candidate area relating to the second vehicle. Stop low-speed vehicle detection device. The stop low-speed vehicle detection device according to claim 6,
The vehicle stop detection means calculates the similarity based on a correlation coefficient between the image information of the vehicle candidate area related to the first vehicle and the image information of the vehicle candidate area related to the second vehicle. . The stop low-speed vehicle detection device according to claim 6,
The vehicle stop detection means is an average value of a differential value of each pixel between differential data of image information of the vehicle candidate area relating to the first vehicle and differential data of image information of the vehicle candidate area relating to the second vehicle. A low-speed vehicle detection device for calculating the similarity based on The stop low-speed vehicle detection device according to claim 6,
The vehicle stop detection means calculates the similarity based on a correlation coefficient between differential data of image information of the vehicle candidate area related to the first vehicle and differential data of image information of the vehicle candidate area related to the second vehicle. Stop low-speed vehicle detection device. The stop low-speed vehicle detection device according to any one of claims 1 to 10,
When a second vehicle having vehicle information that matches or is similar to the information of the first vehicle held in the stop information storage means is not detected for a predetermined time, the stop information storage means Stop low-speed vehicle detection device that discards information. An image input step for inputting a photographed image obtained by photographing a subject including a vehicle on the road;
A vehicle recognition step of detecting a vehicle candidate area corresponding to the vehicle in the captured image and recognizing at least a feature point of the vehicle based on information of the vehicle candidate area;
A vehicle stop detection step for detecting a stop state or a low speed state of the vehicle based on information on the vehicle candidate area detected in the vehicle recognition step or information on the feature points;
A stop information storage step for storing information representing a stopped vehicle candidate when a vehicle stop state or a low speed state is detected in the vehicle stop detection step;
A stop low speed event recognition step for recognizing a stop vehicle or a low speed vehicle and outputting a signal as a stop low speed event based on the detected state in the vehicle stop detection step and information on the stored stop vehicle candidate,
A stop low-speed vehicle detection method comprising: A stop low-speed vehicle detection method according to claim 12,
In the stop information storing step, the target vehicle corresponding to the vehicle detected in the past by the vehicle recognition procedure is not detected in the vehicle recognition procedure, and the stop state or the low speed state is detected in the vehicle stop detection step. Stopped low-speed vehicle detection method for storing corresponding information as a stopped vehicle candidate only when the A stop low-speed vehicle detection method according to claim 12 or 13,
In the stop information storage step, as information indicating the stopped vehicle candidate, a stop that stores at least a vehicle identification number, a position of the detected feature point of the vehicle, and a length of time when the stop state is detected is stored. Low speed vehicle detection method. The stop low-speed vehicle detection method according to claim 14,
When the stop state or low speed state of the second vehicle is detected as a new stop vehicle candidate after the first vehicle in which the vehicle information is stored in the stop information storing step is no longer detected, the stored information is stored. Comparing the distance between the position of the feature point of the first vehicle and the position of the feature point of the second vehicle;
When the compared distance is less than or equal to the predetermined distance, the first vehicle and the second vehicle are regarded as the same, and the stop time for the first vehicle and the stop for the second vehicle are used as stop times of the vehicles. A step of adding time;
A stop low-speed vehicle detection method further comprising: The stop low-speed vehicle detection method according to claim 14,
In the stop information storing step, a stop low-speed vehicle detection method further storing image information of the vehicle candidate area as information representing the stopped vehicle candidate. The stop low-speed vehicle detection method according to claim 16,
When the stop state or low speed state of the second vehicle is detected as a new stop vehicle candidate after the first vehicle in which the vehicle information is stored in the stop information storing step is not detected, the stored information is stored. Comparing the distance between the position of the feature point of the first vehicle and the position of the feature point of the second vehicle;
A similarity calculation step of calculating a similarity between the image information of the vehicle candidate area relating to the first vehicle and the image information of the vehicle candidate area relating to the second vehicle;
When the compared distance is equal to or less than a predetermined value and the similarity is high, the first vehicle and the second vehicle are regarded as the same, and the stop time for the first vehicle is set as the stop time of the vehicle. Adding the time and the stop time for the second vehicle;
A stop low-speed vehicle detection method further comprising: A stopped low-speed vehicle detection method according to claim 17,
In the similarity calculation step, the similarity is calculated based on an average value of the brightness difference of each pixel between the image information of the vehicle candidate area relating to the first vehicle and the image information of the vehicle candidate area relating to the second vehicle. Stop low-speed vehicle detection method. A stopped low-speed vehicle detection method according to claim 17,
In the similarity calculation step, a stopped low-speed vehicle detection method that calculates the similarity based on a correlation coefficient between the image information of the vehicle candidate area relating to the first vehicle and the image information of the vehicle candidate area relating to the second vehicle. . A stopped low-speed vehicle detection method according to claim 17,
In the similarity calculation step, an average value of the differential values of the respective pixels of the differential data of the image information of the vehicle candidate area related to the first vehicle and the differential data of the image information of the vehicle candidate area related to the second vehicle A low-speed vehicle detection method for calculating the similarity based on the vehicle speed. A stopped low-speed vehicle detection method according to claim 17,
In the similarity calculation step, the similarity is calculated based on a correlation coefficient between differential data of image information of the vehicle candidate area related to the first vehicle and differential data of image information of the vehicle candidate area related to the second vehicle. Stop low-speed vehicle detection method. A stop low-speed vehicle detection method according to any one of claims 11 to 21,
If a second vehicle having vehicle information that matches or is similar to the first vehicle information stored in the stop information storage procedure is not detected for a predetermined time, the stored first vehicle information is discarded. A stopped low-speed vehicle detection method further comprising the step of:

Images