JP7284951B2 - Monitoring support device, monitoring support program, and storage medium - Google Patents

Description

TECHNICAL FIELD The present invention relates to a monitoring support device, a monitoring support program, and a storage medium, and in particular, monitoring that measures the running direction of a vehicle without being affected by light, shadow, rain, snow, insects, etc. when used outdoors. It relates to a support device, a monitoring support program, and a storage medium.

In recent years, the number of elderly drivers has increased, and the number of wrong-way accidents on highways has increased. Technology development is awaited for this reverse running monitoring. On the other hand, surveillance cameras have become cheaper, installed in large numbers, and used as surveillance equipment.

JP-A-10-269492

The technology of Patent Document 1 includes "a plurality of vehicle sensors installed on a one-way road, a vehicle traveling direction determining means for determining the traveling direction of the traveling vehicle from the outputs of the plurality of vehicle sensors, and a vehicle traveling direction. reverse-running vehicle determination means for determining whether or not the traveling vehicle is traveling in the opposite direction on the one-way road based on the determination result of the determination means." This is based on the premise that a radio wave transmitting device such as a beacon for transmitting radio waves is installed in the vehicle. Therefore, it is necessary to install radio wave transmitters in all vehicles entering the monitored section, which is difficult to introduce.

In addition, although there are techniques for tracking specific objects of interest (persons, puppies, etc.) in the image of a 2D camera, 2D cameras are affected by light and shadow outdoors, and are located between the camera and the vehicle. , Birds, insects, rain, snow, etc., there are many false detections.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a monitoring support device, a monitoring support program, and a storage medium.

In order to solve the above-mentioned problems, the monitoring support device according to the first invention is
an acquisition unit that acquires an image captured by a stereo camera;
a monitoring plane setting unit for setting a monitoring plane in an image;
a direction setting unit for setting a reverse running direction or a forward direction in an image;
A monitoring unit that sets an object in contact with a monitoring surface as an object of interest, compares the spatial coordinates of the object of interest with the object of interest in at least one frame before or after the image, and obtains the moving direction of the object of interest. and,
an alarm unit that issues an alarm when the moving direction of the target object is the reverse direction or the opposite direction of the forward direction;
have
characterized by having

In addition, the monitoring support device according to the second invention is
an output unit that outputs an image showing the object of interest together with the alarm;
further has

In addition, the monitoring support device according to the third invention is
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even in the opposite case, if the maximum area of the target object on the monitoring plane is less than the first criterion (small objects such as dust and falling objects, or objects such as sheets that have no volume are excluded) , no alarm,
It is characterized by

In addition, the monitoring support device according to the fourth invention is
a three-dimensional object generation unit that collects sliced shapes of the object on the monitoring plane for each frame in which the object of interest contacts and/or passes through the monitoring plane, and generates a three-dimensional object of interest from these objects; ,
have more.

In addition, the monitoring support device according to the fifth invention is
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even if it is the opposite of , if the volume of the 3D object of interest is less than the first criterion (dust, falling objects, sheets, etc. are excluded as having no volume), no alarm is issued.
It is characterized by

In addition, the monitoring support device according to the sixth invention is
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction If the opposite is the case, and if the first standard or more, or the first standard or more but less than the second standard (treated as a first size vehicle such as a motorcycle or a passenger car), the first size vehicle Issue an alarm as a reverse run,
A monitoring support device characterized by:

In addition, the monitoring support device according to the seventh invention is
an acquisition unit that acquires an image captured by a stereo camera;
a monitoring plane setting unit for setting a first monitoring plane and a second monitoring plane in an image;
a direction setting unit for setting a reverse running direction or a forward direction in an image;
An object in contact with the first or second monitoring surface is set as an object of interest, and a time difference between the time when the object of interest first contacts the first monitoring surface and the time when the object first contacts the second monitoring surface is provided. a monitoring unit that obtains the moving direction of the target object using
an alarm unit that issues an alarm when the moving direction of the target object is the reverse direction or the opposite direction of the forward direction;
have

In addition, the monitoring support device according to the eighth invention is
an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
an alarm unit that issues an alarm when the moving direction of the target object is the reverse direction or the opposite direction of the forward direction;
have

In addition, the monitoring support device according to the ninth invention is
an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
merging the object of interest in the first monitoring plane of the first image and the object of interest in the second monitoring plane of the second image to generate an integrated object of interest; and virtual monitoring of the integrated object of interest. a projection unit that projects onto a surface;
a determination unit that determines whether or not the area of the integrated object of interest on the virtual monitoring plane exceeds a predetermined threshold;
If the moving direction of the object of interest is the backward running direction or the reverse of the forward direction, and if the area of the integrated object of interest on the monitoring plane exceeds a predetermined threshold, an alarm is issued. an alarm unit that emits
have

In addition, the monitoring support device according to the tenth invention is
an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
merging the object of interest in the first monitoring plane of the first image and the object of interest in the second monitoring plane of the second image to generate an integrated object of interest; and virtual monitoring of the integrated object of interest. a projection unit that projects onto a surface;
a determination unit that determines whether or not the area (size) of the integrated object of interest on the virtual monitoring plane exceeds a first threshold value and is equal to or less than a second threshold value, or whether or not the second threshold value is exceeded; ,
When the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, and the area of the integrated object of interest on the monitoring plane exceeds the first threshold and reaches the second If it is less than or equal to the threshold, an alarm is issued with information that the object of interest is a vehicle of a first size (e.g., a motorcycle, a small vehicle, or a car, etc.), or the direction of movement of the object of interest is reversed. If it is in the direction of travel, or vice versa in the forward direction, and if it exceeds the second threshold, then the object of interest is a vehicle of a second size (e.g., a truck, bus, or large vehicle). etc.), an alarm unit that issues an alarm,
have

In addition, the monitoring support device according to the eleventh invention is
an acquisition unit that acquires a first image captured by a first stereo camera at a first point and a second image captured by a second stereo camera at a second point;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
If the moving direction of the target object is between the first point and the second point, 1 is added to the number of the target objects, and the moving direction of the target object is between the first point and the first point. , a measuring unit for measuring the number of attention objects between the first point and the second point by subtracting 1 from the number of the attention objects when the direction is from between the second point and the second point;
have

As described above, the solution means of the present invention has been explained as an apparatus, but the present invention can also be realized as a method, a program, and a storage medium recording a program substantially corresponding to these, and the scope of the present invention is as follows. should be understood to include these. Each step of the methods and programs below uses arithmetic processing units such as CPUs and DSPs as necessary for data processing. , HDD, and memory.

For example, the program according to the twelfth invention to be executed by a computer, which implements the present invention as a program,
A monitoring support program that causes one or a plurality of arithmetic processing units to function as the monitoring support device according to any one of the first to eleventh inventions.

According to the present invention, it is possible to measure the traveling direction of a vehicle and properly monitor the vehicle without being affected by light, shadow, rain, snow, insects, etc. in outdoor use.

FIG. 1 is a block diagram showing an outline of a monitoring support device (system) according to one embodiment of the present invention. FIG. 2 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. FIG. 3 is a flow chart showing an example of processing executed by a monitoring support device (system) according to an embodiment of the present invention. FIG. 4 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. FIG. 5 is a flow chart showing an example of processing executed by a monitoring support device (system) according to an embodiment of the present invention. FIG. 6 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. FIG. 7 is a schematic diagram illustrating a technique for generating an integrated object using two stereo cameras. FIG. 8 is a schematic diagram illustrating in more detail the technique of generating an integrated object using two stereo cameras. FIG. 9 is a schematic diagram illustrating a technique for modeling an object from cross-sections of the object passing through the monitoring plane. FIG. 10 is a schematic diagram illustrating a technique for classifying an object that has come into contact with the monitoring surface using this device. FIG. 11 is a schematic diagram of an embodiment in which the present device is realized as a car counter (a device for measuring the number of vehicles by vehicle type). FIG. 12 is a schematic diagram of another embodiment in which the present device is implemented as a car counter (a device for measuring the number of vehicles by vehicle type). FIG. 13 is a schematic diagram showing how the trajectories PW1 to PW4 of the vehicle are tracked after the vehicle enters the rotary.

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an outline of a monitoring support device (system) according to one embodiment of the present invention. As shown in the figure, the monitoring support device MAD1
an acquisition unit ACQ that acquires an image captured by a stereo camera;
a monitoring plane setting unit MPS for setting a monitoring plane in an image;
a direction setting unit DS for setting a reverse running direction or a forward direction in an image;
A monitoring unit that sets an object in contact with a monitoring surface as an object of interest, compares the spatial coordinates of the object of interest with the object of interest in at least one frame before or after the image, and obtains the moving direction of the object of interest. MON and
an alarm unit WAN that issues an alarm when the movement direction of the object of interest is the reverse direction or the opposite direction of the forward direction;
have

Acquisition unit ACQ, monitoring plane setting unit MPS, direction setting unit DS, monitoring unit MON, alarm unit WAN, etc. are preferably configured by program modules loaded into control unit CON, which is an arithmetic unit such as a processor. .

The device has an output unit OUT for outputting an image showing the object of interest together with the warning. The alarm unit WAN may also serve as the output unit.

When the moving direction of the attention object is the reverse running direction instead of the alarm unit WAN issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction. , or even in the case of the reverse of the forward direction, when the maximum area of the object of interest on the monitoring plane is less than the first criterion (small objects such as dust and falling objects, or objects with no volume such as sheets ) does not generate an alarm. According to this configuration, it is possible to prevent false alarms by properly excluding small objects such as dust, falling objects, or sheets.

The control unit CON
A three-dimensional object generation unit GEN that collects sliced shapes of the object on the monitoring plane and generates a three-dimensional object of interest from these objects for each frame in which the object of interest touches and/or passes through the monitoring plane. , and further have

Also, the alarm unit WAN
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even if the opposite is the case, if the volume of the 3D object of interest is less than the first standard (exclude dust, falling objects, sheets, etc. as having no volume), the alarm may not be issued. It is possible. According to this configuration, it is possible to prevent false alarms by appropriately excluding small objects such as dust and fallen objects, or thin objects such as sheets that are thin and have no volume, and which are noisy.

In addition, the alarm unit WAN
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction If the opposite is the case, and if the first standard or more, or the first standard or more but less than the second standard (treated as a first size vehicle such as a motorcycle or a passenger car), the first size vehicle An alarm is issued as a reverse run. According to this configuration, it is possible to classify the size of the wrong-way vehicle and issue an alarm.

The monitoring support device MAD1 also has an input unit IN, an output unit OUT, a communication unit COM, a storage unit STR, and a display unit DIS. In addition, the monitoring support device MAD1 is connected to stereo cameras MC1 and MC2 that take pictures of roads, routes, etc., via a communication unit COM and a network NET, and acquires images. In addition, the monitoring support device MAD1 can be connected to the mobile terminal PDA1 via the communication unit COM and the network NET to exchange information. The output unit OUT can output information stored in the device or generated information to the printer PRN or electronic bulletin board EB1. The display unit DIS can also display information stored or generated in the device. The input unit IN receives operation instructions and information input via the mouse MUS and keyboard KBD, and passes the received information and data to the acquisition unit ACQ. The storage unit STR stores standard information STD storing numerical values of the 1st to 3rd standards and vehicle type information of the standards, and vehicle information CI including a vehicle object to be compared with the modeling object and its vehicle type name. .

Each functional unit included in the control unit is preferably implemented by a program module loaded into the memory space of the control unit. Normally, the PC operates as the present device by downloading software that causes the processor to function as each part of the present device from a storage unit or website, installing the software in the PC, and activating the software. It should be noted that each functional unit provided in the control unit is nothing more than a set of steps that have a certain unity in terms of function. It can be incorporated into other functional units or divided into other independent functional units.

In this way, the generated/extracted information, intermediate data, and acquired data can be transmitted externally, displayed on the display unit, and the generated/extracted information, intermediate data, and acquired data can be stored in the storage unit. Note that doing so is equally possible in other implementations described below. In addition, the present device is a computer such as a general-purpose computer, a special-purpose computer, a server, a PC, a mobile terminal such as a smartphone, or a program module that realizes (executes) the functions and processing procedures (methods) of the present device on a computer. It is preferable to build this device on a computer by holding it in the CPU or storage unit of the computer, or reading it from an external server or storage, and the same applies to each subsequent embodiment. Also, each functional unit may be distributed in separate computers or devices connected by a network. Alternatively, a form in which a plurality of functional units are combined into one, or a part of the processing steps is performed by another functional unit may be used.

FIG. 2 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. Assume that there is a vehicle VH passing through a road RD as shown in the figure. A reverse running direction DIR of the vehicle is set in the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, and a monitoring plane VF is set in the image. This device determines whether the vehicle is traveling in the reverse direction or not using the technique described above each time the vehicle touches the monitoring surface VF. When it is determined that the vehicle is running in the wrong direction, an image of the vehicle and a warning message indicating that the vehicle is running in the wrong direction are displayed on the electronic bulletin board EB1 installed in the direction in which the vehicle is traveling in the reverse direction.

FIG. 3 is a flow chart showing an example of processing executed by a monitoring support device (system) according to an embodiment of the present invention. As shown in the figure, a monitoring plane is set in step S11. The monitoring plane is preferably a vertical plane across the roadway over which the vehicle passes. In step S12, the reverse running direction (or the forward running direction may be used) is set. Then, in step S13, an image is acquired. In step S14, the object in contact with the monitoring surface is set as the object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the movement of the object of interest. ask for directions. Then, in step S15, it is determined whether or not the moving direction of the target object is the reverse running direction, or whether it is the reverse of the forward direction. If the determination condition of step S15 is satisfied, that is, if the moving direction of the object of interest is in the reverse direction or the opposite direction to the forward direction, the process proceeds to step S16, and an alarm is issued. In the event of an alarm, it is possible to display an image of the vehicle in question on a bulletin board or the like, or to broadcast the vehicle type and license plate number of the image. If the determination is not satisfied, the process returns to step S13. Note that once the storing step is performed, it is not necessary to perform it again until the information is changed. Alternatively, the storage step becomes unnecessary by installing a ROM or flash memory storing the information in the device or computer.

FIG. 4 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. Assume that there is a vehicle VH passing through a road RD as shown in the figure. In the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, the vehicle's reverse running direction DIR is set, and in the image, a monitoring plane VF1 and a monitoring plane VF2 are set. This device determines whether the vehicle is traveling in the reverse direction or not using the technique described above each time the vehicle touches the monitoring surface VF. Specifically, an object that touches the first or second monitoring surface is set as an object of interest, and the time when the object of interest first contacts the first monitoring surface and the time when the object of interest first touches the second monitoring surface are calculated. The direction of movement of the object of interest is obtained by using the time difference at the time of contact. When it is determined that the vehicle is running in the wrong direction, an image of the vehicle and a warning message indicating that the vehicle is running in the wrong direction are displayed on the electronic bulletin board EB1 installed in the direction in which the vehicle is traveling in the reverse direction. According to this configuration, since there are two monitoring surfaces, it is possible to easily determine the direction, and furthermore, the two monitoring surfaces make it possible to eliminate contact with flying objects such as sheets and cardboards. That is, false alarms can be suppressed. Since the shape of the flying object changes remarkably when it comes into contact with each contact surface, such objects can be eliminated.

FIG. 5 is a flow chart showing an example of processing executed by a monitoring support device (system) according to an embodiment of the present invention. As shown in the figure, in step S21, first and second monitoring planes are set. The monitoring plane is preferably a vertical plane across the roadway over which the vehicle passes. In step S22, the reverse running direction (or the forward running direction may be used) is set. Then, in step S23, an image is obtained. In step S24, an object that touches the first or second monitoring surface is set as an object of interest, and the time when the object of interest first touches the first monitoring surface and the time when the object first touches the second monitoring surface are calculated. The direction of movement of the object of interest is obtained by using the time difference between the two. Then, in step S25, it is determined whether or not the moving direction of the target object is the reverse running direction, or whether it is the reverse of the forward direction. When the determination condition of step S25 is satisfied, the process proceeds to step S26, and an alarm is issued. At the time of warning, it is possible to display an image of the vehicle on a bulletin board or the like, or to broadcast the vehicle type and number of the image by voice. If the determination condition is not satisfied, the process returns to step S23.

FIG. 6 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse driving monitoring device. Assume that there is a vehicle VH passing through a road RD as shown in the figure. In the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, the reverse running direction DIR of the vehicle is set, and in the image, the monitoring plane VF1 is set. In this configuration, the reverse direction DIR of the vehicle is similarly set in the image of the stereo camera CM2 placed opposite the stereo camera CM1 across the road RD, and in the image, the monitoring Set the plane VF2. The monitoring plane VF1 and the monitoring plane VF2 are set at the same position in terms of spatial coordinates. That is, one of the features of this configuration is to monitor the same monitoring surface from different positions. This device determines whether the vehicle is traveling in the reverse direction or not using the technique described above each time the vehicle touches the monitoring surfaces VF1 and VF2. When it is determined that the vehicle is running in the wrong direction, an image of the vehicle and a warning message indicating that the vehicle is running in the wrong direction are displayed on the electronic bulletin board EB1 installed in the direction in which the vehicle is traveling in the reverse direction. One of the advantages of this configuration is that it eliminates the possibility of missing detection due to obstruction of vision by birds, insects, etc. that pass between the camera and the subject (reversing vehicle). . Even if the field of view of the stereo camera CM1 is blocked by a bird or the like, the possibility that the field of view of the stereo camera CM2 is blocked at the same time is extremely low, so it is possible to reliably prevent omission of detection. .

FIG. 7 is a schematic diagram illustrating a technique for generating an integrated object using two stereo cameras. In this case, a monitoring support device configured as follows is used.

The monitoring support device
an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
merging the object of interest in the first monitoring plane of the first image and the object of interest in the second monitoring plane of the second image to generate an integrated object of interest; and virtual monitoring of the integrated object of interest. a projection unit that projects onto a surface;
a determination unit that determines whether or not the area of the integrated object of interest on the virtual monitoring plane exceeds a predetermined threshold;
If the moving direction of the object of interest is the backward running direction or the reverse of the forward direction, and if the area of the integrated object of interest on the monitoring plane exceeds a predetermined threshold, an alarm is issued. an alarm unit that emits
have

As shown in the figure, when a small vehicle touches the monitoring surface, the stereo camera CM1 recognizes the attention object AO1 on the first monitoring surface VF1 of the first image. Similarly, it is also recognized that a small vehicle has touched the monitoring plane on the monitoring plane VF2 installed at the spatial coordinates of the same plane as the monitoring plane VF1. That is, the stereo camera CM2 recognizes the target object AO2 on the second monitoring plane VF2 of the second image. Then, the projection unit merges the object of interest in the first monitoring plane of the first image and the object of interest in the second monitoring plane of the second image to generate an integrated object of interest, and The object of interest IAO1 is projected onto the virtual monitoring plane MVFI. The advantage of this technique is that the two stereo cameras monitor the same plane surveillance surface from different angles, so that the contours of objects, the entire object, etc. that cannot be captured by one stereo camera can be captured by the other stereo camera. By merging the objects captured by each camera, it is possible to recognize the almost complete outline of the object.

FIG. 8 is a schematic diagram illustrating in more detail the technique of generating an integrated object using two stereo cameras. As shown in the figure, it is possible to recognize only the upper side SDT1 and the right side SDR1 of the object on the monitoring plane VF1. On the other hand, in the monitoring plane VF2, it is possible to recognize only the upper side SDT2 and the right side SDL1 of the object. Then, the apparatus merges these sides to generate an integrated object of interest, and projects the integrated object of interest IAO2 onto the virtual monitoring plane MVFI. With this technique, it is possible to accurately grasp the size of the contact surface in the middle from the time when contact is started with the monitoring surface, and it is possible to estimate the object that is in contact with the monitoring surface. In other words, an object passing through the surveillance plane can only be recognized on two sides (upper side and one lateral side) near the surveillance camera. Therefore, as in this technique, by merging surveillance plane images from two different directions, it is possible to estimate and grasp an accurate cross-section on the surveillance plane of an object passing through the surveillance plane.

FIG. 9 is a schematic diagram illustrating a technique for modeling an object from cross-sections of the object passing through the monitoring plane. This device collects sliced shapes of the object on the monitoring surface for each frame in which the object of interest touches and/or passes through the monitoring surface, and generates a three-dimensional object of interest from these objects. It can be realized by providing a three-dimensional object generation unit. As shown in the figure, in scene SN1, the apparatus recognizes that the nose (tip portion) of vehicle VH1 is in contact with monitoring surface VF1. This scene SN1 is converted into a format in which the object of interest AO1 is projected onto the virtual monitoring plane VFI1. Similarly, in scene SN2, the apparatus recognizes that the body (intermediate portion) of vehicle VH1 is in contact with monitoring plane VF1. This scene SN2 is converted into a format in which the object of interest AO2 is projected onto the virtual monitoring plane VFI2. Then, in scene SN3, the apparatus recognizes that the tail (rear end portion) of vehicle VH1 is in contact with monitoring plane VF1. This scene SN3 is converted into a format in which the object of interest AO3 is projected onto the virtual monitoring plane VFI3. For the sake of illustration and explanation, there is only one intermediate scene. is more than 10, and more than 10 cross-sections of the vehicle (that is, objects of interest) are projected. Finally, a modeling object MDO is generated based on these objects of interest AO1-3 and the estimated vehicle speed (for example, the speed is obtained based on the time between frames and the moving distance of the object of interest between frames). This modeling object MDO is compared with/referred to the vehicle type database stored in the storage unit to identify the vehicle type. In this case, the small vehicle is estimated to be model xx1 of Company A. The more frames per second, the more accurate modeled objects can be generated. Also, in this example, the image was taken with one stereo camera, but of course, it is possible to apply this technique to the image taken with two stereo cameras.

FIG. 10 is a schematic diagram illustrating a technique for classifying an object that has come into contact with the monitoring surface using this device. As shown in the figure, in process IST01, the contact area of the object on the monitoring surface is obtained. Next, in process IST02, the area of the object (or modeling object) on the monitoring plane is obtained, or the size of the object is estimated from the area. Next, in process IST03, the size (or area or volume) of the object is compared with the reference. The reference is a numerical value stored in the storage unit, and includes a first reference, a second reference, and a third reference, which are set as follows.
Less than 1st standard: Garbage, fallen objects, birds, small animals, etc. 1st standard or more, less than 2nd standard: Motorcycles, bicycles 2nd standard or more, less than 3rd standard: Small vehicles {passenger cars, etc.}
Standard 3 or above: Large vehicles (trucks, buses, etc.)
The numerical value of each standard may be area or volume.

In this example, items below the first standard (such as "garbage" as shown) proceed to process IST04 and do not issue an alarm. If the object is equal to or greater than the first criterion but less than the second criterion (such as "motorcycle" as shown in the figure), the process proceeds to process IST05 and an alarm is issued as a motorbike. Vehicles above the second standard and below the third standard (such as small vehicles as shown in the figure) proceed to processing IST06 and are reported as small vehicles (such as passenger cars). If the vehicle exceeds the third criterion (large vehicle, etc.), the process proceeds to step IST07, and is reported as a large vehicle (truck, bus, etc.). In this way, the device can roughly classify objects that run in the wrong direction according to vehicle type.

FIG. 11 is a schematic diagram of an embodiment in which the present device is realized as a car counter (a device for measuring the number of vehicles by vehicle type). This can be realized by the following device configuration.

The monitoring support device
an acquisition unit that acquires a first image captured by a first stereo camera at a first point and a second image captured by a second stereo camera at a second point;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
If the moving direction of the target object is between the first point and the second point, 1 is added to the number of the target objects, and the moving direction of the target object is between the first point and the first point. , a measuring unit for measuring the number of attention objects between the first point and the second point by subtracting 1 from the number of the attention objects when the direction is from between the second point and the second point;
have

As shown in the figure, a stereo camera CM1 is installed on the approach route of a parking area PA on an expressway, a monitoring plane VF1 is set there, and vehicles in the direction of entering the PA are counted (the numerical value is CT1). Similarly, a stereo camera CM2 is installed on the exit route of the parking area PA, a monitoring plane VF2 is set there, and vehicles in the direction exiting the PA are counted (value is CT2). Then, the number of vehicles staying in the PA is calculated by the following formula.
Number of vehicles staying in PA = CT1 - CT2
Since this device can measure by vehicle size, that is, by vehicle type, it is possible to measure the number of stays by vehicle type. In this case, it is possible to estimate the usage rate of parking spaces by vehicle type in the PA from the number of stays by vehicle type, and this information can be displayed on the bulletin board on the main line before the PA. As a result, it is possible to prevent a situation in which there is no parking space for the vehicle type that the driver is driving at the PA that he or she has come to. This technique can also be applied to station roundabouts.

In this example, vehicles VH2, VH3, and VH4 are entering through monitoring plane VF1.
CTT1=3
Then, VH4 passes through the monitoring plane VF2 and exits PA.
CT2=1
Therefore, the number of vehicles staying in PA = CT1-CT2
= 3-1
= 2 units.

As shown in the figure, the "current number of vehicles", "today's cumulative number of vehicles", and the like can be recorded in the car counter CCNT1 and displayed on the display unit or stored in the storage unit. It is possible.

FIG. 12 is a schematic diagram of another embodiment in which the present device is implemented as a car counter (a device for measuring the number of vehicles by vehicle type). The configuration of the device is almost the same as that explained with reference to FIG. 11 unless otherwise specified. An additional device is a bird's-eye view camera BCM that can overlook the rotary ROT. This will be explained later with reference to FIG. In FIG. 12 as well, it is possible to record the "current number of vehicles", "today's cumulative number of vehicles", etc. by vehicle type in the car counter CCNT2, and display it on the display or store it in the storage. be.

FIG. 13 is a schematic diagram showing how the trajectories PW1 to PW4 of the vehicle are tracked after the vehicle enters the rotary. The bird's-eye view camera BCM is for tracking vehicles that have entered the roundabout, one by one, from entry to exit. As shown in FIG. 13, vehicle VH3-1 moves to position PT1 through trajectory PW1. Vehicle VH3-1 can be recorded as it moves from position PT1 to position PT2 through trajectory PW2. Then, the vehicle VH3-1 moves from the position PT2 to the position PT3 through the trajectory PW3, passes through the trajectory PW4, passes through the position PT4, and exits the rotary. In this way, the present device can record and analyze the movements and parking conditions of each vehicle during its stay in the rotary in real time or after the fact. Although one vehicle is tracked in this embodiment, it is also possible to simultaneously track multiple vehicles or vehicles of different types (for example, a motorcycle and a truck). As shown in the figure, the car counter CCNT3 records the "current number of vehicles", "today's cumulative number of vehicles", etc. for each vehicle type, and displays them on the display unit or stores them in the storage unit. It is possible. Furthermore, it is possible to record the "current average staying time", "today's average staying time", etc. for each vehicle type, display it on the display unit, or store it in the storage unit.

Although the present invention has been described with reference to the drawings and examples, it should be noted that various variations and modifications will be readily apparent to those skilled in the art based on this disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, processes and functions included in each unit and each step can be rearranged so as not to be logically inconsistent, and multiple means/units and steps can be combined into one or divided. It is possible. Alternatively, it should be noted that some components, functions, processes, steps, etc. of the apparatus, methods, programs, etc. according to the present invention may be located on a remote server or the like.

ACQ Acquisition unit AO1 Object of interest AO2 Object of interest AO3 Object of interest CI Vehicle information CM1 Stereo camera CM2 Stereo camera COM Communication unit CON Control unit DIR Reverse direction DIS Display unit DS Direction setting unit EB1... Electronic bulletin board GEN... Dimensional object generation part IAO1... Integrated object of interest IAO2... Integrated object of interest IN... Input part IST01... Processing IST02... Processing IST03... Processing IST04... Processing IST05... Processing IST06... Processing IST07... Processing KBD... Keyboard MAD1 Monitoring support device MC1 Stereo camera MDO Modeling object MON Monitoring unit MPS Monitoring surface setting unit MUS Mouse MVFI Virtual monitoring surface NET Network OUT Output unit PA Parking area PDA1 Portable terminal PRN Printer RD Road S11 Step S12 Step S13 Step S14 Step S15 Step S16 Step S21 Step S22 Step S23 Step S24 Step S25 Step S26 Step SDL1 Right side SDR1 Right side SDT1 Top side SDT2 Top side SN1 Scene SN2 Scene SN3 Scene STD Reference information STR Storage unit VF Monitoring surface VF1 Monitoring surface VF2 Monitoring surface VFI1 Virtual monitoring surface VFI2 Virtual monitoring surface VFI3 Virtual monitoring surface VH Vehicle VH1 Vehicle VH2...Vehicle WAN...Alarm part

Claims (16)

an acquisition unit that acquires an image captured by a stereo camera;
a monitoring plane setting unit for setting a monitoring plane in an image;
a direction setting unit for setting a reverse running direction or a forward direction in an image;
A monitoring unit that sets an object in contact with a monitoring surface as an object of interest, compares the spatial coordinates of the object of interest with the object of interest in at least one frame before or after the image, and obtains the moving direction of the object of interest. and,
an alarm unit that issues an alarm when the moving direction of the object of interest is the reverse direction or the reverse of the forward direction;
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even if the reverse is the case, if either the maximum area of the object of interest on the monitoring plane or the volume of the three-dimensional object of interest is less than the first criterion, no alarm is issued,
the maximum area of the object of interest on the monitoring plane is the largest among the areas of the object of interest on the monitoring plane that change over time;
The volume of the three-dimensional object of interest is the volume of the object of interest obtained by collecting sliced shapes of the object of interest on the monitoring plane obtained over time.
Surveillance support device. The monitoring support device according to claim 1,
an output unit that outputs an image showing the object of interest together with the alarm;
A monitoring support device further comprising: The monitoring support device according to claim 1 or 2,
A three-dimensional object generation unit that collects sliced shapes of the object on the monitoring plane for each frame in which the object of interest touches and/or passes through the monitoring plane, and generates the three-dimensional object of interest from these objects. of,
A monitoring support device further comprising. In the monitoring support device according to claim 3,
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction and if the first criterion or more, or the first criterion or more and less than the second criterion, the first size vehicle runs in the opposite direction, and an alarm is issued.
A monitoring support device characterized by: an acquisition unit that acquires an image captured by a stereo camera;
a monitoring plane setting unit for setting a first monitoring plane and a second monitoring plane in an image;
a direction setting unit for setting a reverse running direction or a forward direction in an image;
An object in contact with the first or second monitoring surface is set as an object of interest, and a time difference between the time when the object of interest first contacts the first monitoring surface and the time when the object first contacts the second monitoring surface is provided. a monitoring unit that obtains the moving direction of the target object using
an alarm unit that issues an alarm when the moving direction of the object of interest is the reverse direction or the reverse of the forward direction;
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even if the reverse is the case, if either the maximum area of the object of interest on the monitoring plane or the volume of the three-dimensional object of interest is less than the first criterion, no alarm is issued,
the maximum area of the object of interest on the monitoring plane is the largest among the areas of the object of interest on the monitoring plane that change over time;
The volume of the three-dimensional object of interest is the volume of the object of interest obtained by collecting sliced shapes of the object of interest on the monitoring plane obtained over time.
Surveillance support device. an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
an alarm unit that issues an alarm when the moving direction of the object of interest is the reverse direction or the reverse of the forward direction;
The alarm unit
Instead of issuing an alarm when the moving direction of the attention object is the reverse running direction or when the moving direction is the reverse of the forward direction, when the moving direction of the attention object is the reverse running direction or the forward direction Even if the reverse is the case, if either the maximum area of the object of interest on the monitoring plane or the volume of the three-dimensional object of interest is less than the first criterion, no alarm is issued,
the maximum area of the object of interest on the monitoring plane is the largest among the areas of the object of interest on the monitoring plane that change over time;
The volume of the three-dimensional object of interest is the volume of the object of interest obtained by collecting sliced shapes of the object of interest on the monitoring plane obtained over time.
Surveillance support device. an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
merging an object of interest in a first monitoring plane of a first image and an object of interest in a second monitoring plane of a second image to generate an integrated object of interest; and virtual monitoring of the integrated object of interest. a projection unit that projects onto a surface;
a determination unit that determines whether or not the area of the integrated object of interest on the virtual monitoring plane exceeds a predetermined threshold;
If the moving direction of the object of interest is the backward running direction or the reverse of the forward direction, and if the area of the integrated object of interest on the monitoring plane exceeds a predetermined threshold, an alarm is issued. an alarm unit that emits
A monitoring support device having an acquisition unit that acquires a first image captured by a first stereo camera and a second image captured by a second stereo camera;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image at the same spatial coordinates as the first monitoring plane;
a direction setting unit for setting a reverse running direction or a forward direction in the first and second images;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
merging an object of interest in a first monitoring plane of a first image and an object of interest in a second monitoring plane of a second image to generate an integrated object of interest; and virtual monitoring of the integrated object of interest. a projection unit that projects onto a surface;
a determination unit that determines whether or not the area of the integrated object of interest on the virtual monitoring plane exceeds a first threshold value and is equal to or less than a second threshold value, or whether or not the area exceeds the second threshold value;
When the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, and the area of the integrated object of interest on the monitoring plane exceeds the first threshold and reaches the second If it is less than or equal to the threshold, issue an alarm with the information that the object of interest is a vehicle of the first size, or if the direction of movement of the object of interest is the reverse running direction, or reverse the forward direction. and if it exceeds a second threshold, an alarm unit that issues an alarm together with information that the object of interest is a vehicle of a second size;
A monitoring support device having an acquisition unit that acquires a first image captured by a first stereo camera at a first point and a second image captured by a second stereo camera at a second point;
a monitoring plane setting unit that sets a first monitoring plane in the first image and a second monitoring plane in the second image;
An object in contact with the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. a monitoring unit for determining the direction of movement;
If the moving direction of the target object is between the first point and the second point, 1 is added to the number of the target objects, and the moving direction of the target object is between the first point and the first point. , a measuring unit for measuring the number of objects of interest between the first point and the second point by subtracting 1 from the number of the objects of interest when the direction is from between the first point and the second point;
A monitoring support device having 10. The monitoring support device according to claim 9, wherein the object of interest is a vehicle. The first point is a point on an entry route of the vehicle in a parking area,
The monitoring support device according to claim 10, wherein the second point is a point on an exit route of the vehicle in the parking area. The monitoring support device according to claim 11, wherein the measurement unit measures the number of vehicles staying in the parking area as the number of objects of interest. 12. The monitoring support device according to claim 11, wherein the measurement unit measures the number of vehicles staying in the parking area by vehicle type as the number of objects of interest. 11. The monitoring support apparatus according to claim 10, further comprising an alarm unit that issues an alarm when the movement direction of the object of interest is the reverse direction or the reverse of the forward direction. A monitoring support program that causes one or more arithmetic processing units to function as the monitoring support device according to any one of claims 1 to 14. A computer-readable storage medium storing the monitoring support program according to claim 15.

Images