JP2021100872A - Monitoring device, monitoring system, monitoring method, and program - Google Patents

Abstract

To predict and prevent a human being from falling to a lower part of a step in a place as well as a situation where the human being may cross a vehicle.SOLUTION: A monitoring device comprises a control unit that monitors objects each entering a predetermined area. The control unit classifies the objects moving by entering the predetermined area into persons or vehicles, based on at least one of video data entered from an imaging unit for imaging the objects present in the predetermined area and sensing information entered from a sensor for detecting the objects present in the predetermined area; outputs an alarm signal in response to an alarm at one level out of alarms at a plurality of levels to an alarm output unit for giving notice outside, the alarm signal being selected according to the area a person entered from among a plurality of regions set according to a displacement from a boundary between an upper portion of a step and a lower portion of the step, when a person enters the predetermined area; and outputs a prediction alarm signal for predicting falling of the person to the lower portion of the step, to the alarm output unit, when a relative state between the parson and the vehicle meets a predetermined condition.SELECTED DRAWING: Figure 1

Description

The present invention relates to monitoring devices, monitoring systems, monitoring methods, and programs.

Traditionally, in waste incinerators, it has been necessary to monitor the behavior of workers working on the platform in order to prevent people from falling from the platform to the waste pit. For the monitoring of workers on the platform, direct visual monitoring by the observer or monitoring by the observer through a monitor that displays the captured platform is mainly adopted. In any case, a method is adopted in which the behavior of the worker on the platform is personally monitored by an observer. In addition, when a worker falls into the garbage pit and a guard or other worker discovers the fall, the discoverer or the like presses the emergency stop buttons provided in various parts of the waste incinerator facility. There is also a system that notifies the surroundings of the fall of the worker.

As a method for detecting such a fall, Patent Documents 1 and 2 disclose a technique for monitoring the dynamics of a person on a railway platform with a stereo camera and detecting when the person falls. Further, Patent Document 3 discloses a technique for detecting whether or not a vehicle has fallen into a waste pit in a waste incinerator using a photoelectric switch and a loop coil.

Japanese Unexamined Patent Publication No. 2004-058737 Japanese Unexamined Patent Publication No. 2003-246268 Japanese Unexamined Patent Publication No. 2000-318840

As mentioned above, when monitoring the behavior of people such as workers on the platform, human monitoring by observers may overlook when a person falls under a step such as a garbage pit on the platform. There is. Further, in the prior art, it is only determined whether or not a person or a vehicle has fallen to the lower part of the step, and the person who is a worker and the vehicle intersect with each other, such as on a platform of a waste incinerator. In such places and situations, it was difficult to predict the possibility of a person or vehicle falling to the bottom of a step such as a garbage pit, or to prevent the fall.

The present invention has been made in view of the above, and an object of the present invention is a monitoring device capable of predicting and suppressing a fall of a human step to the lower part in a place or situation where a human and a vehicle intersect. To provide monitoring systems, monitoring methods, and programs.

The monitoring device according to one aspect of the present invention is a monitoring device including a control unit that monitors at least one object that enters a predetermined area above the step at a step of a predetermined height. , The predetermined region is entered based on at least one of the sensing information input from the sensor that detects the object existing in the predetermined region and the video data input from the imaging unit that images the object existing in the predetermined region. A plurality of objects set according to the displacement from the boundary between the upper part of the step and the lower part of the step when the person detects the moving object as a human or a vehicle and the human enters the predetermined area. The alarm signal corresponding to the alarm of one level of the alarms of a plurality of levels selected according to which area of the regions is entered is output to the alarm output unit for notifying the outside. When the relative situation between the human and the vehicle meets a predetermined condition, a predictive warning signal for predicting the fall of the human to the lower part of the step is output to the warning output unit.

In the above invention, the monitoring device according to one aspect of the present invention has the control unit in the predetermined area based on the video data of the object entering the predetermined area captured by the imaging unit and the sensing information. Among the detected objects included in the sensing information by extracting the existing vehicle, the vehicle includes a substantially rectangular parallelepiped spatial region including the existing region of the vehicle extracted based on the video data in the sensing information. Classify as.

In the above-described invention, the monitoring device according to one aspect of the present invention includes vehicle shape data obtained by causing the control unit to recognize a plurality of vehicles in a pattern, and the video data input from the imaging unit. By comparing with the sensing information, the vehicle is extracted from the video data and the sensing information.

In the above invention, the monitoring device according to one aspect of the present invention is obtained by comparing the human shape data obtained by making the control unit recognize a plurality of human postures in a pattern and the sensing information. , It is determined whether or not the object included in the sensing information is the human being.

In the monitoring device according to one aspect of the present invention, in the above invention, the control unit acquires the dynamic locus of the detected object, and based on the dynamic locus, enters and moves into the predetermined region, at least one of the above. Detects one object as a human or a vehicle.

In the above invention, the monitoring device according to one aspect of the present invention is provided with a plurality of the sensors at a position higher than the height of the vehicle, and the control unit receives sensing information output from each of the plurality of sensors. Synchronize with each other.

In the above invention, the monitoring device according to one aspect of the present invention is a distance measuring sensor that measures the distance from the sensor to an object.

In the above invention, in the monitoring device according to one aspect of the present invention, the upper part of the step is the platform in the waste treatment facility, the lower part of the step is the garbage pit in the waste treatment facility, and the vehicle is dust. It is a collection vehicle.

The monitoring system according to one aspect of the present invention is a monitoring system that monitors at least one object that enters a predetermined area above the step at a step of a predetermined height, and detects an object existing in the predetermined area. At least one of a sensor, an imaging unit that images an object existing in the predetermined area, an alarm output unit that notifies an alarm to the outside, sensing information input from the sensor, and video data input from the imaging unit. Based on, the at least one object that enters and moves in the predetermined area is classified as a human or a vehicle, and when the human enters the predetermined area, the boundary between the upper part of the step and the lower part of the step. The alarm signal corresponding to the alarm of one level of the alarms of the plurality of levels selected according to which region of the plurality of regions set according to the displacement from the above is entered. When the relative situation between the human and the vehicle meets a predetermined condition by outputting to the alarm output unit, the alarm output unit is provided with a predictive alarm signal for predicting the fall of the human to the lower part of the step. A monitoring device having a control unit for outputting is provided.

In the monitoring method according to one aspect of the present invention, a control unit having hardware acquires sensing information from a sensor that detects an object existing in a predetermined area above the step at a step at a predetermined height, and obtains sensing information from the sensor. Video data is acquired from an imaging unit that captures an object existing in the region, and based on at least one of the acquired sensing information and the video data, at least one object that enters and moves in the predetermined region is a human or a vehicle. When the human entered the predetermined area, he / she entered any of the plurality of areas set according to the displacement from the boundary between the upper part of the step and the lower part of the step. An alarm signal corresponding to an alarm of one level among a plurality of levels of alarms selected according to the above is transmitted to an alarm output unit for notifying the outside to notify the alarm of the one level. When the relative situation between the human and the vehicle meets a predetermined condition, a predictive warning signal for predicting the fall of the human to the lower part of the step is transmitted to the warning output unit, and the predictive warning signal is transmitted. Notify the prediction alarm corresponding to.

A program according to one aspect of the present invention is a control unit that monitors at least one object that enters a predetermined area above the step at a step of a predetermined height, and a sensor that detects an object existing in the predetermined area. A human being moves at least one object that enters and moves in the predetermined area based on the acquired sensing information and image data acquired from an imaging unit that acquires the sensing information and images an object existing in the predetermined area. Alternatively, it is classified as a vehicle, and when the human enters the predetermined area, it is assigned to any of a plurality of areas set according to the displacement from the boundary between the upper part of the step and the lower part of the step. An alarm signal corresponding to an alarm of one level among a plurality of levels of alarms selected according to the entry is transmitted to an alarm output unit that notifies the outside, and the relative between the person and the vehicle is transmitted. When a certain situation meets a predetermined condition, a prediction warning signal for predicting the fall of the person to the lower part of the step is transmitted to the warning output unit.

According to the monitoring device, monitoring system, monitoring method, and program according to the present invention, in a place or situation where humans and vehicles intersect, such as a platform of a waste treatment facility, to the lower part of a human step. It is possible to predict and suppress a fall.

FIG. 1 is a block diagram showing a configuration of a monitoring system according to an embodiment of the present invention. FIG. 2 is a flowchart for explaining a method of monitoring a person in the monitoring method according to the embodiment of the present invention. FIG. 3 is a flowchart for explaining a vehicle monitoring method in the monitoring method according to the embodiment of the present invention. FIG. 4 is a flowchart for explaining a method of monitoring a person and a vehicle in the monitoring method according to the embodiment of the present invention. FIG. 5 is a side sectional view showing a waste incinerator according to an embodiment of the present invention. FIG. 6 is a top view for explaining an example of a method of setting a platform area according to an embodiment of the present invention. FIG. 7 is an exploded view showing a fixing member of the distance measuring sensor connected to the monitoring device according to the embodiment of the present invention. FIG. 8 is a perspective view showing a fixing member of the distance measuring sensor connected to the monitoring device according to the embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings of the following embodiment, the same or corresponding parts are designated by the same reference numerals. Further, the present invention is not limited to one embodiment described below.

First, a monitoring system according to an embodiment of the present invention will be described. The monitoring system 1 according to the present embodiment is a system for monitoring a person such as a worker on a platform of a waste treatment facility such as a waste incinerator and a vehicle such as a garbage collection vehicle. In this specification, the general waste intermediate treatment plant and the industrial waste treatment plant are collectively referred to as a waste treatment facility, and the waste collection vehicle includes a packer vehicle and a flat vehicle (dumping vehicle). FIG. 1 is a block diagram showing a monitoring system according to the present embodiment. The network 2 is composed of an internet network, a mobile phone network, and the like.

As shown in FIG. 1, the monitoring system 1 includes a monitoring device 10, a human detection unit 20, a vehicle detection unit 30, and an alarm output unit 40. Further, the monitoring system 1 is configured to be able to communicate with the vehicle control center 3 via the network 2. The network 2 is, for example, a public communication network such as the Internet, for example, a LAN (Local Area Network), a WAN (Wide Area Network), a telephone communication network such as a mobile phone, a public line, a VPN (Virtual Private Network), and the like. It consists of one or more combinations such as a dedicated line. The network 2 is a combination of wired communication and wireless communication as appropriate. The vehicle control center 3 is a management center that manages vehicles such as dust collecting vehicles, which will be described later.

(Monitoring device)
The monitoring device 10 includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15. In the monitoring device 10, the control unit 11 determines the necessity of alarm output based on the sensing information input from the human detection unit 20 and the vehicle detection unit 30 to the input unit 13, determines the type of alarm, and outputs the alarm. Various alarm information is output from the unit 14 to the alarm output unit 40. The alarm output unit 40 outputs an alarm according to the input alarm information.

Specifically, the control unit 11 includes a processor having hardware such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor), and a main storage unit such as a RAM (Random Access Memory) and a ROM (Read Only Memory). Neither is shown).

The storage unit 12 is composed of a storage medium selected from a volatile memory such as RAM, a non-volatile memory such as ROM, an EPROM (Erasable Programmable ROM), a hard disk drive (HDD, Hard Disk Drive), and a removable medium. .. The removable media is, for example, a USB (Universal Serial Bus) memory or a disc recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray (registered trademark) Disc). is there. Further, the storage unit 12 may be configured by using a computer-readable recording medium such as a memory card that can be mounted from the outside. The vehicle shape database 12a and the human shape database 12b are stored in the storage unit 12. The vehicle shape database 12a stores vehicle shape data (vehicle shape data) obtained by recognizing a vehicle Ve such as a dust collecting vehicle from various directions by pattern recognition. The human shape database 12b stores human U shape data (human shape data) obtained by recognizing human postures such as standing and walking from various directions by pattern recognition. ..

The storage unit 12 can store an operating system (OS) 121, various programs, various tables, various databases, and the like for executing the operation of the monitoring device 10. These various programs can also be recorded on a computer-readable recording medium such as a hard disk, flash memory, CD-ROM, DVD-ROM, or flexible disk and widely distributed. The various programs also include a monitoring program 122 according to the present embodiment. The control unit 11 loads the program stored in the storage unit 12 into the work area of the main storage unit and executes it, and controls each component or the like through the execution of the program to realize a function that meets a predetermined purpose. it can. In the present embodiment, by executing the monitoring program 122 loaded by the control unit 11, a person who is a worker on the platform of the waste incinerator and a vehicle such as a dust collecting vehicle are monitored.

The input unit 13 is an interface that inputs various information transmitted from, for example, the human detection unit 20 and the vehicle detection unit 30 installed on the platform and outputs the information to the control unit 11. Information may be transmitted from the human detection unit 20 and the vehicle detection unit 30 to the input unit 13 by using wired communication or wireless communication.

The output unit 14 includes a display unit 141 and a printing unit 142, and outputs an alarm signal to an external alarm output unit 40. The display unit 141 is configured to be able to display the information input from the control unit 11. The printing unit 142 is configured to be able to output information input from the control unit 11 such as a printer as a printed matter. The output unit 14 may be an interface for outputting information such as an alarm signal to an external alarm output unit 40 without including the display unit 141 and the printing unit 142.

The communication unit 15 is, for example, a LAN (Local Area Network) interface board, a wireless communication circuit for wireless communication, or the like, and is configured by using a communication module capable of wired communication or wireless communication. The LAN interface board and the wireless communication circuit can be connected to a network 2 such as the Internet, which is a public communication network. Further, the communication unit 15 may be capable of communicating with the outside in accordance with predetermined communication standards such as 4G, 5G, Wi-Fi (Wireless Fidelity) (registered trademark) and Bluetooth (registered trademark).

(Human detection unit)
The human detection unit 20 includes a communication unit 21, a pair of ranging sensors 22a and 22b including, for example, a 3D-LiDAR (3 Dimension Laser Imaging Detection and Ranging) sensor, and infrared cameras 23a and 23b as a pair of imaging units. , And are configured. The communication unit 21 is physically configured in the same manner as the communication unit 15 described above.

The ranging sensor 22a and the infrared camera 23a are, for example, held and installed in a predetermined holding portion on the left side in the vicinity of the input door and the damping box on the platform of the waste incinerator. Further, the distance measuring sensor 22b and the infrared camera 23b are held and installed in a predetermined holding portion on the right side in the vicinity of the insertion door and the damping box, for example. The details of the holding portion will be described later.

The distance measuring sensors 22a and 22b and the infrared cameras 23a and 23b as a plurality of sensors are installed so as to sense the platform by combining the distance measuring sensor 22a and the infrared camera 23a and the distance measuring sensor 22b and the infrared camera 23b as a set. Will be done. The height from the floor on which the distance measuring sensors 22a and 22b and the infrared cameras 23a and 23b are installed is a height that does not come into contact with the upper part of a vehicle such as a dust collecting vehicle and can irradiate a predetermined measurement area with laser light. It is set to. The installation heights of the distance measuring sensors 22a and 22b and the infrared cameras 23a and 23b can be specifically installed at the height of the upper part of the vehicle Ve. The set of the distance measuring sensor 22a and the infrared camera 23a and the set of the distance measuring sensor 22b and the infrared camera 23b are preferably installed at the same height, but may be installed at different heights. As a result, the laser beam can be irradiated to the upper part of the vehicle Ve. By providing a plurality of distance measuring sensors 22a and 22b near the insertion door at a predetermined height, for example, two units, it is possible to reduce the blind spot caused by the vehicle Ve and the human U in sensing.

The range of sensing by the distance measuring sensors 22a and 22b and the infrared cameras 23a and 23b along the horizontal plane can be, for example, a predetermined area on the platform 100 per entrance door. As a result, it is possible to detect the vehicle Ve and the human U who have entered the work area of the platform 100, which is a predetermined area. Further, the range of sensing by the distance measuring sensors 22a and 22b and the infrared cameras 23a and 23b along the height direction is, for example, several hundred mm above the floor of the platform and several meters higher than the height of the vehicle Ve. It can be in the range up to that point. As a result, diffused reflection of laser light and infrared rays by the floor surface can be suppressed, so that the accuracy of sensing by the sensor can be improved.

(Vehicle detection unit)
The vehicle detection unit 30 includes a communication unit 31, infrared cameras 32a and 32b as a pair of image pickup units, and distance measurement sensors 33a and 33b as a plurality of sensors. The infrared cameras 32a and 32b and the distance measuring sensors 33a and 33b are configured in the same manner as the infrared cameras 23a and 23b and the distance measuring sensors 22a and 22b described above, respectively. Here, in FIG. 1, the vehicle detection unit 30 is described separately from the human detection unit 20, but the vehicle detection unit 30 may be installed integrally with the human detection unit 20. That is, in the present embodiment, it is preferable that the infrared cameras 23a and 32a and the distance measuring sensors 22a and 33a are integrally provided in the same housing. Similarly, it is preferable that the infrared cameras 23b and 32b and the distance measuring sensors 22b and 33b are integrally provided in the same housing. Here, the distance measuring sensors 22a and 33a may be composed of one distance measuring sensor, and the distance measuring sensors 22b and 33b may be composed of one distance measuring sensor. Similarly, the infrared cameras 23a and 32a may be composed of one infrared camera, and the infrared cameras 23b and 32b may be composed of one infrared camera. The vehicle detection unit 30 and the human detection unit 20 may be provided separately.

(Alarm output section)
The alarm output unit 40 includes a display 41 as a display unit and a voice speaker 42 as a voice output unit. In addition, you may select and install necessary one of these display 41 and voice speaker 42. Further, at least one display 41 and one voice speaker 42 are provided on the platform 100, and can be provided for each input door on the platform 100 of the waste treatment facility. The alarm output unit 40 outputs at least one of character information, video, image, and sound according to the alarm signal based on the alarm signal supplied from the monitoring device 10.

(Monitoring method)
Next, the monitoring method by the monitoring system 1 configured as described above will be described. In the monitoring method according to the present embodiment, the human detection process and the vehicle detection process are executed in parallel. FIG. 2 is a flowchart for explaining a human monitoring method. FIG. 3 is a flowchart for explaining a vehicle monitoring method. FIG. 4 is a flowchart for explaining a method of monitoring a person and a vehicle.

(Human detection processing)
As shown in FIG. 2, first, in step ST1A, the left sensing information sensed by the distance measuring sensor 22a and the left image data captured by the infrared camera 23a provided on the left side of the input door are the monitoring device 10. Is input to the input unit 13. Similarly, in step ST1B, the right sensing information sensed by the ranging sensor 22b provided on the right side of the input door and the right image data captured by the infrared camera 23b are input to the input unit 13. The input unit 13 supplies the input left sensing information and right sensing information, as well as left video data and right video data, to the control unit 11.

In step ST2, the control unit 11 performs left-right synchronization processing. That is, the control unit 11 performs synchronization processing of the input left sensing information and right sensing information, and synchronization processing of the input left video data and right video data. As a result, the control unit 11 can acquire the sensing information in which the deviation between the left sensing information and the right sensing information is corrected. Similarly, the control unit 11 can acquire the video data in which the deviation between the left video data and the right video data is corrected. The control unit 11 continues to acquire the sensing information and the video data. After that, the detection process proceeds to step ST3 and then to step ST26 shown in FIG. Step ST26 will be described later.

In step ST3, the control unit 11 detects the dynamics based on the acquired sensing information and video data. That is, the control unit 11 determines whether or not a moving object exists in the acquired sensing information and video data. When the control unit 11 determines that the sensing information and the video data do not include a moving object (step ST3: No), the process returns to step ST2. On the other hand, when the control unit 11 determines that the sensing information and the video data include moving objects (step ST3: Yes), the process proceeds to step ST4. The control unit 11 may perform dynamic detection using only one of the sensing information and the video data.

In step ST4, the control unit 11 reads the shape data of the human U from the human shape database 12b of the storage unit 12 and compares the read shape data of the person with the moving object included in the sensing information. Next, in step ST5, the control unit 11 determines whether or not the moving object included in the sensing information is a human being. When the control unit 11 determines that the moving object included in the sensing information is not a human being (step ST5: No), the process returns to step ST2. On the other hand, when the control unit 11 determines that the moving object included in the sensing information is the human U (step ST5: Yes), the process proceeds to step ST6.

In step ST6, the control unit 11 generates a locus (dynamic locus) L1 of the movement of the human U based on the sensing information from the distance measuring sensors 22a and 22b. Here, since the human U performs various tasks, the dynamic locus L1 follows a complicated locus, and the number may be plural. The control unit 11 may display the generated dynamic locus L1 of the human U on the display unit 141. After that, the detection process proceeds to step ST7 and then to step ST31 shown in FIG. The step ST31 will be described later.

In step ST7, the control unit 11 determines whether or not the human U has entered the second area A2. Here, the area set on the platform including the second area will be described. FIG. 5 is a side sectional view showing a dust collecting vehicle and a waste incinerator. FIG. 6 is a top view for explaining an example of a method of setting a platform area.

As shown in FIG. 5, the platform 100 is provided adjacent to the garbage pit 50. Waste 51 is stored in the garbage pit 50. The upper surface of the waste in the state where the waste is stored in the waste pit 50 is the lower part of the step of about several meters to several meters from the platform 100.

As shown in FIG. 6, in the platform 100, a plurality of regions, specifically, for example, three regions can be set according to the displacement from the boundary with the garbage pit 50. The area of the platform 100 that covers the garbage pit 50 can be set as a third area A3 (narrow hatched portion) that is extremely likely to fall. Assuming that the platform 100 side has a positive displacement from the boundary between the platform 100 and the garbage pit 50, the third area A3 is a region where the displacement is negative. The third area A3 can be set, for example, in the area on the garbage pit 50 side from the boundary between the garbage pit 50 and the platform 100. The area farther from the garbage pit 50 than the third area A3 can be set as the second area A2 (wide hatched portion) where there is a high possibility of falling. The second area A2 can be set to a region of several hundred mm in which the displacement is positive from the boundary, including, for example, the wheel fastening of the vehicle Ve provided at the boundary between the garbage pit 50 and the platform 100. Further, the area farther from the garbage pit 50 than the second area A2 can be set as the first area A1 as a room area where there is a possibility of falling only when a predetermined situation appears. The first area A1 (thin spot hatching portion) can be set in, for example, a predetermined area in a normal work area from the boundary with the second area A2 to the side where the vehicle Ve enters.

Returning to FIG. 2, when the control unit 11 determines in step ST7 that the human U has not entered the second area A2 (step ST7: No), the control unit 11 returns to step ST6 and continues the locus processing. On the other hand, when the control unit 11 determines that the human U has entered the second area A2 (step ST7: Yes), the process proceeds to step ST8.

In step ST8, the control unit 11 performs the recording process for several minutes. Specifically, the control unit 11 stores the moving image data as detection data in the storage unit 12 by controlling the infrared cameras 23a, 23b, 32a, 32b and the distance measuring sensors 22a, 22b, 33a, 33b. Next, in step ST9, the control unit 11 outputs a light alarm signal to the output unit 14. The output unit 14 outputs the input light alarm signal to the alarm output unit 40. The light alarm signal is a signal corresponding to a lower level alarm than the most important level alarm when the alarm is set to a plurality of levels. This is because when the human U enters the second area A2, it is less likely to fall into the garbage pit 50 than when the human U enters the third area A3, which will be described later.

Next, in step ST10, the alarm output unit 40 outputs a light alarm from at least one of the display 41 and the voice speaker 42 based on the input light alarm signal. The light alarm can be given, for example, by emitting yellow light from the display 41 or by a voice such as "Be careful of falling" from the voice speaker 42. It should be noted that steps ST8, ST9, and ST10 can be performed in parallel.

Next, in step ST11, the control unit 11 determines whether or not the human U has entered the third area A3. When the control unit 11 determines that the human U has not entered the third area A3 (step ST11: No), the control unit 11 returns to step ST6 and continues the locus processing. On the other hand, when the control unit 11 determines that the human U has entered the third area A3 (step ST11: Yes), the process proceeds to step ST12.

In step ST12, the control unit 11 performs recording processing by the infrared cameras 23a, 23b, 32a, 32b and the distance measuring sensors 22a, 22b, 33a, 33b for several minutes. Next, in step ST13, the control unit 11 outputs a heavy alarm signal to the output unit 14. The heavy alarm signal is a signal corresponding to the alarm of the most important level when the alarm is set to a plurality of levels. That is, the heavy alarm is an alarm having a higher level of importance than the above-mentioned light alarm. This is because when the human U enters the third area A3, there is an extremely high possibility that the human U will fall into the garbage pit 50. The output unit 14 outputs the input heavy alarm signal to the alarm output unit 40 and proceeds to step ST14.

In step ST14, the alarm output unit 40 outputs a heavy alarm from at least one of the display 41 and the voice speaker 42 based on the input heavy alarm signal. For example, the heavy alarm can be given by emitting red light from the display 41, or an announcement such as "It is very dangerous, please evacuate immediately" can be made by voice from the voice speaker 42. It should be noted that step ST12 and steps ST13 and ST14 can be performed in parallel. As described above, the human detection process is completed, but steps ST1 to ST14 are repeatedly executed.

(Vehicle detection process)
Next, the vehicle detection process executed in parallel with the above-mentioned human detection process will be described. As shown in FIG. 3, first, in step ST21A, the left image data which is an infrared image captured by the infrared camera 32a provided on the left side of the input door and the left sensing information sensed by the distance measuring sensor 33a are obtained. , Is input to the input unit 13 of the monitoring device 10. Similarly, in step ST21B, the right image data which is an infrared image captured by the infrared camera 32b provided on the right side of the input door and the right sensing information sensed by the distance measuring sensor 33b are input to the input unit 13. Will be done. The input unit 13 supplies the input left sensing information and right sensing information, and left video data and right video data to the control unit 11.

In step ST22, the control unit 11 performs left-right synchronization processing. That is, the control unit 11 performs synchronization processing of the input left sensing information and right sensing information, and synchronization processing of the input left video data and right video data. As a result, the control unit 11 can acquire the sensing information in which the deviation between the left sensing information and the right sensing information is corrected. Similarly, the control unit 11 can acquire the video data in which the deviation between the left video data and the right video data is corrected. The control unit 11 continues to acquire the sensing information and the video data. After that, the detection process shifts to step ST23.

In step ST23, the control unit 11 detects the dynamics based on the acquired video data. That is, the control unit 11 determines whether or not a moving object exists in the acquired sensing information and video data. When the control unit 11 determines that the sensing information and the video data do not include a moving object (step ST23: No), the process returns to step ST22. On the other hand, when the control unit 11 determines that the sensing information and the video data include a moving object (step ST23: Yes), the process proceeds to step ST24. The above steps ST21A, ST21B, ST22, and ST23 may be performed in parallel with, simultaneously with, or in combination with the above-mentioned steps ST1A, ST1B, ST2, and ST3, respectively. Further, the control unit 11 may perform dynamic detection using only one of the sensing information and the video data.

In step ST24, the control unit 11 reads the vehicle shape data of the vehicle Ve from the vehicle shape database 12a of the storage unit 12, and compares the read vehicle shape data with the moving object included in the sensing information and the video data. Do. A vehicle such as a dust collecting vehicle is made to recognize a pattern by a control unit 11 or the like, and the shape data of the vehicle whose pattern is recognized is stored in the vehicle shape database 12a. The control unit 11 can determine whether or not the vehicle is a vehicle Ve by comparing the pattern-recognized vehicle shape data with the moving object included in the sensing information and the video data. In step ST25, the control unit 11 determines whether or not the moving object included in the sensing information and the video data is the vehicle Ve. When the control unit 11 determines that the moving object included in the sensing information and the video data is not the vehicle Ve (step ST25: No), the process returns to step ST22. On the other hand, when the control unit 11 determines that the moving object included in the sensing information and the video data is the vehicle Ve (step ST25: Yes), the process proceeds to step ST26.

In step ST26, the control unit 11 first synchronizes the sensing information acquired and synchronized from the distance measuring sensors 22a and 22b in step ST2 described above with the video data acquired and synchronized from the infrared cameras 32a and 32b in step ST22. Let me. Subsequently, noise is removed from the vehicle Ve information included in the sensing information based on the vehicle Ve information included in the video data. As a result, the vehicle Ve information included in the sensing information can be accurately extracted.

Next, in step ST27, the control unit 11 performs vehicle range processing for determining the range of the vehicle Ve existing region included in the sensing information. The range of the existence area including the vehicle Ve can be set to, for example, a substantially rectangular parallelepiped shape surrounding the vehicle Ve. That is, the control unit 11 defines the space region of the rectangular parallelepiped including a little outside from the surface recognized as the vehicle Ve as the vehicle Ve (frame enclosing process). This frame enclosure process is also a masking process for masking the vehicle Ve. As a result, it is possible to prevent the control unit 11 from erroneously detecting an accessory such as a hydraulic pump or a protrusion provided on the vehicle Ve as a human U, and to suppress erroneous recognition of the vehicle Ve. In addition, the existence position of the vehicle Ve can be determined.

Next, in step ST28, the control unit 11 generates the dynamic locus L2 of the vehicle Ve based on the sensing information from the distance measuring sensors 22a and 22b. Here, the dynamic locus L2 of the vehicle Ve is basically substantially linear. The control unit 11 may display the generated dynamic locus L2 of the vehicle Ve on the display unit 141. After that, the detection process proceeds to step ST29 shown in FIG.

In step ST29, the control unit 11 determines whether or not the dynamic locus L2 of the vehicle Ve indicates the reverse movement of the vehicle Ve. Here, the retreat of the vehicle Ve means, for example, as shown in FIG. 6, the movement of the vehicle Ve approaching the garbage pit 50 on the platform 100. When the control unit 11 determines that the dynamic locus L2 of the vehicle Ve does not indicate the reverse movement of the vehicle Ve (step ST29: No), step ST29 is repeatedly executed. When the control unit 11 determines that the dynamic locus L2 of the vehicle Ve indicates the backward movement of the vehicle Ve (step ST29: Yes), the process proceeds to step ST30, and the control unit 11 moves from the rear portion of the vehicle Ve to the third. The first area A1 in the range up to the boundary with the two areas A2 is set as the prediction area. This is because when the vehicle Ve retreats, if the human U is located between the vehicle Ve and the garbage pit 50, the retreat of the vehicle Ve may push the human U into the garbage pit 50. is there. Therefore, by setting the area where the possibility of being pushed out occurs as the prediction area, the possibility of falling into the garbage pit 50 can be predicted. The prediction area may be the same area as the first area A1, or may be a wider area including the first area A1 and may be an area overlapping the area behind the vehicle Ve in the first area A1.

Next, in step ST31, the control unit 11 determines whether or not the human U has entered the prediction area based on the dynamic locus L1 of the human U generated in step ST6 described above. When the control unit 11 determines that the human U has not entered the prediction area (step ST31: No), step ST31 is repeatedly executed. On the other hand, when the control unit 11 determines that the human U has entered the prediction area (step ST31: Yes), the process proceeds to step ST32. In this case, the situation in which the vehicle Ve is retreating and the human U is entering the prediction area can be a predetermined condition of the relative situation between the vehicle Ve and the human U. That is, in steps ST29 to ST31, the control unit 11 determines whether or not the relative situation between the vehicle Ve and the human U conforms to a predetermined condition.

After that, in step ST32, the control unit 11 performs the recording process for several minutes. Next, in step ST33, the control unit 11 outputs the prediction alarm signal to the output unit 14. The output unit 14 outputs the input predictive alarm signal to the alarm output unit 40 and proceeds to step ST34. In step ST34, the alarm output unit 40 outputs a predictive alarm from at least one of the display 41 and the voice speaker 42 based on the input predictive alarm signal. For example, the predictive alarm can be given by emitting green light from the display 41, or an announcement such as "There is a person behind, please be careful" can be made by voice from the voice speaker 42. It should be noted that step ST32 and steps ST33 and ST34 can be performed in parallel. As described above, the detection process between the vehicle Ve and the human U is completed, but steps ST21 to ST34 are repeatedly executed in parallel with steps ST1 to ST14.

(Holding member of distance measurement sensor)
Next, the holding member 200 for installing the distance measuring sensors 22a and 22b in the vicinity of the input door of the platform 100 will be described. 7 and 8 are an exploded view of the holding member 200 and a view showing a state at the time of installation, respectively. In the present embodiment, the holding member 200 also holds the infrared cameras 32a and 32b and fixes them in the vicinity of the input door.

As shown in FIG. 7, the holding member 200 includes a holding base 201 shown in FIG. 7A, an inclined extending portion 202 shown in FIG. 7B, and a holding plate 203 shown in FIG. 7C. The holding base 201 is composed of a metal plate or the like that can be attached in the vicinity of the input door. The inclined extending portion 202 is, for example, a trapezoidal rod-shaped or plate-shaped metal member in which a part of a rectangle is formed in a tapered shape on a side surface. The tapered portion (tapered portion) of the inclined extending portion 202 has an acute angle α portion. The acute angle α is set to, for example, 30 °, which is 20 ° or more and 45 ° or less. The holding plate 203 is composed of, for example, a metal plate formed in a rectangular shape.

As shown in FIG. 8, the holding member 200A for holding the distance measuring sensor 22a includes a holding base 201A, an inclined extending portion 202A, and a holding plate 203A. The holding member 200A is attached to the left side of the platform when facing the input door. The tapered portion of the inclined extending portion 202A is tilted to the right side in the front view and fixed by welding at an angle θ with respect to the holding table 201A. The angle θA of the inclined stretching portion 202A with respect to the welding surface of the holding table 201A is set to, for example, 30 ° or more and 60 ° or less, for example, 45 °. The holding plate 203A is welded and fixed to the end portion opposite to the tapered portion along the longitudinal direction of the inclined extending portion 202A. The distance measuring sensor 22a is fixed to the holding plate 203A, and the holding base 201A is attached in the vicinity of the closing door, so that the ranging sensor 22a is fixed to the left side in the vicinity of the closing door.

Similarly, the holding member 200B for holding the distance measuring sensor 22b includes a holding base 201B, an inclined extending portion 202B, and a holding plate 203B. The holding member 200B is attached to the right side of the platform when facing the input door. The tapered portion of the inclined extending portion 202B is tilted to the left side in the front view and fixed by welding at an angle θ with respect to the holding table 201B. The angle θB of the inclined stretching portion 202B with respect to the welded surface of the holding table 201B is set to, for example, 30 ° or more and 60 ° or less, for example, 45 °. The holding plate 203B is welded and fixed to the end portion opposite to the tapered portion along the longitudinal direction of the inclined extending portion 202B. The distance measuring sensor 22b is fixed to the holding plate 203B, and the holding base 201B is attached in the vicinity of the closing door, so that the ranging sensor 22b is fixed to the right side in the vicinity of the closing door.

In the holding members 200A and 200B, the angles θA and θB are preferably equal in size to each other, and in the left and right holding members 200A and 200B, they are preferably inclined inward, but are not necessarily limited. Similarly, the acute angles α in the inclined stretching portions 202A and 202B are preferably equal to each other, but are not necessarily limited. A pan head or the like whose angle can be adjusted may be provided between the holding plates 203A and 203B and the distance measuring sensors 22a and 22b. By configuring the left and right holding members 200A and 200B as described above, the distance measuring sensors 22a and 22b can be installed in a suitable installation state, have good reproducibility, and can emit laser light in an appropriate direction. Can be installed in the vicinity of.

According to the above-described embodiment, the dynamic trajectories L1 and L2 can be detected for the human U and the vehicle Ve to be monitored by using the distance measuring sensors 22a and 22b and the infrared cameras 32a and 32b. Therefore, the position of the human U on the platform 100 and the positional relationship between the human U and the vehicle Ve can be monitored. Therefore, a light warning or a prediction warning can be issued based on these dynamic trajectories L1 and L2, and the fall of the human U from the platform 100 to the garbage pit 50 can be predicted. Therefore, it is possible to make the human U or the driver of the vehicle Ve recognize the possibility of the fall before the human U working on the upper part of the step such as the platform 100 falls to the lower part of the step such as the garbage pit 50. Therefore, the possibility of falling can be prevented and the safety of work at the upper part of the step can be ensured. Further, even if the human U falls to the lower part of the step, it is possible to promptly notify the surroundings, so that it is possible to quickly save lives.

Although the present embodiment has been specifically described above, the present invention is not limited to the above-described embodiment, and various modifications based on the technical idea of the present invention are possible. For example, the content of the announcement and the emission color of the display mentioned in the above-described embodiment are merely examples, and the content of the announcement and the emission color of the display may be different from this.

For example, a process for notifying a "prediction alarm" when a human U enters the first area A1 may be added between the above-mentioned steps ST6 and ST7. In this case, based on the dynamic locus L1 of the human U generated by the control unit 11 in step ST6, when the human U enters the first area A1, the voice speaker 42 says, "Please wear a safety belt." An announcement may be made, or a green light may be emitted from the display 41. As a result, it is possible to notify the human U who approaches the garbage pit 50 of the necessity of ensuring safety.

For example, for comparison with the shape data in steps ST4 and ST24 described above, whether or not it is a vehicle Ve or a human U is determined by an artificial intelligence system equipped with a learned model generated by machine learning such as deep learning. It is also possible to do it.

For example, the above-mentioned first area A1, second area A2, and third area A3 are not necessarily limited to three areas, and can be divided into two areas or four or more areas. is there. Even in these cases, the entry of the human U or the vehicle Ve into each area can be monitored.

For example, in the above-described embodiment, the number of humans U working outside the vehicle Ve on the platform 100 is described as one, but there may be a plurality of humans. In this case, each of the plurality of humans U is individually detected, and an alarm signal suitable for the dynamics of each human U is output. Further, the display 41 may simultaneously emit light of a plurality of colors indicating a plurality of types of alarms. In the voice speaker 42, announcements for notifying a plurality of types of alarms may be alternately and continuously performed.

For example, in the above-described embodiment, the alarm level is set to a three-level alarm of a predictive alarm, a light alarm, and a heavy alarm, but it may be a two-level alarm, and the alarm level may be four or more levels. It may be an alarm.

Further, in the present embodiment, the above-mentioned "part" can be read as "circuit" or the like. For example, the control unit can be read as a control circuit.

In the description of the flowchart in the present specification, the context of the processing between steps has been clarified by using expressions such as "first", "after", and "continued", but in order to implement the present embodiment. The order of processing required for is not uniquely defined by those expressions. That is, the order of processing in the flowchart described in the present specification can be changed within a consistent range.

Further effects and variations can be easily derived by those skilled in the art. The broader aspects of the present disclosure are not limited to the particular details and representative embodiments described and described above. Therefore, various modifications can be made without departing from the spirit or scope of the general concept of the invention as defined by the accompanying claims and their equivalents.

1 Monitoring system 2 Network 3 Vehicle control center 10 Monitoring device 11 Control unit 12 Storage unit 12a Vehicle shape database 12b Human shape database 13 Input unit 14 Output unit 15, 21, 31 Communication unit 20 Human detection unit 22a, 22b, 33a , 33b Distance measurement sensor 23a, 23b, 32a, 32b Infrared camera 30 Vehicle detection unit 40 Alarm output unit 41 Display 42 Voice speaker 50 Garbage pit 51 Waste 100 Platform 122 Monitoring program 141 Display unit 142 Printing unit 200, 200A, 200B Holding member 201, 201A, 201B Holding base 202, 202A, 202B Inclined extension portion 203, 203A, 203B Holding plate

Claims (11)

A monitoring device including a control unit that monitors at least one object that enters a predetermined area above the step at a step of a predetermined height.
The control unit
Entering the predetermined area based on at least one of the sensing information input from the sensor that detects the object existing in the predetermined area and the video data input from the imaging unit that images the object existing in the predetermined area. Detecting at least one moving object as a human or vehicle,
When the human enters the predetermined area, depending on which area of the plurality of areas set according to the displacement from the boundary between the upper part of the step and the lower part of the step has entered. The alarm signal corresponding to the alarm of one level of the selected alarms of a plurality of levels is output to the alarm output unit for notifying the outside.
A monitoring device that outputs a predictive warning signal for predicting a fall of the human to the lower part of the step when the relative situation between the human and the vehicle meets a predetermined condition to the warning output unit. The control unit
Based on the video data of the object entering the predetermined region and the sensing information captured by the imaging unit, the vehicle existing in the predetermined region is extracted.
The first aspect of claim 1, wherein among the detected objects included in the sensing information, a substantially rectangular parallelepiped spatial region including an existing region of the vehicle extracted based on the video data is classified as the vehicle included in the sensing information. Monitoring device. The control unit
By comparing the shape data of the vehicle obtained by pattern-recognizing a plurality of vehicles with the video data and the sensing information input from the imaging unit, the vehicle can be obtained from the video data and the sensing information. The monitoring device according to claim 2 to be extracted. The control unit
A request for determining whether or not the object included in the sensing information is the human by comparing the shape data of the person obtained by pattern-recognizing the postures of a plurality of humans with the sensing information. Item 3. The monitoring device according to any one of Items 1 to 3. The control unit
The invention according to any one of claims 1 to 4, wherein the dynamic locus of the detected object is acquired, and based on the dynamic locus, at least one object that enters and moves in the predetermined region is detected as a human being or a vehicle. Monitoring device. A plurality of the sensors are provided at a position higher than the height of the vehicle.
The monitoring device according to any one of claims 1 to 5, wherein the control unit synchronizes sensing information output from each of the plurality of sensors with each other. The monitoring device according to any one of claims 1 to 6, wherein the sensor is a distance measuring sensor that measures a distance from the sensor to an object. The upper part of the step is a platform in a waste treatment facility, the lower part of the step is a garbage pit in the waste treatment facility, and the vehicle is a dust collecting vehicle. Monitoring device. A monitoring system that monitors at least one object entering a predetermined area above the step at a step of a predetermined height.
A sensor that detects an object existing in the predetermined area,
An imaging unit that images an object existing in the predetermined region,
An alarm output unit that notifies the alarm to the outside and
Based on at least one of the sensing information input from the sensor and the video data input from the imaging unit, the at least one object that enters and moves in the predetermined area is classified into a human or a vehicle, and the human is classified as a human or a vehicle. When entering the predetermined region, it was selected according to which region of the plurality of regions set according to the displacement from the boundary between the upper portion of the step and the lower portion of the step was entered. , An alarm signal corresponding to one level of alarm among a plurality of levels of alarm is output to the alarm output unit, and when the relative situation between the person and the vehicle meets a predetermined condition, the alarm A monitoring system including a monitoring device having a control unit that outputs a prediction warning signal for predicting a fall of the human to the lower part of the step in the output unit. The control unit with hardware
Sensing information is acquired from a sensor that detects an object existing in a predetermined area above the step at a step of a predetermined height.
Video data is acquired from an imaging unit that captures an object existing in the predetermined region, and the image data is acquired.
Based on the acquired sensing information and at least one of the video data, at least one object that enters and moves in the predetermined area is classified as a human or a vehicle.
When the human enters the predetermined area, depending on which area of the plurality of areas set according to the displacement from the boundary between the upper part of the step and the lower part of the step has entered. An alarm signal corresponding to one level of the selected alarms of a plurality of levels is transmitted to an alarm output unit that notifies the outside, and the alarm of the one level is notified.
When the relative situation between the human and the vehicle meets a predetermined condition, a predictive warning signal for predicting the fall of the human to the lower part of the step is transmitted to the warning output unit, and the predictive warning signal is transmitted. A monitoring method that notifies the prediction alarm corresponding to. A control unit that monitors at least one object that enters a predetermined area above the step at a step of a predetermined height.
Sensing information is acquired from a sensor that detects an object existing in the predetermined area, and
Video data is acquired from an imaging unit that captures an object existing in the predetermined region, and the image data is acquired.
Based on the acquired sensing information, the at least one object that enters and moves in the predetermined area is classified as a human or a vehicle.
When the human enters the predetermined area, depending on which area of the plurality of areas set according to the displacement from the boundary between the upper part of the step and the lower part of the step has entered. The alarm signal corresponding to the alarm of one level of the selected alarms of a plurality of levels is transmitted to the alarm output unit that notifies the outside.
A program for transmitting a predictive warning signal for predicting a fall of the human to the lower part of the step to the warning output unit when the relative situation between the human and the vehicle meets a predetermined condition.

Images