JP2022016197A - Monitoring system - Google Patents

Abstract

[Problem] To detect entry of workers into restricted areas with cost consideration and high accuracy. [Solution] A monitoring system includes a camera attached to a crane that photographs the ground below a suspended load, a forming unit that forms a restricted area on the ground photographed by the camera, a detection unit that detects whether a worker has entered the restricted area, and an alarm unit that issues an alarm when the detection unit detects that a worker has entered the restricted area. [Selected Figure] Figure 1

Description

The present invention relates to a monitoring system.

At construction sites, there is a danger that suspended loads will fall, and it is required to prevent accidents caused by falling suspended loads. As a measure to prevent accidents, the so-called slinging 333 movement to maintain the stability of the suspended load during slinging is also carried out, but this alone is not a sufficiently effective measure. There is a need for other means of keeping out under the suspended load.

As a method, for example, there is a technique related to a crane suspension monitoring system that can realize safety measures (see Patent Document 1). In this technology, it is determined by image analysis whether or not a person is present in the dangerous area, and when it is determined that a person is present in the dangerous area around the suspended load, an alarm signal or an emergency stop signal is output. ing.

In addition, there is a work support device for supporting the work of moving a suspended load more safely, and a technique for a work support method (see Patent Document 2). In this technique, based on the image below the boom tip in the boom of the lifting device, a predetermined range is determined based on the lifting emphasis according to the detected behavior of the worker, and the image below the fulcrum is displayed. Based on this, personnel within a predetermined range are detected and a warning is issued. Further, as a similar technique, a technique for determining the range for determining the safety of a worker at the stage until the suspended load is lifted to a predetermined height at which the safety confirmation of the lifted state of the suspended load is carried out. (See Patent Document 3).

In addition, there is a technique related to a crane operation support device that enables an operator to objectively grasp the state of a crane and its suspended load (see Patent Document 4). In this technology, the movable part model is placed in a virtual three-dimensional space based on the measurement result of the posture, and the hook block model is created based on the measurement result of the measurement result including the height to measure the height of the hook block. It is arranged in a three-dimensional space. Further, the suspended load model is arranged in a three-dimensional space based on the measurement result, and the hook block model and the suspended load model, respectively, of the movable part model are displayed by drawing processing.

Japanese Unexamined Patent Publication No. 2020-7134 Japanese Unexamined Patent Publication No. 2018-80053 Japanese Unexamined Patent Publication No. 2018-800054 JP-A-2019-59593

In the techniques of Patent Documents 1 to 3, the worker is detected by the image of the suspended load and some control is performed. The technique of Patent Document 4 makes it easy to grasp the control of the crane. Here, it is assumed that when the lifting work is performed, it is necessary to keep the range where the worker is dangerous to enter within a certain range regardless of the work process. How to define the dangerous area in Patent Document 1 is not specifically defined. Further, in Patent Documents 2 and 3, it is stipulated that the dangerous range changes due to a change in the height of the camera, assuming the influence of a fall based on the lifting priority. The range of is not considered. In lifting work using a crane, the range of the ground imaged by the camera differs depending on the height change. In addition, if the range of the ground in the image taken by the camera is different, it is necessary to consider some consideration about the dangerous range set on the ground. If the area on the ground is different, the size in which the worker is reflected will also be different, but if the size of the detection target changes in this way, it will not be possible to immediately detect the worker's entry into the exclusion zone. Is also a concern. Therefore, it is possible to flexibly respond to various changes that occur in the lifting work using such a crane, and it is possible to reliably inform the workers who are concentrating on the work that they are approaching the suspended load. A monitoring system that can be used is required. In addition, we would like to consider improving the accuracy of the monitoring system and reducing the cost of introduction.

It is an object of the present invention to detect the entry of a worker into an exclusion zone in consideration of cost and with high accuracy in consideration of the above facts.

In order to achieve the above object, the monitoring system of the present invention includes a camera mounted on a crane and photographing the ground under a suspended load, and a forming portion forming an exclusion zone with respect to the ground photographed by the camera. It has a detection unit that detects whether or not a worker has entered the restricted area, and a notification unit that notifies an alarm when the detection unit detects that the worker has entered the restricted area. This makes it possible to detect the entry of workers into the restricted area with consideration for cost and with high accuracy.

Further, in the monitoring system of the present invention, the exclusion zone is circular, and the radius can be changed according to the size of the suspended load. As a result, the size of the suspended load, that is, the size of the off-limits area that requires entry detection can be determined even depending on the danger, so that various suspended loads can be appropriately monitored.

Further, in the surveillance system of the present invention, the camera has a first camera having a wide-angle lens and a second camera having a standard lens having a standard angle of view than the wide-angle lens, and the forming portion comprises. The size of the off-limits area can be kept within a predetermined range by switching between the camera image of the first camera and the camera image of the second camera according to the height of the camera. As a result, the cost can be reduced as compared with the case where a single camera capable of changing the angle of view is provided. Further, by switching the cameras, taking camera images having different angles of view, and switching the camera images, the size of the restricted area can be kept constant to some extent.

Further, in the monitoring system of the present invention, a height detector attached to the crane and detecting the height of the camera is further included, the height detector detects the height of the camera, and the forming portion is formed. By controlling the switching according to the detected height of the camera, the size of the off-limits area can be kept within a predetermined range. As a result, switching of the camera image can be controlled according to the height detected by the height detector.

Further, in the monitoring system of the present invention, the detection unit can perform the detection using a recognition model learned using learning data that defines conditions regarding the movement of the worker and the ground. As a result, the entry of workers can be detected using a recognition model learned under conditions that match the actual environment, so it is possible to robustly detect changes in the environment and detect entry. Expected to improve accuracy.

According to the present invention, it is possible to obtain the effect of detecting the entry of a worker into an exclusion zone with consideration for cost and with high accuracy.

It is a block diagram which shows the structure of a monitoring system. It is a schematic diagram which shows an example of the operation scene of the monitoring system of embodiment of this invention. It is a figure which shows an example of the shooting range by the difference of the angle of view of a camera. It is a schematic diagram which shows an example of the case where the formed exclusion zone is displayed in the camera image which photographed the suspended load from directly above. It is a perspective view which shows an example of a monitoring unit. It is a perspective view which shows an example of a notification part. It is a sequence diagram which shows the monitoring process in the monitoring system which concerns on embodiment of this invention. It is a figure which shows an example of the entry detection.

[Embodiment of the present invention]
Hereinafter, the monitoring system according to the embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the monitoring system 100. As shown in FIG. 1, in the monitoring system 100, the monitoring unit 110 and the notification unit 130 are connected to each other via the network N. In the present embodiment, the network N uses WiFi built in the monitoring unit 110 and the notification unit 130, respectively.

FIG. 2 is a schematic view showing an example of an operating scene of the monitoring system 100 according to the embodiment of the present invention. As shown in FIG. 2, a monitoring unit 110 is attached near the hook 32 of the crane 30, and the monitoring unit 110 is referred to as a camera 112 (note that when referring to a camera in general in the following description, it is also simply referred to as a camera. ) Is built-in. Here, the attachment point of the monitoring unit 110 near the hook 32 is, for example, a hook block. Further, the mounting direction is such that the lens of the camera 112 built in the monitoring unit 110 faces directly downward. As a result, the camera 112 photographs the ground overlooking the suspended load 34. Also, if the ground can be photographed together with the suspended load, the mounting location is not limited to the hook block. The monitoring unit 110 forms an exclusion zone 40 with respect to the ground imaged by the camera 112. When the monitoring unit 110 detects that the worker has entered the restricted area 40, the monitoring unit 110 notifies the notification unit 130, and the notification unit 130 notifies the worker of the danger. The notification unit 130 is arranged at a place where a worker near the crane 30 can see, such as the loading yard side of the crane 30 in FIG. 2. As a result, the notification unit 130 can notify the worker of the danger by notifying the alarm sound, and can also notify the worker of the danger by lighting the lamp.

Next, an example of the configuration of the monitoring unit 110 and the notification unit 130 will be described.

Each configuration of the monitoring unit 110 will be described. As shown in FIG. 1, the monitoring unit 110 includes a camera 112, a barometric pressure sensor 114, and a computer 120. The camera 112 includes a first camera 112A and a second camera 112B. The computer 120 includes a forming unit 122, a detecting unit 124, and a storage unit 126. The computer 120 can be composed of a computer including a CPU, a RAM, a ROM for storing a program for executing each processing unit, and various data (not shown).

It is assumed that the camera 112 is composed of a first camera 112A having a wide-angle lens and a second camera 112B having a standard lens having a standard angle of view than the wide-angle lens. By using the two types of cameras having different angles of view as described above, the size of the exclusion zone formed by the forming portion 122 can be kept within a predetermined range in the camera image to be captured. If the angle of view of the camera is constant, the height of the camera changes during the lifting operation, and the dimensions of the object on the ground in the camera image change significantly. FIG. 3 is a diagram showing an example of a shooting range due to a difference in the angle of view of the camera. As shown in FIG. 3, if the angles of view of the first camera 112A and the second camera 112B are different, a certain range can be captured when the height of the crane 30 changes, and the camera image can be taken. It is possible to keep the above-ground dimensions constant. In addition, by using two or more types of cameras with different angles of view, it is possible to reduce costs without using expensive cameras such as cameras with functions such as zoom function and cameras capable of 3D measurement. Moreover, there is an advantage that the monitoring system 100 with high accuracy can be configured.

As the atmospheric pressure sensor 114, any sensor that can detect a general atmospheric pressure may be used. By detecting the barometric pressure at the start and during the lifting work by the barometric pressure sensor 114, the computer 120 can calculate the height from the detected barometric pressure difference. The barometric pressure sensor 114 is an example of a height detector, and other means capable of detecting the height may be used. For example, an image analysis in a camera image, a laser radar, an external camera that captures lifting work, or the like may be used as the height detector.

The forming unit 122 forms an exclusion zone 40 with respect to the ground imaged by the camera 112. For the exclusion zone 40, for example, a circle having an arbitrary size may be defined according to the assumed size of the suspended load such as a radius of 3 m (diameter 6 m). Further, when the size of the suspended load 34 is predetermined according to the size of the suspended load 34, the size of the suspended load 34 may be calculated from a camera image or the like at the start of the lifting work to determine the size. FIG. 4 is a schematic view showing an example of a case where the formed exclusion zone 40 is displayed in a camera image of the suspended load 34 taken from directly above.

Further, the forming portion 122 uses the camera image of the wide-angle first camera 112A when the height of the crane 30 is lower than the predetermined height, and when the height of the crane 30 reaches the predetermined height. Switch to use the camera image of the second camera 112B with a standard angle of view. The predetermined height referred to here is an arbitrary height that satisfies the range condition for the shooting range on the ground. For example, suppose that a range condition is set for a camera image so that the shooting range should be within 20 m in diameter. This range condition may be a range in which detection by the recognition model can be performed with high accuracy. In this case, for example, the shooting range of the camera image for each height of the first camera 112A is obtained in advance by measurement or the like, and the height within the shooting range of 20 m in diameter is set as a predetermined height. Thereby, the height satisfying the range condition can be determined as a predetermined height. Further, the predetermined height may be adjusted to a height that can be appropriately switched based on the shooting range for each height according to the angle of view of both the first camera 112A and the second camera 112B. For example, it is assumed that the shooting range is set within a diameter of 10 m or more and 20 m or less under the range condition. In this case, if the shooting range of the first camera 112A is the height at the time of the diameter of 18 m and the shooting range of the second camera 112B is the diameter of 10 m, the height is set as a predetermined height. May be good. The height is obtained from the barometric pressure detection result of the barometric pressure sensor 114. The forming unit 122 controls the switching of the camera image by setting a predetermined height so that the restricted area 40 in the camera image is kept within a predetermined range and the size is kept constant. In this way, the forming unit 122 controls switching between the first camera 112A and the second camera 112B according to the height of the detected camera, keeps the exclusion zone 40 in the camera image within a predetermined range, and has a large size. Keep the camera constant. As a result, although the height of the camera changes at the stage of lifting work, the restricted area 40 can be kept within a predetermined range regardless of the height of the camera. The advantage of forming the restricted area 40 so as to be within a predetermined range is that if the range is maintained, the detection unit 124 can accurately detect the entry of workers. The switching process may be processed by the detection unit 124.

The detection unit 124 detects whether or not a worker has entered the restricted area 40 by using the recognition model of the storage unit 126. The detection unit 124 inputs the camera image to the recognition model and obtains the detection result of the worker in the camera image as the output of the recognition model. By collating the detection result of the worker in the camera image with the restricted area 40, it is detected whether or not the worker has entered the restricted area 40. The collation between the camera image and the restricted area 40 may be calculated based on the height calculated from the atmospheric pressure and the angle of view of the camera 112 (first camera 112A or second camera 112B) that captured the camera image. When the detection unit 124 detects that the worker has entered, the detection unit 124 transmits an entry detection signal to the notification unit 130. Further, the detection unit 124 transmits a normal signal to the notification unit 130 when the entry is not detected.

The storage unit 126 stores a pre-learned recognition model for detecting the presence of a worker in the camera image. As the learning method, an image recognition method by DNN (deep neural network) is used. As the recognition model, as the learning data of DNN, a large amount of learning camera images collected from hundreds to tens of thousands of images of the worker working from directly above may be used. In addition, the learning data defines the conditions for the movement of workers and the conditions for the ground. The conditions related to the movement of the worker are, for example, detected by using an image of about 5 to 1 frame per second so as to capture the change in the state according to the movement speed of the worker as a continuously shot learning camera image. I tried to do. In addition, as a condition related to the ground, a learning camera image taken with a plurality of types of ground is used. This is because the recognition rate is high if the ground condition is on black asphalt, but if you learn only with the learning camera image of the ground with such a high recognition rate, the ground where various colors are mixed (for example, concrete) Or in the case of a laid iron plate), the recognition rate may decrease. Therefore, a recognition model corresponding to such various ground conditions is trained. In addition, since the degree of reflection of light on the ground is related not only to the ground but also to the weather conditions, learning camera images taken under various weather conditions are also used.

Each configuration of the notification unit 130 will be described. As shown in FIG. 1, the notification unit 130 includes a first lighting lamp 132, a second lighting lamp 134, an alarm buzzer 136, and a notification control unit 138.

The first lighting lamp 132 and the second lighting lamp 134 are rotation warning lamps that rotate and light in different colors. In the present embodiment, for example, the first lighting lamp 132 is set to light when an entry is detected and to light when another data communication abnormality occurs. The alarm buzzer 136 is a buzzer that notifies a predetermined alarm sound.

When the notification control unit 138 receives the on-site detection signal from the monitoring unit 110, the first lighting lamp 132 is rotated and turned on, and the notification control unit 138 is controlled to notify the alarm sound by the alarm buzzer 136. Further, for example, when the on-site detection signal is received for a long time, both the first lighting lamp 132 and the second lighting lamp 134 may be rotated and turned on to increase the volume of the alarm sound. ..

Next, an example of the actual size configuration of the monitoring unit 110 and the notification unit 130 will be described with reference to the drawings. FIG. 5 is a perspective view showing an example of the monitoring unit 110. As shown in FIG. 5, the monitoring unit 110 can be configured as a rectangular box-type device. A camera 112 (first camera 112A and second camera 112B) is arranged at the bottom of the box of the monitoring unit 110, and the lens of the camera 112 is exposed so as to enable shooting on the ground. There is. In addition, a barometric pressure sensor 114 and a computer 120 are arranged on the opposite side of the lid in the box, and a battery 115 necessary for operating the computer 120 and a WiFi antenna 116 necessary for communication with the notification unit 130 are arranged respectively. There is. It should be noted that the box-shaped shape of the monitoring unit 110 is merely an example, and other shapes may be used. For example, the camera 112, the computer 120, and the like may be of a flat type arranged on the same surface, and may be appropriately attached in the vicinity of the hook 32 of the crane 30.

FIG. 6 is a perspective view showing an example of the notification unit 130. As shown in FIG. 6, the notification unit 130 has a first lighting lamp 132, a second lighting lamp 134, and an alarm buzzer 136 installed on the upper surface as a notification device, and has a notification control unit 138 built-in. The first lighting lamp 132 and the second lighting lamp 134 are provided with semi-transparent gloves arranged so as to have different colors.

Next, the operation of the monitoring system 100 according to the embodiment of the present invention will be described. FIG. 7 is a sequence diagram showing a monitoring process in the monitoring system 100 according to the embodiment of the present invention. The computer 120 of the monitoring unit 110 performs various processes by the CPU reading a program and various data from the ROM and executing the program. The CPU functions as each part of the computer 120. The following monitoring process may be started, for example, by receiving a work start signal for lifting work from an external computer. It is assumed that the barometric pressure sensor 114 has detected the barometric pressure from the start time.

In step S100, the forming unit 122 of the monitoring unit 110 forms an exclusion zone 40 at the start of work. When forming the exclusion zone 40 based on the size of the suspended load 34 calculated from the camera image, step S100 and step S102 may be interchanged. Further, when the size of the restricted area 40 is predetermined regardless of the size of the suspended load 34, the restricted area 40 may be formed in advance.

In step S102, the camera 112 starts shooting on the ground with the start of work and generates a camera image. At the start of the work, the camera image of the first camera 112A is set to be used, but both the first camera 112A and the second camera 112B are photographed so that the camera image can be switched immediately. Generate an image.

In step S104, the forming unit 122 detects the height from the detection result of the atmospheric pressure of the atmospheric pressure sensor 114, and determines whether or not the height is equal to or higher than a predetermined height. If it is determined that the height is equal to or higher than the predetermined height, the process proceeds to step S106, and if it is determined that the height is not higher than the predetermined height, the process proceeds to step S108.

In step S106, the forming unit 122 switches the camera image. In the subsequent steps, processing is performed using the switched camera images. Here, the camera image captured by the first camera 112A is switched to the camera image captured by the second camera 112B.

In step S108, the detection unit 124 detects whether or not a worker has entered the restricted area 40 by using the recognition model of the storage unit 126. If it is determined that the person has entered, the process proceeds to step S110, and if it is not determined that the person has entered, the process proceeds to step S112.
Here, in the detection of the worker's entry by the detection unit 124, the camera image is input to the recognition model, and the detection result of the worker in the camera image is obtained as the output of the recognition model. Then, by collating the detection result of the worker in the camera image with the restricted area 40, it is detected whether or not the worker has entered the restricted area 40.

In step S110, the detection unit 124 transmits the on-site detection signal to the notification unit 130.

In step S112, the detection unit 124 transmits a normal signal to the notification unit 130.

Further, the processing of the monitoring unit 110 shifts to step S122 after the processing of transmission in step S110 or S112.

In step S114, the notification control unit 138 of the notification unit 130 determines whether or not the on-site detection signal has been received. If the entry detection signal is received, the process proceeds to step S116, and if the entry detection signal is not received, the process proceeds to step S118.

In step S116, the notification control unit 138 rotates and lights the first lighting lamp 132, and controls to notify the alarm sound by the alarm buzzer 136. By such control, an alarm to the worker is notified while the on-site detection signal is received from the detection unit 124 of the monitoring unit 110.

In step S118, it is determined whether or not any signal (in this embodiment, an on-site detection signal or a normal signal) has been received within a certain period of time. If it is accepted, the process is terminated, and if it is not accepted, the process proceeds to step S120.

In step S120, the notification control unit 138 rotates and lights the second lighting lamp 134, and controls so as to notify that an abnormality has occurred in the data communication.

In step S122, the detection unit 124 of the monitoring unit 110 determines whether or not the work has been completed. If it is finished, the monitoring process is finished, and if it is not finished, the process returns to step S104 and the process is repeated. The work end determination in the process may be determined by receiving a work end signal or the like from an external computer.

Here, the on-site detection of the detection unit 124 will be described with reference to the drawings. FIG. 8 is a diagram showing an example of on-site detection. As shown on the left side of FIG. 8, when the worker is outside the boundary of the restricted area 40, the detection unit 124 transmits a normal signal as if he / she has not entered the restricted area 40, so that the alarm is not notified. .. As shown on the right side of FIG. 8, when a worker touches or enters the boundary of the exclusion zone 40, the detection unit 124 transmits an entry detection signal as if the worker has entered the exclusion zone 40. Will notify the worker of an alarm.

As described above, according to the monitoring system 100 according to the embodiment of the present invention, it is possible to detect the entry of workers into the restricted area with consideration for cost and with high accuracy.

In addition, it is particularly effective in accident prevention by immediately notifying the entry detection when the worker cannot directly see the suspended load, such as when there is an obstacle between the suspended load and the crane operation room.

The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

For example, in the above-described embodiment, the mode in which the first camera 112A has a wide angle and the second camera 112B has a standard angle of view has been described, but the present invention is not limited thereto. For example, a camera having the same angle of view as the first camera 112A and the second camera 112B may be used, and the viewing angle of the camera image may be widened by the arrangement in the monitoring unit 110. In this case, it is not necessary to switch the camera images, but on-site detection is always performed using both camera images. As a result, even if a worker enters a blind spot such as a suspended load 34 in one camera image, entry can be detected.

30 Crane 32 Hook 34 Suspended load 40 Off-limits area 100 Monitoring system 110 Monitoring unit 112 Camera 112A First camera 112B Second camera 114 Atmospheric pressure sensor 115 Battery 116 Antenna 120 Computer 122 Forming unit 124 Detection unit 126 Storage unit 130 Notification unit 132 1 lighting lamp 134 2nd lighting lamp 136 alarm buzzer 138 notification control unit

Claims (5)

A camera attached to a crane that captures the ground under a suspended load,
A forming portion that forms an exclusion zone with respect to the ground photographed by the camera, and a forming portion.
A detector that detects whether or not a worker has entered the restricted area,
A notification unit that notifies an alarm when it is detected by the detection unit that a worker has entered the restricted area.
Monitoring system with. The monitoring system according to claim 1, wherein the exclusion zone is circular and the radius is changed according to the size of the suspended load. The camera has a first camera having a wide-angle lens and a second camera having a standard lens having a standard angle of view than the wide-angle lens.
The forming portion switches between the camera image of the first camera and the camera image of the second camera according to the height of the camera, and the size of the off-limits area is kept within a predetermined range according to claim 1 or The monitoring system according to claim 2. Further includes a height detector attached to the crane to detect the height of the camera.
The height of the camera is detected by the height detector,
The monitoring system according to claim 3, wherein the forming unit controls the switching according to the height of the detected camera to keep the size of the restricted area within a predetermined range. The monitoring according to any one of claims 1 to 4, wherein the detection unit performs the detection using a recognition model learned using learning data that defines conditions for the movement of the worker and the ground. system.

Images