JP6940906B1 - Peripheral monitoring device for work machines - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a peripheral monitoring device which can be easily installed on a work machine at low cost and can suppress contact between the work machine and a person around the work machine. A peripheral monitoring device 100 for a work machine according to the present disclosure is a predetermined process based on a photographing device 200 including one or more monocular cameras 210 and photographed image data representing a photographed peripheral image of the working machine 10. A processor 303 for executing the above, and an alarm device 400 for outputting a predetermined alarm signal and / and a predetermined alarm. Then, the processor 303 extracts a person image from the photographed image data, and measures the distance between the work machine 10 and the person around it based on the person image and the installation parameters of the photographing device 200. Then, based on the measured distance, the output of the alarm signal and / and the alarm by the alarm device 400 is controlled. [Selection diagram] Fig. 1

Description

The present invention relates to a peripheral monitoring device for a work machine.

At sites where work machines such as excavators and forklifts are used, contact accidents between the work machines and people around them are prevented. For example, a work machine equipped with a mirror or a camera has been put into practical use so that an operator can visually recognize an obstacle existing around the vehicle body of the work machine. Then, there is known a system that displays an image taken by a camera on a display in the driver's cab of a work machine.

Further, as a technique for detecting an obstacle existing around a work machine, it is known to detect an obstacle by using a distance sensor such as a laser radar or a millimeter wave radar. For example, Patent Document 1 discloses a peripheral monitoring device for a moving body that can detect a distance in the entire circumferential direction with respect to the moving direction axis of the moving body. The peripheral monitoring device is provided with a 3D camera having a plurality of imaging units and the like arranged at equal angles around the support axis as a distance sensor.

On the other hand, Patent Document 2 discloses a technique of arranging a plurality of stereo cameras on an upper swing body in a shovel having a lower traveling body and an upper swing body. In this technology, a stereo camera acquires distance information of an object existing around the excavator.

Japanese Unexamined Patent Publication No. 2016-180674 Japanese Unexamined Patent Publication No. 2020-127217

A distance sensor such as a laser radar or a millimeter wave radar irradiates a laser beam or a radio wave in a predetermined direction, and detects the distance from the reflected wave to an object irradiated with the laser beam or the like. Therefore, when trying to detect an object around a work machine such as a shovel or a forklift using such a distance sensor, it is necessary to arrange the distance sensor in a plurality of directions to detect an object around the work machine. The cost for this will be high. Here, according to the technique described in Patent Document 1, the distance can be detected in the all-around direction around the support axis by the 3D camera, but this is only the all-around direction with respect to the support axis. Therefore, even in this case, when trying to detect an object around the work machine, it is necessary to arrange the support shafts in a plurality of directions, and the cost for detecting the object around the work machine becomes high.

Further, according to the technique described in Patent Document 2, although the distance information of the objects existing around the excavator can be acquired based on the image taken by the camera, the camera installed in the excavator is a stereo camera. Therefore, the cost and space for installation can be an issue. As described above, there is still room for improvement in the technique for detecting the surrounding objects of the work machine.

An object of the present disclosure is to provide a peripheral monitoring device that can be easily installed on a work machine at low cost and can suppress contact between the work machine and a person around it.

The peripheral monitoring device of the work machine of the present disclosure is installed in the work machine and acquires a photographing device for photographing the surroundings of the work machine and photographed image data representing the surrounding image of the work machine photographed by the photographing device. A processor that executes a predetermined process based on the captured image data, and an alarm device that outputs a predetermined alarm signal and / or a predetermined alarm are provided. Here, the photographing device includes one or more monocular cameras, and is configured such that the optical axis of the monocular cameras is arranged at a predetermined angle with respect to the ground. Then, the processor extracts the person image from the captured image data, and based on the extracted person image and the installation parameters of the photographing device, the working machine and the surrounding person The alarm signal and / and the output of the alarm are controlled by the alarm device based on the distance measurement process for measuring the distance and the distance between the work machine and a person around it measured by the distance measurement process. The alert control process to be executed and the execution are performed.

Here, the above-mentioned work machine is a construction machine such as an excavator or a machine equipped with a work machine (forklift or the like). Then, in the peripheral monitoring device of the present disclosure installed on such a work machine, a photographing device is arranged on the work machine. This photographing device is arranged so that the optical axis of the monocular camera forms a predetermined angle with respect to the ground. Here, the above-mentioned predetermined angle is defined as an angle at which a person near the ground can be photographed by the photographing device, and is, for example, an angle substantially parallel to the ground or an angle facing downward with respect to a parallel line with the ground. The peripheral monitoring device for the work machine of the present disclosure is realized by a simple configuration such as a photographing device, an image processing device, and an alarm device. Can be easily installed. Further, since the photographing device is composed of a monocular camera, the cost for monitoring the periphery of the work machine can be suppressed as compared with the case where the photographing device is composed of a stereo camera, for example. Further, in such a peripheral monitoring device, distance measurement processing is executed based on the person image extracted by the extraction processing and the installation parameters of the photographing device, and the alarm device is issued based on the result of the distance measurement processing. The output of the alarm signal and / and the alarm is controlled. The alarm output by the alarm device is, for example, a warning light or a warning sound. As a result, contact between the work machine and a person around it can be suppressed. In the alarm control process, when the distance between the work machine and a person around it becomes equal to or less than a predetermined value, the processor causes the work machine and the people around it to move from the alarm device. A signal for instructing a predetermined avoidance operation for avoiding contact may be output. Here, the above-mentioned avoidance operation is an operation in which the operation of the work machine is decelerated, an operation in which the operation of the work machine is stopped, and the like. In such an alarm control process, the signal from the alarm device is input to the control device that controls the operation of the work machine, so that the work machine automatically performs an avoidance operation regardless of the operator's operation. It will be. According to this, it is possible to perform the avoidance operation of the work machine without the operation operation by the operator, and it is possible to more preferably avoid the contact between the work machine and the person around it.

Then, in the peripheral monitoring device of the work machine, in the distance measurement process, the processor acquires the foot coordinates, which are the two-dimensional coordinates of the foot of the person image in the captured image data, and is based on the foot coordinates. Then, a foot angle, which is a predetermined angle between the feet of a person around the work machine and the photographing device, is calculated, and based on the calculated foot angle and the installation height of the photographing device, The distance between the work machine and a person around it may be measured. According to this, it is possible to perform distance measurement processing with high accuracy at low cost by using a monocular camera. According to the distance measurement based on the foot coordinates, the foot coordinates can be obtained relatively accurately, so that the measurement accuracy of the distance can be made as high as possible. Further, in the distance measuring process, the processor acquires an image of the head of the person image in the captured image data, and based on the size of the image of the head, the working machine and the person around it The distance may be measured. Even with this, the monocular camera can perform distance measurement processing with high accuracy at low cost. According to the distance measurement based on the size of the head image, the photographed image data does not include the feet of surrounding persons, or the person is climbing to a high place. Even when not on the ground, the distance between the work machine and the people around it can be measured.

Further, in the peripheral monitoring device of the work machine, in the extraction process, the processor has an input layer that accepts input of predetermined image data and an intermediate layer that extracts a feature amount representing a person from the input image data. The pre-learning model, which is a neural network model having an output layer that outputs an identification result based on the feature amount and is constructed by performing training using data including an image representing a person, is described above. The person image may be extracted by inputting the captured image data. According to this, by using the pre-learning model, it is possible to accurately extract a person image from the captured image data.

In the peripheral monitoring device of the work machine described above, the processor generates a captured image histogram showing the frequency of pixel values included in the captured image data, and is based on the smoothed histogram obtained by smoothing the captured image histogram. An image sharpening process that sharpens the captured image data by synthesizing the acquired smoothed image data and the captured image data at a predetermined ratio may be further executed. According to this, it is possible to extract a person image based on the photographed image data which is sharpened according to the environment. Therefore, the accuracy of extracting a person image from the captured image data can be made as high as possible.

According to the present disclosure, it is possible to provide a peripheral monitoring device that can be easily installed on a work machine at low cost and can suppress contact between the work machine and a person around it.

It is a figure which shows the schematic structure of the peripheral monitoring device of the work machine in 1st Embodiment. It is a flowchart which shows the processing flow which the control part performs in order to avoid the contact between a work machine and a person around it in the peripheral monitoring apparatus which concerns on 1st Embodiment. It is a figure for demonstrating the identification result obtained from the input to the pre-learning model in 1st Embodiment, and the neural network which constitutes the pre-learning model. It is a figure for demonstrating photographed image data and a person image displayed on a display device. It is a figure for demonstrating the relationship between the foot angle, the installation height of the photographing apparatus, and the distance with the surrounding person. It is a flowchart which shows the processing flow performed by the distance measurement processing unit in the foot distance measurement which is one of the distance measurement processing which concerns on 1st Embodiment. It is a figure for demonstrating the subject projected on the sensor of the camera together with the internal parameters of a camera. It is a 1st flowchart which shows the processing flow which the control part performs in order to avoid the contact between a work machine and a person around it in the peripheral monitoring apparatus which concerns on modification 2 of 1st Embodiment. 2 is a second flowchart showing a processing flow performed by a control unit in order to avoid contact between a work machine and a person around the work machine in the peripheral monitoring device according to the second modification of the first embodiment. It is a flowchart which shows the process flow which a control part performs in the image sharpening process which concerns on 2nd Embodiment. It is a figure which shows the modification of the schematic structure of the peripheral monitoring device of a work machine in this disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The configurations of the following embodiments are exemplary, and the present disclosure is not limited to the configurations of the embodiments.

<First Embodiment>
The outline of the peripheral monitoring device for the work machine according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a schematic configuration of a peripheral monitoring device for a work machine according to the present embodiment. The peripheral monitoring device 100 for a work machine according to the present embodiment is a device installed on the work machine 10 for monitoring the periphery of the work machine 10. Here, the work machine 10 is a construction machine such as an excavator or a machine equipped with a work machine (forklift or the like). The peripheral monitoring device 100 includes a photographing device 200, an image processing device 300, and a reporting device 400.

The photographing device 200 is a device installed on the work machine 10 and photographs the surroundings of the work machine 10, and has a function of accepting an input of an image such as a still image or a moving image. Specifically, the charged-coupled devices It is realized by a camera using an image sensor such as (CCD), Metal-oxide-semiconductor (MOS) or Complementary Metal-Oxide-Semiconductor (CMOS). The photographing apparatus 200 of the present disclosure includes one or more monocular cameras 210, and is configured such that the optical axis of the monocular camera 210 is arranged at a predetermined angle with respect to the ground. Here, in the present embodiment, as shown in FIG. 1A, the monocular camera 210 is arranged on the upper surface of the upper swing body 11 of the work machine 10, and the optical axis L2 of the monocular camera 210 is with respect to the ground. It is directed downward at a predetermined acute angle with respect to the parallel line L1. The acute angle is defined as an angle at which a person near the ground can be photographed by the photographing device 200. For example, as shown in FIG. 1A, the monocular camera 210 is arranged on the upper surface of the upper swivel body 11. In the case, it is determined based on the height of the upper surface of the upper swing body 11. Further, if the monocular camera 210 is arranged in the lower region of the upper swivel body 11 (the upper swivel body near the lower traveling body 12), the optical axis L2 of the monocular camera 210 is directed substantially parallel to the ground. May be good. Then, in the present embodiment, as shown in FIG. 1B, four monocular cameras 210 are arranged so as to face four directions (four directions at 90 ° intervals) from the side surface of the upper swing body 11. .. By arranging the four monocular cameras 210 in this way, it is possible to take a picture of the periphery of the work machine 10 without losing the field of view. It should be noted that there is no intention of limiting the photographing apparatus of the present disclosure to a configuration including four monocular cameras. For example, since the direction in which the operator of the work machine 10 faces the front in the driver's cab is a direction in which the operator can directly see, a monocular camera directed in this direction can be omitted.

The image processing device 300 acquires photographed image data representing the surrounding image of the work machine 10 photographed by the photographing device 200, and executes a predetermined process based on the photographed image data.

Here, the image processing device 300 may be any electronic device as long as it has processing capacity for arithmetic processing such as data acquisition, generation, and update, and processing processing, and is, for example, a computer. That is, the image processing device 300 can be configured as a computer having a processor such as a CPU or GPU, a main storage device such as RAM or ROM, and an auxiliary storage device such as EPROM, a hard disk drive, or removable media. The removable media may be, for example, a USB memory or a disc recording medium such as a CD or DVD. The auxiliary storage device stores an operating system (OS), various programs, various tables, and the like.

The alarm device 400 is a device that outputs a predetermined alarm signal and / and a predetermined alarm. Here, the predetermined alarm is a warning light, a warning sound, or the like. In the present embodiment, as shown in FIGS. 1 (a) and 1 (b), as the alarm device 400, for example, a patrol light (registered trademark) may be arranged on the upper surface of the upper swing body 11. Then, the peripheral monitoring device 100 can give a warning to a person around the work machine 10 by, for example, a warning light emitted by Patrite (registered trademark). In addition, the operator of the work machine 10 can be made aware of the approach of a person to the work machine 10. The alarm device 400 may include, for example, a patrol light (registered trademark) and a speaker. In this case, the peripheral monitoring device 100 can give a warning by the warning light and the warning sound.

Further, the peripheral monitoring device 100 may further include a display device 500. Here, the display device 500 is a device configured to be able to display the captured image data taken by the photographing device 200 and / or predetermined data acquired by processing based on the captured image data. Such a display device 500 is provided, for example, in the cab of the work machine 10. As a result, the operator of the work machine 10 can visually recognize the obstacles existing around the vehicle body of the work machine 10 through the display device 500.

As described above, the peripheral monitoring device 100 of the work machine of the present disclosure is realized by a simple configuration such as a photographing device 200, an image processing device 300, and an alarm device 400, and these are arranged in the existing work machine 10. The peripheral monitoring device 100 can be easily installed on the work machine 10 simply by doing so. Further, as described above, since the photographing device 200 is composed of the monocular camera 210, the cost for monitoring the periphery of the work machine 10 is suppressed as compared with the case where the photographing device is composed of, for example, a stereo camera. can do.

Next, the components of the image processing apparatus 300 will be described in detail based on FIG. 1 (c). FIG. 1C is a diagram showing in more detail the components of the image processing device 300 included in the peripheral monitoring device 100 in the first embodiment.

The image processing device 300 has a storage unit 302 and a control unit 303 as functional units, loads the program stored in the auxiliary storage device into the work area of the main storage device, executes the program, and executes each function through the execution of the program. By controlling the units and the like, it is possible to realize each function that meets a predetermined purpose in each functional unit. However, some or all of the functions may be realized by hardware circuits such as ASICs and FPGAs.

The storage unit 302 includes a main storage device and an auxiliary storage device. The main storage device is a memory in which a program executed by the control unit 303 and data used by the control program are expanded. The auxiliary storage device is a device that stores a program executed by the control unit 303 and data used by the control program.

Further, the storage unit 302 may store the pre-learning model described later. This pre-learning model can be used for the extraction process described later. Here, when the function of the image processing device 300 is realized by a hardware circuit such as FPGA, the pre-learning model may be stored in the memory built in the FPGA.

The control unit 303 is a functional unit that controls the processing performed by the image processing device 300. The control unit 303 can be realized by an arithmetic processing unit such as a CPU. The control unit 303 is further composed of four functional units: an acquisition unit 3031, an extraction processing unit 3032, a distance measuring processing unit 3033, and a notification control processing unit 3034. Each functional unit may be realized by executing the stored program by the CPU. The control unit 303 functions as the processor according to the present disclosure by executing the processes of the acquisition unit 3031, the extraction processing unit 3032, the distance measuring processing unit 3033, and the alarm control processing unit 3034. Then, by mounting this processor on the integrated circuit constituting the image processing device 300, the processor that executes the processing of the functional unit is installed in the work machine 10.

Here, the processing flow performed by the control unit 303 will be described with reference to FIG. FIG. 2 is a flowchart showing a processing flow performed by the control unit 303 in order to avoid contact between the work machine 10 and a person around the work machine 10 in the peripheral monitoring device 100 according to the present embodiment. In the present embodiment, when the power of the peripheral monitoring device 100 is turned on, the execution of this flow is started, and the execution is repeatedly executed at a predetermined calculation cycle while the peripheral monitoring device 100 is operating.

In this flow, first, in S101, the acquisition unit 3031 acquires photographed image data representing the surrounding image of the work machine 10 photographed by the photographing device 200. Here, the photographing device 200 and the image processing device 300 have a communication unit, and by connecting these communication units via a network, the acquisition unit 3031 can acquire the above-mentioned captured image data. become. The network may be wireless, wired, or a combination of wireless and wired.

Next, in S102, the extraction processing unit 3032 executes an extraction process for extracting a person image from the captured image data. In this extraction process, when a person exists around the work machine 10 and the person is photographed by the photographing apparatus 200, a person image can be extracted from the photographed image data. The details of this extraction process will be described later. Then, the extraction processing unit 3032 determines in S103 whether or not a person exists around the work machine 10 based on the result of the extraction processing executed in S102. If an affirmative determination is made in S103, the control unit 303 proceeds to the process of S104, and if a negative determination is made in S103, the execution of this flow is terminated.

If an affirmative determination is made in S103, then in S104, the distance measuring processing unit 3033 executes a distance measuring process for measuring the distance between the work machine 10 and a person around it. In this distance measuring process, the distance between the work machine 10 and the people around it is measured based on the person image extracted by the extraction process and the installation parameters of the photographing device 200. The details of this distance measurement process will be described later. Further, in S104, in addition to the distance measuring process described above, the output of the extracted image data, which is the image data of the person image extracted by the extraction process of S102, is executed. At this time, as will be described later, the display device 500 displays the captured image data captured by the photographing device 200 and the person image extracted by the extraction process together with the rectangle.

Next, in S105, the alarm control processing unit 3034 determines whether or not the above distance measured by the distance measuring process is equal to or less than a predetermined value. Here, the above-mentioned predetermined value is a value of a distance at which contact between the work machine 10 and a person around it is feared due to the operation of the work machine 10, and is based on, for example, the size and the operating range of the work machine 10. It is decided in advance. Then, if an affirmative determination is made in S105, the control unit 303 proceeds to the process of S106, and if a negative determination is made in S105, the execution of this flow is terminated.

If a positive determination is made in S105, the alarm control processing unit 3034 then controls the output of the alarm device 400 in S106. In the present embodiment, the alarm control processing unit 3034 outputs a warning light from the alarm device 400 (for example, Patrite (registered trademark)). When the alarm device 400 is composed of, for example, a patrol light (registered trademark) and a speaker, the alarm control processing unit 3034 outputs a warning light from the patrol light (registered trademark) and emits a warning sound from the speaker. It may be output. Then, after the processing of S106, the execution of this flow is completed.

According to the process described above, the distance to the person around the work machine 10 is measured based on the person image extracted by the extraction process, and the alarm control process is performed based on the result of the distance measurement distance. Will be executed. Then, by outputting a predetermined alarm from the alarm device 400, it is possible to give a warning to a person around the work machine 10 in order to avoid contact with the work machine 10. Further, when the operator of the work machine 10 recognizes the output of the alarm from the alarm device 400, the operator can perform an operation operation for avoiding contact between the work machine 10 and a person around it. As a result, contact between the work machine 10 and a person around it can be suppressed.

(Extraction process)
Next, the details of the extraction process executed by the extraction process unit 3032 will be described. In the extraction process of the present embodiment, a person image is extracted by inputting captured image data into the pre-learning model. Then, the pre-learning model is constructed by learning using data including an image representing a person.

Here, FIG. 3 is a diagram for explaining the identification result obtained from the input to the pre-learning model in the present embodiment and the neural network constituting the pre-learning model. In this embodiment, a neural network model generated by deep learning is used as the pre-learning model. The pre-learning model 30 in the present embodiment includes an input layer 31 that accepts input of predetermined image data, and an intermediate layer (hidden layer) 32 that extracts a feature amount representing a person from the image data input to the input layer 31. It has an output layer 33 that outputs an identification result based on a feature amount. In the example of FIG. 3, the pre-learning model 30 has one intermediate layer 32, the output of the input layer 31 is input to the intermediate layer 32, and the output of the intermediate layer 32 is input to the output layer 33. Has been done. However, the number of intermediate layers 32 is not limited to one, and the pre-learning model 30 may have two or more intermediate layers 32.

Also, according to FIG. 3, each layer 31-33 comprises one or more neurons. For example, the number of neurons in the input layer 31 can be set according to the input image data. Further, the number of neurons in the output layer 33 can be set according to the person image which is the identification result.

Then, neurons in adjacent layers are appropriately connected to each other, and a weight (connection load) is set for each connection based on the result of machine learning. In the example of FIG. 3, each neuron is connected to all neurons in the adjacent layer, but the connection of neurons does not have to be limited to such an example and can be appropriately set.

Such a pre-learning model 30 is constructed by performing supervised learning using, for example, supervised learning that is a set of image data including an image representing a person and a label of an image representing a person. Specifically, a set of a feature amount and a label is given to the neural network, and the weight of the connection between neurons is tuned so that the output of the neural network becomes the same as the label. In this way, a pre-learning model for learning the characteristics of the teacher data and estimating the result from the input is inductively acquired.

The pre-learning model 30 may be constructed by performing unsupervised learning. For unsupervised learning, for example, domain adaptation, which is a type of transfer learning, can be used. According to this, the pre-learning model can be acquired without preparing a large amount of labeled teacher data.

Then, in the present embodiment, in addition to the captured image data captured by the photographing device 200, the person image extracted by the extraction process is displayed together with the rectangle on the display device 500 described above. Here, FIG. 4 is a diagram for explaining captured image data and a person image displayed on the display device 500. The operator of the work machine 10 can visually recognize the surroundings of the work machine 10 through the screen SC1 of the display device 500.

FIG. 4A is a diagram showing captured image data displayed on the display device 500. As shown in FIG. 4A, the captured image data SC11 is displayed on the screen SC1 of the display device 500. FIG. 4B is a diagram showing an example in which a person image extracted by the extraction process is displayed on the display device 500 together with the captured image data. As shown in FIG. 4B, the captured image data SC11 and the person image SC12 are displayed on the screen SC1 of the display device 500. In this way, the person image SC12 extracted by the extraction process is displayed together with the rectangle. According to this, the operator of the work machine 10 can easily visually recognize the person image SC12 displayed on the screen SC1 of the display device 500. By extracting the person image using the above-mentioned pre-learning model, the extraction accuracy of the person image from the photographed image data, specifically, the person image SC12 included in the photographed image data SC11 and displayed is represented. The position accuracy of the rectangle is improved.

Here, in the image processing device 300 of the present embodiment, the distortion correction value of the lens of the monocular camera 210 is used to correct the distortion of the captured image data. Then, the captured image data SC11 with distortion correction is displayed on the screen SC1 of the display device 500. In addition, a person image is extracted from the captured image data that has been subjected to distortion correction. Therefore, in constructing the pre-learning model, a distortion-corrected image is used as image data including an image representing a person used for learning. However, there is no intention of limiting the present disclosure to this, and the above-mentioned distortion correction may not be executed in the peripheral monitoring device of the present disclosure. In this case, the captured image data including the distortion is displayed on the display device 500, but if the person image can be accurately extracted from the captured image data including the distortion, the working machine 10 can display the captured image data. The operator can visually recognize the person image displayed together with the rectangle. Therefore, in such a case, in constructing the pre-learning model, an image including distortion is used as image data including an image representing a person used for learning. Then, the person image can be accurately extracted from the captured image data including the distortion. According to this, the image processing device 300 does not need to correct the distortion of the captured image data acquired from the photographing device 200, and thus the processing load of the image processing in the peripheral monitoring device is reduced, so that the work can be performed more quickly. The surroundings of the machine 10 can be monitored.

Further, FIG. 4C is a diagram showing variations of a rectangle representing the person image SC12. As shown in FIG. 4C, the rectangle representing the person image SC12 may be a rectangle SC121 that surrounds the whole body of a person, a rectangle SC122 that surrounds the upper body of a person, or a rectangle SC122 that surrounds the upper body of a person. It may be a rectangle SC123 that surrounds. The operator of the work machine 10 can recognize a person image regardless of the rectangle, but for example, in the case of the rectangle SC121 that surrounds the whole body of a person, it becomes easier to recognize the size of the person. Further, for example, when a plurality of person images are included in the captured image data, the number of people can be easily recognized by using the rectangular SC123 that surrounds the neck of the person. That is, when the captured image data includes a plurality of person images, a rectangular SC123 that surrounds the neck of the person can be preferably used.

In the present embodiment, as described above, the function of the image processing device 300 can be realized by a hardware circuit such as an FPGA. In this case, the pre-learning model can be stored in the memory built into the FPGA. According to this, the extraction process using the pre-learning model can be executed as quickly as possible, and thus the periphery of the work machine 10 can be monitored more quickly.

As described above, according to the extraction process of the present embodiment, by using the pre-learning model, a person image can be accurately extracted from the captured image data, and the extracted person image is suitable for the operator of the work machine 10. Can be visually recognized.

(Distance measurement processing)
Next, the details of the distance measuring process executed by the distance measuring processing unit 3033 will be described. In the distance measuring process of the present embodiment, the distance between the work machine 10 and the person around it is measured based on the person image extracted by the extraction process and the installation parameters of the photographing device 200. Regarding such distance measurement processing, in the present embodiment, foot distance measurement based on the foot image of the person image and head distance measurement based on the head image of the person image are used. Used. In the present embodiment, any one of these distance measuring methods may be used, or both of these distance measuring methods may be used. Foot distance measurement and head distance measurement will be described in detail below.

First, foot distance measurement will be described. In foot distance measurement, the distance measurement processing unit 3033 acquires the foot coordinates, which are the two-dimensional coordinates of the foot of the person image in the captured image data. Then, based on the foot coordinates, the foot angle, which is a predetermined angle between the feet of a person around the work machine 10 and the photographing device 200, is calculated. Then, the distance measuring processing unit 3033 can measure the distance between the work machine 10 and the person around it based on the calculated foot angle and the installation height of the photographing device 200.

Ｄ：周囲人物との距離（ｍｍ）
ｈ：撮影装置２００の設置高さ（ｍｍ）
θ：足元角度（ｄｅｇｒｅｅ） FIG. 5 is a diagram for explaining the relationship between the foot angle, the installation height of the photographing device 200, and the distance to the surrounding people. Here, the surrounding person is a person around the work machine 10, and the distance between the surrounding person and the surrounding person, which is the distance between the surrounding person and the work machine 10, is represented as the distance D in FIG. Then, the installation height of the photographing device 200 from the ground is represented as the height h in FIG. Further, the above-mentioned foot angle between the feet of the surrounding person and the photographing device 200 is represented as the angle θ in FIG. Then, the distance D to the surrounding people can be calculated by the following equation 1.

D: Distance to surrounding people (mm)
h: Installation height (mm) of the photographing device 200
θ: Foot angle (degree)

Then, the distance measuring processing unit 3033 executes the processing flow shown in FIG. 6 in order to calculate the distance D with the surrounding person based on the above equation 1. FIG. 6 is a flowchart showing a processing flow performed by the distance measuring processing unit 3033 in the foot distance measuring, which is one of the distance measuring processes according to the present embodiment. This flow is executed as an internal process of the distance measurement process in S104 shown in FIG. 2 above.

In this flow, first, in S1041, the foot coordinates (two-dimensional coordinates of the foot of the person image in the captured image data) are acquired. This will be described with reference to FIG. FIG. 7 is a diagram for explaining a subject projected on the sensor of the camera together with the internal parameters of the camera. In the example shown in FIG. 7, a surrounding person exists at a position deviated from the optical axis L2 of the camera in the Y-axis direction. In this case, the distance measuring processing unit 3033 acquires the coordinates y of the feet of the surrounding people projected on the sensor as the feet coordinates. As described in the above description of the extraction process, the person image can be displayed together with the rectangle. Therefore, for example, the lower end of the rectangle surrounding the whole body of a person may be acquired as the foot coordinates. Further, when the captured image data for which the foot coordinates are acquired is not distortion-corrected, the acquired foot coordinates can be subjected to the distortion correction processing.

θ：足元角度（ｄｅｇｒｅｅ）
θ１：カメラ中心からの角度（ｄｅｇｒｅｅ）
θ２：撮影装置２００の設置角度（ｄｅｇｒｅｅ）
なお、カメラ中心からの角度θ１は後述する。また、撮影装置２００の設置角度θ２は、図５に示すように、地面との平行線Ｌ１とカメラの光軸Ｌ２との間の角度であって、周辺監視装置１００が設置される作業機械１０に応じて予め定められる。 Then, returning to FIG. 6, next, in S1042, the foot angle (the angle between the foot of a person around the work machine 10 and the photographing device 200) is calculated. Here, according to FIG. 5 above, the foot angle θ is expressed by the following equation 2.

θ: Foot angle (degree)
θ1: Angle from the center of the camera (degree)
θ2: Installation angle of the imaging device 200 (degree)
The angle θ1 from the center of the camera will be described later. Further, as shown in FIG. 5, the installation angle θ2 of the photographing device 200 is an angle between the parallel line L1 with the ground and the optical axis L2 of the camera, and the work machine 10 on which the peripheral monitoring device 100 is installed is installed. It is predetermined according to.

θ１：カメラ中心からの角度（ｄｅｇｒｅｅ）
ｆ：レンズの焦点距離（ｍｍ）
ｙ：足元座標（ｐｉｘｅｌ）
Ｎ：カメラの画素サイズ（ｍｍ／ｐｉｘｅｌ） Here, the angle θ1 from the center of the camera can be calculated by the following equation 3 based on FIG. 7 above.

θ1: Angle from the center of the camera (degree)
f: Focal length of the lens (mm)
y: Foot coordinates (pixel)
N: Camera pixel size (mm / pixel)

Then, returning to FIG. 6, next, in S1043, the installation height of the photographing apparatus 200 is acquired. The installation height of the photographing device 200 is predetermined according to the work machine 10 in which the peripheral monitoring device 100 is installed. The distance measuring processing unit 3033 may perform the processing of S1043 before the processing of S1041.

Next, in S1044, the distance between the work machine 10 and the people around it is measured based on the foot angle calculated by the process of S1042 and the installation height of the photographing device 200 acquired by the process of S1043. NS. Specifically, the distance measuring processing unit 3033 calculates the distance between the work machine 10 and a person around it based on the above equation 1. Then, after the processing of S1044, the control unit 303 proceeds to the processing of S105. As described above, in S104, the output of the extracted image data is executed in addition to the distance measuring process.

Based on the above, the monocular camera 210 can perform distance measurement processing with high accuracy at low cost. In addition, according to such foot distance measurement, since the foot coordinates can be acquired relatively accurately, the above-mentioned distance calculation accuracy can be made as high as possible.

Next, head distance measurement will be described. In the head distance measurement, the distance measurement processing unit 3033 acquires the head image of the person image in the captured image data, and based on the size of the head image, determines the distance between the work machine 10 and the people around it. taking measurement.

Ｄ´：周囲人物との距離（ｍｍ）
Ｄｒｅｆ：基準距離（ｍｍ）
Ｎｒｅｆ：基準距離での縦画素数（ｐｉｘｅｌ）
Ｎｈｅａｄ：頭の画像の縦画素数（ｐｉｘｅｌ）
なお、基準距離Ｄｒｅｆおよび該基準距離での縦画素数Ｎｒｅｆは、予め定められる。そして、このように予め定められた基準と、撮影画像データから取得したの頭の画像の縦画素数Ｎｈｅａｄと、を比較することで、周囲人物との距離Ｄ´が算出される。 Specifically, the distance measuring processing unit 3033 acquires the number of vertical pixels of the head image of the person image in the captured image data. Here, the number of vertical pixels of the head image can be obtained from, for example, the number of vertical pixels of a rectangle surrounding the neck of a person. Then, the present disclosure person has found that the distance can be measured based on the above-mentioned number of vertical pixels representing the size of the head by utilizing the fact that the size of the head of a person has little individual difference. Therefore, the distance measuring processing unit 3033 calculates the distance between the work machine 10 and a person around it based on the following equation 4.

D': Distance to surrounding people (mm)
Dref: Reference distance (mm)
Nref: Number of vertical pixels at reference distance (pixel)
Nhead: Number of vertical pixels (pixel) of the head image
The reference distance Dref and the number of vertical pixels Nref at the reference distance are predetermined. Then, the distance D'with the surrounding person is calculated by comparing the predetermined standard with the number of vertical pixels Nhead of the head image acquired from the captured image data.

Based on the above, the monocular camera 210 can perform distance measurement processing with high accuracy at low cost. According to such head distance measurement, if the captured image data does not include the feet of surrounding people, or if the person is climbing to a high place and the person is not on the ground. Even if there is, the distance between the work machine 10 and the people around it can be calculated.

Ｄｏｕｔ：出力される周囲人物との距離（ｍｍ）
Ｄ：足元測距により算出される周囲人物との距離（ｍｍ）
Ｄ´：頭測距により算出される周囲人物との距離（ｍｍ）
ｗ：ブレンド係数
ここで、ブレンド係数ｗは、下記式６および下記式７に基づいて算出される。

Then, both the foot distance measurement and the head distance measurement described above may be used to calculate the distance between the work machine 10 and a person around it. For example, when the distance D to the surrounding person based on the above formula 1 is calculated by the foot distance measurement and the distance D'to the surrounding person based on the above formula 4 is calculated by the head distance measurement, the distance measurement processing unit 3033. Outputs the distance Dout to the surrounding person calculated based on the following equation 5 as the distance between the work machine 10 and the surrounding person.

Dout: Distance to the output surrounding person (mm)
D: Distance to surrounding people calculated by foot distance measurement (mm)
D': Distance to surrounding people calculated by head distance measurement (mm)
w: Blend coefficient Here, the blend coefficient w is calculated based on the following formula 6 and the following formula 7.

According to this, for example, when the distance D and the distance D'are deviated by 500 mm, the blend coefficient w becomes 0.5, and the distance D to the surrounding person calculated by the foot distance measurement and the head distance measurement are used. The calculated average value of the distance D'with the surrounding person is output as the distance Dout between the work machine 10 and the surrounding person. Further, for example, when the distance D and the distance D'are deviated by 1000 mm or more, the blend coefficient w becomes 1.0, and the distance D'with the surrounding person calculated by the head distance measurement is the same as that of the work machine 10. It is output as the distance Dout with the surrounding people. When the distance D and the distance D'are equal, the blend coefficient w becomes 0, and the distance D with the surrounding person calculated by the foot distance measurement is set as the distance Dout between the work machine 10 and the surrounding person. It is output.

As described above, in the present embodiment, the person image can be accurately extracted from the captured image data by the extraction process, and the distance to the surrounding person can be accurately measured at low cost by the distance measurement process. .. According to the present embodiment, it is possible to provide a peripheral monitoring device that can be easily installed on a work machine at low cost and can suppress contact between the work machine and a person around it.

The peripheral monitoring device 100 of the present embodiment may include a storage device that stores the installation parameters and the like of the photographing device 200 described above. In this case, the storage device can store the installation parameters of the photographing device 200 for each model of the work machine 10. Then, the above-mentioned installation parameters stored in the storage device can be displayed on the display device 500 together with the model of the work machine 10. Then, the user can easily install the peripheral monitoring device 100 according to the model of the work machine 10.

Further, in the above embodiment, the configuration in which the peripheral monitoring device 100 includes the display device 500 has been described as an example, but the peripheral monitoring device of the present disclosure is not intended to be limited to this. The peripheral monitoring device of the present disclosure does not have to include the above-mentioned display device, and even in this case, the distance measuring process is executed based on the person image extracted by the extraction process and the installation parameters of the photographing device. By executing the alarm control process based on the result of the distance measurement process, it is possible to suppress contact between the work machine and a person around it.

<Modification 1 of the first embodiment>
A modification 1 of the first embodiment will be described. In the first embodiment described above, when the distance between the work machine 10 and a person around it becomes a predetermined value or less, a warning light or a warning sound is output from the alarm device 400. As a result, it is possible to give a warning to a person around the work machine 10 in order to avoid contact with the work machine 10. Further, when the operator of the work machine 10 recognizes the output of the alarm from the alarm device 400, the operator can perform an operation operation for avoiding contact between the work machine 10 and a person around it. Here, if the avoidance operation of the work machine 10 can be performed without the operation operation by the operator, the contact between the work machine 10 and a person around the work machine 10 can be more preferably avoided.

Therefore, in the present modification, when the alarm control processing unit 3034 described in the description of the first embodiment has a distance between the work machine 10 and a person around it of a predetermined value or less, the alarm device 400 Therefore, a signal for instructing a predetermined avoidance operation for avoiding contact between the work machine 10 and a person around the work machine 10 is output.

Here, the alarm device 400 of this modification is configured to be able to output a predetermined alarm signal. The predetermined alarm signal is, for example, a signal for instructing the avoidance operation, and the signal is transmitted to the control controller of the work machine 10 by the Controller Area Network (CAN) or the like. Then, the work machine 10 automatically performs the avoidance operation regardless of the operation of the operator. Here, the avoidance operation is an operation in which the operation of the work machine 10 is decelerated, an operation in which the operation of the work machine 10 is stopped, and the like. According to this, it is possible to perform the avoidance operation of the work machine 10 without the operation operation by the operator, and it is possible to more preferably avoid the contact between the work machine 10 and the person around it. The alarm device 400 may be configured to be capable of outputting electrical or mechanical contacts or input / output signals.

Further, the alarm device 400 may have only a function of outputting a signal for instructing such an avoidance operation, and in addition to the function, a warning light or a warning described in the above description of the first embodiment may be provided. It may have a function of outputting sound. Further, as an alarm signal, information indicating that the distance between the work machine 10 and a person around it is equal to or less than a predetermined value may be notified to a predetermined terminal via a predetermined communication means (for example, Ethernet). ..

<Modification 2 of the first embodiment>
The second modification of the first embodiment will be described. In the first embodiment described above, when the power of the peripheral monitoring device 100 is turned on, the flow shown in FIG. 2 is executed. On the other hand, in this modification, when the power of the peripheral monitoring device 100 is turned on, the flow shown in FIG. 8 described below is executed.

In this modification, the processing flow performed by the control unit 303 will be described with reference to FIG. FIG. 8 is a first flowchart showing a processing flow performed by the control unit 303 in order to avoid contact between the work machine 10 and a person around the work machine 10 in the peripheral monitoring device 100 according to the present modification. In this modification, substantially the same processing as that shown in FIG. 2 of the first embodiment described above will be designated by the same reference numerals and detailed description thereof will be omitted.

In the flow shown in FIG. 8, after the processing of S101, it is determined in S201 whether or not a predetermined time has elapsed since the power of the peripheral monitoring device 100 was turned on. Here, the predetermined time is, for example, the time until the pre-learning model or program used for the extraction process is loaded into the work area of the image processing apparatus 300, and is predetermined according to the scale of the model. Then, when an affirmative determination is made in S201, in this case, the extraction process can be executed immediately, and the control unit 303 proceeds to the process in S102. On the other hand, if a negative determination is made in S201, the control unit 303 proceeds to the process of S202.

If a negative determination is made in S201, then in S202, only the captured image data acquired in the process of S101 is output to the display device 500. Then, after the processing of S202, the execution of this flow is completed.

Here, the control unit 303 cannot immediately execute the extraction process until a predetermined time elapses after the power of the peripheral monitoring device 100 is turned on. In such a case, if the process of the control unit 303 is waited until the extraction process can be executed and the operator of the work machine 10 cannot visually recognize the surroundings via the display device 500, the work machine during this period. There is a risk that contact between 10 and a person around it may occur. On the other hand, according to this flow, the data that can be acquired by the extraction process and the distance measurement process is not output to the display device 500 until a predetermined time elapses after the power of the peripheral monitoring device 100 is turned on. Only the captured image data is output to the display device 500. That is, the surrounding image of the work machine 10 is output to the display device 500 as soon as possible after the power of the peripheral monitoring device 100 is turned on. Therefore, the operator of the work machine 10 can visually recognize the surroundings via the display device 500 as soon as possible after the power of the peripheral monitoring device 100 is turned on. According to this, it is possible to more preferably avoid contact between the work machine 10 and a person around it.

When only the captured image data is output to the display device 500 in this way, the rectangle is not displayed on the person included in the captured image data. Then, the operator of the work machine 10 can recognize that the extraction process and the distance measurement process have not been executed yet.

Further, in this modification, when the power of the peripheral monitoring device 100 is turned on, the flow shown in FIG. 9 described below may be executed. FIG. 9 is a second flowchart showing a processing flow performed by the control unit 303 in order to avoid contact between the work machine 10 and a person around the work machine 10 in the peripheral monitoring device 100 according to the present modification. The processes substantially the same as those shown in FIG. 2 of the first embodiment described above are designated by the same reference numerals, and detailed description thereof will be omitted.

In the flow shown in FIG. 9, after the processing of S102, it is determined in S203 whether or not the result of the extraction processing in the processing of S102 can be acquired. Here, when the result of the extraction process cannot be obtained, for example, the pre-learning model or program used for the extraction process has not yet been loaded into the work area of the image processing device 300 (loading to the work area has been completed). Not) if. In this case, in the flow shown in FIG. 9, the process of S102 is executed regardless of whether or not the above loading is completed. If the extraction of surrounding people cannot be performed because the above loading is not completed in S102, the control unit 303 immediately terminates the processing of S102, assuming that the extraction processing cannot be executed. At this time, the control unit 303 may set a flag for determining whether or not the extraction process has been executed. Then, in S203, the above can be determined based on this flag. Then, if an affirmative determination is made in S203, the control unit 303 proceeds to the process of S103, and if a negative determination is made in S203, the control unit 303 proceeds to the process of S204.

If a negative determination is made in S203, then in S204, only the captured image data acquired in the process of S101 is output to the display device 500. Then, after the processing of S204, the execution of this flow is completed.

Even with the above-described processing, the operator of the work machine 10 can visually recognize the surroundings via the display device 500 as soon as possible after the power of the peripheral monitoring device 100 is turned on. It is possible to more preferably avoid contact between the machine 10 and a person around it.

<Second Embodiment>
The second embodiment will be described. In the present embodiment, the control unit 303 further executes the image sharpening process. Specifically, the control unit 303 generates a captured image histogram showing the frequency of pixel values included in the captured image data. Then, the captured image data is sharpened by synthesizing the smoothed image data acquired based on the smoothed smoothed image histogram and the captured image data at a predetermined ratio. This will be described with reference to FIG.

The control unit 303 of the present embodiment executes the processing flow shown in FIG. 10 in order to execute the image sharpening processing. FIG. 10 is a flowchart showing a processing flow performed by the control unit 303 in the image sharpening process according to the present embodiment. This flow is executed after the processing of S101 of the flow shown in FIG. 2 above.

In this flow, after the processing of S101, a captured image histogram is generated in S301. As described above, the captured image histogram is a histogram showing the frequency of pixel values included in the captured image data, and is generated based on a well-known technique.

Next, in S302, smoothed image data is generated. In the process of S302, the control unit 303 first generates a smoothed histogram obtained by smoothing the captured image histogram, and acquires smoothed image data based on the smoothed histogram. Here, the smoothing histogram is generated by stretching the captured image histogram over the entire range of pixel values (256 gradations). For example, a histogram in which the frequency is concentrated in a certain pixel value range is a pixel value. A high-contrast image can be obtained by stretching over the entire range (256 gradations). That is, the smoothed image data is image data in which the contrast of the captured image data is enhanced.

According to such smoothed image data, it is possible to secure the contrast of the image even at night or in foggy weather. However, since the work machine 10 is used in various environments, it is necessary to adjust the contrast according to the environment.

Therefore, in the present embodiment, the sharpening intensity is acquired in the processing of S303. Then, in the process of S304, the smoothed image data is combined with the captured image data at a predetermined ratio. This ratio is a ratio based on the sharpening intensity, and the intensity classification can be set in, for example, three stages of weak, medium, and strong. For example, when the sharpening intensity classification is weak, the smoothed image data and the captured image data are combined at a ratio of 1: 3, and when the sharpening intensity classification is medium, the smoothed image data and the captured image data are combined. When the data and the data are combined at a ratio of 1: 1 and the sharpening intensity classification is strong, the smoothed image data and the captured image data are combined at a ratio of 3: 1. In the processing of S303, the strength value corresponding to such a classification, for example, when the strength classification is weak, the strength value may be acquired as 0.25 (in this case, it is clarified). Twenty-five percent of the captured image data is from the smoothed image data, 75% is from the original captured image data), and any value regardless of classification (value greater than 0 and less than 1) May be obtained. Further, the sharpening intensity may be manually set by the operator of the work machine 10, or may be automatically set according to the environment. Then, after the processing of S304, the control unit 303 proceeds to the processing of S102.

In this way, when the captured image data is sharpened by the processes of S301 to S304 and the extraction process is executed by the next process of S102, it is based on the captured image data that has been sharpened according to the environment. A person image will be extracted. According to this, the accuracy of extracting a person image from the captured image data can be made as high as possible. Further, the operator of the work machine 10 can easily visually recognize an obstacle existing around the vehicle body of the work machine 10 via the display device 500.

Further, the peripheral monitoring device described above can also suitably suppress the contact between the work machine and the person around it.

<Other variants>
The above embodiment is merely an example, and the present disclosure may be appropriately modified and implemented without departing from the gist thereof. For example, the processes and means described in the present disclosure can be freely combined and carried out as long as there is no technical contradiction.

In the above embodiment, an example in which the photographing device 200 is configured to include four monocular cameras 210 has been described, but the photographing device of the present disclosure is not limited to this. FIG. 11 is a diagram showing a modified example of the schematic configuration of the peripheral monitoring device for the work machine in the present disclosure. As shown in FIG. 11, the photographing apparatus of the present disclosure may include three monocular cameras 210 and one hemispherical camera 220. In this case, the hemispherical camera 220 may be arranged at a position (for example, the upper surface of the driver's cab of the work machine 10) where a region where a blind spot is likely to occur can be photographed by a predetermined equipment of the work machine 10.

Further, the processing described as being performed by one device may be shared and executed by a plurality of devices. For example, the extraction processing unit 3032 may be formed in an integrated circuit different from the image processing device 300. At this time, the other integrated circuit is configured to be suitably cooperative with the image processing device 300. Further, the processing described as being performed by different devices may be executed by one device. In an integrated circuit, it is possible to flexibly change what kind of hardware configuration is used to realize each function.

The present disclosure can also be realized by supplying a computer program having the functions described in the above-described embodiment to a computer, and having one or more processors of the computer read and execute the program. Such computer programs may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus. Non-temporary computer-readable storage media include, for example, any type of disc such as a magnetic disc (floppy (registered trademark) disc, hard disk drive (HDD), etc.), an optical disc (CD-ROM, DVD disc, Blu-ray disc, etc.). Includes read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions.

10 ... Work machine 100 ... Peripheral monitoring device 200 ... Imaging device 210 ... Monocular camera 300 ... Image processing device 303 ... Control unit 400 ... Alarm device

Claims (8)

An imaging device installed on a work machine that photographs the surroundings of the work machine,
A processor that acquires photographed image data representing the surrounding image of the work machine photographed by the photographing apparatus and executes a predetermined process based on the photographed image data.
An alarm device that outputs a predetermined alarm signal and / and a predetermined alarm, and
With
The imaging device includes one or more monocular cameras, and is configured such that the optical axis of the monocular camera is arranged at a predetermined angle with respect to the ground.
The processor
An extraction process for extracting a person image from the captured image data, and
A distance measuring process for measuring the distance between the work machine and a person around it based on the extracted person image and the installation parameters of the photographing device.
Based on the distance between the work machine and a person around it measured by the distance measuring process, the alarm signal and / and the alarm control process for controlling the output of the alarm by the alarm device are executed. death,
In the distance measuring process, the processor
The foot coordinates, which are the two-dimensional coordinates of the feet of the person image in the captured image data, are acquired, and the foot coordinates and the angle between the feet of the person around the work machine and the optical axis of the monocular camera are used. Based on the installation angle of the photographing device, the foot angle, which is a predetermined angle between the feet of a person around the work machine and the photographing device, is calculated.
The distance between the work machine and a person around it is measured based on the calculated foot angle and the installation height of the photographing device.
Peripheral monitoring device for work machines. An imaging device installed on a work machine that photographs the surroundings of the work machine,
A processor that acquires photographed image data representing the surrounding image of the work machine photographed by the photographing apparatus and executes a predetermined process based on the photographed image data.
An alarm device that outputs a predetermined alarm signal and / and a predetermined alarm, and
With
The imaging device includes one or more monocular cameras, and is configured such that the optical axis of the monocular camera is arranged at a predetermined angle with respect to the ground.
The processor
An extraction process for extracting a person image from the captured image data, and
A distance measuring process for measuring the distance between the work machine and a person around it based on the extracted person image and the installation parameters of the photographing device.
Based on the distance between the work machine and a person around it measured by the distance measuring process, the alarm signal and / and the alarm control process for controlling the output of the alarm by the alarm device are executed. death,
In the distance measuring process, the processor
The head image of the person image in the captured image data is acquired, and the distance between the work machine and the surrounding person is measured based on the number of pixels of the head image.
Peripheral monitoring device for work machines. In the alarm control process, the processor
When the distance between the work machine and a person around it becomes equal to or less than a predetermined value, the alarm device commands a predetermined avoidance operation for avoiding contact between the work machine and a person around it. Output a signal,
The peripheral monitoring device for a work machine according to claim 1 or 2. In the extraction process, the processor
A neural network model having an input layer that accepts input of predetermined image data, an intermediate layer that extracts a feature amount representing a person from the input image data, and an output layer that outputs an identification result based on the feature amount. The person image is extracted by inputting the photographed image data into the pre-learning model constructed by performing the training using the data including the image representing the person.
The peripheral monitoring device for a work machine according to any one of claims 1 to 3. A display device configured to be able to display the captured image data and / or predetermined data acquired by processing based on the captured image data taken by the photographing device is further provided.
The peripheral monitoring device for a work machine according to any one of claims 1 to 4. The processor
Until a predetermined time elapses after the power of the peripheral monitoring device is turned on, the data that can be acquired by the extraction process and the distance measurement process is not output to the display device, and only the captured image data is displayed on the display device. Execute the output process,
The peripheral monitoring device for a work machine according to claim 5. The processor
A captured image histogram showing the frequency of pixel values included in the captured image data is generated.
Image sharpening that sharpens the captured image data by synthesizing the smoothed image data acquired based on the smoothed smoothed histogram of the captured image histogram and the captured image data at a predetermined ratio. Further execute the conversion process,
The peripheral monitoring device for a work machine according to any one of claims 1 to 6. The processor is mounted on an integrated circuit installed in the working machine.
The peripheral monitoring device for a work machine according to any one of claims 1 to 7.

Images