JP2020051168A - Attachment status display device - Google Patents

Abstract

To provide an attachment status display device that can check the attachment status of a plurality of image sensors and improve workability for adjusting the sensors if necessary.SOLUTION: The attachment status display device includes: an attachment status acquisition unit that is provided at an image sensor and that acquires the attachment status of the image sensor; determination means that determines whether the attachment status acquired by the attachment status acquisition unit is appropriate or not; a display unit 3; and display control means that displays determination results for each image sensor that are determined by the determination means on the display unit 3 at once.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an attachment state display device that displays an attachment state of a plurality of image sensors attached to a work machine.

  Patent Literature 1 discloses a vehicle monitoring system that uses a camera mounted on a vehicle such as an automobile to capture a scene in front of the vehicle and recognizes a traveling state with an image recognition device. The image recognition device inspects the mounting position of the camera. A method for inspecting an in-vehicle camera is disclosed, which determines whether or not it is appropriate and displays the result on a display monitor connected to the in-vehicle navigation device. The operator adjusts the mounting position of the camera according to the result displayed on the display monitor.

JP 2001-95017 A

  By the way, in a working machine such as a hydraulic shovel, a plurality of image sensors such as a camera disclosed in Patent Document 1 are mounted because blind spots are more than in a car. When the mounting state of each of the plurality of image sensors is checked and adjusted as needed, if the mounting states are displayed one by one on the display monitor, workability is poor.

  An object of the present invention is to provide an attachment state display device capable of confirming the attachment state of a plurality of image sensors and improving the workability of an operation of adjusting as necessary.

  The present invention is an attachment state display device attached to each of a plurality of different positions of a work machine, each of which displays an attachment state of a plurality of image sensors for acquiring an image, provided on the image sensor, An attachment state acquisition device that acquires the attachment state of the image sensor, a determination unit that determines whether the attachment state acquired by the attachment state acquisition device is appropriate, a display device, and each of the image sensors by the determination unit And display control means for displaying the judgment result at a time on the display device.

  According to the present invention, the determination result of whether or not the mounting state is appropriate for each image sensor is displayed on the display device at a time. This allows the operator to check at a glance whether the mounting states of the plurality of image sensors are appropriate. Therefore, it is possible to improve the workability of checking the mounting state of each of the plurality of image sensors, and adjusting as necessary.

It is a top view of a working machine. It is a perspective view of an image sensor. It is a circuit diagram of an attachment state display device. It is a perspective view of an information processing terminal. It is a figure showing a screen of a display. It is a flowchart of a display control.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

(Work machine configuration)
The attachment state display device according to the present embodiment is attached to each of a plurality of different positions on a work machine, and respectively displays the attachment states of a plurality of image sensors each acquiring an image. The work machine 10 is, for example, a hydraulic shovel, as shown in FIG. 1, which is a top view of the work machine 10. The work machine 10 includes a lower traveling body 11, an upper revolving superstructure 12, and a work device 13.

  The lower traveling body 11 is a part on which the work machine 10 travels. The upper revolving unit 12 is rotatably provided above the lower traveling unit 11 via a revolving device. A cab (operating cab) 14 is provided at the front of the upper revolving superstructure 12.

  The working device 13 is attached to the upper swing body 12. The working device 13 includes a boom, an arm, and a bucket.

  A plurality of image sensors 20 are attached to the work machine 10, respectively. These image sensors 20 are attached to a plurality of different positions of the work machine 10, respectively, and each acquire an image. In the present embodiment, four image sensors 20 are attached to four different positions A to D, respectively.

  The image sensor 20 is an infrared sensor, a TOF (Time of Flight) camera, or the like. The image acquired by the image sensor 20 includes a distance image. The image sensor 20 measures a distance based on three-dimensional point cloud data obtained by simultaneously measuring a vertical distance by a plurality of infrared sensors arranged in a matrix on a two-dimensional plane, The distance to the target object may be measured by performing stereoscopic vision using two or more cameras.

  As shown in FIG. 2 which is a perspective view of the image sensor 20, the image sensor 20 has a sensor main body 21 and a support member 22. The support member 22 supports the sensor main body 21 so that the pitching angle and the yawing angle of the sensor main body 21 can be adjusted.

(Configuration of mounting status display device)
As shown in FIG. 3 which is a circuit diagram of the attachment state display device 1, the attachment state display device 1 includes an attachment state acquisition device 2, a display device 3, a storage device 4, a communication device 5, a controller 6, have.

  The attachment state acquisition device 2 is provided in the image sensor 20, and acquires the attachment state of the image sensor 20. In the present embodiment, the attachment state acquisition device 2 acquires an image acquired by the image sensor 20 as the attachment state of the image sensor 20.

  Note that the attachment state acquisition device 2 may acquire the attitude of the image sensor 20 as the attachment state of the image sensor 20. Specifically, the pitching angle, the rolling angle, and the yawing angle of the sensor main body 21 may be acquired as the attitude of the image sensor 20.

  In the present embodiment, the display device 3 is provided in the information processing terminal 30. The information processing terminal 30 is a tablet PC, a smartphone, a mobile PC, or the like. Note that the display device 3 may be mounted inside the cab 14.

  The storage device 4 stores, for each image sensor 20, a normal state, which is a normal mounting state of the image sensor 20.

  The communication device 5 can communicate with the communication device 31 of the information processing terminal 30.

  The controller (determination unit) 6 determines whether the attachment state of the image sensor 20 acquired by the attachment state acquisition device 2 is appropriate. More specifically, the controller 6 determines whether the attachment state of the image sensor 20 is appropriate based on the attachment state acquired by the attachment state acquisition device 2 and the normal state stored in the storage device 4.

  Here, as shown in FIG. 4, which is a perspective view of the information processing terminal 30, the controller (display control unit) 6 sets the attachment state acquired by the attachment state acquisition device 2 and the normal state stored in the storage device 4. The superimposed adjustment screen is displayed on the display device 3. Here, description will be made assuming that one of the four image sensors 20 is selected and the mounting state and the normal state of the selected image sensor 20 are displayed on the display device 3 respectively.

  As described above, the attachment state acquired by the attachment state acquisition device 2 is an image acquired by the image sensor 20. If the mounting state of the image sensor 20 is proper, a part of the work machine 10 appears as a structure S in an image acquired by the image sensor 20. On the other hand, the normal state stored in the storage device 4 includes the contour line O of the structure S that will be reflected if the attachment state of the image sensor 20 is proper. Therefore, if the end of the structure S reflected in the attachment state acquired by the attachment state acquisition device 2 matches the contour O of the normal state stored in the storage device 4, the attachment state of the image sensor 20 is appropriate. Will be.

  The controller 6 determines whether or not the end of the structure S shifted to the attachment state acquired by the attachment state acquisition device 2 coincides with the contour line O in the normal state stored in the storage device 4. Then, it is determined that the mounting state of the image sensor 20 is proper.

  In this way, it is determined whether the mounting state is appropriate based on the current mounting state of the image sensor and the normal state that is the normal mounting state of the image sensor. Thereby, the appropriateness of the mounting state can be accurately determined.

  Further, as shown in FIG. 5 showing a screen of the display device 3, the controller (display control means) 6 causes the display device 3 to display a determination screen. On this determination screen, the determination result for each image sensor 20 is displayed at a time. At the center of the determination screen, an image of work machine 10 as viewed from above is displayed, and the determination screen is divided into four parts. Is displayed. FIG. 5 illustrates a case where the determination result of the image sensor 20 attached to the position A is “NG” and the determination result of the three image sensors 20 attached to the positions B to D is “OK”. .

  As described above, the determination result of whether or not the mounting state is appropriate for each image sensor 20 is displayed on the display device 3 at a time. This allows the operator to check at a glance whether the mounting states of the plurality of image sensors 20 are appropriate. Therefore, the workability of checking the mounting state of the plurality of image sensors 20 and adjusting as necessary can be improved.

  Further, the adjustment screen shown in FIG. 4 and the determination screen shown in FIG. 5 are displayed on the display device 3 of the information processing terminal 30. The operator confirms the mounting state on the determination screen, and adjusts the mounting state while looking at the adjustment screen.

  Here, in the working machine 10, the image sensor 20 is often installed outside the cab 14 and at a location that is inaccessible from the ground. Therefore, when the image sensor 20 is provided outside the cab 14 and the display device 3 is provided inside the cab 14, the worker checks the mounting state while reciprocating between the inside of the cab 14 and the outside of the cab 14. Adjustments have to be made, which takes time. Therefore, by displaying the result of the determination as to whether or not the mounting state is appropriate on the display device 3 of the information processing terminal 30, the worker can carry the information processing terminal 30 while the mounting state is displayed on the determination screen shown in FIG. It is possible to easily confirm whether it is appropriate or not. In addition, by displaying the adjustment screen shown in FIG. 4 on the display device 3 of the information processing terminal 30, the worker can easily adjust the mounting state while carrying the information processing terminal 30.

  As shown in FIG. 3, the information processing terminal 30 has a terminal controller (position information obtaining means) 32 for obtaining its own position information. The position information is a GPS signal, a GNSS signal, or the like. The position information may be obtained based on a change in radio wave intensity between the communication device 5 and the communication device 31, a reflection time of sound, a change in frequency due to reflection, or a reflection time.

  The controller (selection means) 6 selects the target image sensor 20 from the plurality of image sensors 20. In the present embodiment, the controller 6 selects the image sensor 20 closest to the position information of the information processing terminal 30.

  As shown in FIG. 4, the controller (display control means) 6 causes the display device 3 to display the selected mounting state and the normal state of the image sensor 20. Therefore, the worker can easily confirm and adjust the mounting state by comparing the mounting state with the normal state.

  Further, the mounting state and the normal state of the image sensor 20 closest to the position of the information processing terminal 30 are displayed on the display device 3 of the information processing terminal 30. That is, the adjustment screen (see FIG. 4) of the image sensor 20 to which the worker carrying the information processing terminal 30 is closest is displayed on the display device 3 of the information processing terminal 30. Therefore, the labor for selecting the target image sensor 20 can be omitted, and the mounting state can be adjusted quickly.

  Note that a configuration in which the displacement of the mounting state of the image sensor 20 is automatically adjusted may be employed. That is, in FIG. 2, the image sensor 20 includes an actuator capable of rotating the sensor main body 21 and the support member 22, and the controller 6 controls these actuators to automatically adjust the pitching angle and the yawing angle of the sensor main body 21. May be.

  In the present embodiment, as shown in FIG. 3, the controller 6 determines whether or not the mounting state is appropriate, and displays the determination result on the display device 3 at a time. That is, the work machine 10 is provided with a determination unit that determines whether the attachment state is appropriate and a display control unit that displays the determination result on the display device 3 at a time. Thus, the configuration of the image sensor 20 can be simplified and the cost of the image sensor 20 can be reduced as compared with the case where at least one of the determination unit and the display control unit is provided in the image sensor 20.

  Note that the terminal controller 32 of the information processing terminal 30 may be configured to perform at least one of the determination as to whether the attachment state is appropriate and the control to display the determination result on the display device 3 at a time.

(Operation of the mounting status display device)
Next, the operation of the mounting state display device 1 will be described with reference to FIG. 6 which is a flowchart of the display control.

  First, the controller 6 acquires the attachment state of the image sensor 20 via the attachment state acquisition device 2 (Step S1). Next, the controller 6 determines whether or not the mounting state is appropriate (Step S2). Then, the controller 6 causes the display device 3 to display the determination screen shown in FIG. 5 (Step S3).

  Next, the controller 6 selects the image sensor 20 closest to the information processing terminal 30 (Step S4). Then, the controller 6 causes the display device 3 to display the selected mounting state and the normal state of the image sensor 20 as an adjustment screen shown in FIG. 4 (step S5). Then, this flow ends. The operator adjusts the mounting state as necessary while viewing the adjustment screen.

(effect)
As described above, according to the mounting state display device 1 according to the present embodiment, the determination result of whether or not the mounting state is appropriate for each image sensor 20 is displayed on the display device 3 at a time. This allows the operator to check at a glance whether the mounting states of the plurality of image sensors 20 are appropriate. Therefore, the workability of checking the mounting state of the plurality of image sensors 20 and adjusting as necessary can be improved.

  The display device 3 is provided in the information processing terminal 30. In the working machine 10, the image sensor 20 is often installed outside the cab 14 and at a location that is inaccessible from the ground. Therefore, when the image sensor 20 is provided outside the cab 14 and the display device 3 is provided inside the cab 14, the worker checks the mounting state while reciprocating between the inside of the cab 14 and the outside of the cab 14. Adjustments have to be made, which takes time. Therefore, by displaying the determination result of whether or not the attachment state is appropriate on the display device 3 of the information processing terminal 30, the operator can easily confirm whether or not the attachment state is appropriate while carrying the information processing terminal 30. Can be done.

  In addition, it is determined whether the mounting state is appropriate based on the current mounting state of the image sensor 20 and the normal state that is the normal mounting state of the image sensor 20. Thereby, the appropriateness of the mounting state can be accurately determined.

  In addition, the display device 3 displays the attachment state and the normal state of the selected image sensor 20. Therefore, the worker can easily confirm and adjust the mounting state by comparing the mounting state with the normal state.

  The image sensor 20 closest to the information processing terminal 30 is selected from the plurality of image sensors 20. Then, the mounting state and the normal state of the image sensor 20 closest to the information processing terminal 30 are displayed on the display device 3 of the information processing terminal 30. Therefore, the labor for selecting the target image sensor 20 can be omitted, and the mounting state can be adjusted quickly.

  Further, the determination means and the display control means are provided in the work machine 10. Thus, the configuration of the image sensor 20 can be simplified and the cost of the image sensor 20 can be reduced as compared with the case where at least one of the determination unit and the display control unit is provided in the image sensor 20.

  As described above, the embodiments of the present invention have been described, but they are merely specific examples, and do not particularly limit the present invention. The specific configuration and the like can be appropriately changed in design. Also, the actions and effects described in the embodiments of the invention merely enumerate the most preferable actions and effects resulting from the present invention, and the actions and effects according to the present invention are described in the embodiments of the present invention. It is not limited to what was done.

  For example, when the attachment state acquisition device 2 acquires the orientation of the image sensor 20 as the attachment state of the image sensor 20, the pitching angle, the rolling angle, and the like of the sensor body 21 acquired by the attachment state acquisition device 2 Based on the yawing angle and the regular pitching angle, rolling angle, and yawing angle of the sensor body 21 stored in the storage device 4, it may be determined whether or not the mounting state is appropriate. Further, in the adjustment screen shown in FIG. 4, instead of the image acquired by the image sensor 20 or the outline O of the structure S, the pitching angle, rolling angle, and The numerical value of the yaw angle and the numerical values of the regular pitching angle, rolling angle, and yaw angle of the sensor body 21 stored in the storage device 4 may be displayed at the same time.

REFERENCE SIGNS LIST 1 attachment state display device 2 attachment state acquisition device 3 display device 4 storage device 5 communication device 6 controller (determination means, display control means, selection means)
DESCRIPTION OF SYMBOLS 10 Working machine 11 Lower traveling body 12 Upper revolving superstructure 13 Working device 14 Cab 20 Image sensor 21 Sensor main body 22 Support member 30 Information processing terminal 31 Communication device 32 Terminal controller (position information acquisition means)

Claims (6)

An attachment state display device attached to each of a plurality of different positions of the work machine, each of which displays an attachment state of a plurality of image sensors for acquiring an image,
An attachment state acquisition device that is provided on the image sensor and acquires the attachment state of the image sensor.
Determining means for determining whether or not the attachment state acquired by the attachment state acquisition device is appropriate;
A display device;
Display control means for displaying the determination result of each image sensor by the determination means on the display device at once,
A mounting state display device comprising:   The mounting state display device according to claim 1, wherein the display device is provided in an information processing terminal. A normal state that is the normal mounting state of the image sensor, a storage device that stores for each image sensor,
The said determination means determines whether the said attachment state is appropriate based on the said attachment state which the said attachment state acquisition device acquired, and the said normal state which the said storage device stores. 3. The mounting state display device according to 1 or 2. Further comprising a selection means for selecting the target image sensor from the plurality of image sensors,
The mounting state display device according to claim 3, wherein the display control means causes the display device to display the mounting state and the normal state of the image sensor selected by the selection means, respectively. The display device is provided in the information processing terminal,
The information processing terminal has position information acquisition means for acquiring its own position information,
The mounting state display device according to claim 4, wherein the selection unit selects the image sensor closest to the position information.   The mounting state display device according to claim 1, wherein the determination unit and the display control unit are provided in the work machine.

Images