EP4269710A1

EP4269710A1 - Container measurement system

Info

Publication number: EP4269710A1
Application number: EP21930330.2A
Authority: EP
Inventors: Yukihiro HOSO; Sho Fujiwara; Shingun KYU
Original assignee: Kobelco Construction Machinery Co Ltd
Current assignee: Kobelco Construction Machinery Co Ltd
Priority date: 2021-03-08
Filing date: 2021-12-02
Publication date: 2023-11-01
Also published as: WO2022190484A1; JP7452473B2; CN116761919A; JP2022136849A; US20240112459A1; EP4269710A4; CN116761919B

Abstract

A container measurement system enables accurate calculation of three-dimensional information about a container. The container measurement system includes: a distance image acquisition part (40) provided in a working machine that performs a loading work to a container for acquiring a distance image of the container; a calculation part (66) that calculates, on the basis of the acquired distance image, three-dimensional information including a three-dimensional position and a three-dimensional shape of the container; a storage part (70) that stores three-dimensional shapes of various kinds of containers; a specifying part (68) that specifies a kind of the container subjected to the calculation on the basis of the calculated three-dimensional shape of the container and the three-dimensional shapes of the various kinds of containers stored in the storage part (70); and a correction part (69) that corrects the calculated three-dimensional information about the container.

Description

Technical Field

The present invention relates to a container measurement system for calculating three-dimensional information about a container.

Background Art

For instance, Patent Literature 1 describes a technology of calculating three-dimensional information about a container. The technology described in the literature includes photographing four feature points of the container by a camera and calculating a three-dimensional position for each of the feature points.

Citation List

Patent Literature

Patent Literature 1: Japanese Unexamined Patent Publication No. 2000-064359
The technology described in Patent Literature 1 faces a difficulty in stably calculating the three-dimensional information on a feature point when a camera fails to appropriately photograph the feature point. As a result, it is difficult to stably calculate three-dimensional information about the container.
Under the circumstances, a possible way is to calculate three-dimensional information on a feature point by extracting the feature point with use of a publicly known deep learning technique even when the camera fails to appropriately photograph the feature point as described above. However, even use of the deep learning technique may fail to accurately extract the feature point on some occasions.

Summary of Invention

An object of the present invention is to provide a container measurement system that enables accurate calculation of three-dimensional information about a container.
The present invention provides a container measurement system. The container measurement system includes: a distance image acquisition part provided in a working machine that performs a loading work to a container for acquiring a distance image of the container; a calculation part that calculates, on the basis of the acquired distance image, three-dimensional information including a three-dimensional position and a three-dimensional shape of the container; a storage part that stores three-dimensional shapes of various kinds of containers; a specifying part that specifies a kind of the container subjected to the calculation on the basis of the three-dimensional shape of the container calculated by the calculation part and the three-dimensional shapes of the various kinds of containers stored in the storage part; and a correction part that corrects, on the basis of the three-dimensional shape of the kind of the container specified by the specifying part, the three-dimensional information about the container calculated by the calculation part.

Brief Description of Drawings

Fig. 1 is a side view of a transport vehicle and a working machine in an embodiment of the present invention.
Fig. 2 is a view of the transport vehicle seen from the working machine in the embodiment of the present invention.
Fig. 3 is a block diagram of a container measurement system according to the embodiment of the present invention.
Fig. 4 is an illustration of three-dimensional information about a container in the embodiment of the present invention.
Fig. 5 is an explanatory view of a way of checking a feature point in the embodiment of the present invention.
Fig. 6 is an illustration of a first superimposed image by the container measurement system according to the embodiment of the present invention.
Fig. 7 is an illustration of a second superimposed image by the container measurement system according to the embodiment of the present invention.
Fig. 8 is a flowchart of a container measurement process in the embodiment of the present invention.

Description of Embodiments

Hereinafter, a preferable embodiment of the present invention will be described with reference to the accompanying drawings.

Configuration of a transport vehicle

A container measurement system in an embodiment of the present invention aims at calculating three-dimensional information about a container. Fig. 1 is a side view of a transport vehicle 10 and a working machine 20 in the embodiment. As shown in Fig. 1, the transport vehicle 10 includes a container 12. The transport vehicle 10 is intended for transporting an object (load object) loaded by the working machine 20, and may be a dump truck or another truck.
The transport vehicle 10 includes a main body 11 and the container 12. The main body 11 is travelable on the ground and supports the container 12. The main body 11 has an operating compartment 11a. The container 12 is located in the rear of the operating compartment 11a in the transport vehicle 10.
The container 12 serves as a load bed of the transport vehicle 10. The container 12 has, for example, a shape like a box shape without a cover (see Fig. 2). The container 12 has a flat part 13. The container 12 accommodates the load object. The load object to be accommodated in the container 12 may be, for example, soil and sand, or waste or industrial waste. The container 12 may be movable with respect to the main body 11, or may be fixed to the main body 11. The container 12 may be, for example, a container directly arrangeable on the ground in place of the load bed of the transport vehicle 10.
Hereinafter, directions concerning the transport vehicle 10 are defined as follows, specifically, a direction from the operating compartment 11a to the container 12 is defined as "transport vehicle rearward direction", and a direction from the container 12 to the operating compartment 11a is defined as "transport vehicle forward direction".
Fig. 2 is a view of the transport vehicle 10 seen from the working machine 20 in the embodiment. As shown in Fig. 2, the flat part 13 indicates a flat or substantially flat part of the container 12. The flat part 13 may have a flat shape or substantially flat shape as a whole. The flat part 13 may be uneven, or may have a gently curving surface.
The flat part 13 has a floor 13a, a tail gate wall 13b, side gate walls 13c, and a guard wall 13d. The floor 13a defines a bottom surface (lower surface) of the container 12. The tail gate wall 13b defines a rear surface of the container 12 in a transport vehicle rearward portion of the container 12 and extends upward from the floor 13a in the transport vehicle rearward portion. The side gate walls 13c respectively define left and right surfaces of the container 12 and extend upward from left and right ends of the floor 13a. The guard wall 13d defines a front surface of the container 12 in a transport vehicle forward portion and extends from the floor 13a in the transport vehicle forward portion. The guard wall 13d extends upward further than the side gate walls 13c and further than the tail gate wall 13b.

Configuration of the working machine

As shown in Fig. 1, the working machine 20 performs a work (loading work) of loading a load object to the container 12. The working machine 20 may be configured to, for example, scoop the load object, or sandwich and grasp the load object. Examples of the working machine 20 include a construction machine that performs a construction work, e.g., a hydraulic excavator.
The working machine 20 includes a lower traveling body 21, an upper slewing body 23, and an attachment 25. The lower traveling body 21 causes the working machine 20 to travel. The lower traveling body 21 includes, for example, a crawler. The upper slewing body 23 is slewably mounted on the lower traveling body 21. The upper slewing body 23 has a cab 23a.
The attachment 25 transfers the load object. The attachment 25 has a boom 25a, an arm 25b, and a leading end attachment 25c. The boom 25a is tiltably attached to the upper slewing body 23 (rotatably upward and downward). The arm 25b is rotatably attached to the boom 25a (in a pulling or pushing manner). The leading end attachment 25c is provided on a distal end of the attachment 25 and rotatably attached to the arm 25b. The leading end attachment 25c may be a bucket to scoop the load object (e.g., soil and sand), or a device (e.g., grapple) to sandwich and grasp the load object.

Configuration of the container measurement system

The container measurement system is a system for measuring three-dimensional information about the container 12. The term "three-dimensional information about the container 12" represents information including a three-dimensional position (three-dimensional coordinate) and a three-dimensional shape. The container measurement system measures a position, a direction, and a shape of the container 12 with respect to the working machine 20. As an example, a predetermined origin point or reference point is set in the working machine 20.
Fig. 3 is a block diagram of the container measurement system 30. As shown in Fig. 3, the container measurement system 30 includes a distance image acquisition part 40, a display (display device) 50, a controller 60, and a data storage part 70.
The distance image acquisition part 40 acquires a distance image D showing the container 12 as shown in Fig. 2. The distance image D includes distance information (depth information). Specifically, the distance image D corresponds to a distribution of distances respectively from measurement points around the working machine 20 to the origin point. The distance image acquisition part 40 is provided in the working machine 20. The distance image acquisition part 40 is arranged at such a position as to acquire the distance image D of the container 12 and a periphery of the container 12 when the working machine 20 performs the loading work. For instance, the distance image acquisition part 40 may be arranged or provided inside the cab 23a, may be arranged outside the cab 23a, or may be arranged on a top surface of the cab 23a as exemplified in Fig. 1. The distance image acquisition part 40 may automatically follow the container 12 to acquire the distance image D of the container 12.
As shown in Fig. 3, the distance image acquisition part 40 has a two-dimensional information acquisition section 42 and a three-dimensional information acquisition section 43. The two-dimensional information acquisition section 42 acquires (captures) two-dimensional information (image) showing the container 12. The two-dimensional information acquisition section 42 is, for example, a camera.
The three-dimensional information acquisition section 43 acquires three-dimensional information (point cloud data) about the container 12. The three-dimensional information acquisition section 43 measures a distance from the three-dimensional information acquisition section 43 to each component (to be described in detail later) of the container 12. Specifically, for instance, the three-dimensional information acquisition section 43 may include LiDAR (Light Detection and Ranging, or laser Imaging Detection and Ranging), may include a stereo camera, or may include TOD (Time Of Flight) sensor.
The display 50 may be provided in the operating compartment 11a of the transport vehicle 10, may be provided in the cab 23a of the working machine 20, or may be provided outside the transport vehicle 10 and outside the working machine 20.
The controller 60 executes input and output of a signal, determination, computation like calculation, and storage of information. The controller 60 has an image processing part 61, a position coordinate estimation part 62, a three-dimensional coordinate acquisition part 63, a coordinate conversion part 64, a first superimposing part 65, and a feature point position detection part 66.
The image processing part 61 preprocesses the image acquired by the two-dimensional information acquisition section 42. The preprocessing includes removing a noise from the image and emphasizing an edge in the image.
The position coordinate estimation part 62 estimates a shape of the container 12 from the image preprocessed by the image processing part 61. Specifically, as shown in Fig. 2, the position coordinate estimation part 62 extracts (recognizes, estimates) positions of feature points F (F1 to F8) in the image and positions of links L(L1 to L9) in the image. The position coordinate estimation part 62 extracts each feature point F and each link L by executing, for example, a program for extracting a specific shape from an image. For instance, the program adopts a deep learning technique. In this manner, the position coordinate estimation part 62 acquires two-dimensional information including the image showing the container 12, the position of the feature point F in the image, and the position of the link L in the image.
As shown in Fig. 2, each feature point F corresponds to a point on a corner of the flat part 13. Each link L represents a line connecting two feature points F to each other, and corresponds to, for example, a side line of the flat part 13. In the example shown in Fig. 2, the positions of the feature points F include two positions (F1, F2) on upper ends of the guard wall 13d, two positions (F3, F4) on upper ends of the side gate walls 13c, 13c in the transport vehicle forward portion, and four positions (F5 to F8) on corners of the tail gate wall 13b. For instance, when a lower end of the guard wall 13d is reflected in the image, the position of the lower end of the guard wall 13d may be extracted as the feature point F (see Fig. 1).
In the example shown in Fig. 2, the positions of the links L include positions (L1 to L4) of four sides of a quadrangle in a portion of the guard wall 13d that is located at a higher position than the side gate walls 13c. The positions of the links L include positions (L5, L6) of upper sides of the side gate walls 13c and four sides (L7 to L10) of the tail gate wall 13b, specifically, four sides of a quadrangle substantially overlapping the tail gate wall 13b.
Referring back to Fig. 3, the three-dimensional coordinate acquisition part 63 acquires a three-dimensional coordinate of each piece of the point cloud data captured by the three-dimensional information acquisition section 43. The coordinate conversion part 64 acquires a two-dimensional coordinate of each piece of the point cloud data through perspective projection transformation of the three-dimensional coordinate of each piece of the point cloud data. The coordinate system of the two-dimensional coordinate agrees with a coordinate system of the image acquired by the two-dimensional information acquisition section 42.
The first superimposing part 65 superimposes the two-dimensional coordinate of each piece of the point cloud data acquired by the coordinate conversion part 64 and the shape of the container 12 estimated by the position coordinate estimation part 62 one on another. In this manner, a point corresponding to each feature point F shown in Fig. 2 is acquirable from the point cloud data.
The two-dimensional coordinate of each piece of the point cloud data and the estimated shape of the container 12 may be superimposed one on another by using a coordinate system of a machine coordinate. The machine coordinate has an origin at a specific position in the working machine 20. In use of the coordinate system of the machine coordinate, the coordinate system of the two-dimensional coordinate of each piece of the point cloud data is set to agree with the coordinate system of the machine coordinate, and the coordinate system of the image acquired by the two-dimensional information acquisition section 42 is set to agree with the coordinate system of the machine coordinate (so that the positions of the systems are relative to each other).
The feature point position detection part (calculation part) 66 detects (calculates) a three-dimensional coordinate of each feature point F and each link L shown in Fig. 2. Specifically, the feature point position detection part 66 detects a three-dimensional coordinate of a point corresponding to a specific feature point F shown in Fig. 2 from point cloud data superimposed onto the shape of the container 12 by the first superimposing part 65 to calculate a three-dimensional coordinate of a corner of the flat part 13 corresponding to the position of the feature point F. As a result, three-dimensional information on a corresponding link L connecting the feature point F is also calculated. The calculation leads to acquisition of the three-dimensional information about the container 12 (specifically, the feature point F and the link L).
Fig. 4 is an illustration of three-dimensional information about the container 12 detected by the feature point position detection part 66. In the example shown in Fig. 4, the three-dimensional information has an error in the feature point F1 surrounded by a circular sign "o" among eight feature points F (F1 to F8). It is seen from this perspective that even use of the deep learning technique may fail to accurately extract the feature point on some occasions.
As shown in Fig. 3, the controller 60 has a container shape detection part 67, a container specifying part 68, a feature point correction part 69, a second superimposing part 71, a third superimposing part 72, and a display control part 73.
The container shape detection part 67 detects a three-dimensional shape of the container 12 from the three-dimensional coordinate of the feature point F and the three-dimensional information on the link L each calculated by the feature point position detection part 66.
The container shape detection part 67 checks each of the eight feature points F (F1 to F8) concerning the detected three-dimensional shape of the container 12. The container shape detection part 67 then adopts the detected three-dimensional shape of the container 12 when three or more feature points F respectively have correct three-dimensional coordinates. By contrast, the container shape detection part 67 avoids adopting the detected three-dimensional shape of the container 12 when less than three feature points F have correct three-dimensional coordinates. In this case, the feature point position detection part 66 detects (calculates) a three-dimensional coordinate of each feature point F and each link L again.
A way of checking each of the eight feature points F will be described below. Fig. 5 is an explanatory view of the way of checking each feature point. As shown in Fig. 5, the feature point F1 is defined as a reference, and an inner product of a vector A from the feature point F1 to the feature point F2 and a vector B from the feature point F1 to the feature point F3 is obtained. An angle between the vectors is defined as "0" to determine whether cos θ indicates zero.
The three-dimensional coordinate of the feature point F1 is determined to be correct when the cos θ indicates zero, and the three-dimensional coordinate of the feature point F1 is determined to be incorrect when the cos θ does not indicate zero. Here, the three-dimensional coordinate of each of the eight feature points F has an origin point set at an appropriate position of the working machine 20.
Next, the feature point F2 is defined as a reference, and an inner product of a vector C from the feature point F2 to the feature point F4 and a vector (-A) from the feature point F2 to the feature point F1 is obtained, the vector -A being in the opposite direction to the vector A. Similarly, the aforementioned determination is made. The determination is made for each of the eight features F1 to F8.
Referring back to Fig. 3, the data storage part (storage part) 70 stores three-dimensional shapes of various kinds of containers in advance. The container specifying part (specifying part) 68 specifies a kind of the container 12 subjected to the calculation on the basis of the calculated three-dimensional shape of the container 12 and the three-dimensional shapes of the various kinds of containers stored in the data storage part 70.
The container specifying part 68 specifies the kind of the container by selecting a container having a three-dimensional shape closest to the calculated three-dimensional shape of the container 12 among the three-dimensional shapes of the various kinds of containers stored in the data storage part 70.
In a case where the three-dimensional information about the container calculated from the distance image has an error, no container having a three-dimensional shape matching the three-dimensional shape of the container 12 calculated from the distance image may be found among the three-dimensional shapes of the various kinds of containers stored in the data storage part 70. Even in this case, the kind of the container 12 can be preferably specified by selecting a container having a three-dimensional shape closest to the calculated three-dimensional shape of the container 12.
The feature point correction part (correction part) 69 corrects the calculated three-dimensional information about the container 12 on the basis of the three-dimensional shape of the specified kind of the container among the three-dimensional shapes of the various kinds of containers stored in the data storage part 70. As described above, even when the three-dimensional information about the container 12 calculated from the distance image has an error, correction of the three-dimensional information about the container 12 leads to a success in accurate calculation of the three-dimensional information about the container 12.
The three-dimensional information about the container 12 is adoptable for various controls. For instance, the information is adoptable for automatic operation of the working machine 20 or adoptable for assistive operation of the working machine 20. The information is further adoptable for a control of avoiding a collision between the working machine 20 and the container 12, a control of automatically changing the positions of the working machine 20 and container 12 relative to each other, and a control of notifying an operator of the positions of the working machine 20 and the container 12 relative to each other. In addition, the information is adoptable for calculation of a locus of the leading end attachment 25c from a current position of the leading end attachment 25c to a position (discharge position, e.g., soil discharge position) for discharging the load object.
The second superimposing part (superimposing part) 71 superimposes the corrected three-dimensional information about the container 12 onto the image preprocessed by the image processing part 61 to generate a first superimposed image. As shown in Fig. 2, an AR marker 80 is provided as a marking member on a corner of the flat part 13 that corresponds to the position of the feature point F5. The second superimposing part 71 sets the image at the corrected three-dimensional position of the container 12 with reference to the AR marker 80. The set image and position are superimposed to generate the first superimposed image. The marking member for the setting may be a light (e.g., LED) to be tuned on.
In this manner, the image is set to the corrected three-dimensional position of the container 12 with reference to the AR marker 80. This consequently attains improvement in the accuracy of the setting and achieves a reduction in the burden required for the setting.
Referring back to Fig. 3, the third superimposing part (superimposing part) 72 superimposes the corrected three-dimensional information about the container 12 onto the point cloud data acquired by the three-dimensional coordinate acquisition part 63 to generate a second superimposed image. The third superimposing part 72 superimposes the corrected three-dimensional information about the container 12 onto the point cloud data to generate the second superimposed image after setting the point cloud data at the position of the corrected three-dimensional information. The setting may be executed with reference to the AR marker.
The display control part 73 causes the display 50 to display the first superimposed image generated by the second superimposing part 71. Fig. 6 shows an example of the first superimposed image. The display control part 73 further causes the display 50 to display the second superimposed image generated by the third superimposing part 72. Fig. 7 shows an example of the second superimposed image.
For instance, when the operator remotely manipulates the working machine 20, the operator can recognize a depth to the container 12 by visually confirming the first superimposed image and the second superimposed image.

Operation of the container measurement system

Fig. 8 is a flowchart of a container measurement process. Next, the operation of the container measurement system 30 will be described with reference to Fig. 8.
First, the controller 60 acquires an image from the two-dimensional information acquisition section 42 (step S1), and preprocesses the acquired image (step S2). The controller 60 then estimates a shape of the container 12 from the preprocessed image by using a deep learning technique (step S3).
Simultaneously with steps S1 to S3, the controller 60 acquires point cloud data from the three-dimensional information acquisition section 43 (step S4), and acquires a three-dimensional coordinate of each piece of the acquired point cloud data (step S5). The controller 60 then acquires a two-dimensional coordinate of each piece of the point cloud data through perspective projection transformation of the three-dimensional coordinate of each piece of the point cloud data (step S6).
Subsequently, the controller 60 superimposes the two-dimensional coordinate of each piece of the point cloud data and the shape of the container 12 one on another (step S7). The controller 60 subsequently calculates a three-dimensional coordinate of each feature point F and each link L (step S8). The controller 60 then detects a three-dimensional shape of the container 12 from the three-dimensional coordinate of the feature point F and the three-dimensional information on the link L (step S9).
Next, the controller 60 checks each of the eight feature points F (F1 to F8) (step S10). The controller 60 subsequently determines whether three or more correct feature points F respectively have correct three-dimensional coordinates (step S11). When determining that less than three correct feature points have correct three-dimensional coordinates in step S11 (NO in step S11), the controller 60 returns to step S8 to calculate a three-dimensional coordinate of each feature point F and each link L again.
By contrast, when determining that three or more correct feature points respectively have the three-dimensional coordinates in step S11 (YES in step S11), the controller 60 specifies a kind of the container 12 on the basis of the calculated three-dimensional shape of the container 12 and three-dimensional shapes of various kinds of containers stored in the data storage part 70 (step S 12). The controller 60 then corrects, on the basis of the three-dimensional shape of the specified kind of the container, the calculated three-dimensional information about the container 12 (step S13).
Next, the controller 60 generates a first superimposed image and a second superimposed image (step S14). Specifically, the controller 60 superimposes the corrected three-dimensional information about the container 12 onto the preprocessed image to generate the first superimposed image. The controller 60 further superimposes the corrected three-dimensional information about the container 12 onto the acquired point cloud data to generate the second superimposed image.
Subsequently, the controller 60 causes the display 50 to display the first superimposed image and the second superimposed image (step S15). This process is then finished.
As described heretofore, in the container measurement system 30 according to the embodiment, three-dimensional information including a three-dimensional position and a three-dimensional shape of the container 12 is calculated on the basis of an acquired distance image of the container 12. Further, a kind of the container 12 subjected to the calculation is specified on the basis of the calculated three-dimensional shape of the container 12 and three-dimensional shapes of various kinds of containers stored in the data storage part 70. When the kind of the container 12 is specified, the calculated three-dimensional information about the container 12 is corrected on the basis of the three-dimensional shape of the specified kind of the container. As described above, even when the three-dimensional information about the container 12 calculated from the distance image of the container 12 has an error, correction of the three-dimensional information about the container 12 leads to a success in accurate calculation of the three-dimensional information about the container 12.
The kind of the container 12 subjected to the calculation is specified by selecting a container having a three-dimensional shape closest to the calculated three-dimensional shape of the container 12 among the three-dimensional shapes of the various kinds of containers stored in the data storage part 70. In a case where the three-dimensional information about the container 12 calculated from the distance image of the container 12 has an error, no container having a three-dimensional shape matching the three-dimensional shape of the container 12 calculated from the distance image may be found among the three-dimensional shapes of the various kinds of containers stored in the data storage part 70. Even in this case, selecting a container having a three-dimensional shape closest to the calculated three-dimensional shape of the container 12 succeeds in preferably specifying the kind of the container 12 subjected to the calculation.
The corrected three-dimensional information about the container 12 is superimposed onto the distance image to generate a superimposed image (a first superimposed image, a second superimposed image) after the acquired distance image of the container 12 is set at the corrected three-dimensional position of the container 12. Then, the display 50 displays the superimposed image thereon. For instance, when the operator remotely manipulates the working machine 20, the operator can recognize a depth to the container 12 by visually confirming the superimposed image.
The distance image of the container 12 is set to the corrected three-dimensional position of the container 12 with reference to the marking member (the AR marker 80) provided to the container 12. This consequently attains improvement in the accuracy of the setting and achieves a reduction in the burden required for the setting.
The embodiment of the present invention is described heretofore, but the embodiment is merely described as a detailed example of the present invention without particularly limiting the present invention. It is the matter of design choice for changes in the details of the configuration. Furthermore, the operations and effects described in the embodiments of the present invention are merely listed as optimal operations and effects attained by the present invention, and thus should not be limited thereto.
The present invention provides a container measurement system. The container measurement system includes: a distance image acquisition part provided in a working machine that performs a loading work to a container for acquiring a distance image of the container; a calculation part that calculates, on the basis of the acquired distance image, three-dimensional information including a three-dimensional position and a three-dimensional shape of the container; a storage part that stores three-dimensional shapes of various kinds of containers; a specifying part that specifies a kind of the container subjected to the calculation on the basis of the three-dimensional shape of the container calculated by the calculation part and the three-dimensional shapes of the various kinds of containers stored in the storage part; and a correction part that corrects, on the basis of the three-dimensional shape of the kind of the container specified by the specifying part, the three-dimensional information about the container calculated by the calculation part.
In the configuration, the specifying part may specify the kind of the container subjected to the calculation by selecting a container having a three-dimensional shape closest to the three-dimensional shape of the container calculated by the calculation part among the three-dimensional shapes of the various kinds of containers stored in the storage part.
The configuration may further include: a display device; a superimposing part that superimposes the corrected three-dimensional information about the container onto the distance image to generate a superimposed image after setting the distance image of the container acquired by the distance image acquisition part at a three-dimensional position of the container corrected by the correction part; and a display control part that causes the display device to display the superimposed image.
The configuration may further include a marker provided on the container so as to be captured in the distance image. The superimposing part may set the distance image at the corrected three-dimensional position of the container with reference to the marker.
According to the present invention, three-dimensional information including a three-dimensional position and a three-dimensional shape of a container is calculated on the basis of an acquired distance image of the container. Further, a kind of the container subjected to the calculation is specified on the basis of the calculated three-dimensional shape of the container and three-dimensional shapes of various kinds of containers stored in a storage part. When the kind of the container is specified, the calculated three-dimensional information about the container is corrected on the basis of the three-dimensional shape of the specified kind of the container. Conclusively, even when the three-dimensional information about the container calculated from the distance image of the container has an error, correction of the three-dimensional information about the container leads to a success in accurate calculation of the three-dimensional information about the container.

Claims

A container measurement system, comprising:
a distance image acquisition part provided in a working machine that performs a loading work to a container for acquiring a distance image of the container;

a calculation part that calculates, on the basis of the acquired distance image, three-dimensional information including a three-dimensional position and a three-dimensional shape of the container;

a storage part that stores three-dimensional shapes of various kinds of containers;

a specifying part that specifies a kind of the container subjected to the calculation on the basis of the three-dimensional shape of the container calculated by the calculation part and the three-dimensional shapes of the various kinds of containers stored in the storage part; and

a correction part that corrects, on the basis of the three-dimensional shape of the kind of the container specified by the specifying part, the three-dimensional information about the container calculated by the calculation part.
The container measurement system according to claim 1, wherein the specifying part specifies the kind of the container subjected to the calculation by selecting a container having a three-dimensional shape closest to the three-dimensional shape of the container calculated by the calculation part among the three-dimensional shapes of the various kinds of containers stored in the storage part.
The container measurement system according to claim 1 or 2, further comprising:
a display device;

a superimposing part that superimposes the corrected three-dimensional information about the container onto the distance image to generate a superimposed image after setting the distance image of the container acquired by the distance image acquisition part at a three-dimensional position of the container corrected by the correction part; and

a display control part that causes the display device to display the superimposed image.
The container measurement system according to claim 3, further comprising a marker provided on the container so as to be captured in the distance image, wherein
the superimposing part sets the distance image at the corrected three-dimensional position of the container with reference to the marker.