JP2019196927A - Volume estimation device, volume estimation system and volume estimation method - Google Patents

Abstract

To provide a volume estimation device and volume estimation method that enable a simple configuration to estimate a volume with high accuracy.SOLUTION: A volume estimation system 1 is configured to acquire depth image data D1, D2 including depth information for each pixel from a depth camera 10 photographing a three-dimensional space on a top surface 52 of a conveyor 50 to be loaded with a conveyed object M; on the basis of the depth information of a pixel of the depth image data D1 on an unladen state where the conveyed object M is not loaded on the top surface 52, acquire a distance from the depth camera 10 to an area corresponding to the pixel as an unladen distance L1; on the basis of the depth information of a pixel of the depth image data D2 on a running time of the conveyor 50, acquire a distance from the depth camera 10 to an area corresponding to the pixel as a running time distance L2; and estimate a volume of the conveyed object M on the basis of the unladen distance L1 and the running time distance L2.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a volume estimation device, a volume estimation system, and a volume estimation method. In particular, the present invention relates to a volume estimation device, a volume estimation system, and a volume estimation method that are suitable for estimating the volume of waste or earth and sand transported by a transport device such as a conveyor or a dump truck.

  In recent years, from the viewpoint of effective use of resources, it is required to reuse waste as recyclable resources. For example, waste such as building waste is revived as a valuable resource by sorting by material and crushing it so that it can be easily reused.

  The waste is subjected to sorting and crushing at the treatment plant. In the treatment plant, the amount of waste carried in and out is measured for waste management.

  Patent Document 1 discloses an invention relating to a load amount calculation device for calculating the amount of waste loaded on a vehicle. In this apparatus, a plurality of measurement locations are distributed in advance on the loading surface of the loading platform of the vehicle, and the distance to the upper surface of the load is measured by a plurality of laser distance meters. Then, the distance to the loading surface of the loading platform is measured by at least one laser distance meter, the loading height from the loading surface of the loading platform is obtained, and the loading capacity is calculated.

JP 2012-78114 A

  The load amount calculation device disclosed in Patent Document 1 requires a plurality of laser distance meters according to measurement locations. Therefore, in order to increase the calculation accuracy of the load amount, it has been necessary to increase the number of measurement points and to arrange a large number of laser distance meters. In addition, for example, a laser rangefinder that measures the distance to the loading surface in order to cope with a transport device having a non-flat loading surface, such as a dump truck with an uneven loading surface or a conveyor having a V-shaped cross section. A lot more had to be placed. Therefore, there is a problem that the device configuration is complicated and the device cost is increased.

  Accordingly, an object of the present invention is to provide a volume estimation device, a volume estimation system, and a volume estimation method that can accurately estimate a volume with a simple configuration.

  In order to achieve the above object, a volume estimation apparatus according to an aspect of the present invention is a volume estimation apparatus that estimates a volume of a conveyed object that is conveyed by a conveying apparatus, and the conveying apparatus on which the conveyed object is placed A depth image data acquisition unit that acquires depth image data including depth information for each pixel from a depth camera that captures a three-dimensional space on the transfer surface, and a transfer target to the transfer surface acquired by the depth image data acquisition unit Based on the depth information of the pixels of the depth image data in an unloaded state in which no object is placed, an unloaded distance acquisition unit that acquires a distance from the depth camera to the area corresponding to the pixel as an unloaded distance And corresponding to the pixel from the depth camera based on the depth information of the pixel of the depth image data during operation of the transport device acquired by the depth image data acquisition unit. An operating distance acquisition unit that acquires the distance to the region as an operating distance, and a volume estimation unit that estimates the volume of the conveyed object based on the empty distance and the operating distance. It is characterized by being.

  In the present invention, the area of the region corresponding to the pixel A is acquired as the first area based on the unloaded distance for the pixel of the depth image data in the unloaded state (referred to as “pixel A”), In addition, based on the operating distance for the pixel corresponding to the pixel A (referred to as “pixel B”) in the depth image data during operation of the transport device, the area of the region corresponding to the pixel B is set to the second area. An area acquisition unit that acquires an area is further included, and the volume estimation unit includes the first area, the second area, the unloaded distance for the pixel A, and the operating time for the pixel B. It is preferable to estimate the volume of the transported object based on the distance.

  In the present invention, the volume estimation device calculates the volume of the object to be transported based on the empty distance, the operating distance, and a reference distance to a reference plane orthogonal to the imaging direction of the depth camera. It is preferable to estimate.

  In the present invention, the area of the region corresponding to the pixel A is acquired as the first area based on the unloaded distance for the pixel of the depth image data in the unloaded state (referred to as “pixel A”), And, based on the operating distance for the pixel (referred to as “pixel B”) of the depth image data during operation of the transport device, the area of the region corresponding to the pixel B is acquired as a second area, and An area acquisition unit that acquires, as a third area, an area of a region corresponding to a pixel of the depth image data when the reference plane is captured by the depth camera, and the volume estimation unit includes the first area The transported object based on the second area, the third area, the empty distance for the pixel A, the operating distance for the pixel B, and the reference distance. Estimate the volume of Door is preferable.

  In order to achieve the above object, a volume estimation system according to another aspect of the present invention is a volume estimation system that estimates the volume of a transported object that is transported by a transporting device, on which the transported object is placed. A depth camera that captures a three-dimensional space on the transport surface of the transport device and the volume estimation device are provided.

  In order to achieve the above object, a volume estimation method according to another aspect of the present invention is a volume estimation method for estimating a volume of a transported object transported by a transport apparatus, on which the transported object is placed. The three-dimensional space on the transport surface of the transport device is photographed with a depth camera, depth image data including depth information is obtained for each pixel, and the unloaded object in which no object is placed on the transport surface Based on the depth image data, the distance from the depth camera to the area corresponding to the pixel of the depth image data is obtained as an unloaded distance, and based on the depth image data during operation of the transport device, the depth The distance from the camera to the area corresponding to the pixel of the depth image data is acquired as an operating distance, and the volume of the object to be transported is estimated based on the empty distance and the operating distance. You .

  According to the present invention, depth image data including depth information is acquired for each pixel from a depth camera that captures a three-dimensional space on a conveyance surface of a conveyance device on which an object to be conveyed is placed. Based on the depth information of the pixels of the depth image data in an unloaded state in which an object to be transported is not placed on the transport surface, the distance from the depth camera to the area corresponding to the pixel is acquired as an unloaded distance. Based on the depth information of the pixels of the depth image data during operation of the transport device, the distance from the depth camera to the area corresponding to the pixels is acquired as the operating distance. The volume of the transported object is estimated based on the empty time distance and the operating time distance. Thus, since the depth image data acquired from the depth camera includes depth information for each pixel, the distance from the depth camera to the area corresponding to each pixel can be acquired by using the depth image data. . Therefore, the height of the conveyed object can be obtained from the distance for the area corresponding to each pixel, and the volume of the conveyed object can be estimated. Therefore, it is possible to accurately estimate the volume of the conveyed object with a simple configuration.

It is a figure explaining the volume estimation system which concerns on one Embodiment of this invention. It is a top view of the conveyor which the volume estimation system of FIG. 1 has. It is sectional drawing of the conveyor which the volume estimation system of FIG. 1 has. It is a figure which shows the functional block of the volume estimation apparatus which the volume estimation system of FIG. 1 has. It is a figure which shows typically the three-dimensional space image | photographed with the depth camera which the volume estimation system of FIG. 1 has. It is a flowchart which shows an example of the process (volume estimation process) based on this invention performed in the computer of the volume estimation apparatus of FIG. It is a figure which shows typically a mode that a volume is calculated using the depth image data of an unloaded state. It is a figure which shows typically a mode that a volume is calculated using the depth image data in operation of the conveyor. It is a figure which shows typically a mode that the volume of a to-be-conveyed object is estimated from the volume calculated using the depth image data of the empty state and the depth image data in operation of the conveyor. It is a flowchart which shows another example of the process (volume estimation process) based on this invention performed in the computer of the volume estimation apparatus of FIG. It is the figure which applied the volume estimation system which concerns on this invention to the structure which estimates the volume of the to-be-conveyed object mounted on the loading platform of a dump truck.

  Hereinafter, a volume estimation system according to an embodiment of the present invention will be described with reference to FIGS. The volume estimation system of this embodiment estimates the volume of the waste conveyed by a conveying apparatus. Of course, the present invention can be used for estimating the volume of earth and sand in addition to these.

  FIG. 1 is a diagram illustrating a volume estimation system according to an embodiment of the present invention. 2 and 3 are a plan view and a cross-sectional view of a conveyor included in the volume estimation system of FIG. FIG. 4 is a diagram showing functional blocks of the volume estimation device included in the volume estimation system of FIG. FIG. 5 is a diagram schematically showing a three-dimensional space photographed by a depth camera included in the volume estimation system of FIG. FIG. 6 is a flowchart showing an example of processing (volume estimation processing) according to the present invention, which is executed in the computer of the volume estimation device of FIG. FIG. 7 is a diagram schematically illustrating how the volume is calculated using the depth image data in an unloaded state. FIG. 8 is a diagram schematically illustrating how the volume is calculated using the depth image data during operation of the conveyor. FIG. 9 is a diagram schematically illustrating a state in which the volume of the object to be transported is estimated from the volume calculated using the depth image data in an unloaded state and the volume calculated using the depth image data during operation of the conveyor. is there.

  The volume estimation system 1 according to the present embodiment estimates the volume of an object M to be transported by a conveyor 50 as a transport device shown in FIGS.

  The conveyor 50 has a lane 51 having a V-shaped cross section, and conveys an object to be conveyed M placed on the upper surface 52 of the lane 51 serving as a conveying surface in one direction (conveying direction F). The conveyor 50 is installed on the flat floor G of the building in the waste disposal site. The conveyor 50 is connected to a downstream of a crusher (not shown), and conveys the transported object M, which is waste crushed by the crusher, on the upper surface 52. Various conveyors such as a belt type conveyor and a bucket type conveyor can be adopted as the conveyor 50.

  As shown in FIGS. 1 to 3, the volume estimation system 1 includes a depth camera 10 and a volume estimation device 20.

  The depth camera 10 is disposed above the conveyor 50 and faces the upper surface 52 of the lane 51 in the vertical direction. The depth camera 10 captures a three-dimensional space on the upper surface 52 and generates depth image data. Specifically, the depth camera 10 detects the ultrasonic waves and laser light reflected on the upper surface 52 or the surface of the object M placed on the upper surface 52, and the depth camera 10 for the imaging region R in the three-dimensional space. Information related to the distance from the position (referred to as “depth information”) is acquired, and depth image data including depth information is generated for each pixel. The depth camera 10 sequentially photographs in accordance with the speed at which the lane 51 travels in the transport direction F so that the imaging regions R do not overlap and do not leave a gap.

  The depth image data is composed of a large number of pixels lined up in the vertical direction and the horizontal direction, and is specifically a set of data corresponding to the large number of pixels. In the present embodiment, the depth image data is, for example, a grayscale image in which each pixel is represented by shading according to depth. The depth image data is expressed as light and shade of 256 gradations for each pixel so that it is lighter as the distance is shorter and darker as the distance is longer. The depth image data may be a color image in which each pixel is represented by a color corresponding to the depth. The vertical direction of the depth image data coincides with the conveyance direction F, and the horizontal direction coincides with the direction across the conveyor 50. The depth image data may be data including depth information for each pixel, and the visual recognition mode is arbitrary. In the present embodiment, the depth image data has a configuration in which a large number of pixels are arranged in a row in the horizontal direction (that is, a plurality of rows). However, in addition to this, only one row in which a large number of pixels are arranged in the horizontal direction is provided. It may have a configuration (that is, a single row).

  FIG. 4 schematically shows a three-dimensional space photographed by the depth camera 10. The depth camera 10 is located at the apex (position P0) of the quadrangular pyramid-shaped three-dimensional space, and the depth image data is generated by photographing the three-dimensional space. Considering the case where the planar imaging areas R1 and R2 are imaged in the three-dimensional space, the imaging area R1 at the position P1 that is a distance L1 from the position P0 is a position that is a distance L2 (L2 <L1) from the position P0. The area is larger than the planar imaging region R2 in P2. Therefore, the area r1 corresponding to the pixel of the depth image data D1 generated when the shooting area R1 is shot is larger in area than the area r2 corresponding to the pixel of the depth image data D2 generated when the shooting area R2 is shot. Is big. This also applies to one piece of depth image data D. That is, the area corresponding to the pixel including depth information indicating a position far from the depth camera 10 has a larger area than the area corresponding to the pixel including depth information indicating a position close to the depth camera 10. Therefore, it can be seen that the area corresponding to the pixel of the depth image data D has an area corresponding to the distance from the depth camera 10. The area r1 corresponding to the pixel (pixel A) of the depth image data D1 in the shooting area R1 is set as the bottom, and the area r2 corresponding to the pixel (pixel B) corresponding to the pixel A in the depth image data D2 of the shooting area R2 is set as the top. When the photographing region R1 is photographed by obtaining the volume of the quadrangular pyramid T with the difference value between the distance L1 from the depth camera 10 to the pixel A and the distance L2 from the depth camera 10 to the pixel B as the bottom. And the amount of change in volume in the three-dimensional space when the imaging region R2 is imaged.

  By applying this principle and using the depth image data, the volume of the transported object M in the three-dimensional space can be estimated.

  The 1st volume estimation method is shown. (1) The depth image data D1 obtained by photographing the state without the transported object M and the depth image data D2 obtained by capturing the state with the transported object M are acquired. (2) For each pixel of the depth image data D1 and the depth image data D2, the distance from the depth camera 10 to the region corresponding to the pixel is acquired based on the depth information included in the pixel. (3) The area (first area) of the region corresponding to the pixel (pixel A) of the depth image data D1 is acquired based on the distance from the depth camera 10. (4) Based on the distance from the depth camera 10, the area (second area) of the region corresponding to the pixel (pixel B) corresponding to the pixel A in the depth image data D2 is acquired. (5) The difference value (height) between the distance to the area corresponding to the pixel A and the distance to the area corresponding to the pixel B is acquired. Then, (6) the volume of the quadrangular pyramid is calculated using the first area (lower base area), the second area (upper base area), and the height. (7) The volume of the square frustum T is calculated for all pixels, and the total value of these volumes is set as the estimated volume of the conveyed object M. In the first volume estimation method, when the angle of view of the depth camera 10 is relatively narrow, it is assumed that the areas of the regions corresponding to the pixels of the depth image data D1 and the depth image data D2 are all the same. The volume may be simply estimated based on (that is, the distance from the depth camera 10).

  The 2nd volume estimation method is shown. In the second volume estimation method, the volume of the object is estimated by further using a reference plane orthogonal to the imaging direction of the depth camera 10. It is assumed that the reference plane is separated from the depth camera 10 by a reference distance. (1) The depth image data D1 obtained by photographing the state without the transported object M and the depth image data D2 obtained by capturing the state with the transported object M are acquired. (2) For each pixel of the depth image data D1 and the depth image data D2, the distance from the depth camera 10 to the area corresponding to the pixel is acquired based on the depth information included in the pixel. (3) The area (first area) of the region corresponding to the pixel (pixel A) of the depth image data D1 is acquired based on the distance from the depth camera 10. (4) Based on the distance from the depth camera 10, the area (second area) of the region corresponding to the pixel (pixel B) of the depth image data D2 is acquired. (5) The area (third area) of the region corresponding to the pixel of the depth image data D3 when the reference plane is captured by the depth camera 10 is acquired. (6) The reference distance, the distance to the area corresponding to the pixel A, and the difference value (first height) are acquired. (7) A difference value (second height) between the reference distance and the distance to the area corresponding to the pixel B is acquired. (8) The volume of the quadrangular pyramid is calculated using the first area (upper base area), the third area (lower base area), and the first height. (7) The volume of the truncated pyramid is calculated for all the pixels of the depth image data D1, and the total value (first total value) of these volumes is calculated. (9) The volume of the truncated pyramid is calculated using the second area (upper base area), the third area (lower base area), and the second height. (10) The volume of the truncated pyramid is calculated for all the pixels of the depth image data D2, and the total value (second total value) of these volumes is calculated. (11) The difference value between the first total value and the second total value is set as the estimated volume of the conveyed object M.

  In the second volume estimation method, when the angle of view of the depth camera 10 is relatively narrow, the areas corresponding to the pixels of the depth image data D1, the depth image data D2, and the depth image data D3 are all the same. As described above, the volume may be simply estimated based on the first height and the second height (that is, the distance from the depth camera 10). In this way, the difference value (first height) between the reference distance Lg and the unloaded distance L1 is calculated for the region corresponding to the pixel of the unloaded depth image data D1, and the working depth image By calculating the difference value (second height) between the reference distance Lg and the operating distance L2 for the area corresponding to the pixel of the data D2, the pixel of the unloaded depth image data D1 and the operating depth image data are calculated. The volume of the conveyed object M can be estimated more easily without considering the correspondence with the pixel D2.

  In the volume estimation apparatus 20 of the present embodiment, the latter second volume estimation method is adopted, and the floor G at the position Pg that is a reference distance Lg away from the position P0 of the depth camera 10 is used as the reference plane.

  The volume estimation apparatus 20 is configured by a tablet terminal, for example. The volume estimation device 20 may be configured with a personal computer or the like. The volume estimation device 20 includes a computer that is a central processing unit (CPU), a memory used as a work area of the computer, a storage device that stores programs and data executed by the computer, and a display with a touch panel. ing. Moreover, the volume estimation apparatus 20 has a communication apparatus which communicates with the depth camera 10 by short-distance radio, a communication cable, etc., for example.

  The computer of the volume estimation device 20 executes a program stored in a storage device or the like, thereby obtaining a depth image data acquisition unit 21, an unloaded distance acquisition unit 22, an operating distance acquisition unit 23, an area acquisition unit 24, and , Functions as the volume estimation unit 25.

  The depth image data acquisition unit 21 acquires depth image data generated by the depth camera 10. The depth image data acquisition unit 21 acquires the depth image data D1 in an unloaded state in which the object M is not placed on the upper surface 52 of the conveyor 50 and the depth image data D2 during operation of the conveyor 50.

  The unloaded distance acquisition unit 22 calculates the distance from the depth camera 10 to the region corresponding to the pixel based on the depth information of the pixel of the unloaded depth image data D1 acquired by the depth image data acquisition unit 21. Acquired as an unloaded distance L1. The unloaded distance acquisition unit 22 acquires the unloaded distance L1 for all the pixels included in the depth image data D1.

  The operating distance acquisition unit 23 is a distance from the depth camera 10 to the region corresponding to the pixel based on the depth information of the pixel of the depth image data D2 during operation of the conveyor 50 acquired by the depth image data acquisition unit 21. Is obtained as the operating distance L2. The operating distance acquisition unit 23 acquires the operating distance L2 for all the pixels included in the depth image data D2.

  The area acquisition unit 24 acquires the area of the region corresponding to this pixel as the first area S1 based on the unloaded distance L1 for the pixel of the unloaded depth image data D1. The area acquisition unit 24 acquires the area of the region corresponding to this pixel as the second area S2 based on the operating distance L2 for the pixel of the depth image data D2 during operation of the conveyor 50. The area acquisition unit 24 acquires, as the third area S3, the area of the region corresponding to the pixel of the depth image data D3 when the depth camera 10 captures the floor G that is the reference plane. The area acquisition unit 24 acquires the first area S1, the second area S2, and the third area S3 for all the pixels of the depth image data D1, the depth image data D2, and the depth image data D3.

  The area acquisition unit 24 has a function for calculating the area of the region corresponding to the pixels of the depth image data D1, D2, and D3 by inputting the distance from the position P0 of the depth camera 10. Alternatively, the area acquisition unit 24 may have a table that stores the distance from the position P0 of the depth camera 10 and the area of the region corresponding to the pixel of the depth image data D in association with each other. In the present embodiment, since the floor surface G is a flat surface, the third areas S3 for all the pixels included in the depth image data D3 are the same. The area acquisition unit 24 is distorted in depth image data caused by photographing with the depth camera 10 (for example, deformed into a barrel shape from the central part to the peripheral part due to the influence of the lens shape, or the peripheral part of the depth image data is extended. The area of each pixel may be acquired by correcting distortion or the like.

  The volume estimation unit 25 includes a first area S1 for the pixel (pixel A) in the depth image data D1, a second area S2 for the pixel (pixel B) corresponding to the pixel A in the depth image data D2, and a third Based on the area S3, the empty distance L1 for the pixel A, the operating distance L2 for the pixel B, and the reference distance Lg, the volume of the conveyed object M is estimated.

  Specifically, the volume estimation unit 25 acquires a difference value between the reference distance Lg and the unloaded distance L1 for the pixel A as the first height H1. The volume estimation unit 25 calculates the volume of the truncated pyramid T1 using the first area S1 for the pixel A as the upper base area, the third area S3 as the lower base area, and the first height H1 as the height. To do. The volume estimation unit 25 calculates the volume of the quadrangular frustum T1 for all the pixels of the depth image data D1, and calculates the total value (first total value K1) of these volumes.

  In addition, the volume estimation unit 25 acquires a difference value between the reference distance Lg and the operating distance L2 for the pixel B as the second height H2. The volume estimation unit 25 calculates the volume of the truncated pyramid T2 using the second area S2 for the pixel B as the upper base area, the third area S3 as the lower base area, and the second height H2 as the height. To do. The volume estimation unit 25 calculates the volume of the quadrangular frustum T2 for all the pixels of the depth image data D2, and calculates the total value (second total value K2) of these volumes.

  Then, the volume estimation unit 25 calculates the volume of the conveyed object M by subtracting the first total value K1 from the second total value K2.

  The volume estimation device 20 has a display operation unit 26. The display operation unit 26 includes a display device such as a liquid crystal display or an organic EL display, and a touch panel overlaid on the display device. The display operation unit 26 has a software switch combined with an operation icon displayed on the display device and a touch panel, and the volume estimation device 20 is operated by this software switch. On the display device of the display operation unit 26, for example, an image based on the depth image data D1 and the depth image data D2 and the volume of the transported object M estimated by the volume estimation unit 25 are displayed.

  Next, an example of the process (volume estimation process) according to the present invention executed in the computer of the volume estimation apparatus 20 of the volume estimation system 1 described above will be described with reference to the flowchart of FIG.

  The computer of the volume estimation device 20 generates depth image data D1 generated by photographing an empty state in which nothing is placed on the upper surface 52 of the conveyor 50 (no object M is placed) by the depth camera 10. Is acquired (S110). The unloaded distance L1 and the first area S1 are acquired for the region r1 corresponding to each pixel of the depth image data D1 (S120, S130). A third area S3 is acquired for a region r3 corresponding to each pixel of the depth image data D3 when the floor surface G is photographed by the depth camera 10 (S140). For the region r1 corresponding to each pixel of the depth image data D1, a difference value between the reference distance Lg and the unloaded distance L1 is calculated and acquired as the first height H1 (S150). Using the first area S1 (upper base area), the third area S3 (lower base area), and the first height H1 (height), the volume of the quadrangular frustum T1 is calculated (S160). Iterate until the volume of the square frustum T1 is calculated for all the pixels of the depth image data D1 (N in S170), and when the calculation is completed (Y in S170), the total value of these volumes (first total value K1) is calculated. (S180). FIG. 7 schematically shows how the volume of the quadrangular frustum T1 is calculated. The shaded portion in FIG. 7 is an aggregate of the square frustum T1.

  Next, the computer of the volume estimation apparatus 20 waits for the start of operation of the conveyor 50 (N in S190). When the conveyor 50 starts operating (Y in S190), the depth camera 10 captures the operating conveyor 50. The generated depth image data D2 is acquired (S200). The operating distance L2 and the second area S2 are acquired for the region r2 corresponding to each pixel of the depth image data D2 (S210, S220). The third area S3 is acquired for the region r3 corresponding to the pixel of the depth image data D3 when the floor surface G is photographed by the depth camera 10 (S230). For the region r2 corresponding to each pixel of the depth image data D2, the difference value between the reference distance Lg and the operating distance L2 is calculated and acquired as the second height H2 (S240). Using the second area S2 (upper base area), the third area S3 (lower base area), and the second height H2 (height), the volume of the truncated pyramid T2 is calculated (S250). Iterate until the volume of the square frustum T2 is calculated for all the pixels of the depth image data D2 (N in S260), and when the calculation is completed (Y in S260), the total value of these volumes (second total value K2) is calculated. (S270). FIG. 8 schematically shows how the volume of the quadrangular frustum T2 is calculated. The shaded portion in FIG. 8 is an aggregate of the square frustum T2.

  The computer of the volume estimation device 20 calculates the volume of the conveyed object M by subtracting the first total value K1 from the second total value K2 (S280). FIG. 9 schematically shows how the volume of the conveyed object M is calculated. The shaded portion in FIG. 9 is an aggregate of a quadrangular frustum T3 obtained by subtracting the corresponding quadrangular frustum T1 from the quadrangular frustum T2.

  Then, the computer of the volume estimation apparatus 20 calculates the volume of the conveyed object M by repeating S200 to S280 until the conveyor 50 stops operating (N in S290), and when the conveyor 50 stops operating (Y in S290). Then, the process of this flowchart ends.

  As described above, according to the volume estimation system 1 of the present embodiment, depth image data D1 including depth information for each pixel from the depth camera 10 that captures the three-dimensional space on the upper surface 52 of the conveyor 50 on which the object M is placed. , D2 is acquired. Based on the depth information of the pixel of the depth image data D1 in the unloaded state where the object M is not placed on the upper surface 52, the distance from the depth camera 10 to the area corresponding to the pixel is acquired as the unloaded distance L1. To do. Based on the depth information of the pixel of the depth image data D2 during operation of the conveyor 50, the distance from the depth camera 10 to the area corresponding to the pixel is acquired as the operating distance L2. Based on the unloaded distance L1 and the operating distance L2, the volume of the conveyed object M is estimated. Since it did in this way, since the depth image data D1 and D2 acquired from the depth camera 10 contain depth information for every pixel, it corresponds to each pixel from the depth camera 10 by using the depth image data D1 and D2. It is possible to acquire the distances to the area to be operated (empty distance L1, operating distance L2). Therefore, the height of the transported object M can be obtained from the distance for the area corresponding to each pixel, and the volume of the transported object M can be estimated. Therefore, it is possible to accurately estimate the volume of the conveyed object with a simple configuration.

  Further, the area of the region corresponding to the pixel A is acquired as the first area S1 based on the unloaded distance L1 for the pixel of the depth image data D1 in the unloaded state (referred to as “pixel A”). Based on the operating distance L2 for the pixel (referred to as “pixel B”) of the depth image data D2 during operation of the conveyor 50, the area of the region corresponding to the pixel B is acquired as the second area S2. The area of the region corresponding to the pixel of the depth image data D3 when the floor surface G is photographed by the depth camera 10 is acquired as the third area S3. And based on the first area S1, the second area S2, the third area S3, the empty distance L1 for the pixel A, the operating distance L2 for the pixel B, and the reference distance Lg, The volume of the conveyed object M is estimated. In the depth image data D1, D2, and D3, the area of the region corresponding to each pixel changes according to the distance from the depth camera 10. That is, the closer to the depth camera 10, the smaller the area corresponding to the pixel, and the farther from the depth camera 10, the larger the area corresponding to the pixel. Therefore, based on the distance from the depth camera 10 (empty distance L1, operating distance L2, and reference distance Lg), the area of the region corresponding to the pixels of the depth image data D1, D2, D3 (first area S1, By obtaining the second area S2 and the third area S3) and estimating the volume of the conveyed object M using these areas, the estimation accuracy can be effectively increased. Furthermore, the first area S1 of the region corresponding to the pixel of the depth image data D1 in the unloaded state and the third area S3 corresponding to the pixel of the depth image data D3 of the floor surface G are calculated, and the depth image data in operation By calculating the second area S2 of the region corresponding to the pixel of D2 and the third area S3 corresponding to the pixel of the depth image data D3 on the floor G, the pixel of the depth image data D1 in an unloaded state is in operation. The volume of the conveyed object can be estimated more easily without considering the correspondence with the pixels of the depth image data D2.

  In the volume estimation system 1 according to the above-described embodiment, the second volume estimation method is adopted. However, the first volume estimation method may be adopted. Refer to the flowchart of FIG. 10 for another example of the processing (volume estimation processing) according to the present invention executed by the computer of the volume estimation device 20 when the first volume estimation method is adopted in the volume estimation system 1. To explain.

  The computer of the volume estimation device 20 generates depth image data D1 generated by photographing an empty state in which nothing is placed on the upper surface 52 of the conveyor 50 (no object M is placed) by the depth camera 10. Is acquired (T110). The unloaded distance L1 and the first area S1 are acquired for the region r1 corresponding to each pixel of the depth image data D1 (T120, T130).

  Next, the computer of the volume estimation apparatus 20 waits for the operation start of the conveyor 50 (N in T140), and when the conveyor 50 starts operating (Y in T140), the depth camera 10 captures the operating conveyor 50. The generated depth image data D2 is acquired (T150). The operating distance L2 and the second area S2 are acquired for the region r2 corresponding to each pixel of the depth image data D2 (T160, T170). The unloading distance L1 for the region r1 corresponding to the pixel (pixel A) in the depth image data D1, and the operating distance L2 for the region r2 corresponding to the pixel (pixel B) corresponding to the pixel A in the depth image data D2 Is obtained as the height H (T180). Using the first area S1 (lower bottom area), the second area S2 (upper bottom area), and the height H, the volume of the quadrangular frustum T is calculated (T190). Repeat until the volume of the square frustum T is calculated for all the pixels of the depth image data D2 (N in T200). When the calculation is completed (Y in T200), the volume of the conveyed object M is calculated by adding these volumes. (T210).

  And the computer of the volume estimation apparatus 20 calculates the volume of the to-be-conveyed object M by repeating T150-T210 until the conveyor 50 stops operation (N in T220), and if the conveyor 50 stops operation (Y in T220). Then, the process of this flowchart ends.

  According to such a configuration, the area of the region corresponding to the pixel A is set to the first area S1 based on the unloaded distance L1 for the pixel (referred to as “pixel A”) of the depth image data D1 in the unloaded state. Get as. Based on the operating distance L2 for the pixel (referred to as “pixel B”) corresponding to the pixel A in the depth image data D2 during operation of the conveyor 50, the area of the region corresponding to the pixel B is acquired as the second area S2. To do. Then, based on the first area S1, the second area S2, the empty distance L1 for the pixel A, and the operating distance L2 for the pixel B, the volume of the conveyed object M is estimated. In the depth image data D <b> 1 and D <b> 2, the area of the region corresponding to each pixel changes according to the distance from the depth camera 10. That is, the closer to the depth camera 10, the smaller the area corresponding to the pixel, and the farther from the depth camera 10, the larger the area corresponding to the pixel. Therefore, based on the distance from the depth camera 10 (empty distance L1 and operating distance L2), the areas (first area S1 and second area S2) corresponding to the pixels of the depth image data D1 and D2 are determined. By acquiring and estimating the volume of the conveyed object using these areas, the estimation accuracy can be effectively increased.

  Further, in the above-described embodiment, the volume of the object M to be transported that is transported by the conveyor 50 as the transport device is estimated, but besides this, for example, as shown in FIG. The volume estimation system of the present invention may be applied to the estimation of sediment such as earth and sand loaded on the loading platform 71. In this configuration, the inner surface of the loading platform 71 of the dump truck 70 corresponds to the transport surface. Also in this structure, there exists an effect similar to the volume estimation system 1 mentioned above.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the configurations of these embodiments. Those in which those skilled in the art appropriately added, deleted, and changed the design of the above-described embodiments, or combinations of the features of the embodiments as appropriate, also fall within the scope of the present invention without departing from the spirit of the present invention. Included in the range.

DESCRIPTION OF SYMBOLS 1 ... Volume estimation system 10 ... Depth camera 20 ... Volume estimation apparatus 21 ... Depth image data acquisition part 22 ... Unloaded distance acquisition part 23 ... Operating distance acquisition part 24 ... Area acquisition part 25 ... Volume estimation part 26 ... Display operation Section 50 ... Conveyor 51 ... Lane 52 ... Upper surface 70 ... Dump truck 71 ... Loading platform F ... Conveying direction G ... Floor surface R, R1, R2 ... Shooting area L1 ... Empty distance L2 ... Operating distance Lg ... Reference distance S1 ... 1st area S2 ... 2nd area S3 ... 3rd area H1 ... 1st height H2 ... 2nd height

Claims (6)

A volume estimation device for estimating a volume of an object to be conveyed conveyed by a conveyance device,
A depth image data acquisition unit that acquires depth image data including depth information for each pixel from a depth camera that captures a three-dimensional space on a transport surface of the transport device on which the transported object is placed;
From the depth camera to the area corresponding to the pixel based on the depth information of the pixel of the depth image data in an unloaded state in which the object to be conveyed is not placed on the conveyance surface acquired by the depth image data acquisition unit An unloading distance acquisition unit that acquires the distance of
Based on the depth information of the pixel of the depth image data during operation of the transport device acquired by the depth image data acquisition unit, the distance from the depth camera to the area corresponding to the pixel is acquired as the operating distance An operating distance acquisition unit;
A volume estimation device comprising: a volume estimation unit configured to estimate a volume of the object to be transported based on the empty distance and the operating distance. Based on the unloaded distance for the pixel of the depth image data in the unloaded state (referred to as “pixel A”), the area of the region corresponding to the pixel A is acquired as a first area, and the transport Based on the operating distance for the pixel corresponding to the pixel A (referred to as “pixel B”) in the depth image data during operation of the apparatus, the area of the region corresponding to the pixel B is acquired as the second area. It further has an area acquisition unit,
The volume estimator is configured to determine whether the object to be transported is based on the first area, the second area, the empty distance for the pixel A, and the operating distance for the pixel B. The volume estimation apparatus according to claim 1, wherein the volume is estimated.   The volume estimation device estimates the volume of the object to be transported based on the empty distance, the operating distance, and a reference distance to a reference plane orthogonal to the imaging direction of the depth camera. The volume estimation apparatus according to claim 1, wherein Based on the unloaded distance for the pixel of the depth image data in the unloaded state (referred to as “pixel A”), the area of the region corresponding to the pixel A is acquired as a first area, and the transport The area of the region corresponding to the pixel B is acquired as a second area based on the operating distance for the pixel (referred to as “pixel B”) of the depth image data during operation of the apparatus, and the depth camera An area acquisition unit that acquires an area of a region corresponding to a pixel of the depth image data when the reference plane is imaged as a third area;
The volume estimation unit includes the first area, the second area, the third area, the empty distance for the pixel A, the operating distance for the pixel B, and the reference distance. The volume estimation device according to claim 3, wherein the volume of the conveyed object is estimated based on A volume estimation system for estimating a volume of an object to be conveyed conveyed by a conveying device,
A depth camera that captures a three-dimensional space on a transport surface of the transport device on which the transported object is placed;
A volume estimation system comprising: the volume estimation device according to any one of claims 1 to 4. A volume estimation method for estimating a volume of an object to be conveyed conveyed by a conveying device,
Taking a three-dimensional space on the transport surface of the transport device on which the transported object is placed with a depth camera, and obtaining depth image data including depth information for each pixel,
Based on the depth image data in an empty state in which an object to be conveyed is not placed on the conveyance surface, a distance from the depth camera to an area corresponding to a pixel of the depth image data is acquired as an unloaded distance.
Based on the depth image data during operation of the transport device, to obtain the distance from the depth camera to the area corresponding to the pixels of the depth image data as the operating distance,
A volume estimation device that estimates the volume of the object to be transported based on the empty distance and the operating distance.

Images