JP2020060451A - Luggage space monitoring system and luggage space monitoring method - Google Patents

Abstract

To grasp an available state of a floor surface for arranging luggage by grasping an available area to be occupied by the luggage in a luggage space.SOLUTION: A luggage space monitoring device 10 includes: an acquisition part 11 for acquiring a distance image acquired by each of two or more sensors provided other than the floor surface of a luggage space; a generation part 12 for generating luggage voxels showing a position in the luggage space of the surface of the luggage in the luggage space and floor surface voxels showing the position of the floor surface on the basis of the distance image, arrangement positions of the sensors and imaging directions; an interpolation part 13 for generating interpolation voxels in a space area to be a dead angle from the sensors; an estimation part 14 for estimating a space area occupied by the luggage, luggage area information showing a space area having a possibility of being occupied by the luggage, and available floor information on a floor surface on which the luggage can be arranged on the basis of the luggage voxels, the interpolation voxels and the floor surface voxels; and an output part 15 for outputting the luggage area information and the available floor information.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a luggage compartment monitoring system and a luggage compartment monitoring method.

  In order to improve the efficiency of physical distribution by vehicles, it is required to accurately grasp the state of the cargo in the luggage compartment of the vehicle. Especially, the capacity of the luggage and the floor of an empty floor where the luggage is not placed. It is required to grasp the area. For example, in order to visually recognize the situation in the luggage compartment, an optical camera is provided in the luggage compartment to acquire an image of the luggage compartment. Further, there is known a technique of measuring the volume of a measurement target using three three-dimensional measurement sensors. (For example, refer to Patent Document 1).

JP, 2003-247805, A

  With the conventional method of acquiring an image with an optical camera, it is not possible to accurately grasp the capacity of the luggage and the vacant floor area in the luggage compartment. Further, when the technique described in Patent Document 1 is applied and a measurement sensor is provided in the luggage compartment to measure the volume of the luggage, many sensors are required, and the measurement is complicated, The cost for measurement was high. On the other hand, in order to improve the efficiency of logistics, it is also necessary to grasp the exact volume of the luggage, but it is possible to accurately and easily grasp how much luggage can be loaded in the luggage compartment where the luggage is loaded. It is also required to do.

  Therefore, the present invention accurately grasps the state of the cargo in the luggage compartment, and in particular, accurately grasps the space area occupied by the luggage in the luggage compartment, so as to determine the availability of the floor surface for further luggage distribution. The purpose is to grasp.

  A luggage compartment monitoring system according to an aspect of the present invention is a luggage compartment monitoring system that monitors an internal state of a luggage compartment, and includes at least two sensors provided on a surface other than a floor forming a lower surface of the luggage compartment. Based on the acquisition unit that acquires the distance image acquired by each, information on the distance to the object shown in the distance image, and the placement position and imaging direction of the sensor in the luggage space, the luggage existing in the luggage compartment. A parcel voxel that is information indicating the position of the surface of the inside of the luggage compartment space, and a floor voxel that is the information indicating the position of the floor surface, and a space area that is a blind spot from the sensor in the luggage compartment space. Based on the interpolator that generates the interpolated voxels and the luggage voxels, the interpolated voxels, and the floor voxels, the space area occupied by the luggage in the space of the luggage compartment and the luggage can be occupied. Comprising cargo area information indicating a spatial region where there is a gender, and an estimation unit that estimates a free floor information on the floor can be placed luggage, and an output unit for outputting the cargo area information and free floor information.

  A luggage compartment monitoring method according to an aspect of the present invention is a luggage compartment monitoring method in a luggage compartment monitoring system that monitors an internal state of a luggage compartment, and is provided at least on a floor other than a floor forming a lower surface of the luggage compartment. Based on the acquisition step of acquiring the distance image acquired by each of the two or more sensors, the information on the distance to the object shown in the distance image, and the arrangement position and the imaging direction of the sensor in the luggage space, A generation step of generating a luggage voxel that is information indicating the position of the surface of luggage existing in the room in the luggage compartment space and a floor voxel that is information indicating the position of the floor surface, and a blind spot from the sensor in the luggage compartment space. Interpolation step to generate interpolated voxels in the space area that becomes and the baggage is occupied in the space of the luggage compartment based on the baggage voxels, the interpolated voxels and the floor voxels. Outputting baggage area information and vacant floor information, and an estimation step of estimating baggage area information indicating a space area that may be occupied by baggage and luggage, and vacant floor information regarding a floor surface on which luggage can be placed And an output step to perform.

  A luggage compartment monitoring program according to an aspect of the present invention is a luggage compartment monitoring program for causing a computer to function as a luggage compartment monitoring system that monitors an internal state of the luggage compartment. An acquisition function of acquiring a distance image acquired by each of at least two or more sensors provided on other than the constituent floor surface, information on a distance to an object shown in the distance image, and a sensor in the luggage space. Generation that generates a luggage voxel that is information indicating the position in the luggage space of the surface of luggage existing in the luggage space and a floor voxel that is information indicating the position of the floor surface, based on the arrangement position and the imaging direction The function and the interpolation function to generate interpolation voxels from the sensor in the space area that becomes a blind spot in the luggage space, and based on the baggage voxel, the interpolation voxel and the floor voxel. And estimating the space area information indicating the space area occupied by the luggage and the space area possibly occupied by the luggage in the space of the luggage compartment, and the empty floor information regarding the floor surface on which the luggage can be arranged. An estimation function and an output function for outputting luggage area information and empty floor information are realized.

  According to the above-mentioned form, a baggage voxel that is information indicating the position of the baggage surface can be obtained based on the distance image. Further, based on the information indicating the blind spots of the luggage voxel and the sensor, etc., it is possible to obtain an interpolated voxel indicating a space area in which the luggage may not be practically placed. It is possible to obtain baggage region information indicating the space region that is occupied and the space region that may be occupied by the package. Therefore, it becomes possible to output the vacant floor information indicating the floor surface on which the luggage can be placed.

  In the luggage compartment monitoring system according to another aspect, the interpolation voxel may include a luggage interpolation voxel that occupies a space region from the luggage voxel to a floor surface vertically below the luggage voxel.

  Even if there is a space below the upper surface of the luggage that is not occupied by the luggage due to the shape of the luggage, it is not possible to place more luggage in that space. According to the above aspect, the luggage interpolation voxel is generated by treating such a space area in which the luggage cannot be arranged as a space area substantially occupied by the luggage. You can properly understand the availability of the floor.

  In a luggage compartment monitoring system according to another aspect, at least two or more sensors include a first sensor and a second sensor, and the acquisition unit includes a first distance image and a second distance image acquired by the first sensor. The second distance image acquired by the sensor of the first distance image is acquired, and the estimation unit determines the first baggage interpolation voxel that is the baggage interpolation voxel based on the first distance image and the baggage voxel or the floor based on the second distance image. When the area voxel overlaps, the baggage area information and the empty floor information may be estimated by assuming that the first baggage interpolation voxel is arranged at the overlapping position.

  The space area in which the baggage interpolation voxel is generated is an area in which no further baggage can be placed. According to the above aspect, even if the space voxel or the floor voxel based on the second range image is generated, if the space region overlaps with the baggage interpolation voxel based on the first range image in the space region. Since the space area is an area in which the luggage cannot be arranged, it is treated as the arrangement of the luggage interpolation vector. Therefore, it is possible to properly grasp the availability of the floor surface for distributing the luggage.

  In the luggage compartment monitoring system according to another aspect, the interpolation voxel includes a blind spot interpolation voxel that occupies a space area of a predetermined height above the floor surface that is a blind spot from the sensor due to the angle of view of the luggage or the sensor. Good.

  Since there is a possibility that luggage is placed on the floor that is the blind spot of the sensor, it is inappropriate to treat the floor that is the blind spot as a floor on which more luggage can be placed. According to the above-described aspect, the blind spot interpolation voxel is generated by treating the space area of the predetermined height above the floor surface, which is the blind spot of the sensor, as the space area substantially occupied by the luggage, and thus the luggage is It is possible to properly understand the availability of the floor surface for arranging.

  In the luggage compartment monitoring system according to another aspect, the estimation unit determines that the first blind spot interpolation voxel, which is a blind spot interpolation voxel based on the first distance image, and the luggage voxel or floor voxel based on the second distance image. In the case of overlap, the baggage voxel or floor voxel based on the second distance image may be arranged at the overlapping position, and the baggage area information and the empty floor information may be estimated.

  The spatial area in which the blind spot interpolation voxels are generated is an area in which it is unclear whether or not luggage can be further arranged. Therefore, in the area in which the first blind spot interpolation voxels based on the first distance image are generated, When the luggage voxels or floor voxels based on the second range image are generated in an overlapping manner, the situation of the region is correctly reflected by the luggage voxels or floor voxels based on the second range image. According to the above aspect, in such a case, since the luggage area information and the empty floor information are estimated as the luggage voxel or the floor voxel based on the second distance image is arranged in the overlapping area, the luggage is distributed. It is possible to properly understand the availability of the floor surface to do so.

  In the luggage compartment monitoring system according to another aspect, the estimation unit uses at least one of the volume information occupied by the luggage voxels and the interpolation voxels and the floor area information occupied by the luggage voxels and the interpolation voxels as the luggage area. It may be estimated as information.

  According to the above aspect, at least one of the volume information occupied by the luggage voxel and the interpolation voxel and the floor area information occupied by the luggage voxel and the interpolation voxel is estimated as the luggage area information, so that the luggage is arranged. You can properly understand the availability of the floor surface.

  In the luggage compartment monitoring system according to another aspect, the estimating unit may estimate the area information of the floor surface not occupied by the luggage voxels and the interpolation voxels as the empty floor information.

  According to the above aspect, the area information of the floor surface not occupied by the luggage voxels and the interpolation voxels is estimated as the vacant floor information, so that the vacant situation of the floor surface for allocating the luggage can be appropriately grasped.

  In the luggage compartment monitoring system according to another aspect, the output unit may output an image visually showing the luggage compartment space and the luggage voxels, the interpolation voxels, and the floor surface voxels as the luggage area information and the empty floor information. .

  According to the above-described embodiment, an image visually showing the luggage compartment space, the luggage voxels, the interpolation voxels, and the floor surface voxels is output, so that the state of the cargo in the luggage compartment can be accurately grasped.

  According to one aspect of the present invention, by accurately grasping the state of the cargo in the luggage compartment, in particular by accurately grasping the space area occupied by the luggage in the luggage compartment, a floor surface for further luggage placement It becomes possible to grasp the availability of the.

It is a figure which shows the apparatus structure of the system containing the luggage compartment monitoring apparatus which concerns on this embodiment. It is a figure which shows typically the application example of the luggage compartment monitoring system of this embodiment. It is a block diagram which shows the functional structure of a luggage compartment monitoring apparatus. It is a figure which shows the hardware constitutions of a luggage compartment monitoring apparatus. It is a figure explaining generation of conversion information for converting a distance image into a point cloud. It is a figure explaining the process which transforms a range image into the point group showing the object represented by the range image. It is a figure which shows the production | generation process of a voxel. It is a figure which shows the production | generation process of an interpolation voxel. It is a figure which shows typically estimation of baggage area information and empty floor information. FIG. 9A is a diagram showing the luggage compartment upper surface VU. FIG. 9B is a diagram showing the luggage compartment side surface VS. It is a figure which shows the 1st example which the interpolation voxel and the baggage voxel or the floor surface voxel overlap. It is a figure which shows the 2nd example which the interpolation voxel and the baggage voxel or the floor surface voxel overlap. It is a figure which shows the example of the image which shows a voxel visually. FIG. 12A is a diagram showing an example of an image displaying the space in the luggage compartment and the luggage voxels. FIG. 12B is a diagram showing an example of an image displaying the space of the luggage compartment and the luggage interpolation voxels. FIG. 12C is a diagram showing an example of an image displaying the space of the luggage compartment, the blind spot interpolation voxels, and the floor surface voxels. It is a flowchart which shows the processing content of the luggage compartment monitoring method implemented in a luggage compartment monitoring system. It is a figure which shows the structure of a luggage compartment monitoring program.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference symbols, without redundant description.

  FIG. 1 is a diagram showing a device configuration of a system including a luggage compartment monitoring device according to the present embodiment. The system 1 shown in FIG. 1 is configured to include a luggage compartment monitoring device 10 (a luggage compartment monitoring system) and a terminal T mounted on a vehicle V. The vehicle V has a luggage compartment for carrying luggage. Two or more sensors C are provided in the luggage compartment of the vehicle V. The terminal T of the present embodiment transmits the information acquired by the sensor C to the luggage compartment monitoring device 10 via the network N. The network N is a wireless network. The luggage compartment monitoring device 10 constitutes a luggage compartment monitoring system that monitors the internal state of the luggage compartment. Although three vehicles V and terminals T are shown in FIG. 1, the numbers thereof are not limited.

  FIG. 2 is a diagram schematically showing an application example of the luggage compartment monitoring system of the present embodiment. As shown in FIG. 2, the luggage compartment monitoring system is exemplified as a system that monitors the luggage compartment R of the vehicle V. The luggage compartment R is provided with at least two sensors C other than the floor surface. In the example shown in FIG. 2, the luggage compartment R has two sensors C1 and C2 provided on the top surface. The sensors C1 and C2 have their imaging directions directed vertically downward.

  The sensor C is a sensor that acquires a distance image including information on the distance to the target, and is configured by, for example, a so-called TOF (Time Of Flight) sensor. The distance image acquired by the sensor C has information on the distance to the object in each pixel.

  The sensor C acquires a distance image of the luggage compartment R in which the luggage B is placed. The distance image acquired by the sensor C includes, in each pixel, information indicating the distance from the sensor C to the surface of the luggage B or the floor surface of the luggage compartment R.

  In the present embodiment, the luggage compartment monitoring system is exemplified as a system that monitors the luggage compartment of the vehicle V, but the luggage compartment to which the application is applied is not limited to the luggage compartment of the vehicle as long as the luggage is arranged in the space. Instead, it can be applied to a luggage compartment of a warehouse, for example.

  FIG. 3 is a block diagram showing a functional configuration of the luggage compartment monitoring device 10. The luggage compartment monitoring device 10 is composed of, for example, a computer such as a server. As illustrated in FIG. 3, the luggage compartment monitoring device 10 of the present embodiment functionally includes an acquisition unit 11, a generation unit 12, an interpolation unit 13, an estimation unit 14, and an output unit 15.

  FIG. 4 is a hardware configuration diagram of the luggage compartment monitoring device 10. Physically, the luggage compartment monitoring device 10 is, as shown in FIG. 4, a main storage device 102 including a CPU 101, a memory such as a RAM and a ROM, an auxiliary storage device 103 including a hard disk, and a communication control device 104. It is configured as a computer system including The luggage compartment monitoring device 10 may further include an input device 105 such as a keyboard, a touch panel, and a mouse that are input devices, and an output device 106 such as a display.

  The functions shown in FIG. 3 operate the communication control device 104 and the like under the control of the CPU 101 by loading predetermined computer software on the hardware such as the CPU 101 and the main storage device 102 and the like shown in FIG. At the same time, it is realized by reading and writing data in the main storage device 102 and the auxiliary storage device 103. Data and databases required for processing are stored in the main storage device 102 and the auxiliary storage device 103. The terminal T may also be configured as a computer system having the hardware configuration shown in FIG. In addition, in the present embodiment, the functional units 11 to 15 are configured in the luggage compartment monitoring device 10, but may be configured to be distributed to a plurality of computers.

  The terminal T mounted on the vehicle V may be configured as a computer system. The device forming the terminal T is not limited, and may be, for example, a computer terminal installed in the vehicle V, a high-performance mobile phone (smartphone), a tablet computer, a mobile phone, a personal digital assistant (PDA), or the like. It may be a mobile terminal.

Each functional unit of the luggage compartment monitoring device 10 will be described with reference to FIG. 3 again. The acquisition unit 11 acquires the range image acquired by each of the sensors C. As described with reference to FIG. 2, since the sensor C is provided on the top surface of the luggage compartment R with the vertically downward direction as the imaging direction, the load placed on the floor of the luggage compartment R and the floor surface can be detected. A range image within the range of the angle of view is acquired as a detection target.
Since the sensor C is provided on the top surface of the luggage compartment R, the sensor C detects the surface of the luggage B exposed in the direction of the sensor C, that is, the upper surface. Each pixel of the range image contains information on the direction and distance to the object.

  The generation unit 12 positions the surface of the luggage existing in the luggage compartment in the luggage compartment space based on the information on the distance to the object shown in the distance image and the arrangement position and the imaging direction of the sensor in the luggage compartment space. And a floor voxel that is information that indicates the position of the floor surface. Hereinafter, generation of voxels will be specifically described.

  First, the generation unit 12 converts the distance image into a point cloud representing the target represented in the distance image. The conversion processing of the point group, that is, the generation processing of the point group will be described with reference to FIGS. 5 and 6. FIG. 5 is a diagram illustrating generation of conversion information for converting a distance image into a point cloud.

  The point group generated here is generated with respect to the three-dimensional coordinate system DR representing the luggage space. The generation unit 12 generates, as conversion information, a conversion matrix for converting the three-dimensional coordinate system DP of the distance image into the three-dimensional coordinate system DR of the luggage space.

  The three-dimensional coordinate system DR of the luggage compartment space is set such that the left front side of the floor of the luggage compartment R is the origin, the right direction is the X axis, the rear direction is the Y axis, and the upward direction is the Z axis. Since the sensor C is provided on the top surface of the luggage compartment R with the vertically downward direction as the imaging direction, the coordinate system DC in the sensor C has the vertical downward direction as the Z axis, and the X axis and the X axis on the plane orthogonal to the Z axis. It is set so that the Y axis is along. The distance image DI acquired by the sensor C set in this way has a coordinate system DP having an X axis and a Y axis which are orthogonal to each other on the image plane, and a Z axis along the image depth direction.

  The generation unit 12 acquires the previously measured position information of the sensor C in the luggage compartment R and the vector vc indicating the imaging direction of the sensor C. Next, the production | generation part 12 acquires the distance image which imaged the floor surface of the luggage compartment R where the luggage is not placed with the sensor C, and produces | generates the point group ds0 showing a floor surface based on the acquired distance image. The generation unit 12 acquires a vector vu indicating the normal line of the floor surface by performing a predetermined analysis process (for example, a well-known plane detection process such as RANSAC) on the point group ds0. Then, the generation unit 12 calculates posture information indicating the posture of the sensor C based on the vector vc indicating the imaging direction of the sensor C and the vector vu indicating the normal line of the floor surface, and further calculates the posture information and the sensor C. Based on the position information, conversion information, which is a conversion matrix for converting the three-dimensional coordinate system DP of the distance image into the three-dimensional coordinate system DR of the luggage space, is generated.

  FIG. 6 is a diagram illustrating a process of converting a range image into a point cloud representing an object represented in the range image. The production | generation part 12 acquires the distance image DI1 which imaged the luggage compartment R of an empty state. For example, the generation unit 12 converts the information of each pixel represented in the distance image DI1 into the point group ds1 using the conversion matrix based on the distance image DI1 obtained by capturing the luggage compartment R in which no luggage is placed. As illustrated, the transformed and generated point group ds1 is arranged on a plane corresponding to the floor surface (Z = 0) in the coordinate system DR representing the space of the luggage compartment R of the vehicle.

  In addition, when the distance image DI2 obtained by capturing the luggage compartment R in which the luggage is placed is acquired, the generation unit 12 uses the conversion matrix based on the distance image DI2 to calculate the pixel of each pixel represented in the distance image DI2. The information is converted into the point group ds2. The point group ds2-1 of the point group ds2 corresponds to the surface of the luggage. As illustrated, the point group ds2-1 is arranged at the position of the surface (upper surface) of the package in the coordinate system DR representing the luggage space. Of the point group ds2, the point group ds2-2 corresponds to the floor surface that is exposed above without luggage being placed. As illustrated, the point group ds2-2 is arranged at a position corresponding to the floor surface (Z = 0) exposed above in the coordinate system DR representing the space of the luggage compartment R of the vehicle.

  The generation unit 12 indicates the position of the luggage voxel, which is the information indicating the position of the surface of the luggage existing in the luggage compartment in the luggage compartment space, and the position of the floor surface based on the point group generated based on the distance image. A floor voxel that is information is generated. FIG. 7 is a diagram showing a voxel generation process.

  Specifically, the generation unit 12 generates a voxel at a position corresponding to each point based on the position information of each point included in the point group in the coordinate system DR of the luggage compartment R. As illustrated in FIG. 7, the generation unit 12 generates the luggage voxel vx2 based on, for example, the point group ds2-1 corresponding to the surface of the luggage. Further, the generation unit 12 generates the floor surface voxel vx1 based on the point group ds2-2 indicating the position of the floor surface.

  In the present embodiment, one unit of voxel indicates the position of one point in the three-dimensional space. Therefore, in this case, one unit of voxel can be regarded as having no concept of volume. The baggage voxel and the floor voxel are generated as a set of a plurality of voxels. The luggage voxel and the floor surface voxel correspond to the point cloud generated based on the range image, and thus correspond to the surface of the luggage and the surface of the floor, respectively. Since the voxels arranged so as to correspond to the surface do not have the concept of thickness (or height), they can have the concept of area, but do not have the concept of volume. Since a set of voxels distributed three-dimensionally occupies a certain spatial region in a three-dimensional space, the concept of volume can be involved. Therefore, the interpolation voxels (baggage interpolation voxels, blind spot interpolation voxels) described later can have the concept of volume. In this embodiment, one voxel does not have the concept of volume, but it is assumed that one unit of voxel has the minimum unit volume in the three-dimensional space representing the luggage space in the luggage monitoring system. Good.

  Referring again to FIG. 3, the interpolation unit 13 generates interpolation voxels from the sensor C in the luggage space in a space area that is a blind spot. FIG. 8 is a diagram showing a process of generating interpolation voxels. Specifically, the interpolation unit 13 generates, as an interpolation voxel, a baggage interpolation voxel that occupies a space region from the baggage voxel to a floor surface vertically below the baggage voxel. The interpolation unit 13 also generates, as interpolation voxels, blind spot interpolation voxels that occupy a space area of a predetermined height above the floor surface that is a blind spot from the sensor C due to the angle of view of the luggage or the sensor C.

  In the example illustrated in FIG. 8, the generation unit 12 generates the luggage voxels vx31 and 32 and the floor surface voxel vf1 in the space coordinate system of the luggage compartment R. Further, the sensor C has an angle of view VA. In such a case, the interpolation unit 13 generates a baggage interpolation voxel vi11 that occupies a space area from the baggage voxel vx31 to the floor surface vertically below the baggage voxel vx31. Similarly, the interpolation unit 13 generates a baggage interpolation voxel vi12 that occupies a space area from the baggage voxel vx32 to the floor surface vertically below the baggage voxel vx32.

  Further, the floor surface f1 becomes a blind spot of the sensor C due to the presence of luggage. In this case, the interpolation unit 13 generates blind spot interpolation voxels vi21 and vi22 that occupy a spatial area of a predetermined height above the floor surface f1. The predetermined height of the blind spot interpolation voxel is set to, for example, the position of the luggage voxel located closest to the floor surface that is the blind spot. Therefore, the heights of the blind spot interpolation voxels vi21 and vi22 are set to the heights corresponding to the positions of the luggage voxels vx31 and vx32, respectively.

  Further, the floor surface f2 becomes a blind spot of the sensor C due to the angle of view of the sensor C. In this case, the interpolation unit 13 generates a blind spot interpolation voxel vi23 that occupies a spatial area of a predetermined height above the floor surface f2. The height of the blind spot interpolation voxel vi23 is set to a height corresponding to the position of the luggage voxel vx31 located closest to the floor surface f2, for example.

  The estimation unit 14 is, based on the luggage voxels, the interpolation voxels, and the floor surface voxels, luggage area information indicating a space area occupied by the luggage and a space area possibly occupied by the luggage in the space of the luggage compartment, In addition, vacant floor information regarding the floor surface on which the luggage can be placed is estimated.

  Since the spatial area in which the interpolated voxels are generated is a spatial area in which luggage may exist, in the present embodiment, the spatial area in which the interpolated voxels are generated is regarded as a spatial area substantially occupied by luggage. I am going to handle it. Based on such handling, specifically, the estimation unit 14 determines at least one of the volume information occupied by the luggage voxels and the interpolation voxels, and the floor area information occupied by the luggage voxels and the interpolation voxels. , Estimated as luggage area information. Further, the estimation unit 14 estimates area information of the floor surface not occupied by the luggage voxels and the interpolation voxels as empty floor information.

  FIG. 9 is a diagram schematically showing the estimation of luggage area information and empty floor information. FIG. 9A is a diagram showing the luggage compartment upper surface VU. FIG. 9B is a diagram showing the luggage compartment side surface VS. In the example shown on the luggage compartment upper surface VU and the vehicle side surface VS, the generated luggage voxel vx, luggage interpolation voxel vi1, blind spot interpolation voxel vi2, and floor surface voxel vf are shown. The estimation unit 14 may calculate the volume information of the luggage voxel vx, the luggage interpolation voxel vi1, and the blind spot interpolation voxel vi2 as the luggage area information. In addition, the estimation unit 14 may calculate the area information of the floor surface occupied by the baggage voxel vx, the baggage interpolation voxel vi1, and the blind spot interpolation voxel vi2 as baggage region information. Further, the estimation unit 14 uses, as empty floor information, area information of the floor surface occupied by the floor surface voxels vf and not occupied by the luggage voxel vx, the luggage interpolation voxel vi1 and the blind spot interpolation voxel vi2. It may be estimated.

  Next, with reference to FIGS. 10 and 11, handling of interpolated voxels based on distance images from a plurality of sensors C will be described. FIG. 10 is a diagram showing a first example in which interpolation voxels overlap with baggage voxels or floor voxels. In the example shown in FIG. 10, the first sensor C1 and the second sensor C2 are provided in the luggage compartment, and the first distance image acquired by the first sensor C1 and the second sensor C2 are acquired. The second distance image is acquired by the acquisition unit 11. Then, FIG. 10 shows the baggage voxel vx41 and the baggage interpolation voxel vi14 generated based on the first distance image, and the baggage voxel vx42 generated based on the second distance image. The voxel vi14 and the baggage voxel vx42 overlap. In such a case, the estimation unit 14 determines that the baggage interpolation voxel vi14 is arranged in the space area on the area fd4, which is the overlapping position of the baggage interpolation voxel vi14 and the baggage voxel vx42, and the baggage area information and the vacant space. Estimate floor information.

  That is, the space area in which the baggage interpolation voxel is generated is an area in which no baggage can be further arranged. As described with reference to the example shown in FIG. 10, even if the space area in which the luggage voxel based on the second distance image is generated overlaps with the luggage interpolation voxel based on the first distance image in the spatial area. Is treated as the space where the baggage interpolation vector is arranged, because the space region is a region where the baggage cannot be arranged. Even when the floor voxel based on the second distance image overlaps with the baggage interpolation voxel vi14 instead of the baggage voxel vx42, similarly, the baggage interpolation voxel vi14 is arranged in the overlapping region, The luggage area information and the empty floor information are estimated.

  FIG. 11 is a diagram showing a second example in which interpolation voxels overlap with baggage voxels or floor voxels. In the example shown in FIG. 11, the first sensor C1 and the second sensor C2 are provided in the luggage compartment, and the first distance image acquired by the first sensor C1 and the second sensor C2 are acquired. The second distance image is acquired by the acquisition unit 11. Then, FIG. 11 shows the luggage voxel vx51, the luggage interpolation voxel vi15 and the blind spot interpolation voxel vi25 generated based on the first distance image, and the luggage voxel vx52 generated based on the second distance image. The blind spot interpolation voxel vi25 and the baggage voxel vx52 overlap. In such a case, the estimation unit 14 determines that the luggage voxel vx52 is arranged in the space area on the area fd5 including the overlapping position of the blind spot interpolation voxel vi25 and the luggage voxel vx52, and determines the luggage area information and the empty floor. Estimate the information.

  That is, the spatial area in which the blind spot interpolation voxel is generated is an area in which it is unclear whether or not more luggage can be placed. As described with reference to the example illustrated in FIG. 11, when the luggage voxel based on the second distance image overlaps the area where the first blind spot interpolation voxel based on the first distance image is generated, the spatial area This situation is correctly reflected by the luggage voxel based on the second range image. In such a case, the luggage area information and the vacant floor information are estimated assuming that the luggage voxel based on the second distance image is arranged in the overlapping area, and therefore the vacant floor condition for allocating the luggage is appropriate. Can understand. Even if the floor surface voxel based on the second distance image overlaps with the blind spot interpolation voxel vi25 instead of the luggage voxel vx52, the luggage voxel vx52 is similarly arranged in the overlapping area as the luggage voxel vx52. Area information and vacant floor information are estimated.

  Referring again to FIG. 3, the output unit 15 outputs the luggage area information and the empty floor information. The output unit 15 can output the luggage area information and the empty floor information in various modes. For example, the output unit 15 may output by displaying the luggage area information and the vacant floor information on the display of a predetermined computer. Further, the output unit 15 may output the luggage area information and the empty floor information by storing the information in a predetermined storage unit for a predetermined process performed later.

  Further, the output unit 15 may output an image visually showing the luggage space, the luggage voxels, the interpolation voxels, and the floor surface voxels as the luggage area information and the empty floor information. FIG. 12 is a diagram showing an example of an image visually showing voxels. FIG. 12A is a diagram showing an example of an image displaying the space of the luggage compartment R and the luggage voxels. FIG. 12B is a diagram showing an example of an image displaying the space of the luggage compartment R and the luggage interpolation voxels. FIG. 12C is a diagram showing an example of an image displaying the space of the luggage compartment R, the blind spot interpolation voxels, and the floor voxels.

  As illustrated in FIG. 12A, the output unit 15 may output an image pc1 visually showing the space of the luggage compartment R and the luggage voxel vx6 as luggage area information and empty floor information. Further, as shown in FIG. 12B, the output unit 15 may output an image pc2 visually showing the space of the luggage compartment R and the luggage interpolation voxel vi16 as luggage area information and empty floor information. Further, as shown in FIG. 12C, the output unit 15 outputs an image pc3 visually showing the space of the luggage compartment R, the blind spot interpolation voxels vi26, and the floor surface voxels vf6 as luggage area information and empty floor information. May be. By outputting such an image, it becomes possible to accurately grasp the state of the cargo in the luggage compartment.

  Next, the operation of the luggage compartment monitoring device 10 of the present exemplary embodiment will be described with reference to FIG. 13. FIG. 13 is a flowchart showing the processing contents of the luggage compartment monitoring method implemented in the luggage compartment monitoring device 10.

  In step S1, the acquisition unit 11 acquires the range image acquired by the sensor C. Next, in step S2, the generation unit 12 generates a load voxel and a floor voxel based on the distance image and the sensor arrangement position and the imaging direction in the luggage space.

  Subsequently, in step S3, the interpolation unit 13 generates interpolation voxels from the sensor C in the dead space in the space area of the luggage compartment space. The interpolation voxels include baggage interpolation voxels and blind spot interpolation voxels.

  In step S4, the estimation unit 14 indicates a space area occupied by the luggage and a space area possibly occupied by the luggage in the space of the luggage compartment based on the luggage voxels, the interpolation voxels, and the floor voxels. Estimate the luggage area information and the empty floor information regarding the floor on which the luggage can be placed. Then, in step S5, the output unit 15 outputs the luggage area information and the empty floor information.

  Next, a luggage compartment monitoring program for causing a computer to function as the luggage compartment monitoring device 10 will be described with reference to FIG. 14. The luggage compartment monitoring program P1 includes a main module m10, an acquisition module m11, a generation module m12, an interpolation module m13, an estimation module m14, and an output module m15.

The main module m10 is a part that integrally controls the luggage compartment monitoring process. The functions realized by executing the acquisition module m11, the generation module m12, the interpolation module m13, the estimation module m14, and the output module m15 are respectively the acquisition unit 11, the generation unit 12, and the acquisition unit 11 of the luggage compartment monitoring device 10 illustrated in FIG. Interpolation unit 13, estimation unit 14, and output unit 15
Is similar to the function of.

  The luggage compartment monitoring program P1 is provided by a storage medium M1 such as a magnetic disk, an optical disk, or a semiconductor memory, for example. The luggage compartment monitoring program P1 may be provided via a communication network as a computer data signal superimposed on a carrier wave.

  According to the luggage compartment monitoring device 10, the luggage compartment monitoring method, and the luggage compartment monitoring program P1 of the present embodiment described above, a luggage voxel that is information indicating the position of the luggage surface is obtained based on the distance image. Further, based on the information indicating the blind spots of the luggage voxel and the sensor, etc., it is possible to obtain an interpolated voxel indicating a space area in which the luggage may not be practically placed. It is possible to obtain baggage region information indicating the space region that is occupied and the space region that may be occupied by the package. Therefore, it becomes possible to output the vacant floor information indicating the floor surface on which the luggage can be placed.

  The present invention has been described in detail above based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the gist thereof.

  DESCRIPTION OF SYMBOLS 1 ... System, 10 ... Luggage compartment monitoring device, 11 ... Acquisition part, 12 ... Generation part, 13 ... Interpolation part, 14 ... Estimation part, 15 ... Output part, C ... Sensor, M1 ... Storage medium, m10 ... Main module, m11 ... Acquisition module, m12 ... Generation module, m13 ... Interpolation module, m14 ... Estimation module, m15 ... Output module, N ... Network, P1 ... Luggage compartment monitoring program, R ... Luggage compartment, T ... Terminal, V ... Vehicle.

Claims (10)

A luggage compartment monitoring system for monitoring the internal condition of the luggage compartment,
An acquisition unit that acquires a distance image acquired by each of at least two sensors provided on a surface other than the floor that constitutes the lower surface of the luggage compartment;
Based on the information of the distance to the object shown in the distance image, and the position in the luggage compartment space of the surface of the luggage existing in the luggage compartment, based on the arrangement position and the imaging direction of the sensor in the luggage compartment space. A certain baggage voxel, and a generation unit that generates a floor surface voxel that is information indicating the position of the floor surface,
An interpolation unit that generates interpolation voxels in a space area that is a blind spot from the sensor in the luggage space,
Based on the luggage voxel, the interpolation voxel and the floor surface voxel, luggage area information indicating a space area occupied by luggage and a space area possibly occupied by luggage in the space of the luggage room, and , An estimation unit that estimates empty floor information about a floor on which luggage can be placed,
An output unit that outputs the luggage area information and the empty floor information,
Luggage compartment monitoring system. The interpolation voxel includes a load interpolation voxel that occupies a space area from the load voxel to the floor surface vertically below the load voxel.
The luggage compartment monitoring system according to claim 1. The interpolation voxel includes a blind spot interpolation voxel that occupies a space area of a predetermined height above a floor surface that is a blind spot from the sensor due to an angle of view of the luggage or the sensor,
The luggage compartment monitoring system according to claim 2. The at least two or more sensors include a first sensor and a second sensor,
The acquisition unit acquires a first distance image acquired by the first sensor and a second distance image acquired by the second sensor,
If the first baggage interpolation voxel, which is the baggage interpolation voxel based on the first distance image, and the baggage voxel or the floor voxel based on the second distance image overlap, the estimation unit performs the duplication. Assuming that the first baggage interpolation voxel is arranged at the position, the baggage area information and the empty floor information are estimated.
The luggage compartment monitoring system according to claim 3. The estimation unit, when the blind spot interpolation voxels that are the blind spot interpolation voxels based on the first distance image, and the luggage voxels or the floor surface voxels based on the second distance image overlap, Assuming that the luggage voxel or the floor voxel based on the second distance image is arranged at the overlapping position, the luggage area information and the empty floor information are estimated.
The luggage compartment monitoring system according to claim 4. The estimation unit estimates, as the luggage area information, at least one of volume information occupied by the luggage voxels and the interpolation voxels, and floor area information occupied by the luggage voxels and the interpolation voxels.
The luggage compartment monitoring system according to any one of claims 1 to 5. The estimation unit estimates area information of a floor surface not occupied by the luggage voxel and the interpolation voxel as the empty floor information,
The luggage compartment monitoring system according to any one of claims 1 to 6. The output unit outputs an image visually showing the luggage space and the luggage voxels, the interpolation voxels and the floor surface voxels as the luggage area information and the empty floor information,
The luggage compartment monitoring system according to any one of claims 1 to 7. A luggage compartment monitoring method in a luggage compartment monitoring system for monitoring the internal state of a luggage compartment,
An acquisition step of acquiring a distance image acquired by each of at least two or more sensors provided on a surface other than a floor constituting the lower surface of the luggage compartment;
Based on the information of the distance to the object shown in the distance image, and the position in the luggage compartment space of the surface of the luggage existing in the luggage compartment, based on the arrangement position and the imaging direction of the sensor in the luggage compartment space. A luggage voxel, and a generation step of generating a floor voxel that is information indicating the position of the floor,
An interpolation step of generating an interpolation voxel in the space area that is a blind spot from the sensor in the luggage space;
Based on the luggage voxel, the interpolation voxel and the floor surface voxel, luggage area information indicating a space area occupied by luggage and a space area possibly occupied by luggage in the space of the luggage room, and An estimation step of estimating empty floor information about a floor on which luggage can be placed,
An output step of outputting the luggage area information and the empty floor information,
And a luggage compartment monitoring method. A cargo compartment monitoring program for causing a computer to function as a cargo compartment monitoring system for monitoring an internal state of a cargo compartment,
On the computer,
An acquisition function for acquiring a distance image acquired by each of at least two or more sensors provided on a surface other than a floor constituting the lower surface of the luggage compartment;
Based on the information of the distance to the object shown in the distance image, and the position of the sensor in the luggage space and the imaging direction, information indicating the position of the surface of the luggage existing in the luggage space in the luggage space. A certain baggage voxel, and a generation function for generating a floor surface voxel that is information indicating the position of the floor surface,
An interpolation function for generating interpolation voxels in a space area that is a blind spot from the sensor in the luggage space,
Based on the luggage voxel, the interpolation voxel and the floor surface voxel, luggage area information indicating a space area occupied by luggage and a space area possibly occupied by luggage in the space of the luggage room, and , An estimation function that estimates empty floor information about the floor on which luggage can be placed,
An output function for outputting the luggage area information and the empty floor information,
A cargo compartment monitoring program that realizes

Images