KR102403460B1 - Method, Apparatus, and Computer-readable Medium for Automatic 3D Modeling Method for Warehouse Using Lidar Sensing Data and WMS Data - Google Patents

Abstract

The present invention relates to a method, apparatus, and computer-readable medium for performing automatic 3D modeling for a warehouse using LiDAR sensing data and WMS data, which updates warehouse modeling quickly and immediately. More specifically, the method comprises: a step of receiving LiDAR sensing data from a LiDAR sensing one or more objects located inside a warehouse; a step of generating surface data on the basis of the LiDAR sensing data; a step of dividing 3D surface data into one or more groups; and a step of performing 3D modeling on the one or more objects located inside the warehouse on the basis of WMS data including identification information, shape information, size information, and 3D modeling data of goods stored in the warehouse.

Description

{Method, Apparatus, and Computer-readable Medium for Automatic 3D Modeling Method for Warehouse Using Lidar Sensing Data and WMS Data}

The present invention relates to an automatic three-dimensional modeling method, apparatus, and computer-readable medium of a warehouse using lidar sensing data and WMS data, and more particularly, one or more objects located in the warehouse from lidar Receives lidar sensing data sensed with ida, generates surface data and divides 3D surface data into one or more groups based on the lidar sensing data, identification information, shape information, and Based on WMS data including size information and 3D modeling data, performing 3D modeling on one or more objects located inside the warehouse, automatic 3D modeling of a logistics warehouse using lidar sensing data and WMS data It relates to a method, an apparatus and a computer-readable medium.

Recently, autonomous driving technology using Lidar, which uses a technology that can detect the distance from an object and various physical properties by illuminating a laser on a target, is attracting attention.

Lidar stands for 'Light Detection And Ranging' or 'Laser Imaging, Detection and Ranging'. It is a technology that detects distance, direction, speed, temperature, material distribution and concentration characteristics by measuring time and intensity. LiDAR performs tasks and collects the results in a so-called 'point cloud', which can act as a three-dimensional map of the real world in real time.

Meanwhile, using radar, lidar, or camera in mobility such as a forklift operated in a warehouse, etc. A lot of research on a smart warehouse is also in progress. Related patents include technology related to an automatically operated logistics robot device, such as Korean Patent Registration No. 10-2248439.

However, in order for smart mobility such as the automated logistics robot device to be operated in a logistics warehouse, computerized data on the warehouse environment is required. There was trouble. In this way, in order to operate a smart warehouse, a technology that scans the warehouse using lidar sensing data and WMS data to automatically 3D model the warehouse and view logistics information for each modeled object This is a necessary situation.

Republic of Korea Patent Registration No. 10-2248439 (2021.04.29.)

The present invention relates to an automatic three-dimensional modeling method, apparatus, and computer-readable medium of a warehouse using lidar sensing data and WMS data, and more particularly, one or more objects located inside the warehouse from lidar Receives lidar sensing data sensed by , and divides surface data generation and 3D surface data into one or more groups based on the lidar sensing data, and identification information, shape information, and size of logistics stored in the corresponding warehouse Based on WMS data including information and 3D modeling data, performing 3D modeling on one or more objects located inside the warehouse, an automatic 3D modeling method of a logistics warehouse using lidar sensing data and WMS data , an apparatus and a computer-readable medium.

In order to solve the above problems, an embodiment of the present invention is an automatic three-dimensional modeling method of a warehouse using lidar sensing data and WMS data, performed by a computing device having one or more processors and one or more memories. , The WMS data includes identification information, shape information, size information, and 3D modeling data for each of the logistics stored in the logistics warehouse, and receives the lidar sensing data scanning one or more objects in the logistics warehouse, Lidar sensing data receiving step of generating point cloud data; a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data; By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step; By applying the shape information and size information of each logistics included in the WMS data to the object attribute information of each object included in the 3D surface data derived in the object classification step, each included in the 3D surface data a WMS fitting data generating step of generating WMS fitting data including identification information on the WMS data of logistics corresponding to the location of the object in the global coordinate system and the corresponding object; and a modeling generation step of generating logistics warehouse modeling data by applying the 3D modeling data of the corresponding logistics of the WMS data to the WMS fitting data;

In an embodiment of the present invention, the step of receiving the lidar sensing data includes: receiving, by a SLAM module, the lidar sensing data from the lidar; and generating, by the SLAM module, point cloud data that is an aggregate of a plurality of points having position coordinates in the global coordinate system for one or more objects inside the distribution warehouse using the lidar sensing data. have.

In an embodiment of the present invention, each point included in the point cloud data includes vector information, and the surface data generation step is based on the vector information of each point, and exists in the point cloud data. A surface generation operation for filling a plurality of empty spaces may be performed, and the surface generation operation may be performed through a deep learning algorithm.

In an embodiment of the present invention, the object classification step includes: for each object included in the divided three-dimensional surface data, using the first machine learning model, a first group corresponding to logistics; A second group corresponding to the pallet supporting the logistics; A third group corresponding to a rack for storing logistics; and a fourth group that does not correspond to any of the first group, the second group, and the third group.

In an embodiment of the present invention, the modeling generating step recognizes global coordinates of a plurality of identification points of 3D surface data of objects corresponding to the first group, the second group, and the third group. step; mapping the 3D modeling data of the corresponding logistics included in the WMS data according to the global coordinates with respect to the object corresponding to the first group; and modeling the objects corresponding to the second group and the third group using the first machine learning model in accordance with the global coordinates.

In an embodiment of the present invention, the WMS fitting data generating step uses a second machine learning model to combine each of the plurality of 3D surface data classified into the first group as a result of the object classification step with the WMS data After comparing, calculating a degree of similarity with each of the three-dimensional surface data classified into the first group and the logistics information in the WMS data, each of the three-dimensional surface data classified into the first group based on the logistics information showing the highest similarity Detailed modeling can be performed on 3D surface data.

In order to solve the above problems, an embodiment of the present invention is an automatic three-dimensional modeling device for a logistics warehouse using lidar sensing data and WMS data, implemented with a computing device having one or more processors and one or more memories, The WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each logistics stored in the warehouse, and the three-dimensional modeling device is a lidar scanning one or more objects inside the warehouse. A lidar sensing data receiving step of receiving sensing data and generating point cloud data; a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data; By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step; Each object included in the three-dimensional surface data by applying the shape information and size information of each logistics to the WMS data to the object attribute information of each object included in the three-dimensional surface data derived in the object classification step WMS fitting data generating step including the location in the global coordinate system of the WMS data and identification information on the WMS data; and a modeling generation step of generating logistics warehouse modeling data by applying the 3D modeling data of the corresponding logistics of the WMS data to the WMS fitting data;

In order to solve the above problems, an embodiment of the present invention implements an automatic three-dimensional modeling method of a logistics warehouse using lidar sensing data and WMS data, which is performed by a computing device having one or more processors and one or more memories A computer-readable medium, wherein the WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each of the materials stored in the distribution warehouse, wherein the computer-readable medium includes: Stores instructions for causing the device to perform the following steps, the steps comprising: receiving lidar sensing data scanning one or more objects in the warehouse, and receiving lidar sensing data to generate point cloud data; a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data; By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step; Each object included in the three-dimensional surface data by applying the shape information and size information of each logistics to the WMS data to the object attribute information of each object included in the three-dimensional surface data derived in the object classification step WMS fitting data generating step including the location in the global coordinate system of the WMS data and identification information on the WMS data; and a modeling generation step of generating logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data;

According to an embodiment of the present invention, when three-dimensional modeling of the interior of a logistics warehouse, a person does not need to model each object one by one, and by automatically modeling the necessary objects only with lidar sensing data and WMS data, It can have the effect of being able to quickly and immediately update your warehouse modeling.

According to an embodiment of the present invention, by using a machine learning model to determine what kind of logistics the corresponding logistics is with only the external information of the lidar sensing data using a machine learning model, and then performing 3D modeling on the corresponding logistics, manpower and time consumption can be minimized. possible effect can be exerted.

According to an embodiment of the present invention, in operating a smart warehouse, it is possible to intuitively check the inventory of the logistics warehouse.

According to an embodiment of the present invention, it is possible to exhibit the effect of helping the autonomous driving learning or operation of smart mobility operated in a logistics warehouse.

According to an embodiment of the present invention, it is possible to achieve the effect of obtaining accurate three-dimensional modeling data by materializing unclear object information obtained through lidar sensing data through WMS data.

According to an embodiment of the present invention, by generating learning data for machine learning through simulation using the lidar sensing data, it is possible to exhibit the effect of increasing the accuracy of object classification using the machine learning model.

According to an embodiment of the present invention, by using the attribute information of the logistics included in the WMS data, if you touch the logistics modeled through the manager terminal remotely, you can see the attribute information for the logistics, through this, the logistics It can exert the effect of performing inventory logistics management of the warehouse.

1 schematically shows the configuration of an automatic three-dimensional modeling method of a distribution warehouse according to an embodiment of the present invention and a three-dimensional modeling apparatus implementing the method.
2 schematically shows a process of performing a lidar sensing data receiving step according to an embodiment of the present invention.
3 schematically illustrates a point cloud according to an embodiment of the present invention.
4 schematically shows a process of performing a surface data generation step according to an embodiment of the present invention.
5 schematically shows the execution process and execution result of the object classification step according to an embodiment of the present invention.
6 schematically shows WMS data according to an embodiment of the present invention.
7 schematically shows a process of performing the WMS fitting data generation step according to an embodiment of the present invention.
8 schematically illustrates a process of determining the logistics corresponding to the object by using the second machine learning model on the 3D surface data of the object whose properties are unclear according to an embodiment of the present invention.
9 schematically shows a process of performing a modeling performing step according to an embodiment of the present invention.
10 schematically shows the results of the modeling performing step according to an embodiment of the present invention.
11 schematically illustrates an internal configuration of a computing device according to an embodiment of the present invention.

Hereinafter, various embodiments and/or aspects are disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. However, it will also be recognized by one of ordinary skill in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain illustrative aspects of one or more aspects. These aspects are illustrative, however, and some of various methods may be employed in the principles of the various aspects, and the descriptions set forth are intended to include all such aspects and their equivalents.

Further, various aspects and features will be presented by a system that may include a number of devices, components and/or modules, and the like. It is also noted that various systems may include additional apparatuses, components, and/or modules, etc. and/or may not include all of the apparatuses, components, modules, etc. discussed with respect to the drawings. must be understood and recognized.

As used herein, “embodiment”, “example”, “aspect”, “exemplary”, etc. may not be construed as an advantage or advantage in any aspect or design being described over other aspects or designs. . The terms '~part', 'component', 'module', 'system', 'interface', etc. used below generally mean a computer-related entity, for example, hardware, hardware A combination of and software may mean software.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements, and/or groups thereof. should be understood as not

Also, terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those of ordinary skill in the art to which the present invention belongs. have the same meaning as Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in an embodiment of the present invention, an ideal or excessively formal meaning is not interpreted as

1 schematically shows a configuration of an automatic three-dimensional modeling method of a distribution warehouse according to an embodiment of the present invention and a three-dimensional modeling apparatus 10000 implementing the method.

Schematically, Fig. 1 (a) schematically shows an automatic three-dimensional modeling method of a warehouse using lidar sensing data and WMS data, and Fig. 1 (b) is an automatic three-dimensional modeling method of the logistics warehouse. A 3D modeling apparatus 10000 to perform is schematically shown.

As shown in FIG. 1, as an automatic three-dimensional modeling method of a warehouse using lidar sensing data and WMS data, performed by a computing device having one or more processors and one or more memories, the WMS data is stored in the warehouse. LiDAR that includes identification information, shape information, size information, and three-dimensional modeling data for each stored logistics, receives lidar sensing data that scans one or more objects in the warehouse, and generates point cloud data sensing data receiving step (S100); a surface data generation step (S200) of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data; Using the first machine learning model 1310, each object included in the 3D surface data is classified, and an object including shape information and size information for each object included in the 3D surface data object classification step of deriving attribute information (S300); By applying the shape information and size information of each logistics included in the WMS data to the object property information of each object included in the 3D surface data derived in the object classification step (S300), the 3D surface data WMS fitting data generation step (S400) of generating WMS fitting data including identification information on the WMS data of logistics corresponding to the location in the global coordinate system of each object included and the corresponding object; and a modeling generation step (S500) of generating logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data.

Specifically, in the lidar sensing data receiving step (S100), the point cloud data is generated by receiving the lidar sensing data from the lidar. More specifically, the lidar may be mounted on a mobility such as a forklift that can move around the warehouse, and the lidar mounted on the mobility emits a laser pulse while moving around the warehouse, and the emitted laser It generates lidar sensing data measuring the time and intensity of a pulse from striking an object to being reflected and returned to the lidar.

The lidar sensing data is transmitted to the lidar sensing data receiving unit 1100 of the 3D modeling apparatus 10000, and the lidar sensing data receiving unit 1100 receives the lidar sensing data. By using the corresponding lidar sensing data, the lidar sensing data receiving step (S100) of generating point cloud data is performed. A point cloud refers to an aggregate of a plurality of points in a three-dimensional space, and the point cloud data includes information about the point cloud. A more detailed description of the lidar sensing data receiving step (S100) will be described later with reference to FIG. 2 .

In the surface data generation step (S200), 3D surface data for one or more objects inside the distribution warehouse is generated based on the point cloud data generated as a result of performing the lidar sensing data receiving step (S100). More specifically, the point cloud data shows the outlines of all areas that light can reach as a plurality of points, and the surface data generation unit 1200 of the 3D modeling apparatus 10000 is, in a 3D environment, Surface data generation step (S200) of generating three-dimensional surface data by filling the plurality of empty spaces between the plurality of points using the point cloud data shown as a plurality of points using a deep learning algorithm (S200) ) is performed. A more detailed description of the surface data generating step (S200) will be described later with reference to FIG. 4 .

In the object classification step ( S300 ), each object included in the 3D surface data generated as a result of performing the surface data generation step ( S200 ) is divided using the first machine learning model ( 1310 ). In other words, since the three-dimensional surface data has only information about a seamless surface that is not differentiated with respect to one or more objects existing in a distribution warehouse including boxes, pallets, racks, and forklifts, , through the object classification step (S300), using the first machine learning model 1310 to classify the surface data corresponding to each object existing in the three-dimensional surface data.

As described above, the object classifying unit 1300 of the 3D modeling apparatus 10000 divides the 3D surface data into one or more objects, and then for each object, shape information and size information of the corresponding object. An object classification step (S300) of deriving object attribute information including The shape information, for example, corresponds to information that can distinguish whether it is a rectangular parallelepiped shape or a cylindrical shape, and the size information corresponds to length information of the width, length, and height of the box in the shape of a rectangular parallelepiped.

In the WMS fitting data generation step (S400), after finding the shape information and size information of the logistics corresponding to the object attribute information generated as a result of the object classification step (S300) in the WMS data, the shape information and size of the corresponding logistics The information is applied to the object attribute information.

More specifically, for example, in the object classification step (S300), when an object having object attribute information having a rectangular parallelepiped shape, a width of 30 cm, a vertical length of 50 cm, and a height of 28 cm is distinguished, the WMS fitting data In the generating step (S400), a logistic corresponding to a box having a rectangular parallelepiped shape, a width of 30 cm, a length of 50 cm, and a height of 28 cm is searched for on the WMS data. It is preferable to search for logistics perfectly matching the object attribute information, but for a high search rate, it is possible to search for logistics within a preset error. If the search result in the WMS data shows that the shape information of logistics corresponding to the object attribute information is a cuboid, and the size information corresponds to a horizontal length of 32 cm, a vertical length of 50 cm, and a height of 30 cm, the object classification step The size information and shape information of the object divided in (S300) is applied to a rectangular box having a horizontal length of 32 cm, a vertical length of 50 cm, and a height of 30 cm.

In the WMS fitting data generation step (S400), after applying the WMS data to the object attribute information of the object divided in the object classification step (S300), location information in the global coordinate system of the object and logistics corresponding to the object WMS fitting data including identification information of The global coordinate system is a coordinate system generated by mapping the internal space of the logistics warehouse to a three-dimensional spatial coordinate system, and the positions of all objects existing in the internal space of the logistics warehouse can be displayed using the global coordinate system. In other words, in the WMS fitting data generation step (S400), the position of the object divided in the object classification step (S300), that is, the global coordinates of the object can be grasped using the global coordinate system. In addition, location information in the global coordinate system corresponds to three-dimensional spatial coordinates.

Additionally, in the WMS fitting data generation step (S400), as described above, the identification information of the corresponding logistics can be grasped for the logistics on the WMS data corresponding to the object attribute information of the object divided in the object classification step (S300). have. The WMS fitting data generation unit 1400 of the three-dimensional modeling apparatus 10000, the global coordinates of the object; and WMS fitting data generating step (S400) for generating WMS fitting data including; and identification information of logistics corresponding to the object. Logistics identification information corresponding to the object is used in the modeling generation unit 1500 to be described later, and a more detailed description of the corresponding matter will be described later with reference to FIG. 9 .

In the modeling generation step (S500), with respect to the WMS fitting data generated as a result of the performance of the WMS fitting data generation step (S400), the three-dimensional surface data of the object corresponding to the WMS fitting data 3 of the logistics corresponding to the object The dimensional modeling data is applied, and at this time, the identification information of the corresponding logistics in the WMS fitting data is used. The three-dimensional modeling data of the logistics is included in the WMS data.

As a result of performing the modeling generation step (S500), more specific 3D modeling data is applied to the 3D surface data, and when the modeling generation step (S500) is performed with respect to one or more objects in the logistics warehouse, an embodiment of the present invention As an example, a result as shown in FIG. 10 is generated.

2 schematically shows a process of performing the lidar sensing data receiving step S100 according to an embodiment of the present invention, and FIG. 3 schematically shows a point cloud according to an embodiment of the present invention.

As shown in Figure 2, the lidar sensing data receiving step (S100), the SLAM module 1110 receiving the lidar sensing data from the lidar; and generating, by the SLAM module 1110, point cloud data that is an aggregate of a plurality of points having location coordinates in the global coordinate system for one or more objects inside the distribution warehouse using the lidar sensing data; include

Specifically, as described above in the description of FIG. 1 , the lidar sensing data receiving unit 1100 of the 3D modeling apparatus 10000 receives the lidar sensing data from the lidar and generates point cloud data. The lidar sensing data is information about the time and intensity it takes for the lidar to emit a laser pulse inside the warehouse, and the emitted laser pulse is reflected after colliding with one or more objects in the warehouse and returned. include In addition, the lidar sensing data receiver 1100 may include a SLAM module 1110 , and may generate the point cloud data through the SLAM module 1110 .

The SLAM (Simultaneous Localization and Mapping) module is a module that simultaneously performs location recognition and mapping using a deep learning algorithm, receives lidar sensing data from lidar, and stores the lidar sensing data in a preset 3D space. Multiple points can be mapped. That is, each of the plurality of points includes location information of the global coordinate system. The deep learning algorithms used in the SLAM module 1110 include Loam, Lego-LOAM, and IMLS-SLAM, and the SLAM module ( 1110), the results of mapping a plurality of points to the global coordinate system in the warehouse are shown in FIG. 3 .

3 is an embodiment of a point cloud generated after scanning a logistics warehouse with a lidar, and shows a visualization of the point cloud data.

As shown in Fig. 3, in the 3D point cloud generated as a result of the reception of the lidar sensing data receiving step (S100), a closer object is densely sampled compared to a far object, and the farther the object is, the more it is reflected. The strength of the return signal is weakened and contains large noise.

In addition, the point cloud obtained by using the lidar may also include reflectance intensity information indicating the intensity of the reflected signal along with the three-dimensional coordinates. As shown in FIG. 3 , the portions marked with green and blue colors of the point cloud mean that the reflectivity intensity is relatively low, and the parts marked with red color mean that the reflectivity intensity is relatively high. Factors affecting the reflectivity intensity include a medium through which a laser pulse emitted from the lidar passes, a shade where the surface of an object meets the pulse, and a reflectivity of the surface of the object. In the surface data generating step (S200) to be described later, 3D surface data may be generated based on density information and reflectivity intensity information included in the point cloud data.

4 schematically shows a process of performing the surface data generation step S200 according to an embodiment of the present invention.

As shown in Fig. 4, the automatic three-dimensional modeling method of the warehouse includes a surface data generation step (S200) of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data; Including, each point included in the point cloud data includes vector information, and the surface data generation step (S200) is based on the vector information of each point, which is present in the point cloud data. A surface creation operation for filling a plurality of empty spaces is performed, and the surface creation operation is performed through a deep learning algorithm.

Schematically, Fig. 4 (a) schematically shows an object existing in real space, and Fig. 4 (b) is a lidar sensing data receiver after sensing the object of Fig. 4 (a) with lidar. The point cloud generated in (1100) is schematically shown, and Fig. 4 (c) is a three-dimensional surface generated by performing the surface data generation step (S200) with respect to the point cloud shown in Fig. 4 (b). The data is shown schematically.

Specifically, with reference to FIGS. 3 and 4 (b), the plurality of points included in the point cloud data generated from the lidar are not dense enough to form a single surface, and each of the plurality of points is between each other. It does not contain information about relationships and information about specific forms. That is, through the surface data generating step ( S200 ), with respect to a plurality of points included in the point cloud data, a surface creation operation for filling an empty space between the plurality of points may be performed.

The object shown in (a) of FIG. 4 schematically shows an arbitrary object existing in the distribution warehouse, and may correspond to a distribution box or the like. The lidar senses the object and transmits the lidar sensing data to the lidar sensing data receiving unit 1100, and the lidar sensing data receiving unit 1100, with reference to the description of FIG. 2, the SLAM module 1110 ), it is possible to generate point cloud data based on the lidar sensing data.

The point cloud shown in (b) of FIG. 4 is schematically illustrated by visualizing the point cloud data generated by the lidar sensing data receiver 1100 as described above. As shown in (b) of FIG. 4 , a plurality of points may be mapped in a 3D space in a form similar to the object of FIG. 4 (a). On the other hand, since the point cloud cannot be recognized as an object by the first machine learning model 1310 or the second machine learning model 1410 to be described later, as shown in FIG. In order to have it, surface creation work must be performed.

As shown in (c) of FIG. 4 , the surface data generating unit 1200 of the 3D modeling apparatus 10000 receives the point cloud data, and a vector included in a plurality of points of the point cloud data. A surface creation operation is performed to fill a plurality of empty spaces existing in the point cloud data based on the information.

The surface generation operation in the surface data generation step (S200) is performed through a deep learning algorithm. More specifically, by performing the surface data generation step (S200) using the SIREN algorithm, Implicit Neural Representation (Implicit Neural Representation) that smoothly connects each point to generate a continuous face is performed. can do.

Through the above method, the 3D modeling apparatus 10000 performs the surface data generating step S200 for each object, The WMS fitting data generation step (S400) and the modeling generation step (S500), which will be described later, may be performed with respect to the three-dimensional surface data.

5 schematically shows the process and results of the object classification step (S300) according to an embodiment of the present invention.

As shown in FIG. 5 , the automatic three-dimensional modeling method of the logistics warehouse uses a first machine learning model 1310 to classify each object included in the three-dimensional surface data, and the three-dimensional surface data object classification step (S300) of deriving object attribute information including shape information and size information for each object included in; For each object included in, using the first machine learning model 1310, a first group corresponding to logistics; A second group corresponding to the pallet supporting the logistics; A third group corresponding to a rack for storing logistics; and a fourth group that does not correspond to any of the first group, the second group, and the third group.

Schematically, Fig. 5 (a) schematically shows the execution process of the object classification step (S300), and Fig. 5 (b) schematically shows the execution result of the object classification step (S300).

Specifically, the process of performing the object classifying step S300 shown in FIG. 5A is performed by the object classifying unit 1300 of the 3D modeling apparatus 10000, and the object classifying unit 1300 is A first machine learning model 1310 is included. The object classifying unit 1300 receives 3D surface data from the surface data generating unit 1200, and uses the first machine learning model 1310 to identify each object included in the 3D surface data. In addition, as an embodiment of the present invention, each of the objects may be classified into the first group, the second group, the third group, and the fourth group.

As described above, the first group is a group corresponding to distribution, and corresponds to distribution boxes of various shapes and sizes, and the like. For example, a distribution box in the form of a rectangular parallelepiped or a distribution box in the form of a cylinder may be included in the first group. The second group is a group corresponding to a pallet supporting the distribution box, and includes pallets of various lengths and widths. The third group is a group corresponding to a rack. A rack corresponds to a skeletal structure that can store a distribution box. The fourth group is a group of objects not belonging to the first to third groups, and for example, a person, a floor, a wall, a forklift, and a conveyor belt may be included in the fourth group.

Preferably, the first machine learning model 1310 performs machine learning on a plurality of images for the first to third groups before performing the object classification step S300. On the other hand, since the first machine learning model 1310 cannot derive any information other than the external shape information of the three-dimensional surface data to be distinguished, more preferably, the plurality of first to third groups Machine learning is performed on the external image. On the other hand, as an embodiment of the present invention, a simulation may be generated by using the lidar sensing data received from the lidar or the three-dimensional surface data, and by generating learning data for machine learning through the simulation, the An effect of increasing the accuracy of object classification using the first machine learning model 1310 may be exhibited.

As shown in Fig. 5 (a), in the object classification step (S300), the 3D surface data for the entire distribution warehouse is included in the 3D surface data using the first machine learning model 1310. It is possible to classify each object to be used, classify each divided object into the first to fourth groups, and then derive object attribute information for each object corresponding to the first group.

The object attribute information includes shape information and size information of an object corresponding to the first group. The shape information includes information about the shape of the distribution box, as described above in the description of FIG. 1, and the size information includes information about the size of the distribution box.

5 (b) schematically shows the result of the object classifying step (S300), and for intuitive understanding, each After classifying the objects, each of the objects was divided into first to fourth groups using the first machine learning model 1310, and each group is shown in a different color.

As shown in (b) of Figure 5, the part shown in red is a part corresponding to the first group, and the object divided by the distribution box is shown in red. The parts shown in blue are parts corresponding to the second to third groups, and the pallets and racks provided in the distribution warehouse are shown in blue. Figure 5 (b) is an embodiment of the present invention, the second group and the third group, which are parts irrelevant to logistics, are shown in the same color, and as another embodiment of the present invention, the second group Although the object corresponding to and the object corresponding to the third group may be shown in different colors, preferably, the second group and the third group are for optimization of the three-dimensional modeling method as in the above embodiment. are treated as the same color.

A part shown in yellow is a part corresponding to the fourth group, and as described above, objects that do not correspond to the first to third groups are divided into a fourth group. Meanwhile, in the present invention, although both the person and the floor shown in FIG. 5(b) are shown in yellow, they are shown in different colors for better understanding of the description. In addition, as an embodiment of the present invention, the green part shown in FIG. 5(b) is an area in which objects are not distinguished through the first machine learning model 1310, and objects are obtained through other 3D surface data. Objects included in the corresponding area can be distinguished only when the classification step S300 is performed. .

6 schematically shows WMS data according to an embodiment of the present invention.

As shown in FIG. 6 , the WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each of the materials stored in the distribution warehouse.

Specifically, the WMS data is preferably stored in a database separately provided inside the 3D modeling apparatus 10000 as an embodiment of the present invention, and as another embodiment of the present invention, stored in a separate system and communicates with the 3D modeling apparatus 10000, the 3D modeling apparatus 10000 may receive the WMS data.

A plurality of logistics information included in the WMS data is information recorded when a distribution box is stored in the warehouse. It corresponds to a file sampled based on the data, and the shape information and size information is information generated by recording the shape and size of the corresponding distribution. The identification information of the corresponding distribution is information given to manage a plurality of distributions existing in the distribution warehouse as unique identification information of the corresponding distribution.

As shown in Fig. 6, it can be seen that the logistics A to which the identification information is given as a type 1 box is packaged in a rectangular box having a width, a length, and a height of 40 cm, 20 cm, and 25 cm, respectively. On the other hand, a distribution box of a cylindrical shape like the 3-type box shown in FIG. 6 may also be input to the WMS data. As another embodiment of the present invention, the identification information is expressed in binary and can be converted into an identification code such as a barcode or QR code.

7 schematically shows a process of performing the WMS fitting data generation step (S400) according to an embodiment of the present invention.

As shown in Figure 7, the automatic three-dimensional modeling method of the logistics warehouse is included in the WMS data in the object attribute information of each object included in the three-dimensional surface data derived in the object classification step (S300). WMS fitting data generation step of generating WMS fitting data including identification information on the location in the global coordinate system of each object included in the three-dimensional surface data and identification information on the WMS data by applying the shape information and size information of each logistics (S400); includes.

Specifically, in the WMS fitting data generation step (S400), with reference to the description of FIG. 5, the size information and shape of each object included in the 3D surface data derived in the object classification step (S300) For object attribute information including information, size information and shape information included in WMS data are respectively applied. In other words, the size information of the WMS data is applied to the size information of the object attribute information, and the shape information of the WMS data is applied to the shape information of the object attribute information. In this case, the object corresponding to the object attribute information and the logistics corresponding to the size information and shape information of the WMS data have a corresponding relationship. As described above, the object of the 3D surface data corresponding to each other and the logistics on the WMS data can be found in a process described later in FIG. 8 .

The WMS fitting data generation unit 1400 of the three-dimensional modeling apparatus 10000 applies the size information and shape information of the WMS data to the object attribute information, and then applies the object attribute information to the object attribute information to the position of the object in the global coordinate system. A WMS fitting data generation step (S400) of generating WMS fitting data by further applying the information and identification information of the WMS data is performed.

That is, the WMS fitting data includes, with respect to the object included in the three-dimensional surface data, location information in the global coordinate system of the object, size information of logistics existing on the WMS data corresponding to the object, shape information, and Includes identification information. A schematic diagram of an embodiment of the configuration of the WMS fitting data is as shown in FIG. 9 (a).

FIG. 8 schematically illustrates a process of determining the logistics corresponding to the object using the second machine learning model 1410 for 3D surface data of an object whose properties are unclear according to an embodiment of the present invention.

As shown in FIG. 8 , the WMS fitting data generation step (S400) is performed using a second machine learning model 1410, and as a result of performing the object classification step (S300), a plurality of 3 Comparing each of the dimensional surface data with the WMS data, calculating the similarity with each of the 3D surface data classified into the first group and the logistics information in the WMS data, based on the logistics information showing the highest degree of similarity Detailed modeling is performed on each 3D surface data classified into the first group.

Specifically, as shown in Figure 8, in the WMS fitting data generation step (S400), the object included in the 3D surface data divided in the object classification step (S300) and a plurality of logistics existing on the WMS data can be compared through the second machine learning model 1410, and the logistics most similar to the object included in the three-dimensional surface data can be found.

As an embodiment of the present invention, an object marked with ? shown in FIG. 8 corresponds to an arbitrary object included in the 3D surface data. On the other hand, the arbitrary object corresponds to the object divided into the first group by the object classification step (S300). The second machine learning model 1410 is included in the WMS fitting data generation unit 1400, and the second machine learning model 1410 generates WMS data before performing the WMS fitting data generation step (S400). It is desirable to learn The second machine learning model 1410, having finished machine learning on the WMS data, may grasp object attribute information of the arbitrary object, that is, size information and shape information of the arbitrary object.

The second machine learning model 1410 identifies the object attribute information of the arbitrary object, and considers size information and shape information of all logistics existing on the WMS data, and has the highest similarity to the arbitrary object. Logistics can be calculated. Preferably, to the user of the present invention, the first logistics with the highest similarity, the second logistics with the highest similarity excluding the first logistics, and the first logistics with the highest degree of similarity except for the first logistics and the second logistics A degree of similarity between the third logistics and each logistics may be provided. As shown in FIG. 8 , for the arbitrary object, the second machine learning model 1410 has a similarity of 75% with logistics A that exists on the WMS data, and C that exists on the WMS data. The similarity with logistics is 17%, and the similarity with logistics B existing on the WMS data is calculated as 8%. As a result, the second machine learning model 1410 of the WMS fitting data generator 1400 determines that the arbitrary object is logistics A with the highest similarity among the logistics on the WMS data.

When the WMS fitting data generation unit 1400 finds the distribution on the WMS data corresponding to an arbitrary object included in the 3D surface data, as described above in the description of FIG. 7 , the object of the arbitrary object WMS fitting data is created by applying the location information of the object in the global coordinate system, size information, shape information, and identification information of the WMS data of logistics corresponding to the object to the attribute information.

9 schematically shows a process of performing a modeling performing step according to an embodiment of the present invention.

As shown in Figure 9, the automatic three-dimensional modeling method of the warehouse is a modeling generation step of generating the logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data ( S500); and, in the modeling generating step (S500), the global coordinates of a plurality of identification points of the three-dimensional surface data of the object corresponding to the first group, the second group, and the third group. recognizing; mapping the 3D modeling data of the corresponding logistics included in the WMS data according to the global coordinates with respect to the object corresponding to the first group; and modeling the objects corresponding to the second group and the third group using the first machine learning model 1310 according to the global coordinates.

Schematic, Fig. 9 (a) schematically shows the configuration of WMS fitting data, Figure 9 (b) schematically shows the configuration of the warehouse modeling data.

Specifically, the WMS fitting data shown in (a) of FIG. 9 includes position information of the global coordinate system of the object included in the three-dimensional surface data, that is, the global coordinate, as described above, and corresponds to the object. Includes identification information of WMS data. The modeling generating unit 1500 of the three-dimensional modeling apparatus 10000 performs a modeling generating step (S500) of recognizing global coordinates of a plurality of identification points existing in the WMS fitting data.

As shown in (a) of FIG. 9, an object included in the three-dimensional surface data has one or more identification points for distinguishing the object, and in order to distinguish the object in the object classification step S300, at least three An identification point is required, and preferably, when there are 5 or more identification points, the object classification step ( S300 ) for the corresponding object can be smoothly performed. As an embodiment of the present invention, through the five identification points shown in (a) of FIG. 9, the first machine learning model 1310 may determine that the corresponding object is an object belonging to the first group, The first machine learning model 1310 may determine size information and shape information of the corresponding object.

The modeling generation unit 1500 may extract a three-dimensional modeling file of logistics corresponding to the WMS fitting data from the WMS data, using the identification information present in the WMS fitting data, to the WMS fitting data. The result of applying the 3D modeling file is shown in FIG. 9(b).

As an embodiment of the present invention, after setting three reference points in the global coordinates of the object of the WMS fitting data, the 3D modeling file may be applied to the WMS fitting data. The three reference points may be set by selecting three of the plurality of identification points. By mapping the reference points of the 3D modeling file corresponding to the three reference points, the entire object of the WMS fitting data can be modeled as 3D modeling data of logistics corresponding to the object. In the embodiment, when the global coordinates do not exist on one surface, it is possible to exhibit the effect of performing the modeling generation step ( S500 ) through a small amount of calculation processing.

As another embodiment of the present invention, a three-dimensional modeling file can be applied to the WMS fitting data by generating a center of gravity vector of the corresponding object using all global coordinates of the object of the WMS fitting data. The center of gravity vector of the corresponding object can be calculated through the vectors of each of the plurality of global coordinates of the WMS fitting data, and by matching the center of gravity vector of the object with the center of gravity of the 3D modeling file on the WMS data, the WMS The 3D modeling data of the logistics can be mapped according to the global coordinates of the fitting data. The other embodiment can exhibit the effect of performing the modeling generation step (S500) regardless of the location of the global coordinates.

Additionally, for the objects belonging to the second group and the third group, since detailed modeling is not required, a rough modeling file is generated through the first machine learning model 1310, and the second group and the By recognizing the global coordinates of a plurality of identification points of the three-dimensional surface data of the object corresponding to the third group, and applying the modeling file to the corresponding global coordinates, modeling of the objects corresponding to the second to third groups A generation step ( S500 ) may be performed. On the other hand, it is preferable not to perform the modeling generation step (S500) for the object corresponding to the fourth group.

10 schematically shows the results of the modeling performing step according to an embodiment of the present invention.

The three-dimensional modeling file of the entire logistics warehouse shown in FIG. 10 is, as an embodiment of the present invention, the logistics warehouse modeling data as shown in FIG. 9 (b) for each of one or more objects in the logistics warehouse. It can be created by mapping each to the global coordinates of the logistics warehouse, and the pallet and rack shown in FIG. 10, as described above in the description of FIG. 9, the modeling file obtained through the first machine learning model 1310 It can be created by mapping to the global coordinates of the warehouse.

Through the three-dimensional modeling file of the entire logistics warehouse as shown in FIG. 10, the user of the present invention can exhibit the effect of intuitively checking the inventory of the corresponding logistics warehouse in operating the smart warehouse, an embodiment of the present invention For example, the user of the present invention may receive the 3D modeling file shown in FIG. 10 through the user's terminal, and when selecting a specific logistics in the 3D modeling file, the specific logistics using WMS data information, for example, the type of the specific logistics, the date of stocking in the warehouse, and the current quantity, can be checked through a separate interface displayed on the user's terminal.

11 schematically illustrates an internal configuration of a computing device according to an embodiment of the present invention.

11, the computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, an input/output subsystem ( It may include at least an I/O subsystem) 11400 , a power circuit 11500 , and a communication circuit 11600 . In this case, the internal configuration of the computing device 11000 may include the modeling device 10000 shown in FIG. 1 , or the computing device 11000 corresponds to an embodiment of the modeling device 10000 shown in FIG. 1 . The processor 11100 may be configured in plurality, and may include the first processor module and the second processor module described above.

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. . The memory 11200 may include a software module, an instruction set, or other various data required for the operation of the computing device 11000 .

In this case, access to the memory 11200 from other components such as the processor 11100 or the peripheral device interface 11300 may be controlled by the processor 11100 .

Peripheral interface 11300 may couple input and/or output peripherals of computing device 11000 to processor 11100 and memory 11200 . The processor 11100 may execute a software module or an instruction set stored in the memory 11200 to perform various functions for the computing device 11000 and process data.

The input/output subsystem may couple various input/output peripherals to the peripheral interface 11300 . For example, the input/output subsystem may include a controller for coupling a peripheral device such as a monitor or keyboard, mouse, printer, or a touch screen or sensor as required to the peripheral interface 11300 . According to another aspect, input/output peripherals may be coupled to peripheral interface 11300 without going through an input/output subsystem.

The power circuit 11500 may supply power to all or some of the components of the terminal. For example, the power circuit 11500 may include a power management system, one or more power sources such as batteries or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or a power source. It may include any other components for creation, management, and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, if necessary, the communication circuit 11600 may transmit and receive an RF signal, also known as an electromagnetic signal, including an RF circuit, thereby enabling communication with other computing devices.

This embodiment of FIG. 11 is only an example of the computing device 11000, and the computing device 11000 may omit some components shown in FIG. 11 or further include additional components not shown in FIG. 11, or 2 It may have a configuration or arrangement that combines two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. 11 , and various communication methods (WiFi, 3G, LTE) are provided in the communication circuit 11600 . , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 11000 may be implemented as hardware, software, or a combination of both hardware and software including an integrated circuit specialized for one or more signal processing or applications.

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. The application to which the present invention is applied may be installed in the computing device 11000 through a file provided by the file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file in response to a request of the computing device 11000 .

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computing devices, and may be stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

According to an embodiment of the present invention, when three-dimensional modeling of the interior of a logistics warehouse, a person does not need to model each object one by one, and by automatically modeling the necessary objects only with lidar sensing data and WMS data, It can have the effect of being able to quickly and immediately update your warehouse modeling.

According to an embodiment of the present invention, by using a machine learning model to determine what kind of logistics the corresponding logistics is with only the external information of the lidar sensing data using a machine learning model, and then performing 3D modeling on the corresponding logistics, manpower and time consumption can be minimized. possible effect can be exerted.

According to an embodiment of the present invention, in operating a smart warehouse, it is possible to intuitively check the inventory of the logistics warehouse.

According to an embodiment of the present invention, it is possible to exhibit the effect of helping the autonomous driving learning or operation of smart mobility operated in a logistics warehouse.

According to an embodiment of the present invention, it is possible to achieve the effect of obtaining accurate three-dimensional modeling data by materializing unclear object information obtained through lidar sensing data through WMS data.

According to an embodiment of the present invention, by generating learning data for machine learning through simulation using the lidar sensing data, it is possible to exhibit the effect of increasing the accuracy of object classification using the machine learning model.

According to an embodiment of the present invention, by using the attribute information of the logistics included in the WMS data, if you touch the logistics modeled through the manager terminal remotely, you can see the attribute information for the logistics, through this, the logistics It can exert the effect of performing inventory logistics management of the warehouse.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result. Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (8)

As an automatic three-dimensional modeling method of a warehouse using lidar sensing data and WMS data, performed by a computing device having one or more processors and one or more memories,
The WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each logistics stored in the warehouse,
A lidar sensing data receiving step of receiving the lidar sensing data by scanning one or more objects in the warehouse, and generating point cloud data;
a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data;
By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step;
By applying the shape information and size information of each logistics included in the WMS data to the object attribute information of each object included in the 3D surface data derived in the object classification step, each included in the 3D surface data a WMS fitting data generating step of generating WMS fitting data including the location of the object in the global coordinate system and identification information on the WMS data; and
A modeling generation step of generating logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data;
The WMS fitting data generation step is,
Using the second machine learning model, each of the plurality of 3D surface data classified into the first group as a result of performing the object classification step is compared with the WMS data, and each 3D surface data classified into the first group After calculating the degree of similarity with the logistics information in the WMS data, detailed modeling is performed on each three-dimensional surface data classified into the first group based on the logistics information showing the highest degree of similarity. 3D modeling method.
The method according to claim 1,
The lidar sensing data receiving step includes:
receiving, by the SLAM module, the lidar sensing data from the lidar; and
Creating, by the SLAM module, point cloud data that is an aggregate of a plurality of points having position coordinates in the global coordinate system for one or more objects inside the warehouse using the lidar sensing data; of an automatic 3D modeling method.
3. The method according to claim 2,
Each point included in the point cloud data includes vector information,
The surface data generation step is
Based on the vector information of each point, a surface creation operation is performed to fill a plurality of empty spaces existing in the point cloud data, and the surface creation operation is performed through a deep learning algorithm. modeling method.
The method according to claim 1,
The object classification step is
Using the first machine learning model for the three-dimensional surface data,
The first group corresponding to logistics;
A second group corresponding to the pallet supporting the logistics;
A third group corresponding to a rack for storing logistics; and
The first group, the second group, and the fourth group that does not correspond to any of the third group; an automatic three-dimensional modeling method of a distribution warehouse.
5. The method according to claim 4,
The modeling generation step is
recognizing global coordinates of a plurality of identification points of three-dimensional surface data of objects corresponding to the first group, the second group, and the third group;
mapping the three-dimensional modeling data of the corresponding logistics included in the WMS data to the global coordinates for the object corresponding to the first group; and
For the objects corresponding to the second group and the third group, specifying through the first machine learning model in accordance with global coordinates; comprising, an automatic three-dimensional modeling method of a warehouse.
delete As an automatic three-dimensional modeling device for a logistics warehouse using lidar sensing data and WMS data, implemented as a computing device having one or more processors and one or more memories,
The WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each logistics stored in the warehouse,
The three-dimensional modeling device,
A lidar sensing data receiving step of receiving the lidar sensing data by scanning one or more objects in the warehouse, and generating point cloud data;
a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data;
By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step;
Each object included in the three-dimensional surface data by applying the shape information and size information of each logistics to the WMS data to the object attribute information of each object included in the three-dimensional surface data derived in the object classification step a WMS fitting data generation step of generating WMS fitting data including the location in the global coordinate system of the WMS data and identification information on the WMS data; and
A modeling generation step of generating logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data;
The WMS fitting data generation step is,
Using the second machine learning model, each of the plurality of 3D surface data classified into the first group as a result of performing the object classification step is compared with the WMS data, and each 3D surface data classified into the first group After calculating the degree of similarity with the logistics information in the WMS data, detailed modeling is performed on each three-dimensional surface data classified into the first group based on the logistics information showing the highest degree of similarity. 3D modeling device.
A computer-readable medium for implementing an automatic three-dimensional modeling method of a warehouse using lidar sensing data and WMS data, performed by a computing device having one or more processors and one or more memories, comprising:
The WMS data includes identification information, shape information, size information, and three-dimensional modeling data for each logistics stored in the warehouse,
The computer-readable medium stores instructions for causing a computing device to perform the following steps:
A lidar sensing data receiving step of receiving the lidar sensing data by scanning one or more objects in the warehouse, and generating point cloud data;
a surface data generation step of generating three-dimensional surface data for one or more objects inside the warehouse based on the point cloud data;
By using the first machine learning model, each object included in the three-dimensional surface data is divided, and object property information including shape information and size information for each object included in the three-dimensional surface data is obtained. deriving object classification step;
Each object included in the three-dimensional surface data by applying the shape information and size information of each logistics to the WMS data to the object attribute information of each object included in the three-dimensional surface data derived in the object classification step a WMS fitting data generation step of generating WMS fitting data including the location in the global coordinate system of the WMS data and identification information on the WMS data; and
A modeling generation step of generating logistics warehouse modeling data by applying the three-dimensional modeling data of the corresponding logistics of the WMS data to the WMS fitting data;
The WMS fitting data generation step is,
Using the second machine learning model, each of the plurality of 3D surface data classified into the first group as a result of performing the object classification step is compared with the WMS data, and each 3D surface data classified into the first group After calculating the degree of similarity with the logistics information in the WMS data, a three-dimensional modeling computer that performs detailed modeling on each three-dimensional surface data classified into the first group based on the logistics information showing the highest similarity - readable media.

Images