WO2023068762A1

WO2023068762A1 - Method and device for acquiring information about solid particulate material having fluidity loaded in space having predetermined shape

Info

Publication number: WO2023068762A1
Application number: PCT/KR2022/015869
Authority: WO
Inventors: 지인찬; 오은송
Original assignee: (주)딥인사이트
Priority date: 2021-10-18
Filing date: 2022-10-18
Publication date: 2023-04-27

Abstract

According to some embodiments disclosed herein, disclosed is a method for acquiring information about a solid particulate material having fluidity and loaded in a space having a predetermined shape. The method comprises the steps of: acquiring a plurality of depth images of the solid particulate material, the depth images being captured by a plurality of 3D sensing cameras; generating a matched image from the plurality of depth images on the basis of capture position information about each of the plurality of 3D cameras, wherein the matched image includes a plurality of captured areas and uncaptured areas each corresponding to each of the plurality of depth images; and generating a corrected image by estimating the uncaptured areas from the captured areas on the basis of a correction algorithm.

Description

Method and apparatus for obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape

The present invention relates to computing technology, and more particularly, to a method and apparatus for obtaining information on solid particulate matter having fluidity such as grain loaded in a space having a predetermined shape, such as a grain storage tower.

A rice processing complex (RPC) refers to a facility that handles the entire grain process, from importation of grains to sorting, weighing, quality inspection, drying, storage, and milling, to product shipment, sales, and by-product processing. do. In RPC, there are a number of grain silos that store grains such as rice. Each grain silo is filled with different amounts of grain because each grain silo is different in terms of the amount of grain that is brought in and the amount that is discharged, when grain is brought in, when when grain is discharged, and the like.

Grain stored in the grain storage tower inevitably loses weight during processes such as drying, storage, and processing. Weight loss means that the weight of grains decreases due to moisture loss, removal of foreign substances, and grain respiration. Accurately grasping the weight loss of grain is very important in the grain processing process.

The best way to know the weight loss of grains stored in the grain storage tower is to actually measure it, but continuous measurement on RPC is practically difficult, and a lot of manpower and time are required for one measurement operation. Therefore, methods capable of appropriately estimating in real time the volume of grains loaded in the grain storage tower in the RPC are required.

[Prior art literature]

[Patent Literature]

(Patent Document 1) KR 20180130815 A

The present disclosure has been made in response to the above-described background art, and is intended to provide a method and apparatus for acquiring information on solid particulate matter having fluidity such as grain loaded in a space having a predetermined shape such as a grain storage tower. am.

A method for obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape for realizing the above object is disclosed. The method includes: acquiring a plurality of depth images of the solid particulate material by a plurality of 3D sensing cameras; generating a matched image from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras - the matched image includes a plurality of photographed areas and non-photographed areas corresponding to each of the plurality of depth images; Ham -; and generating a corrected image by estimating the non-captured area from the captured area based on a correction algorithm. can include

Alternatively, the method further comprises: determining height information of the solid particulate material based on the calibrated image; may further include.

Alternatively, the method includes: determining volume information of the solid particulate material based on height information of the solid particulate material and the predetermined shape; may further include.

Alternatively, generating a corrected image by estimating the uncaptured area based on the plurality of captured areas may include: determining depth information on the uncaptured area by using depth change information on the plurality of captured areas; doing; can include

Alternatively, the determining of depth information on the non-photographed area using depth change information on the plurality of captured areas may include: generating a plurality of divided images by dividing the matched image; determining depth information on an uncaptured area included in the first split image by using depth change information on a captured area included in a first split image among the plurality of split images; and determining depth information on an uncaptured area included in the second split image by using depth change information on a captured area included in a second split image different from the first split image among the plurality of split images. ; can include

Alternatively, generating a corrected image by estimating the non-captured area based on the plurality of captured areas may include: applying an image smoothing algorithm to the corrected image; can include

Alternatively, the calibration algorithm may include polynomial fitting.

Alternatively, a space having a predetermined shape may contain a grain silo, and the solid particulate material may contain grain.

Alternatively, the plurality of 3D sensing cameras may include three 3D sensing cameras positioned at an angle of 120 degrees from each other.

A computer program stored in a computer readable storage medium for realizing the above object is disclosed. When the computer program is executed on one or more processors, the operations for performing a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape are performed, and the operations include: a plurality of obtaining a plurality of depth images of the solid particulate material by a 3D sensing camera; generating a matched image from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras, wherein the matched image includes a plurality of captured areas and non-photographed areas corresponding to each of the plurality of depth images; Ham -; and generating a corrected image by estimating the non-captured area from the captured area based on a correction algorithm.

Disclosed is a computing device that performs a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape for realizing the above object. The computing device includes: a memory including computer-executable components; a processor executing the following computer-executable components stored in a memory; The processor acquires a plurality of depth images obtained by photographing the solid particulate material by a plurality of 3D sensing cameras, and obtains a plurality of depth images from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras. generating a registered image, wherein the registered image includes a plurality of captured regions and uncaptured regions corresponding to each of the plurality of depth images, and estimating the uncaptured region from the captured region based on a correction algorithm; A correction image can be created.

The present disclosure may provide a method and apparatus for acquiring information on a solid particulate material having fluidity such as grain loaded in a space having a predetermined shape such as a grain storage tower.

FIG. 1 is a block diagram of a computing device performing a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape according to some embodiments of the present disclosure.

2 is a diagram for explaining a 3D sensing camera according to some embodiments of the present disclosure.

3 is a diagram for explaining a plurality of 3D sensing cameras installed in a space having a predetermined shape according to some embodiments of the present disclosure.

4 is a schematic diagram for explaining a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape according to some embodiments of the present disclosure.

5 is a schematic diagram illustrating a step of generating a corrected image according to some embodiments of the present disclosure.

6 shows an example registered image and corrected image in accordance with some embodiments of the present disclosure.

7 is a flow chart of a method for obtaining information about a solid particulate material having fluidity loaded in a space having a predetermined shape according to some embodiments of the present disclosure.

8 is a block diagram of a computing device according to some embodiments of the present disclosure.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is apparent that these embodiments may be practiced without these specific details.

The terms “component,” “module,” “system,” and the like, as used herein, refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or an execution of software. For example, a component may be, but is not limited to, a procedure, processor, object, thread of execution, program, and/or computer running on a processor. For example, both an application running on a computing device and a computing device may be components. One or more components may reside within a processor and/or thread of execution. A component can be localized within a single computer. A component may be distributed between two or more computers. Also, these components can execute from various computer readable media having various data structures stored thereon. Components may be connected, for example, via signals with one or more packets of data (e.g., data and/or signals from one component interacting with another component in a local system, distributed system) to other systems and over a network such as the Internet. data being transmitted) may communicate via local and/or remote processes.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless otherwise specified or clear from the context, “X employs A or B” is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or, if X uses both A and B, "X uses either A or B" may apply to either of these cases. Also, the term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" should be understood to mean that the features and/or components are present. However, it should be understood that the terms "comprises" and/or "comprising" do not exclude the presence or addition of one or more other features, elements, and/or groups thereof. Also, unless otherwise specified or where the context clearly indicates that a singular form is indicated, the singular in this specification and claims should generally be construed to mean "one or more".

In addition, the term “at least one of A or B” should be interpreted as meaning “when only A is included”, “when only B is included”, and “when A and B are combined”.

Those skilled in the art will further understand that the various illustrative logical blocks, components, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented using electronic hardware, computer software, or combinations of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of this disclosure.

The description of the presented embodiments is provided to enable any person skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art of this disclosure. The general principles defined herein may be applied to other embodiments without departing from the scope of this disclosure. Thus, the present invention is not limited to the embodiments presented herein. The present invention is to be accorded the widest scope consistent with the principles and novel features set forth herein.

As shown in FIG. 1 , the computing device 100 may include a processor 110 , a memory 130 , and a network unit 150 . The configuration of the computing device 100 shown in FIG. 1 is only a simplified example. In some embodiments of the present disclosure, the computing device 100 may include other components for performing a computing environment of the computing device 100, and only some of the disclosed components may constitute the computing device 100.

The processor 110 may include one or more cores, and includes a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. unit), data analysis and processing, and processors for deep learning. The processor 110 may read a computer program stored in the memory 130 to perform data conversion, calculation, generation, and the like for a method of obtaining information on a solid particulate material according to some embodiments of the present disclosure. In addition, according to some embodiments of the present disclosure, the processor 110 may process an operation for implementing a method in which at least one of the CPU, GPGPU, and TPU of the processor 110 obtains information on the solid particulate material. . For example, a CPU and a GPGPU may together process calculations for implementing a method for obtaining information on solid particulate matter. In addition, in some embodiments of the present disclosure, data conversion, calculation, generation, learning of a network function, and data using a network function for performing a method of obtaining information on a solid particulate material by using processors of a plurality of computing devices together. classification can be handled. In addition, a computer program executed in a computing device according to some embodiments of the present disclosure may be a CPU, GPGPU or TPU executable program.

According to some embodiments of the present disclosure, the memory 130 may store any type of information generated or determined by the processor 110 and any type of information received by the network unit 150 . For example, the memory 130 may store data generated in the process of performing an operation for performing a method of obtaining information on solid particulate material by the processor 110 . Also, the memory 130 may store data received from the outside during the process of performing a method of obtaining information on the solid particulate material by the processor 110 . For example, the memory 130 may store a depth image obtained from a 3D sensing camera. However, it is not limited thereto, and the memory 130 may store various information for performing a method of obtaining information on a solid particulate material according to some embodiments of the present disclosure.

According to some embodiments of the present disclosure, the memory 130 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg, SD or XD memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) -Only Memory), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The computing device 100 may operate in relation to a web storage that performs a storage function of the memory 130 on the Internet. The above description of the memory is only an example, and the present disclosure is not limited thereto.

The network unit 150 according to some embodiments of the present disclosure may use any type of known wired or wireless communication system.

The network unit 150 may transmit and receive information processed by the processor 110, a user interface, and the like through communication with other terminals. For example, the network unit 150 may provide a user interface generated by the processor 110 to a client (eg, a user terminal). In addition, the network unit 150 may receive an external input of a user authorized as a client and transmit it to the processor 110 . At this time, the processor 110 may process operations such as outputting, correcting, changing, adding, and the like of information provided through the user interface based on the user's external input received from the network unit 150 .

Specifically, for example, the network unit 150 may transmit/receive various types of information for performing a method of acquiring information on a solid particulate material through the memory 130 according to some embodiments of the present disclosure. For example, the network unit 150 may transmit and receive a depth image or a correction image. In addition, the network unit 150 may externally transmit some data generated in the process of performing a method of obtaining information on solid particulate matter described below to be stored in a database.

Meanwhile, the computing device 100 according to some embodiments of the present disclosure may include a server as a computing system that transmits and receives information through communication with a client. In this case, the client may be any type of terminal capable of accessing the server. For example, the computing device 100 that is a server may receive a query from a user terminal and generate a single information processing result corresponding to the query. In this case, the computing device 100 serving as a server may provide a user interface including a processing result to the user terminal. At this time, the user terminal may output a user interface received from the computing device 100 as a server, and receive or process information through interaction with the user.

In an additional embodiment, the computing device 100 may include any type of terminal that receives data resources generated by any server and performs additional information processing.

2 is a diagram for explaining a 3D sensing camera according to some embodiments of the present disclosure. 3 is a diagram for explaining a plurality of 3D sensing cameras installed in a space having a predetermined shape according to some embodiments of the present disclosure.

Since a camera or sensing device such as a 3D sensing camera has a limited field of view, an area capable of being photographed may be limited. Referring to FIG. 2 , a photographing area according to a distance of a 3D sensing camera having a viewing angle of 60 °H x 49.5 °V is shown. As shown in FIG. 2 , the capturing area of the 3D sensing camera may increase as the capturing distance from the object to be imaged increases. However, it may be difficult to increase the photographing distance in order to photograph a large area due to limitations in space in which the 3D sensing camera is installed and limitations in performance of the camera, such as resolution. Accordingly, a plurality of 3D sensing cameras need to be used to photograph a space having a large area.

Referring to FIG. 3 , an exemplary embodiment is shown in which three 3D sensing cameras are installed in a grain silo to measure the height or volume of grains stacked in the silo. When there are no grains accumulated in the grain silos, the three 3D sensing cameras can capture the entire area of the silos as the shooting distance increases. However, as the height of the grains accumulated in the grain storage tower increases, the distance between the camera and the surface of the grains becomes shorter, so the area that can be captured by the 3D sensing camera may gradually decrease. As the area capable of being photographed decreases, it becomes difficult to accurately measure the height at which all grains are piled up due to non-photographed parts, and this problem makes it impossible to accurately measure the amount (or volume) of all grains stored in the grain storage tower. The present disclosure provides a method and apparatus capable of accurately measuring the height and volume of grains accumulated in a large space such as a grain storage tower using a limited number of 3D sensing cameras by estimating the non-photographable area using the photographing area. can provide

4 is a schematic diagram for explaining a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape according to some embodiments of the present disclosure. 5 is a diagram for explaining a step of generating a corrected image according to some embodiments of the present disclosure. 6 shows an example registered image and corrected image in accordance with some embodiments of the present disclosure.

Hereinafter, with reference to FIGS. 4 to 6 , a processor 110 for performing a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape according to some embodiments of the present disclosure. An exemplary operation of is described.

In some embodiments of the present disclosure, the processor 110 may acquire a plurality of depth images obtained by photographing a solid particulate material by a plurality of 3D sensing cameras.

The processor 110 may acquire a plurality of depth images from a plurality of 3D sensing cameras installed in a space having a predetermined shape. For example, the processor 110 may receive depth images from a plurality of 3D sensing cameras through the network unit 150 .

In some examples, the space having a predetermined shape includes, but is not limited to, a storage tower having a cylindrical shape. In some instances, the solid particulate material may include various solid materials such as grain, sand, gravel, boxes, and the like. The solid particulate material does not have a fixed shape and may have various shapes depending on the order, method, and amount of loading in a space having a predetermined shape. In the following, an example is described for grain loaded in a grain silos, but the method of the present disclosure can be used for a variety of solid particulate materials loaded in a variety of spaces.

In some embodiments according to the present disclosure, three 3D sensing cameras may be installed in the grain silos. The three 3D sensing cameras may overlap each other and be positioned at a certain distance and at a certain angle so as to secure as much shooting area as possible while reducing the shooting area. For example, three 3D sensing cameras may be installed near the ceiling of a grain silo at the same height and at an angle of 120 degrees from each other. However, it is not limited thereto, and the number and installation positions of the 3D sensing cameras may vary.

According to some embodiments of the present disclosure, the processor 110 may generate a matching image from a plurality of depth images based on photographing position information of each of a plurality of 3D sensing cameras. In this case, the matched image may include a plurality of captured areas and non-photographed areas corresponding to each of the plurality of depth images.

In some instances, the processor 110 may measure the amount (or volume) of grain stored in the grain silos by measuring the height of the entire grain surface 220 loaded in the silos having a cylindrical shape. To this end, the processor 110 may generate the matched image 200 using a plurality of depth images obtained by partially photographing the surface 220 of all grains loaded in the grain storage tower.

Referring to (a) of FIG. 4 , an exemplary matched image 200 generated using a plurality of depth images is shown. The processor 110 may generate the matched image 200 from a plurality of depth images by using information on the photographing location of each of the plurality of 3D sensing cameras. For example, the processor 110 may generate a matched image by correcting depth information included in the depth image according to photographing location information of a 3D sensing camera using an image matching algorithm. In this case, each of the plurality of capturing

areas

210a, 210b, and 210c may correspond to one depth image. For example, the first capturing area 210a may be created using depth information included in a first depth image captured by the first 3D sensing camera. Also, the second capturing area 210b may be generated using depth information included in a second depth image captured by the second 3D sensing camera. Also, the third capturing area 210c may be generated using depth information included in a third depth image captured by a third 3D sensing camera. In this case, the matched image 200 may include a plurality of captured

areas

210a, 210b, and 210c having depth information and an unphotographed area having no depth information.

According to some embodiments of the present disclosure, the processor 110 may generate a corrected image by estimating a non-photographed area from a captured area based on a correction algorithm.

In order to accurately measure the amount (or volume) of grain stored in the grain silos, it may be necessary to obtain depth information for the entire grain surface 120 loaded in the silos. To this end, the processor 110 may estimate the non-photographed area using depth information of the captured area using a correction algorithm. When the solid granular material having fluidity such as grain is loaded into the grain storage tower, the average height of the entire grain surface 120 may tend to increase while the granular material in a higher position naturally moves to a lower position according to gravity. . In this case, the depth information of the entire grain surface 120 may change with a generally continuous gradient. Accordingly, the correction algorithm may predict depth information on the non-photographed area by using the change value of the depth information on the captured area.

In some examples, the calibration algorithm may include polynomial fitting. It is not limited thereto, and the correction algorithm may include various interpolation algorithms. As another example, the correction algorithm may include an algorithm using a neural network.

Referring to FIG. 5 , a specific embodiment of generating a corrected image is described.

According to some embodiments of the present disclosure, the processor 110 may determine depth information on an uncaptured area by using depth change information on a plurality of captured areas.

As described above, the depth information appearing on the entire grain surface 120 of the solid particulate material having grain-like fluidity may have a generally continuous slope. Using this tendency, a correction algorithm such as polynomial fitting may predict depth information on an uncaptured area using a change value (depth change information) of depth information on a capture area.

In some examples, when determining depth information on an uncaptured area using depth change information on a plurality of captured areas, the processor 110 may generate a plurality of divided images by dividing the matched image 300. there is. In order to increase the processing speed for generating the corrected image 300, the registered image 200 may be divided into a predetermined number of divided images and processed. For example, referring to (a) of FIG. 5 , the matched image 200 may be divided into four divided

images

230a, 230b, 230c, and 240d.

When a plurality of divided images are acquired, the processor 110 uses the depth change information on the captured region included in the first divided image among the plurality of divided images to determine the depth on the non-photographed region included in the first divided image. information can be determined. Further, depth information may be determined on an uncaptured area included in the second split image by using depth change information on a captured area included in a second split image different from the first split image among the plurality of split images.

Specifically, the processor 110 may determine depth information of an uncaptured area using depth change information on a captured area included in each of a plurality of divided images by a correction algorithm. For example, referring to (b) of FIG. 5 , the processor 110 uses the depth change information of the first capturing area 210a included in the first divided image 230a by a correction algorithm to obtain a first image. Depth information of an unphotographed area may be determined on the divided image 230a. Similarly, the processor 110 uses the depth change information of the second captured area 210b included in the second split image 230b by the correction algorithm to determine the non-photographed area on the second split image 230b. Depth information of can be determined. In addition, the processor 110 may use the depth change information of the first and

third capturing areas

210a and 210c included in the third split image 230c by a correction algorithm to capture the third split image 230c. ), it is possible to determine depth information of an unphotographed area. In addition, the processor 110 uses the depth change information of the second and

third capturing regions

210a and 210c included in the fourth divided image 230d by a correction algorithm to capture the fourth divided image 230d. ), it is possible to determine depth information of an unphotographed area. Referring to (c) of FIG. 5 , a corrected image in which depth information of an uncaptured area is determined using depth change information of a capture area included in each of four divided images is shown.

According to some embodiments of the present disclosure, when generating a corrected image by estimating an uncaptured area based on a plurality of captured areas, the processor 110 may apply an image smoothing algorithm to the corrected image. there is.

When generating the depth information of a non-captured area using depth change information of a plurality of different captured areas, the depth information of the non-captured area is discontinuous, such as a boundary line generated by estimating the depth information from different captured areas A region that changes with a slope may occur. As described above, since the depth information appearing on the entire grain surface 120 of the solid particulate material having grain-like fluidity generally has a continuous slope, the area where the depth information changes to a discontinuous slope needs to be corrected there is. To this end, the processor 110 may apply an image smoothing algorithm to the corrected image 300 . The corrected image 300 to which the image smoothing algorithm is applied may have depth information that changes with a continuous gradient, similar to grains actually loaded in a grain storage tower. Referring to (d) of FIG. 5 , an exemplary corrected image 300 to which an image smoothing algorithm is applied is shown.

According to some embodiments of the present disclosure, the processor 110 may determine height information of the solid particulate material based on the calibrated image. Also, the processor 110 may determine volume information of the solid particulate material based on height information and a predetermined shape of the solid particulate material.

The processor 110 may obtain information about the solid particulate material by using the depth information of the corrected image 300 . For example, since the corrected image 300 has depth information of the entire grain surface 120 of the grains loaded in the grain storage tower, the processor 110 can determine height information of the solid particulate material that is the grain. First, by taking only the region 220 corresponding to the shape of the grain silos from the corrected image to which the image smoothing algorithm is applied, depth information of the entire grain surface 120 of the grains loaded in the silos can be obtained. Also, the processor 110 may determine volume information of the solid particulate material based on the height information and the predetermined shape of the solid particulate material. Since the solid particulate material can be stacked on top of each other with almost no vacant space on a space having a predetermined shape (eg, a grain storage tower), the processor 110 uses the height information of the solid particulate material and the predetermined shape. to determine the total volume of the solid particulate material. For example, the processor 110 may determine the average height using height information of grains. Further, the processor 110 may obtain information on the total volume of grains loaded in the grain storage tower by multiplying the average height by the cylindrical area of the grain storage tower. However, it is not limited thereto, and the processor 110 may obtain information on the solid particulate material using various methods.

Referring to FIG. 6A , a matched image shown as a 3D diagram showing the volume of some grains loaded in a grain storage tower is shown using depth information of a photographed area obtained by three depth images. Referring to FIG. 6B, the volume of all grains loaded in the grain storage tower obtained by determining the depth information of the non-photographed region by a correction algorithm using the depth change information of the photographed region obtained by three depth images A calibrated image shown in 3D drawing is shown. As can be seen with reference to FIG. 6 , depth information of an uncaptured area may be obtained (or predicted) using depth information of a captured area. As described above, the present disclosure can accurately measure the height and volume of grains piled up in a large space such as a grain storage tower using a limited number of 3D sensing cameras by estimating an area that cannot be filmed using a filming area. A method and apparatus may be provided.

According to some embodiments of the present disclosure, a method for obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape includes a plurality of depth images of a solid particulate material photographed by a plurality of 3D sensing cameras. It may include an acquiring step (s100).

According to some embodiments of the present disclosure, the method may include generating a matched image from a plurality of depth images based on photographing position information of each of a plurality of 3D sensing cameras ( S200 ). The matching image may include a plurality of captured areas and non-photographed areas corresponding to each of a plurality of depth images.

In some instances, a space having a predetermined shape includes a grain silo, and the solid particulate material may include grain. In some examples, the plurality of 3D sensing cameras may include three 3D sensing cameras positioned at an angle of 120 degrees from each other.

According to some embodiments of the present disclosure, the method may include generating a corrected image by estimating the non-photographed area from the captured area based on a correction algorithm (S300).

Alternatively, generating a corrected image by estimating the uncaptured area based on the plurality of captured areas (s200) includes determining depth information on the uncaptured area using depth change information on the plurality of captured areas. and applying an image smoothing algorithm to the corrected image.

Alternatively, determining depth information on an uncaptured area using depth change information on a plurality of captured areas may include: generating a plurality of divided images by dividing the matched image; first of the plurality of divided images; Determining depth information on an uncaptured area included in a first split image by using depth change information on a captured area included in the split image, and a second split image different from the first split image among the plurality of split images Determining depth information on an uncaptured area included in the second divided image by using depth change information on a captured area included in .

In some examples, the calibration algorithm may include polynomial fitting.

According to some embodiments of the present disclosure, the method may further include determining height information of the solid particulate material based on the calibrated image.

According to some embodiments of the present disclosure, the method may further include determining volume information of the solid particulate material based on height information and a predetermined shape of the solid particulate material.

The steps of the method for generating candidate drug information described above are only presented for explanation, and some steps may be omitted or additional steps may be added. Also, the steps described above may be performed in any order.

8 depicts a simplified and general schematic diagram of an example computing environment in which some embodiments of the present disclosure may be implemented.

Although the present disclosure has generally been described above in terms of computer executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in conjunction with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Further, it will be appreciated by those skilled in the art that the methods of the present disclosure may be used in single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like (each of which is It will be appreciated that other computer system configurations may be implemented, including those that may be operative in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer readable media. Any medium that can be accessed by a computer can be a computer readable medium. Computer readable media includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer readable media may include computer readable storage media and computer readable transmission media. Computer readable storage media are volatile and nonvolatile media, transitory and non-transitory, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device; or any other medium that can be accessed by a computer and used to store desired information.

Computer readable transmission media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal, such as other transport mechanisms, and includes any information delivery media. do. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information within the signal. By way of example, and not limitation, computer readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer readable transmission media.

An exemplary environment 1100 implementing various aspects of the present disclosure is shown including a computer 1102, which includes a processing unit 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to system memory 1106 , to processing unit 1104 . Processing unit 1104 may be any of a variety of commercially available processors. Dual processor and other multiprocessor architectures may also be used as the processing unit 1104.

System bus 1108 may be any of several types of bus structures that may additionally be interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read only memory (ROM) 1110 and random access memory (RAM) 1112 . A basic input/output system (BIOS) is stored in non-volatile memory 1110, such as ROM, EPROM, or EEPROM, and is a basic set of information that helps transfer information between components within computer 1102, such as during startup. contains routines. RAM 1112 may also include high-speed RAM, such as static RAM, for caching data.

Computer 1102 also includes an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA) - this internal hard disk drive 1114 can also be configured for external use within a suitable chassis (not shown). Yes-, magnetic floppy disk drive (FDD) 1116 (e.g., for reading from or writing to removable diskette 1118), and optical disk drive 1120 (e.g., CD-ROM) for reading disc 1122 or reading from or writing to other high capacity optical media such as DVDs). The hard disk drive 1114, magnetic disk drive 1116, and optical disk drive 1120 are connected to the system bus 1108 by a hard disk drive interface 1124, magnetic disk drive interface 1126, and optical drive interface 1128, respectively. ) can be connected to The interface 1124 for external drive implementation includes at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of computer readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art can use zip drives, magnetic cassettes, flash memory cards, cartridges, etc. It will be appreciated that other tangible media readable by a computer, such as the like, may also be used in the exemplary operating environment and any such media may include computer executable instructions for performing the methods of the present disclosure.

A number of program modules may be stored on the drive and RAM 1112, including an operating system 1130, one or more application programs 1132, other program modules 1134, and program data 1136. All or portions of the operating system, applications, modules and/or data may also be cached in RAM 1112. It will be appreciated that the present disclosure may be implemented in a variety of commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1102 through one or more wired/wireless input devices, such as a keyboard 1138 and a pointing device such as a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, a parallel port, IEEE 1394 serial port, game port, USB port, IR interface, may be connected by other interfaces such as the like.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, computers typically include other peripheral output devices (not shown) such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communications. Remote computer(s) 1148 may be a workstation, computing device computer, router, personal computer, handheld computer, microprocessor-based entertainment device, peer device, or other common network node, and generally includes It includes many or all of the components described for, but for simplicity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, such as a wide area network (WAN) 1154 . Such LAN and WAN networking environments are common in offices and corporations and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to worldwide computer networks, such as the Internet.

When used in a LAN networking environment, computer 1102 connects to local network 1152 through wired and/or wireless communication network interfaces or adapters 1156. Adapter 1156 may facilitate wired or wireless communications to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, computer 1102 may include a modem 1158, be connected to a communicating computing device on WAN 1154, or establish communications over WAN 1154, such as over the Internet. have other means. A modem 1158, which may be internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for computer 1102, or portions thereof, may be stored on remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between computers may be used.

Computer 1102 is any wireless device or entity that is deployed and operating in wireless communication, eg, printers, scanners, desktop and/or portable computers, portable data assistants (PDAs), communication satellites, wireless detectable tags associated with It operates to communicate with arbitrary equipment or places and telephones. This includes at least Wi-Fi and Bluetooth wireless technologies. Thus, the communication may be a predefined structure as in conventional networks or simply an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) makes it possible to connect to the Internet without wires. Wi-Fi is a wireless technology, such as a cell phone, that allows such devices, eg, computers, to transmit and receive data both indoors and outdoors, i.e. anywhere within coverage of a base station. Wi-Fi networks use a radio technology called IEEE 802.11 (a,b,g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or on products that include both bands (dual band). there is.

Those skilled in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, instructions, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields s or particles, or any combination thereof.

Those skilled in the art will understand that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein are electronic hardware, (for convenience) , may be implemented by various forms of program or design code (referred to herein as “software”) or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" includes a computer program or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash memory. device (eg, EEPROM, card, stick, key drive, etc.), but is not limited thereto. Additionally, various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is to be understood that the specific order or hierarchy of steps in the processes presented is an example of exemplary approaches. Based upon design priorities, it is to be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of this disclosure. The accompanying method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art of this disclosure. The general principles defined herein may be applied to other embodiments without departing from the scope of this disclosure. Thus, the present disclosure is not to be limited to the embodiments presented herein, but is to be interpreted in the widest scope consistent with the principles and novel features presented herein.

As described above, the related contents have been described in the best mode for carrying out the invention.

Claims

A method for acquiring information on a solid particulate material having fluidity loaded in a space having a predetermined shape, comprising:

acquiring a plurality of depth images of the solid particulate material by a plurality of 3D sensing cameras;

generating a matched image from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras - the matched image includes a plurality of photographed areas and non-photographed areas corresponding to each of the plurality of depth images; Ham -; and

generating a corrected image by estimating the non-captured area from the captured area based on a correction algorithm;

including,

method.
According to claim 1,

determining height information of the solid particulate material based on the corrected image;

Including more,

method.
According to claim 2,

determining volume information of the solid particulate material based on height information of the solid particulate material and the predetermined shape;

Including more,

method.
According to claim 1,

The step of generating a corrected image by estimating the non-captured area based on the plurality of captured areas:

determining depth information on the uncaptured area by using depth change information on the plurality of captured areas;

including,

method.
According to claim 4,

The step of determining depth information on the uncaptured area using depth change information on the plurality of captured areas:

generating a plurality of divided images by dividing the registered image;

determining depth information on an uncaptured area included in the first split image by using depth change information on a captured area included in a first split image among the plurality of split images; and

determining depth information on an uncaptured area included in the second split image by using depth change information on a captured area included in a second split image different from the first split image among the plurality of split images;

including,

method.
According to claim 4 or 5,

The step of generating a corrected image by estimating the non-captured area based on the plurality of captured areas:

applying an image smoothing algorithm to the corrected image;

including,

method.
According to claim 4 or 5,

The correction algorithm includes polynomial fitting,

method.
According to claim 1,

A space having a predetermined shape includes a grain silo,

The solid particulate material comprises grain,

method.
According to claim 1,

The plurality of 3D sensing cameras include three 3D sensing cameras positioned at an angle of 120 degrees from each other,

method.
A computer program stored on a computer-readable storage medium, which, when executed on one or more processors, performs operations for performing a method of obtaining information on a solid particulate material having fluidity loaded in a space having a predetermined shape. and the operations are:

obtaining a plurality of depth images of the solid particulate material by a plurality of 3D sensing cameras;

generating a matched image from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras, wherein the matched image includes a plurality of captured areas and non-photographed areas corresponding to each of the plurality of depth images; Ham -; and

generating a corrected image by estimating the non-captured area from the captured area based on a correction algorithm;

including,

A computer program stored on a computer-readable storage medium.
A computing device performing a method of obtaining information about a solid particulate material having fluidity loaded in a space having a predetermined shape, the computing device comprising:

memory containing computer-executable components;

a processor executing the following computer-executable components stored in a memory; including,

the processor,

Obtaining a plurality of depth images of the solid particulate material by a plurality of 3D sensing cameras;

generating a matched image from the plurality of depth images based on photographing position information of each of the plurality of 3D sensing cameras, the matched image including a plurality of photographed regions and non-photographed regions corresponding to each of the plurality of depth images; -, and

generating a corrected image by estimating the non-captured area from the captured area based on a correction algorithm;

computing device.