JP7288568B1 - Automatic measurement system - Google Patents

Abstract

A two-dimensional code for parcels read by a camera is used in association with point cloud data for automatically measuring parcels. An automatic measuring system 100 includes a passage detection sensor 3, which is a sensor provided at a rectangular entrance of a baggage storage area and detects passage of a conveying means through the entrance, and four rectangular entrances. A ToF camera 1 for photographing an arbitrary subject is provided at each corner, and when a passage detection sensor detects passage of a transport means through a doorway, infrared light is emitted to the baggage and the infrared light is reflected from the baggage. Four ToF (Time of Flight) cameras 1 for receiving infrared light, a camera 2 capable of photographing an arbitrary subject and provided on either the left or right side of the entrance, and a ToF camera and an information management server 4 that manages images taken by the camera 1 and images taken by the camera 2. - 特許庁[Selection drawing] Fig. 1

Description

The present invention relates to an automatic measurement system that automatically measures a package being transported by a carrier vehicle or other transport means and that generates an image showing the three-dimensional shape of the package.

In recent years, with the globalization of transactions and the increase in demand for internet mail-order sales, the amount of packages to be delivered has been steadily increasing.
Under such circumstances, the number of packages being carried in and out of storage warehouses is naturally increasing.

When delivering packages, it is necessary to know the width, height, and depth of each package.
Although some companies have introduced an automatic measuring system to measure packages in such storage warehouses to improve efficiency, there are still many companies that still rely on manual measuring.

As the volume of distribution increases, measuring a very large number of packages one by one is a heavy workload, and some of the packages are quite heavy. It also doubles the fatigue of the person.
If the cargo to be handled can be automatically measured, the human resources involved in the measurement work will be freed up, and the human resources can be allocated to other work. It is thought that there is.

Furthermore, in recent years, QR (Quick Response) codes (registered trademark) have been used everywhere.
In the field of logistics as well, it is common practice to attach a QR code (registered trademark) to the packaging of a package, and to store and deliver the package by reading the package information embedded in this QR code (registered trademark). It has become.
The "package management by QR code (registered trademark)" function plays a role in supporting the efficiency of physical distribution in comparison with the above-mentioned "automatic measurement" function.

In recent years, against the background of remarkable improvements in camera technology, various "automatic measurement techniques" that apply cameras to baggage processing sites have been provided (see Patent Documents 1 and 2, for example).
According to the automatic measuring apparatus disclosed in Patent Document 1, portable three-dimensional fixed-length solid rulers 101 and 102 placed side by side in close contact with an object 100 to be measured and the object are photographed by a CCD camera or the like. The length and coordinates of each side of the object are determined with reference to the ruler (Ref. 1, paragraph 7).

Further, according to the container measurement system disclosed in Patent Document 2, two-dimensional image information including the container 12 is acquired by the two-dimensional information acquisition unit 42 (camera or the like) (Publication 2, paragraph 17), and further The three-dimensional information of the container 12 is acquired by the three-dimensional information acquiring unit 43 (3D sensor/distance measuring means) (ibid. 2, Paragraph 17).
Then, the calculation unit 53 calculates the three-dimensional position of the plane portion 13 that constitutes the container 12 based on the distance image D of the container 12, and calculates the three-dimensional information of the container 12 based on this three-dimensional position ( Ibid., Paragraph 2, Ibid., Paragraph 19).
The three-dimensional information of the container 12 includes three-dimensional position information of the feature point F (corner portion of the flat portion 13) and three-dimensional position information of the link L (side portion of the flat portion 13), It includes information on the dimensions of the link L, and includes information on the three-dimensional shape of the portion surrounded by the link L (the surface portion of the flat portion 13) (Patent Document 2, paragraph 38).

In addition, a "package management technology using a QR code (registered trademark)" using a camera has also been provided (see, for example, Patent Document 3).
According to the label reading system disclosed in Patent Literature 3, the passage sensor 6 is positioned between the arrangement position of the QR code photographing device 3 and the arrangement position of the pile photographing device 4 in the extending direction of the travel route F1. It is arranged and detects the passage of the forklift 10 on the traveling route F1 (the same document 3, paragraph 37).
The information processing terminal 8 acquires the information of the QR code (registered trademark) QRC from the information of the images received from the cameras 35a to 35d and 36a to 36d of the QR code photographing device 3, and obtains the information of the QR code (registered trademark) QRC. Information such as the type and number of parts contained in each parts box 2 is acquired from the information (Ref. 3, paragraph 42).
Then, the information processing terminal 8, in the image of the entire cargo pile photographed by the cargo pile photographing units 41 and 42 as described in paragraph 3 and paragraph 78 of the same document, reads the properly read QR code QRC and the QR code QRC that could not be read. can be specified (the same document 3, paragraph 80).

JP 2012-137304 A JP 2021-056543 A JP 2020-064334 A

The technique of Patent Document 1 assumes the use of a fixed-length solid ruler, and cannot be applied in an environment in which the object and the identification ruler cannot be captured in the same photographed image.

According to the technique of Patent Document 2, three-dimensional information of the container 12 can be obtained from the two-dimensional image information including the container 12 photographed by the two-dimensional information obtaining unit 42 (Reference 2, FIG. 2).
However, in Patent Literature 2, a camera (two-dimensional information acquisition unit 42) is provided only at one location (Patent Literature 2, FIG. 1), and a three-dimensional image showing the three-dimensional shape of the container 12 cannot be generated.

Further, according to the technique of Patent Document 3, a plurality of QR codes (registered trademark) attached to a plurality of packages are photographed in the same image, and a plurality of QR codes corresponding to the individual packages reflected in the image are captured. Information can be read from the code (registered trademark), and the efficiency of cargo management operations can be greatly improved.
However, the use of the QR code (registered trademark) in Document 3 is limited to linking it with two-dimensional image information, and the information of the QR code (registered trademark) is used as point cloud data (on the photographed image) obtained by automatically measuring the package. On the other hand, it is not assumed to be used in association with data added with distance information from the camera to the subject in the image).

The present invention has been made in view of the above problems, and an object of the present invention is to utilize a two-dimensional code for baggage read by a camera in association with point cloud data for automatically measuring the baggage. .

[First invention]
Therefore, in order to solve the above problems, the automatic measurement system according to the first invention of the present application includes:
An automatic measuring system capable of automatically measuring the size of a package transported by a transport vehicle or other transport means,

a passage detection sensor provided at a rectangular doorway of a luggage storage location, the sensor detecting passage of the conveying means through the doorway;
A ToF (Time of Flight) camera for photographing an arbitrary subject provided one each at each of the four corners of the rectangular doorway, when the passage detection sensor detects passage of the doorway by the conveying means four ToF (Time of Flight) cameras that irradiate the baggage with infrared light and receive infrared light reflected from the baggage;
a camera capable of photographing an arbitrary subject, the camera being provided on either the left or right side of the doorway;
an information management server that manages images captured by the ToF camera and images captured by the camera;
and

Each of the four ToF (Time of Flight) cameras,
a photographing means for photographing a parcel vehicle image in which the parcels being conveyed by the conveying means and the parcels being conveyed by the conveying means are photographed when the passage detection sensor detects passage of the entrance/exit of the conveying means;
Based on the light reception time required from when the passage detection sensor detects the passage of the entrance/exit of the transportation means to when the infrared light is emitted to the baggage until the infrared light reflected by the same baggage is received. a distance calculation means for obtaining the depth distance between the subject portion reflected in each pixel of the ToF camera and the ToF camera;
generating a data set of each pixel of the cargo vehicle image including depth coordinates in the cargo vehicle image determined according to the depth distance obtained by the distance calculating means and a predetermined color indicating a division according to each depth distance; a point cloud data generating means for generating point cloud data by coloring each pixel of the cargo vehicle image with a predetermined color corresponding to the depth distance of each pixel based on the generated data set;
Point cloud data transmission means for transmitting the point cloud data to an information management server;
has

The camera is
Code photographing for photographing a code image in which a package being transported by the transport means and the transport means and a two-dimensional code affixed to the same load are photographed when the passage detection sensor detects passage of the entrance/exit of the transport means. means and
code image transmission means for transmitting the code image to an information management server;
has

The information management server is
Measure the width, height, and depth of the cargo reflected in the point cloud data in the four point cloud data generated from the four cargo vehicle images of the same subject taken by each ToF camera from each corner of the entrance/exit. death,
Synthesizing the four pieces of point cloud data to generate a single package stereoscopic image showing the three-dimensional shape of the package reflected in the point cloud data,
a measuring process of associating and storing the measured width, height, and depth of the luggage indicated by the stereoscopic image of the luggage;

recognizing a two-dimensional code attached to each package reflected in the code image and reading package information indicated by the two-dimensional code;
recognizing individual packages appearing in the code image based on the recognized two-dimensional code;
a code reading process for determining whether or not a plurality of vertically stacked packages are reflected in the code image based on a two-dimensional code affixed to each package;

When it is determined that a plurality of packages stacked up and down are reflected in the code image,
identifying a code relative position in the three-dimensional image of the baggage corresponding to the position of the two-dimensional code reflected in the code image;
The two-dimensional code reflected in the code relative position and the package information indicated by the two-dimensional code are stored in association with the package stereoscopic image for which the code relative position is specified.

According to the first aspect of the present application, when the passage detection sensor 3 detects that the conveying means (the conveying vehicle VH in FIG. 2) has passed through the entrance GW, the camera 2 for photographing the two-dimensional code detects the two-dimensional code CD (see FIG. 4), four ToF cameras 1-1 to 1-4 (FIG. 3) shoot the luggage vehicle image P _CV (see FIG. 8A).
Further, each of the four ToF cameras 1-1 to 1-4 generates "point cloud data PD" (see FIG. 8(b)).
The information management server 4 receives the code image P _CD (FIG. 11) captured by the camera 2 and the point cloud data PD generated by the ToF camera 1, and performs two-dimensional code recognition (decoding) processing and three-dimensional point cloud data analysis (dimension measurement) processing.

By doing so, the first invention obtains from four luggage vehicle images _PCV showing the same subject photographed by four ToF cameras 1-1 to 1-4 from each corner of the entrance/exit GW of the luggage storage area. Based on the generated point cloud data PD, the size (width, height, depth) of the package reflected in the point cloud data PD is measured.
As a result, it is possible to "automatically measure the size of the cargo CR while it is being transported by the transport vehicle VH" without human intervention, and furthermore, human resources freed from the measuring work can perform other tasks. is also possible.
In addition, it is possible to proceed with the measurement of the load CR (loads CR-1 and CR-2 in the example of FIG. 2) being transported by the transport vehicle VH without any hindrance to the work of transporting the load by the transport vehicle VH.
Incidentally, the above-mentioned "conveying means" may be a conveying vehicle VH typified by a forklift, a belt conveyor, or the like.

Further, according to the first invention, the information management server 4 stores four parcel vehicle images P _CV (FIGS. 6A to 6B) captured by the ToF cameras 1-1-1-4. 7(a) to 7(b)), and by synthesizing the four pieces of point cloud data PD (FIG. 8(b)), the three-dimensional image of the package CR reflected in the point cloud data PD is obtained. "One parcel stereoscopic image _P3D " (see FIGS. 17(a) and 17(b)) showing a three-dimensional shape is generated.

In addition, in the first invention, a code image photographed by a camera is used without visually discriminating the individual packages by a worker engaged in handling work of the packages CR (packages CR-1 and CR-2 in FIG. 2). Based on the _PCD , it is possible to "automatically discriminate individual packages CR being transported by the transport vehicle VH".
In this way, by being able to identify each package, the number of packages CR can be automatically counted, and furthermore, the number of personnel required to load these packages and the number of delivery vehicles required for delivery can be determined. It is also possible to automatically predict the number of vehicles.

Furthermore, according to the first invention, when it is determined that a plurality of vertically stacked packages are reflected in the code image P _CD (FIG. 11), they are reflected in the code image P _CD (FIG. 11). Then, the relative positions of the codes in the stereoscopic image P _3D (FIGS. 17A and 17B) corresponding to the positions of the two-dimensional codes CD (CD-1 and CD-2 in FIG. 11) are identified.
Also, the two-dimensional code CD reflected in the code image PCD and the package information _ICR indicated by the two-dimensional code _CD are stored in association with the three-dimensional package image _P3D showing the three-dimensional shape of the package. do.
Here, the "parcel information I _CR " is information indicating attributes related to the parcel CR transported by the transport vehicle VH.
Specific examples of the above-mentioned "parcel information _ICR " include information such as the name of the product handled as the parcel CR, the product number, the number of products, and the weight of the product.

[Second invention]
In order to solve the above problems, an automatic measurement system according to a second invention of the present application is the automatic measurement system according to the first invention,
The ToF camera is
Even if the package to be measured is a low-reflection object that absorbs infrared light, it is configured so that point cloud data can be generated from the photographed image of the package vehicle.

Black or dark colored objects (low reflectance objects) absorb infrared light.
Therefore, in general, even if a ToF camera is used to photograph a low-reflectance object, it is difficult to obtain point cloud data of the low-reflectance object reflected in the captured image.
According to the trial by the inventor of the present application, even if ToF cameras 1 are installed at two corners above the entrance/exit GW of the baggage storage area, based on the luggage vehicle image _PCV taken by these two ToF cameras 1, It was not possible to obtain point cloud data PD for packages made of low-reflection objects.

However, the present inventor increased the number of ToF cameras 1 to 4 and installed a total of 4 ToF cameras 1 at each of the four corners of the rectangular entrance GW. , it became possible to generate point cloud data PD even for packages CR consisting of black or dark low-reflectivity objects, and automatic measurement of low-reflectivity objects could also be realized.
This is accompanied by an increase in the number of ToF cameras 1, and the total amount of infrared light emitted from the ToF cameras 1 to the low-reflecting object increases, and the infrared light reflected by the low-reflecting object increases. It is considered that the light exceeded the detection limit of the light receiving means 15 (FIG. 5).

According to the automatic measurement system according to the second invention, four ToF cameras 1 are installed at each of the four corners of the rectangular entrance GW. Even if there is, the point cloud data DP can be acquired, and the same parcel CR can be automatically measured based on the data DP.
Note that the ToF cameras 1 are arranged at the corners of a rectangle at approximately equal intervals so as to surround the subject (parcel CR) as the center (the center of the rectangular doorway GW).
In this example, the number of ToF cameras 1 is four, but the number of ToF cameras 1 may be increased to five or more as long as the ToF cameras 1 are installed at each corner of the entrance/exit GW.

The technical merits of adopting the ToF camera 1 are that the device (ToF camera 1) itself is equipped with the light irradiating means 14 and the light receiving means 15, so that "the device configuration can be downsized", and Since infrared light is used, it is possible to take pictures even in a dark place where luggage is stored.

[Third invention]
In order to solve the above problems, an automatic measurement system according to a third invention of the present application is an automatic measurement system according to the first or second invention,

When the transportation means is a forklift,
The information management server
A process of cutting the data portion in which the cargo and the forklift claw portion are integrated in the point cloud data by the cutting surface at the cutting position at predetermined intervals along the axial direction of the forklift claw portion having a long shape;
A process of calculating a cross-sectional area cut by a cutting plane at each of the cutting positions;
A process of sequentially comparing the cross-sectional areas cut by each cutting surface at two cutting positions that are consecutively back and forth to calculate the cutting area ratio;
a process of comparing the previously obtained cut area ratio with a predetermined load detection threshold value, starting from the first cut position where cutting by the cut surface was started;
This is a process of determining the contact position where the forklift claw portion contacts the load based on the magnitude relationship between the cut area ratio and the load detection threshold, and when the cut area ratio at a certain cut position exceeds the load detection threshold, the cut position is A process of determining that it is the contact position of the forklift claw portion and the load;
A process of removing from the point cloud data the claw image area in which the claw portion of the forklift cut off by each cutting plane from the leading cutting position to the contact position is reflected;
was configured to execute

According to the automatic measuring system according to the third invention, the information management server 4 is cut by the cutting planes CS ₀ to CS _T from the leading cutting position x ₀ where cutting is started by the cutting plane CS _n to the contact position x _T. The "claw image region RG _FK in which the forklift claw portion is reflected" (see FIG. 19A) is removed from the point cloud data PD.
As a result, even when the forklift claw portion is included in the point cloud data PD in harmony with the cargo CR, the point cloud data of the forklift claw portion can be removed, and the measurement accuracy of the cargo CR can be further improved. .

[Fourth invention]
In order to solve the above problems, an automatic measurement system according to a fourth invention of the present application is an automatic measurement system according to the first or second invention,

The information management server
a process of generating a horizontal plane projection view of the baggage image area reflected in the point cloud data when viewed from the height direction of the baggage;
a process of generating a horizontal projection diagram by rotating the horizontal projection diagram on the horizontal plane by a predetermined unit angle about the axis in the height direction of the package;
A process of determining the area of a circumscribing rectangle that circumscribes the horizontal projection view based on the maximum and minimum values of the horizontal coordinates of the endpoints of the horizontal projection view;
A process of determining the area of a circumscribing rectangle that circumscribes the horizontal map based on the maximum and minimum values of the horizontal coordinates of the endpoints of the horizontal map;
a process of determining that the direction of each side of the horizontal plane map is parallel to the horizontal plane coordinate axis of the ToF camera at the rotation angle when the area of the circumscribing rectangle is the minimum;
was configured to execute

According to the automatic measurement system according to the fourth invention, the information management server 4 generates (See FIG. 22(d)), the direction of each side of the XY plane map obtained by rotating the luggage image area RG _CR is parallel to the horizontal plane coordinate axes (X axis and Y axis) of the ToF camera 1. I judge.
This makes it possible to correct the inclination of the point cloud data of the package CR with respect to the coordinate axes of the ToF camera 1 so as to make them parallel to each other, thereby further improving the measurement accuracy of the package CR.

[Fifth invention]
In order to solve the above problems, an automatic measurement system according to a fifth invention of the present application is an automatic measurement system according to the first or second invention,

The information management server
A process of calculating the number of point clouds existing in a predetermined noise determination range centering on each pixel included in the point cloud data;
a process of determining that the point group within the noise determination range is a noise component when the number of point groups present in the noise determination range is smaller than a predetermined noise determination criterion;
A process of removing the point cloud determined as the noise component;
was configured to execute

According to the automatic measurement system according to the fifth invention, based on the number of point groups included in the noise determination range RN _NS (see FIG. 24(b)), it is determined as the noise component NS (see FIG. 24(a)) remove the point cloud.
By doing so, the shape of the _three- dimensional package image P3D generated from the point cloud data PD can be brought closer to the accurate shape of the actual package CR.

1 is a schematic diagram showing the overall configuration of an automatic measurement system according to an embodiment; FIG. FIG. 2 is a schematic diagram showing an example of the arrangement configuration of ToF cameras at the entrance/exit of a baggage storage area; FIG. 4 is a schematic diagram showing the photographable range of a ToF camera placed at the entrance/exit of a baggage storage area; FIG. 3 is a schematic diagram showing an example of the arrangement form of the ToF camera, the camera, and the passage detection sensor when viewed from the entrance/right side of the baggage storage area. 2 is a block diagram showing an electrical configuration of a ToF camera; FIG. Schematic diagrams showing cargo vehicle images captured by a ToF camera, where (a) is a cargo vehicle image captured by the ToF camera located at the upper right corner of the entrance, and (b) is a cargo vehicle image captured by the ToF camera located at the upper left corner of the entrance. It is an image. Schematic diagrams showing cargo vehicle images captured by a ToF camera, where (a) is a cargo vehicle image captured by a ToF camera located at the lower right corner of an entrance, and (b) is a cargo vehicle image captured by a ToF camera located at the lower left corner of an entrance. vehicle image. (a) is an example of an actual cargo vehicle image captured by a ToF camera, and (b) is point cloud data generated based on this actual cargo vehicle image. (a) is a schematic diagram showing the positional relationship between the installation position of the ToF camera and the cargo vehicle image captured by the ToF camera; is a schematic diagram showing depth coordinates in . 3 is a block diagram showing the electrical configuration of the camera; FIG. FIG. 4 is a diagram showing an example of a code image captured by a camera; 3 is a block diagram showing an electrical configuration of an information management server; FIG. It is a figure which shows an example of the data structure of package image information. FIG. 3 is a diagram showing an example of the data structure of code image information; (a) is a schematic diagram showing the positional relationship between the point cloud data generated from the luggage vehicle image taken by the ToF camera and the ToF camera installation position, and (b) in the luggage image area showing the luggage reflected in the point cloud data It is a schematic diagram which shows the state which specified the end point. FIG. 4 is a schematic diagram showing a process of synthesizing point cloud data to generate a luggage stereoscopic image; FIG. 4 is a diagram showing one package stereoscopic image obtained by synthesizing four pieces of point cloud data, (a) being a first example of the package stereoscopic image, and (b) being a first example of the package stereoscopic image. 2 is an example. FIG. 4 is a sequence diagram showing the operation of each component in the automatic measurement system when measuring a package; (a) is an example of the point cloud data in which the cargo and the forklift claw part are reflected harmoniously, and (b) is an example of the point cloud data in which the cargo and the forklift claw part are reflected harmoniously, cut at a predetermined cutting plane. It is a schematic diagram of when the (a) is a schematic diagram showing the cross-sectional area of the point cloud data in which the cargo and the forklift claw portion are integrally reflected, and (b) is a schematic diagram showing the cross-sectional area of the point cloud data only for the forklift claw portion. . It is an operation sequence at the time of removal processing of a forklift claw portion. (a) is an example of the baggage image area (baggage point cloud data), (b) is an XY plane projection view of the baggage image area when viewed from the Z-axis direction, and (c) is an XY plane projection view at a predetermined angle. FIG. 2D is a schematic diagram showing a first example of an XY plane projection diagram when rotated, and a second example of an XY plane projection diagram when the XY plane projection diagram is rotated by a predetermined angle. It is an operation sequence when correcting the inclination of the point cloud data of the package with respect to the coordinate axes of the ToF camera. (a) is a schematic diagram showing an example of point cloud data before noise components are removed, and (b) is an example of point cloud data after noise components are removed. It is an operation sequence when noise components are removed from point cloud data.

The automatic measurement system of the present invention will be described below with reference to FIGS. 1 to 25. FIG.
In the embodiment of the present invention, the two-dimensional code CD for the package read by the camera 2 is used in association with the point cloud data PD for automatically measuring the package CR. This is an example of configuring an automatic measuring system 100 capable of "automatically measuring the size of a load being transported" by transport means.
This content will be described in detail below.

FIG. 1 is a block diagram showing the configuration of an automatic measurement system 100 according to this embodiment.
As shown in FIG. 1, the automatic measurement system 100 includes a ToF camera 1, a camera 2, a passage detection sensor 3, an information management server 4, a controller 5, and a relay server 6.

As shown in FIG. 1, in this system 100, ToF cameras 1 (1-1 to 1-4), cameras 2, passage detection sensors 3, information management server 4, controller 5, relay server 6, are connected so that they can communicate with each other.

[Network usage form]
Also, as shown in FIG. 1, in this example, three communication networks, ie, the Internet 200 that can be used for public communication, a wired LAN (Local Area Network) 300, and a wireless LAN 400 are used.
In this example, the Internet 200 and the local communication network (wired LAN 300/wireless LAN 400) are connected via the relay server 6 so as to be able to communicate with each other.
Furthermore, the controller 5 is communicably connected to the relay server 6 .
For the Internet 200, a high security level can be ensured by using, for example, VPN (Virtual Private Network) technology.

[Connection between components]
Furthermore, the form of connection between components in the configuration of this example will be specifically described.
The ToF cameras 1-1 to 1-4 are communicably connected to the relay server 6 via the wired LAN 300. FIG.
The camera 2 and passage detection sensor 3 are communicably connected to the controller 5 .
Also, the camera 2 can communicate with the relay server 6 via the wireless LAN 400 .
The information management server 4 is connected to the Internet 200 outside the local communication network (wired LAN 300/wireless LAN 400).

[Role of each component]
In this system 100, a "passage detection sensor 3" is installed at the entrance/exit GW of the baggage storage location to detect that the transport vehicle VH that conveys the baggage CR passes through the entrance/exit (FIGS. 2, 3, and 4).
In addition, "fixed points of ToF cameras 1-1 to 1-4 are installed one by one" at each of the four corners of the rectangular gateway GW (Figs. 2, 3 and 4).
Note that the ToF cameras 1-1 to 1-4 are connected to the wired LAN 300. FIG.

[ToF (Time of Flight) camera]
The ToF cameras 1-1 to 1-4 are cameras capable of imaging arbitrary subjects.
In this example, when the passage detection sensor 3 detects that the transport vehicle VH has passed through the entrance/exit GW, the ToF cameras 1-1 to 1-4 capture both the transport vehicle VH and the package transported by the same vehicle VH. A cargo vehicle image P _CV (see FIG. 8(a)) is photographed.

Furthermore, the ToF cameras 1-1 to 1-4 generate point cloud data PD (see FIG. 8(b)) from the cargo vehicle image _PCV described above.
This point cloud data PD is used in the information management server 4 to generate a package stereoscopic image P _3D .

The imaging directions of the ToF cameras 1-1 to 1-4 are all directed to the rectangular center of the gateway GW.
Therefore, all of the four ToF cameras 1 are capable of "capturing the subject positioned at the center of the entrance GW near the center of the captured image" (FIG. 3).
In this example, since the main purpose is to measure the cargo CR, the cargo vehicle image _PCV in which the cargo CR is captured near the center of the captured image of the ToF camera 1 is captured.
Note that α ₁ to α ₄ in FIG. 3 represent the photographable range (angle of view) of the ToF cameras 1-1 to 1-4.
Furthermore, the ToF cameras 1-1 to 1-4 are arranged so that there is no blind spot during photography (so that there is no range where photography is impossible at the entrance/exit GW).

The ToF camera 1 has a function of irradiating the cargo CR with infrared light when the passage detection sensor 3 detects passage of the transport vehicle through the entrance, and a function of receiving the infrared light reflected by the cargo CR. ing.
The ToF camera 1 determines the "depth distance L _D between the subject (mainly _the baggage CR) and the ToF camera 1" (Fig. 9 ( b) Calculate cf.

Specifically, the depth distance _LD can be calculated by Equation 1 below.
Note that c in Equation 1 is the speed of light.

Then, the ToF camera 1 generates "point cloud data PD" (FIG. 8(b)) based on the depth distance L _D and the cargo vehicle image P _CV (FIG. 8(a)).
The “point cloud data PD” referred to here refers to image data in which each pixel of the parcel vehicle image _PCV is colored with a predetermined color corresponding to the depth distance _LD at each pixel.

Prior to generating the point cloud data PD, the ToF camera 1 captures (i) the depth coordinates (x, y, z) of each pixel in the cargo vehicle image _PCV determined according to the depth distance _LD , and (ii) A data set unique to each pixel of the parcel vehicle image _PCV is generated, which is composed of a predetermined color assigned to each pixel indicating a division corresponding to each depth distance _LD .

The above-mentioned "depth coordinate" refers to the depth coordinate origin (0, 0, 0) in the depth coordinate system of FIG. These are coordinates indicating the pixel position of the vehicle image _PCV .

Based on the depth coordinates (x, y, z) of each pixel of the cargo vehicle image _PCV and the data set of the predetermined color, the ToF camera 1 captures the cargo vehicle image PCV with a predetermined color corresponding to the depth distance _LD at each pixel. By coloring each pixel of _{the CV} , "point cloud data PD" as shown in FIG. 8B is generated.
More specifically, the “point cloud data PD” is information on the depth distance _LD from the ToF camera 1 to the subject (parcel CR) captured in the parcel vehicle image _PCV , which is associated with the parcel vehicle image _PCV . In addition, it is image data obtained by coloring the cargo vehicle image _PCV with a predetermined color corresponding to the depth distance _LD .

The four ToF cameras 1-1, 1-2, 1-3, and 1-4 independently generate point cloud data PD ₁ , PD ₂ , PD ₃ , and PD ₄ , respectively.
Also, the four ToF cameras 1-1, 1-2, 1-3, and 1-4 transmit point cloud data PD ₁ , PD ₂ , PD ₃ , and PD ₄ .

As the ToF camera 1, not only a dedicated imaging device with a ToF function but also a portable multifunctional electronic device such as a smartphone with a camera with a ToF function can be adopted.
When processing cargo vehicle images _PCV and generating point cloud data PD, a considerable processing load is placed on the CPU (Central Processing Unit), but the processing performance of smartphones is rapidly improving at a rapid pace. , a comfortable operating speed can be ensured even when executing image processing of the cargo vehicle image _PCV and the point cloud data PD.

[camera]
The camera 2 is a camera that captures an arbitrary subject, and in this example, it plays a role of capturing a two-dimensional code CD (FIG. 4).
This camera 2 is provided at the entrance/exit GW, and the number thereof is "one" in this example.

In this example, a QR (Quick Response) code (registered trademark) is used as the two-dimensional code CD photographed by the camera 2 .
In the automatic measurement system 100, the two-dimensional code CD is affixed to each package CR.
Furthermore, in the system 100, the position where the two-dimensional code CD is pasted on the package CR is determined in advance at a predetermined position.
In this example, the two-dimensional code CD is attached to the lower left corner of the side surface of the package CR (Fig. 4).
Also, in this example, the camera 2 is provided on the right side of the entrance/exit GW so that the two-dimensional code CD can be photographed corresponding to the sticking position of the two-dimensional code CD.

When the passage detection sensor 3 detects that the transport vehicle VH has passed through the entrance/exit GW, the camera 2 detects the transport vehicle VH, the package CR being transported by the transport vehicle VH, and the two-dimensional code CD attached to the package CR. A code image _PCD in which is photographed.
The camera 2 then transmits the code image _PCD to the information management server 4 .

[Passing detection sensor]
The passage detection sensor 3 is a sensor that detects passage of the transport vehicle VH through the entrance/exit GW.
In this example, when the passage detection sensor 3 detects passage of the transport vehicle VH through the entrance/exit GW, it outputs a “passage detection signal” indicating the detection to the controller 5 .
The controller 5 transfers the passage detection signal from the passage detection sensor 3 to the camera 2, and also transfers the same passage detection signal to the ToF cameras 1-1 to 1-4 via the relay server 6 and the wired LAN 300. FIG.

As shown in FIG. 2, the passage detection sensor 3 is provided at the entrance/exit GW of the baggage storage area.
In this example, a photoelectric sensor (having a set of a light projecting unit that emits light and a light receiving unit that receives the emitted light) is employed as the passage detection sensor 3 .
Further, the transportation vehicle VH is provided with a "light reflector RF" in order to enhance the reflectivity of the light emitted from the passage detection sensor 3 (FIG. 2). Examples of optical reflectors RF include highly reflective mirrors.

[Information management server]
The information management server 4 manages a “cargo vehicle image P _CV ” that is an image captured by the ToF camera 1 and a “code image P _CD ” that is an image captured by the camera 2 .
The information management server 4 may be owned by, for example, a company that manages and operates a luggage storage location.

The information management server 4 stores these data in four point cloud data PD generated from four parcel vehicle images _PCV each showing the same subject photographed by each ToF camera 1 from each corner of the parcel storage area/entrance GW. Measure the "width, height, and depth of the baggage CR" reflected in the PD.

The information management server 4 synthesizes the four point cloud data PD to generate a "single parcel stereoscopic image _P3D " showing the three-dimensional shape of the parcel reflected in the point cloud data PD. do.

The server 4 also recognizes the two-dimensional code CD attached to each package reflected in the code image _PCD , and reads the package information _ICR indicated by the two-dimensional code.
In the example of FIG. 2, two-dimensional codes CD-1 and CD-2 are recognized.
Based on the recognized two-dimensional code CD, the information management server 4 recognizes the individual packages CR (CR-1 and CR-2 in the example of FIG. 2) appearing in the code image _PCD .
Further, the server 4 determines whether or not a plurality of vertically stacked parcels are reflected in the code image _PCD based on the two-dimensional code CD attached to each parcel CR.

When the information management server 4 determines that a plurality of vertically stacked packages are reflected in the code image P _CD (FIG. 11), the information management server 4 determines the position of the two-dimensional code CD reflected in this image P _CD . Then, the code relative position of the two-dimensional code CD in the luggage stereoscopic image P _3D (FIGS. 17(a) and 17(b)) is identified.
Also, the two-dimensional code CD reflected in the code image PCD and the package information _ICR indicated by the two-dimensional code _CD are associated with the three-dimensional package image _P3D showing the three-dimensional shape of the package CR. are stored in the package image information (FIG. 13).

Next, each configuration of the ToF camera 1, the camera 2, and the information management server 4 will be described in order.

[ToF camera]
As shown in FIG. 5 , the ToF camera 1 has a control section 11 , a storage section 12 , an imaging section 13 , a light irradiation section 14 , a light receiving section 15 and a communication section 16 .
In addition, the ToF camera 1 includes an output section 18 such as a small liquid crystal display (not shown) and an input section 17 including a shutter (not shown).
In addition, the ToF camera 1 includes a control unit 11 including a CPU (central processing unit) that executes arbitrary arithmetic processing, a storage unit 12 including ROM, RAM, etc., and an input/output path for various signals or information. A communication unit 16 including a communication port (not shown) is provided.

The communication unit 16 is connected to the wired LAN 300 and performs various communications with the information management server 4 and the controller 5 via the relay server 6 .
A network interface/modem or the like is used as the communication unit 16 .

The control unit 11 has a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and loads application programs stored in the ROM/storage unit 12 into the RAM. and execute it, thereby realizing various logical means.

The ToF camera 1 implements passage determination means 111 , photographing instruction means 112 , distance calculation means 113 , point cloud data generation means 114 and point cloud data transmission means 115 .

When the passage determination means 111 receives a "passage detection signal" from the passage detection sensor 3 indicating that the transport vehicle VH has passed through the entrance/exit GW, it determines that the transport vehicle VH has passed through the entrance/exit GW.

The photographing instruction means 112 sends to the photographing means 13 a "parcel vehicle photographing instruction" instructing execution of photographing operation when the passage determining means 111 determines that "the transport vehicle VH has passed through the entrance GW".

[Calculation of Depth Coordinates in Cargo Vehicle Image]
The distance calculation means 113 obtains the required light receiving time T _FL from when the light emitting means 14 irradiates the infrared light until the light receiving means 15 receives the light reflected by the subject (baggage CR etc.), and calculates the required light receiving time T _FL . is multiplied by the speed of light to obtain the “depth distance L _D from the ToF camera 1 to the subject” based on the round trip distance (twice the depth distance L _D ).
Subsequently, the distance calculation means 113 calculates the depth distance LD from the obtained depth distance L _D in a three-dimensional space (X-axis/ The "depth coordinates" (x, y, z) in the orthogonal coordinates consisting of the Y-axis and Z-axis are derived.

As shown in FIG. _9B , the distance calculation means 113 calculates the depth coordinates (hereinafter referred to as "reference depth coordinates") (x ₀ , y ₀ , z ₀ ) are calculated.
The reference depth coordinates (x ₀ , y ₀ , z ₀ ) are obtained from the depth distance _LD from the ToF camera 1 to the pixel intersecting the optical axis of the camera 1 and the depth coordinate origin (0, 0, 0).
Further, the means 113 calculates the depth coordinates (x, y, z) of each pixel on the parcel vehicle image _PCV based on the positional relationship shown in _FIG . Obtained from the coordinate origin (0, 0, 0).

[Generation of point cloud data]
The point cloud data generating means 114 generates an image (point cloud data PD) in which each pixel in the parcel vehicle image _PCV is colored with a predetermined color that indicates a division according to the depth distance _LD .
In this example, the "point cloud data PD" is determined by (i) the depth distance _LD between the subject (parcel CR, transport vehicle VH, etc.) reflected in each pixel in the parcel vehicle image _PCV and the ToF camera 1. It contains depth coordinates and (ii) predetermined color data indicating a division corresponding to each depth distance _LD .
Note that the same means 114 determines a predetermined color indicating a division corresponding to each depth distance _LD obtained by the distance calculating means 113 for each pixel on the parcel vehicle image _PCV .

Then, the point cloud data generation means 114 generates point cloud data PD from the depth coordinates (x, y, z) of each pixel and the data set of the predetermined color.
In this example, each pixel of the cargo vehicle image _PCV is colored with a predetermined color that indicates a division corresponding to the depth distance _LD of each pixel in the cargo vehicle image _PCV , thereby obtaining the image shown in FIG. "Point cloud data PD" is generated.
Note that the point cloud data generating means 114 provided in the ToF cameras 1-1, 1-2, 1-3, and 1-4 generates each of the point cloud data PD ₁ , PD ₂ , and PD from the individually photographed luggage vehicle images _{PCV. 3.PD} Generate ₄ ;

The point cloud data transmission means 115 uses the communication unit 16 to transmit the point cloud data PD to the information management server 4 .
In this example, the point cloud data PD is once sent to the relay server 6 and transferred by the server 6 to the information management server 4 via the Internet 200 .

The storage unit 12 stores the parcel vehicle image _PCV photographed by the photographing means 13 and the point cloud data PD generated by the point cloud data generating means 114 .

When the photographing means 112 sends a parcel vehicle photographing instruction, the photographing means 13 captures a "cargo vehicle image P _CV " (FIG. 6(a)) in which the conveying vehicle VH and the parcel CR being conveyed by the same vehicle VH are captured. ), FIG. 6(b), FIG. 7(a), FIG. 7(b), and FIG. 8(a)).

The light irradiation means 14 is a light source that irradiates the cargo CR with infrared light.
In this example, the light irradiation means 14 irradiates the cargo CR with infrared light when the passage determination means 111 determines that "the transport vehicle VH has passed through the entrance GW".

The light receiving means 15 receives the infrared light emitted by the light emitting means 14 and reflected by the cargo CR.

[camera]
Next, the camera 2 will be explained.
FIG. 10 is a block diagram showing the electrical configuration of the camera 2 according to this embodiment.
As shown in FIG. 10, the device body of the camera 2 includes a control unit 21 including a CPU for executing arithmetic processing, a storage unit 22 including ROM, RAM, etc., and a communication unit forming an input/output path for various signals or information. A communication unit 24 including a port (not shown) is provided.
The camera 2 also includes an output section 26 such as a small liquid crystal display (not shown) and an input section 25 including a shutter (not shown).

The communication unit 24 is connected to the wireless LAN 400 and the controller 5 and performs various communications with the information management server 4 and the passage detection sensor 3 . As the communication unit 24, a network interface/modem or the like is used.
The camera 2 may be connected to a printer (not shown) via a predetermined interface such as a USB (Universal Serial Bus) connection terminal or the communication section 24, and the photographed code image _PCD may be printed out on a paper medium.

The control unit 21 of FIG. 10 has a CPU, ROM, RAM, etc., loads an application program stored in the ROM/storage unit 22 into the RAM and executes it, thereby realizing various logical means.
In the camera 2 of this example, a code photographing instruction means 211 and a code image transmitting means 212, which are subdivided according to function, are realized in order to photograph a plurality of two-dimensional codes.

The code photographing instruction means 211 instructs execution of the photographing operation when it receives a "passage detection signal" (a signal indicating that the transport vehicle VH has passed through the entrance GW) output from the passage detection sensor 3. code photographing instruction” is sent to the code photographing means 23.

The code image transmitting means 212 transmits the code image _PCD photographed by the code photographing means 23 to the information management server 4 .

When the code photographing means 23 receives the code photographing instruction from the code photographing instructing means 211, the code photographing means 23 scans the conveying vehicle VH, the package CR being conveyed by the vehicle VH, and the two-dimensional code affixed to a predetermined position on the same package CR. A “code image P _CD ” in which the CD is reflected is photographed.
In the example of FIG. 11, two vertically stacked packages CR-1 and CR-2 are photographed while being transported. Two-dimensional codes CD-1 and CD-2 are affixed to predetermined positions CR-1 and CR-2 of each package.

[Information management server]
FIG. 12 is a block diagram showing the electrical configuration of the information management server 4 according to this embodiment.
As shown in FIG. 12, the information management server 4 includes an output unit 44 such as a liquid crystal display and a printer (not shown), and an input unit 43 including a keyboard, mouse (not shown), touch panel (not shown), and the like. ing.
In addition, the device main body of the information management server 4 includes a control unit 41 including a CPU (Central Processing Unit) that executes arbitrary arithmetic processing, a hard disk drive (HDD), a ROM (Read Only Memory), and a RAM. (random access memory), etc., and a communication unit 45 including a communication port (not shown) forming an input/output path for various signals or information.

The communication unit 45 is connected to the Internet 200, which is a public communication network, and performs various communications with the ToF cameras 1-1 to 1-4 and the camera 2 via the relay server 6. FIG.
As the communication unit 45, a network interface/modem or the like is used.

The control unit 41 has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and an application program (image management program) stored in the ROM/storage unit 42 423) is loaded into RAM and executed, thereby realizing various logical means.

The information management server 4 implements image information management means 411 , parcel measurement means 412 , stereoscopic image generation means 413 , code reading means 414 , parcel recognition means 415 , piling determination means 416 and code position specifying means 417 .

Image information management means 411 is means for managing package image information 421 (FIG. 13) and code image information 422 (FIG. 14).
The management means 411 performs information registration and deletion of registered contents with respect to package image information 421 and code image information 422 .

As shown in FIG. 13, the parcel image information 421 is information specifying the correspondence relationship between the parcel CR transported by the transport vehicle VH and the attributes of the parcel CR.
In the example of FIG. 13, in parcel image information 421, image data (point cloud data PD ₁ to PD ₄ , parcel stereoscopic image P _3D , code image P _CD ) and parcel information I _CR are associated.

Also, as shown in FIG. 14, the code image information 422 is information specifying the correspondence relationship between the two-dimensional code CD attached to the package CR transported by the transport vehicle VH and the attributes related to the code CD.
In the example of FIG. 14, the code image information 422 includes the code image _PCD , the two-dimensional code CD attached to the package CR reflected in the code image _PCD , and the package information read from the two-dimensional code CD. and _ICR are associated.

[Luggage measurement]
The parcel measuring means 412 measures four point cloud data PD each generated from four parcel vehicle images _PCV showing the same subject photographed by the ToF cameras 1-1 to 1-4, and measures the reflection in these data PD. Measure the size (width, height, depth) of the cargo CR.
In this example, the same means 412 identifies the "region in which the package CR is reflected" (hereinafter referred to as the "package image area RG _CR ") in the point cloud data PD as shown in FIG. The actual size (height, width, depth) of the parcel _{CR is measured by obtaining the distance between each pixel in the peripheral portion of the image area RG CR} (see FIG. 15(b)).

The parcel measuring means 412 first identifies end points (for example, each vertex of a cube) of the parcel image area RG _CR in the point cloud data PD of FIG. 15(a).
In the example of FIG. 15(b), vertices A, B, C, D, E, and H are specified as end points of the luggage image area RG _CR .

After identifying each end point of the load CR (baggage image region RG _CR ) in the point cloud data PD, the parcel measuring means 412 acquires the depth coordinates (x, y, z) of each pixel in which each end point is captured. .
Note that the depth coordinates in the point cloud data PD are associated with the point cloud data PD itself in a referable state.

In the example of FIG. 15B, the depth coordinates at the six end points of vertices A, B, C, D, E, and H are acquired.
As a result, the depths of the sides AE and DH corresponding to the width of the package CR, the sides AD and BC corresponding to the height of the package CR, and the sides AB and CD corresponding to the depth of the package CR are determined. Each distance (length of each of side AE, side DH, side AD, side BC, side AB, and side CD) can be calculated based on the coordinates.

In FIG. 15B, the depth coordinates of vertex C are ( _xC , _yC , _zC ), and the depth coordinates of vertex D are ( _xD , _yD , _zD ).
In this case, the distance of the side CD (the side between the vertex C and the vertex D) forming the depth of the package can be calculated by Equation 2 below.
That is, the distance of the side CD is the difference “x _C −x _{D ”} between the X-coordinate components, the difference “y _C −y _D ” between the Y-coordinate components, and the difference between the Z-coordinate components It is determined by the square root of the sum of squares of the difference "z _C -z _D ".
It should be noted that other distances between end points in the luggage image area RG _CR are also calculated by the same calculation as in Equation 2.

In this way, the parcel measuring means 412 uses the depth distance _LD between the subject captured at each of the vertices A, B, C, D, E, and H and the ToF camera 1 to determine the actual width and width of the parcel CR. Calculate height/depth (e.g. length in meters).
As a result, the actual size (width, height, depth) of the cargo CR can be measured.

[Generation of 3D image of luggage]
The stereoscopic image generating means 413 synthesizes four point cloud data PD created from the four parcel vehicle images _PCV photographed by the ToF cameras 1-1 to 1-4.
Through this synthesizing process, the parcel stereoscopic image generating means 413 generates a “single parcel stereoscopic image P _3D ” showing the three-dimensional shape of the parcel CR reflected in each point cloud data PD.
The “parcel stereoscopic image P _3D ” is an image of the parcel CR having parallax images when viewed from any direction in all directions.

In this example, one “parcel stereoscopic image P _3D ” is generated from point cloud data PD obtained from four parcel vehicle images P _CV from multiple viewpoints (ToF cameras 1-1 to 1-4 arranged at intervals). As a method for doing so, a visual volume intersection method is used to obtain three-dimensional voxel data (data expressing the cargo CR as a set of small cubes) of the cargo CR based on a plurality of point cloud data PD.
Note that the method of generating the _3D baggage image P is not limited to the visual volume intersection method, and any method such as the stereo matching method can be employed.

When generating one parcel stereoscopic image P _3D from four pieces of point cloud data PD using a known visual volume intersection method, first, the stereoscopic image generation means 413 generates parcel CR A silhouette image (a binary image in which the cargo image area RG _CR in which the cargo CR is reflected is represented in black and white) is extracted.
Next, the same means 413 stereoscopically back-projects the silhouette images centering on the ToF cameras 1-1 to 1-4, and extracts voxel data (aggregated data of small cubes) of the cargo CR from the overlapped portion of the visual volume. Generate.
Then, by drawing small triangular surfaces on the surface of each voxel data of the package CR, a three-dimensional package image _P3D showing the surface shape (three-dimensional shape) of the package CR can be generated.

[Reading two-dimensional code]
The code reading means 414 recognizes the two-dimensional code CD attached to each package CR reflected in the code image _PCD and reads the package information _ICR indicated by the code CD.
In this example, since the two-dimensional code CD is a QR code (registered trademark), the two-dimensional code CD is recognized based on a finder pattern (so-called clipping symbol) for position detection.

The parcel recognition means 415 "recognizes each parcel CR reflected" in the code image _PCD .
Furthermore, the parcel recognition means 415 reads the two-dimensional code CD for each parcel CR recognized by the code reading means 414 for each parcel reflected in the code image _PCD , and It is stored in the code image information 422 (FIG. 14) in association with the read parcel information _ICR .

Here, the automatic measurement system 100 is operated so as to attach the two-dimensional code CD to a predetermined position of the package CR (in the example of FIG. 11, the lower left side of the side surface of the package CR).
Therefore, by focusing on the two-dimensional code CD reflected in the code image _PCD , the parcel recognition means 415 recognizes each parcel CR to which the respective two-dimensional code CD is affixed even while the parcel CR is being conveyed. can be distinguished and recognized.

The parcel information _ICR read by the code reading means 414 includes, for example, the name of the product handled as the parcel CR, the product number, the number of products, the weight of the product, and the like.

The pile discrimination means 416 in FIG. 12 determines whether or not a plurality of packages CR stacked vertically are reflected in the code image _PCD based on the two-dimensional code CD attached to each package CR. to discriminate.
As described above, the two-dimensional code CD is affixed to each package CR in this example at a predetermined position (lower left side of the side _surface ). By doing so, it is possible to determine whether or not there is a load CR conveyed in a state of being stacked vertically.
Also, by referring to the position of the two-dimensional code CD, it is possible to determine the positional relationship (hierarchical relationship) of the stacked packages CR.

[Identification of two-dimensional code position in baggage stereoscopic image]
Furthermore, in this example, when it is determined that a plurality of vertically stacked packages CR are reflected in the code image P _CD , the code position specifying means 417 selects the code image P in the package stereoscopic image P _3D . The position of the two-dimensional code CD reflected on _{the CD} is specified.

The means 417 identifies the position of the two-dimensional code CD reflected in the _three -dimensional baggage image P3D (the same code as the two-dimensional code CD reflected in the code image _PCD ).
Then, the means 417 converts the "coordinates of the image position of the two-dimensional code CD reflected in the code image P _CD " to the "two-dimensional code reflected in the three-dimensional package image P _3D " generated by the stereoscopic image generation means 413. "coordinates of the image position on the CD".
In addition, the code position specifying means 417 reads the two-dimensional code CD recognized by the code reading means 414 for the two-dimensional code CD reflected in the three-dimensional package image P _3D and the package information I _CR read by the same code reading means 414) are associated with each other and stored in the parcel image information 421 (FIG. 13).

By introducing the application (image management program 423) according to the present embodiment into any multifunctional electronic device, various characteristic functions of the present invention can be added, and the information management server 4 can be configured. be.
Further, according to the present embodiment, the information management server 4 can be configured simply and in a short time simply by installing a dedicated application in the multifunctional electronic device.

[Embodiment 1: Operation sequence]
Next, the operation of the automatic measurement system 100 according to this embodiment will be described.
[When measuring luggage]
In the following, the operation of each component in this system 100 when automatically measuring the size of the package while the transport vehicle VH is still transporting the package CR will be described.

When the transport vehicle VH passes through a predetermined position (the entrance/exit GW of the baggage storage area), the passage detection sensor 3 operates.
When the passage detection sensor 3 detects that the transport vehicle VH has passed through the entrance GW, it outputs a "passage detection signal" indicating that the vehicle VH has passed through the entrance GW to the controller 5 .

In this example, the passing detection signal is transmitted from the controller 5 to the relay server 6 once, and then sent from the relay server 6 to the four ToF cameras 1-1 connected to the same server 6 via the wired LAN 300 (FIG. 1). 1 to 1-4.
The controller 5 also transmits this passage detection signal to the camera 2 for photographing the two-dimensional code.

[Operation of ToF camera]
When receiving the passage detection signal from the passage detection sensor 3, the ToF camera 1 determines that the transport vehicle VH has passed through the entrance GW.
When it is determined that the transport vehicle VH has passed through the entrance GW in this way, the ToF camera 1 captures the cargo vehicle image P _CV (FIG. 8A) in which the transport vehicle VH and the cargo CR are both captured (FIG. 8A). 18.Step S101).
Note that the photographing operation in step S101 is performed by sending an instruction to photograph the luggage vehicle from the control unit 11 to the photographing means 13 inside the ToF camera 1 .

In this example, when the ToF camera 1 determines that the transport vehicle VH has passed through the entrance/exit GW by receiving the passage detection signal, the ToF camera 1 irradiates the parcel CR with infrared light (step S102).
After that, the ToF camera 1 receives the infrared light reflected by the cargo CR (step S103).

Subsequently, the ToF camera 1 calculates _the required light receiving time _TFL until _the emitted infrared light is received. A “depth distance L _D ” between the subject part and the ToF camera 1 is obtained.
Further, based on the obtained depth distance _LD , the ToF camera 1 sets the position of the ToF camera 1 as the depth coordinate origin (0, 0, 0) as shown in FIG. ” (x, y, z).
In addition, the ToF camera 1 determines a predetermined color indicating a division corresponding to each depth distance _LD for all pixels on the cargo vehicle image _PCV .

Then, the ToF camera 1 generates point cloud data PD based on the depth coordinates (x, y, z) of each pixel and a data set of a predetermined color corresponding to the depth distance _LD (FIG. 18, step S104). .
In this example, the ToF camera 1 generates "point _cloud data _PD " as shown in FIG. do.

Further, the ToF camera 1 transmits the generated point cloud data PD to the information management server 4 (FIG. 18, step S105).
In this example, the point cloud data PD is once transmitted from the ToF camera 1 to the relay server 6 via the wired LAN 300 and then transferred from the relay server 6 to the information management server 4 via the Internet 200 .
The processing of steps S101 to S105 by the ToF camera 1 is repeatedly executed each time the passage detection sensor 3 detects passage of the transport vehicle VH through the entrance GW.

[Operation of camera for photographing two-dimensional code]
On the other hand, when the camera 2 receives the passage detection signal from the passage detection sensor 3, it sends a "code photographing instruction" to the code photographing means 23, so that the transport vehicle VH, the package CR and the two-dimensional code CD are photographed. A "code image P _CD " is photographed (step S106).

Subsequently, the camera 2 transmits the photographed code image _PCD to the information management server 4 (step S107).
In this example, the camera 2 once transmits the code image _PCD to the relay server 6 via the wireless LAN 400 , and then the relay server 6 transfers the code image _PCD to the information management server 4 via the Internet 200 .
The processing of steps S106 and S107 by the camera 2 is also repeated each time the passage detection sensor 3 detects passage of the transport vehicle VH through the entrance GW.

[Operation of information management server]
When the point cloud data PD and the code image _PCD are received, the information management server 4 starts two-dimensional code recognition (decoding) processing and three-dimensional point cloud data analysis (dimension measurement) processing.

[Luggage measurement]
The information management server 4 identifies the "parcel image area RG _CR " in which the parcel CR is reflected in the point cloud data PD, and calculates the actual size of the parcel _CR ( Measure height, width and depth.

The information management server 4 first identifies end points (for example, vertices of a cube) of the luggage image area RG _CR in the point cloud data PD of FIG. 15(a) (FIG. 15(b)).
Subsequently, the server 4 acquires the depth coordinates (x, y, z) at each of the end points (pixels) identified above, and uses Equation 2 based on the depth coordinates to obtain the distance between the end points (the distance between the sides). length).
Then, the information management server 4 uses the depth distance _LD between the subject captured in the pixels of each end point of the package image area RG _CR and the ToF camera 1 to determine the actual width, height, and depth of the package CR (for example, , length in meters).
As a result, the actual size (width, height, depth) of the cargo CR is measured (FIG. 18, step S108).

[Generation of 3D image of luggage]
Next, the information management server 4 generates a “single parcel stereoscopic image P _3D ” showing the three-dimensional shape of the parcel CR reflected in each point cloud data PD (step S109).

This parcel stereoscopic image _P3D is generated by synthesizing four pieces of point cloud data PD respectively generated from four parcel vehicle images _PCV .
In this example, as a method of generating one package stereoscopic image P _3D from four point cloud data PD obtained from multiple viewpoints (ToF cameras 1-1 to 1-4 arranged at intervals), a plurality of package vehicle images A visual volume intersection method is used to obtain voxel data of the cargo CR based on the _PCV .

[Reading two-dimensional code]
The information management server 4 recognizes the two-dimensional code CD attached to each package CR reflected in the code image _PCD (step S110), and reads the package information _ICR indicated by the code CD.
When a QR code (registered trademark) is used as the two-dimensional code CD, a finder pattern is used for recognition of the same code CD.

Further, the information management server 4 recognizes each package CR reflected in the code image _PCD (step S111).
The information management server 4 associates each package in the code image _PCD recognized by itself with the two-dimensional code CD attached to each package and the package information _ICR read from each two-dimensional code CD. Stored in the code image information 422 .

The information management server 4 determines whether or not a plurality of vertically stacked parcels CD are reflected in the code image _PCD based on the positional relationship of the two-dimensional code CDs attached to each parcel. (step S112).

When the information management server 4 determines that a plurality of vertically stacked packages are reflected in the code image _PCD , the information management server 4 determines that the "two-dimensional image shown in the code image _PCD " in the package stereoscopic image _P3D is displayed. position of the code CD" is specified (step S113).
Then, the server 4 converts the coordinates of the image position of the two-dimensional code appearing in the code image _PCD into the coordinates of the image position of the two-dimensional code appearing in the _{three-dimensional} baggage image P3D.
The processing of steps S108 to S113 in the information management server 4 is repeatedly executed each time the passage detection sensor 3 detects passage of the transport vehicle VH through the entrance/exit GW.
Thus, in the automatic measuring system 100, a series of operations for performing "automatic measurement of the load CR" and "reading of the two-dimensional code CD" while the load CR is being transported by the transport vehicle VH are completed. do.

As described above, according to the automatic measurement system 100 according to the present embodiment, the package CR is transported by the transport vehicle VH without human intervention (and without any hindrance to the transport operation of the package). can be automatically measured, freeing up human resources from measuring work.

Further, according to the present invention, the information management server 4 synthesizes four pieces of point cloud data PD generated by the four ToF cameras 1-1 to 1-4, respectively, so that these point cloud data PD A single package stereoscopic image _P3D showing the three-dimensional shape of the captured package CR can be generated.

Furthermore, according to the present invention, a worker who is engaged in the handling work of the packages CR does not need to visually identify each package, and based on the code image _PCD photographed by the camera, the transport vehicle VH is being transported. Individual packages CR can be automatically identified.
Therefore, it is possible to automatically count the number of parcels CR, which can contribute to automatic prediction of the number of workers required for loading the parcels and the number of delivery vehicles required for delivery.

In addition, according to the present invention, based on the code image _PCD , the presence or absence of vertically stacked packages CR is automatically determined. The position of the two-dimensional code CD attached to the parcel CR can be specified in _P3D .

Furthermore, in the present invention, by repeating trials by the inventor of the present application, one ToF camera 1-1 to 1-4 is installed at each of the four corners of the rectangular entrance GW (four in total). Generation of point cloud data PD and automatic measurement have also been realized for packages CR made of black or dark colored low-reflection objects.

[Modification: Removal of claws from integrated parts of cargo and forklift claws]
Forklifts are frequently used as transport vehicles for cargo CR in cargo storage locations such as factories and warehouses.
Generally, a forklift is provided with an elongated claw portion for supporting and loading a load CR.
In the system of the present application, when the point cloud data PD is generated, the claw portion of the forklift is included in the package CR, and the size of the measured package CR may differ from the actual size of the package CR.
In order to avoid such a situation, in the present invention, the claw portion of the forklift and the load CR may be discriminated, and the point cloud data of the claw portion reflected in the data PD may be removed from the point cloud data PD.

In the present application, the information management server 4 realizes the point cloud data removal processing of the forklift claw part reflected in the point cloud data PD by the following method.
In the following, an example of removing the “claw image region RG _FK in which the claw portion of the forklift is reflected” from the point cloud data PD shown in FIG. 19A will be described.

First, as shown in FIG. 19B, the information management server 4 divides the processing target (the data portion where the cargo CR and the forklift claw portion are integrated) in the point cloud data PD at predetermined intervals along the X axis (for example, 1 mm) at the cutting position _{x n} (step S21 in FIG. 21).
In the example of FIG. 19(b), a cross-sectional image (see FIG. 20(a)) in which the cargo CR and the forklift claw portion are reflected can be acquired on the cutting plane _CS2 .
In addition, since the load CR is not reflected on the cutting plane _CS1 in FIG. 19(b), a cross-sectional image (see FIG. 20(b)) in which only the forklift claw portion is reflected can be acquired.
The direction of the X-axis is "the same direction as the axial direction (stretching direction) of the forklift claw".

Next, the information management server 4 sequentially calculates the cross-sectional area _Axn cut by the cutting surface _CSn at each cutting position _xn (step S22 in FIG. 21).
In this example, the cross-sectional area Ax ₂ of the cross-sectional image (see FIG. 20(a)) in which the load CR and the forklift claw portion are reflected by the cutting plane CS ₂ in FIG. 19(b) is "200000". .
Further, the cross-sectional area _Ax1 of the cross-sectional image (see FIG. 20(b)) in which the forklift claw portion cut off by the cutting plane _CS1 in FIG. 19(b) is reflected is "2000".

Then, the information management server 4 sequentially compares the cross-sectional areas Ax n and Ax n+1 cut by the cutting planes CS _{n and} CS _{n+1 at} the two cutting positions x _n and x _n+1 that are successively adjacent to each other. .
That is, the value of the cut area ratio “Ax _n+1 /Ax _n ” is sequentially calculated for each cut position x _n along the X-axis (step S23 in FIG. 21).
When the cut surface CS ₁ and the cut surface CS ₂ in FIG. 19B are in a continuous anteroposterior relationship, the value of the cut area ratio “Ax ₂ /Ax ₁ ” is “100”.

Furthermore, the information management server 4 starts from the top cutting position x ₀ where cutting was started by the cutting surface CS _n , and sets the previously obtained cutting area ratio "Ax _n+1 /Ax _n " to the predetermined parcel detection threshold TH _CR. (step S24 in FIG. 21).
For the cargo detection threshold TH _CR , a numerical value suitable for distinguishing between the cargo CR and the forklift claw portion is set by carrying out a field test at an actual cargo storage place.

Subsequently, the information management server 4 determines the contact position _xT where the forklift claw contacts the cargo _CR based on the size relationship between the cutting area ratio "Axn ₊₁ / _Axn " and the cargo detection threshold THCR (see FIG. 21). step S25).
In this example, when the value of the cutting area ratio “Ax _n+1 /Ax _n ” exceeds the load detection threshold TH _CR (for example, 60), the cutting position x _n is the contact position x _T between the forklift claw portion and the load CR. It is determined that

Then, the information management server 4 _stores the "claw with the forklift claw portion reflected" cut by the cutting planes CS ₀ to CS _T from the first cutting position x ₀ where cutting by the cutting plane CS _n is started to the contact position xT. The image region RG _FK ” is removed from the point cloud data PD (step S26 in FIG. 21).
This completes the series of operations for removing the forklift claw portion reflected in the point cloud data PD.

As a result, even when the forklift claw portion is included in the point cloud data PD in harmony with the cargo CR, the point cloud data of the forklift claw portion can be removed, and the measurement accuracy of the cargo CR can be further improved. .

[Modification: Tilt correction function that makes the point cloud data of the package parallel to the coordinate axis of the ToF camera]
As described above, forklifts are often used to transport the cargo CR at the cargo storage location.
In a general forklift, the upper surface of the claw portion that contacts the cargo CR is gently inclined in order to prevent the cargo CR from slipping down from the claw portion.
Therefore, the cargo CR captured in the cargo vehicle image _PCV captured by the ToF cameras 1-1 to 1-4 is not horizontal to the coordinate axes of the ToF cameras 1-1 to 1-4, but is tilted. There are many.
In addition, during a busy transportation operation, it is often possible that the cargo CR is loaded on the forklift claw portion in a state deviated from the ToF camera coordinate axis.

Since the "point cloud data PD" is generated based on the parcel vehicle image _PCV , if the posture of the parcel CR reflected in the parcel vehicle image _PCV is originally inclined, the posture of the parcel CR is inherited as it is to the point cloud data PD.
Therefore, the present system 100 is provided with a “tilt correction function” that makes the point cloud data of the package parallel to the coordinate axes of the ToF camera 1 .

The tilt correction function as described above is realized by the following method in the information management server 4 of the present application.
Below, the posture of the package image area RG _CR (point cloud data of package CR) shown in FIG. A case will be described as an example.

The end points (pixels corresponding to the vertices of the cube) of the luggage image area RG _CR in FIG. 22(a) each have depth coordinates (x, y, z).
Therefore, first _, the information management server 4 determines the "baggage image area RG _CR XY plane projection view of ” is generated (step S31 in FIG. 23).
An "XY plane projection view (horizontal plane projection view)" shown in FIG. 22(b) is obtained from the parcel image area RG _CR of FIG. 22(a).
The direction of the Z-axis is the same as the "height direction of the load CR" (vertical direction).

Next, the information management server 4 converts the XY plane projection view of FIG. 22(b) obtained from the package image area RG _CR into the XY plane with the Z axis (the axis in the height direction of the package CR) as the central axis of rotation. (Horizontal plane) is rotated by a predetermined unit angle (step S32 in FIG. 23).
As a result, the information management server 4 generates an "XY plane map" (horizontal plane map) (step S32).
The information management server 4 in this example rotates the XY plane projection view of FIG. 22B on the XY plane by a predetermined unit angle (for example, 1°) around the Z axis.

The server 4 performs the rotation processing of the XY plane projection view (horizontal plane projection view) in step S32 of FIG. 23 within a predetermined rotation range (eg, −45° to +45°).
In this example, when the XY plane projection view of FIG. 22(b) is rotated by a rotation angle (+2°), the "XY plane map (horizontal plane map)" of FIG. 22(c) is obtained. and
Further, in this example, when the XY plane projection view of FIG. 22(b) is rotated by a rotation angle (+5°), the XY plane map (horizontal plane map) of FIG. 22(d) is obtained. and

The information management server 4 calculates the maximum and minimum values (X-axis maximum value, X-axis minimum value, Y-axis maximum value and Y-axis minimum value) are specified (step S33 in FIG. 23).

Subsequently, the server 4 obtains the area of the circumscribing rectangle (the rectangle surrounded by the dashed line in FIG. 22(b)) that circumscribes the XY plane projection view (step S33).
The area of the circumscribing rectangle can be calculated from the formula (maximum value of X-axis-minimum value of X-axis).times.(maximum value of Y-axis-minimum value of Y-axis).

Furthermore, the same server 4 rotates the XY plane projection view each time it is rotated by a predetermined unit angle within the above-mentioned predetermined rotation range, and each XY plane projection view (FIGS. 22(c) and 22(d) obtained by rotating the XY plane projection view )), the maximum and minimum values (X-axis maximum value, X-axis minimum value, Y-axis maximum value, Y-axis minimum value) of the horizontal plane coordinates of the mapping diagram are also specified (step S34 in FIG. 23).
Then, the "area of the circumscribing rectangle" of each XY plane map is calculated (at step S34).

Then, the information management server 4 determines that the direction of each side of the XY plane mapping diagram obtained by rotating the baggage image area RG _CR is the horizontal plane coordinate axis of the ToF camera 1 at the rotation angle when the "area of the circumscribing rectangle" is the smallest. It is judged to be in a parallel state aligned with (X-axis and Y-axis) (step S35 in FIG. 23).
This makes it possible to correct the inclination of the point cloud data of the package CR with respect to the coordinate axes of the ToF camera 1 so as to make them parallel to each other, thereby further improving the measurement accuracy of the package CR.

[Modification: Function to remove noise components from point cloud data]
The "point cloud data PD" generated from the cargo vehicle image _PCV captured by the ToF camera 1 may include an object unrelated to the cargo CR in the cargo storage area.
Since such an object unrelated to the parcel CR can become a noise component NS in the point cloud data PD, it adversely affects the accuracy of the shape of the "parcel stereoscopic image P _3D " generated from the point cloud data PD (for example, distortion).
In order to avoid such a situation in advance, the system 100 has a function of removing the noise component NS from the point cloud data PD.

In the present application, the noise removal function as described above is realized by the following method in the information management server 4 .
In the following, an example of removing the noise component NS included in the point cloud data PD shown in FIG. 24A will be described.

In this method, the information management server 4 removes the noise component NS from the point cloud data PD using a predetermined noise criterion TH _NS .
In this example, by comparing the "point cloud density" (the number of point clouds included in the noise determination range RN _NS indicated by the dashed line in FIG. 24(a)) with the noise determination reference TH _NS , determine whether
The noise judgment reference TH _NS is set to a numerical value suitable for judging the noise component NS by performing a field test at an actual luggage storage location.
In this example, the value of the noise criterion TH _NS is "5".

The information management server 4 calculates “the number of point clouds existing in the noise determination range RN _NS ” centering on each pixel (depth coordinates (x, y, z)) for all pixels included in the point cloud data PD. Calculate (step S41 in FIG. 25).

In this example, as the noise determination range RN _NS , the inside of a "circle of radius r" with each pixel in the point cloud data PD as the center of the circle is employed.
Since this radius r is a parameter that depends on the luggage storage location, it is suitably set according to the site of the luggage storage location.
Note that the shape of the noise determination range RN _NS may be arbitrary, and may be an elliptical shape or a polygonal shape.

Next, when "the number of point groups existing in the noise determination range RN _NS" is smaller than the noise determination reference TH _NS , the information management server 4 determines that the point groups within the noise determination range RN _NS are noise components NS. It is determined that there is (step S42 in FIG. 25).
The server 4 then removes the point group (the point group determined as the noise component NS) in these pixels included in the noise determination range RN _NS (step S43 in FIG. 25).
By doing so, it is possible to obtain clearer point cloud data PD (see FIG. 24(b)) from which the noise component NS _is removed. can be closer to the exact shape of the actual cargo CR.

Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art are possible in the configuration and details of the present invention without departing from the gist of the present invention.

1 ToF (Time of Flight) camera 2 Camera 3 Passage detection sensor 4 Information management server 5 Controller 6 Relay server 100 Automatic measurement system 200 Internet 300 Wired LAN
400 wireless LAN
CR Cargo VH Conveying vehicle P _CV cargo vehicle image PD Point cloud data P _3D cargo stereoscopic image CD Two-dimensional code P _CD code image

Claims (5)

An automatic measuring system capable of automatically measuring the size of a package transported by a transport vehicle or other transport means,

a passage detection sensor provided at a rectangular doorway of a luggage storage location, the sensor detecting passage of the conveying means through the doorway;
A ToF (Time of Flight) camera for photographing an arbitrary subject provided one each at each of the four corners of the rectangular doorway, when the passage detection sensor detects passage of the doorway by the conveying means four ToF (Time of Flight) cameras that irradiate the baggage with infrared light and receive infrared light reflected from the baggage;
a camera capable of photographing an arbitrary subject, the camera being provided on either the left or right side of the doorway;
an information management server that manages images captured by the ToF camera and images captured by the camera;
and

Each of the four ToF (Time of Flight) cameras,
a photographing means for photographing a parcel vehicle image in which the parcels being conveyed by the conveying means and the parcels being conveyed by the conveying means are photographed when the passage detection sensor detects passage of the entrance/exit of the conveying means;
Based on the light reception time required from when the passage detection sensor detects the passage of the entrance/exit of the transportation means to when the infrared light is emitted to the baggage until the infrared light reflected by the same baggage is received. a distance calculation means for obtaining the depth distance between the subject portion reflected in each pixel of the ToF camera and the ToF camera;
generating a data set of each pixel of the cargo vehicle image including depth coordinates in the cargo vehicle image determined according to the depth distance obtained by the distance calculating means and a predetermined color indicating a division according to each depth distance; a point cloud data generating means for generating point cloud data by coloring each pixel of the cargo vehicle image with a predetermined color corresponding to the depth distance of each pixel based on the generated data set;
Point cloud data transmission means for transmitting the point cloud data to an information management server;
has

The camera is
Code photographing for photographing a code image in which a package being transported by the transport means and the transport means and a two-dimensional code affixed to the same load are photographed when the passage detection sensor detects passage of the entrance/exit of the transport means. means and
code image transmission means for transmitting the code image to an information management server;
has

The information management server is
Measure the width, height, and depth of the cargo reflected in the point cloud data in the four point cloud data generated from the four cargo vehicle images of the same subject taken by each ToF camera from each corner of the entrance/exit. death,
Synthesizing the four pieces of point cloud data to generate a single package stereoscopic image showing the three-dimensional shape of the package reflected in the point cloud data,
a measuring process of associating and storing the measured width, height, and depth of the luggage indicated by the stereoscopic image of the luggage;

recognizing a two-dimensional code attached to each package reflected in the code image and reading package information indicated by the two-dimensional code;
recognizing individual packages appearing in the code image based on the recognized two-dimensional code;
a code reading process for determining whether or not a plurality of vertically stacked packages are reflected in the code image based on a two-dimensional code affixed to each package;

When it is determined that a plurality of packages stacked up and down are reflected in the code image,
identifying a code relative position in the three-dimensional image of the baggage corresponding to the position of the two-dimensional code reflected in the code image;
It is characterized in that the two-dimensional code reflected in the code relative position and the baggage information indicated by the two-dimensional code are stored in association with the three-dimensional image of the package in which the code relative position is specified. automatic measurement system.
The automatic measurement system according to claim 1,

The ToF (Time of Flight) camera is
1. An automatic measurement system characterized in that it is configured to generate point cloud data from photographed baggage vehicle images even when the baggage to be measured is a low-reflection object that absorbs infrared light.
The automatic measurement system according to claim 1 or 2,

When the transportation means is a forklift,
The information management server
A process of cutting the data portion in which the cargo and the forklift claw portion are integrated in the point cloud data by the cutting surface at the cutting position at predetermined intervals along the axial direction of the forklift claw portion having a long shape;
A process of calculating a cross-sectional area cut by a cutting plane at each of the cutting positions;
A process of sequentially comparing the cross-sectional areas cut by each cutting surface at two cutting positions that are consecutively back and forth to calculate the cutting area ratio;
a process of comparing the previously obtained cut area ratio with a predetermined load detection threshold value, starting from the first cut position where cutting by the cut surface was started;
This is a process of determining the contact position where the forklift claw portion contacts the load based on the magnitude relationship between the cut area ratio and the load detection threshold, and when the cut area ratio at a certain cut position exceeds the load detection threshold, the cut position is A process of determining that it is the contact position of the forklift claw portion and the load;
A process of removing from the point cloud data the claw image area in which the claw portion of the forklift cut off by each cutting plane from the leading cutting position to the contact position is reflected;
An automatic weighing system, characterized in that it is configured to perform
The automatic measurement system according to claim 1 or 2,

The information management server
a process of generating a horizontal plane projection view of the baggage image area reflected in the point cloud data when viewed from the height direction of the baggage;
a process of generating a horizontal projection diagram by rotating the horizontal projection diagram on the horizontal plane by a predetermined unit angle about the axis in the height direction of the package;
A process of determining the area of a circumscribing rectangle that circumscribes the horizontal projection view based on the maximum and minimum values of the horizontal coordinates of the endpoints of the horizontal projection view;
A process of determining the area of a circumscribing rectangle that circumscribes the horizontal map based on the maximum and minimum values of the horizontal coordinates of the endpoints of the horizontal map;
a process of determining that the direction of each side of the horizontal plane map is parallel to the horizontal plane coordinate axis of the ToF camera at the rotation angle when the area of the circumscribing rectangle is the minimum;
An automatic weighing system, characterized in that it is configured to perform
The automatic measurement system according to claim 1 or 2,

The information management server
A process of calculating the number of point clouds existing in a predetermined noise determination range centering on each pixel included in the point cloud data;
a process of determining that the point group within the noise determination range is a noise component when the number of point groups present in the noise determination range is smaller than a predetermined noise determination criterion;
A process of removing the point cloud determined as the noise component;
An automatic weighing system, characterized in that it is configured to perform

Images