JP7386681B2 - Scrap grade determination system, scrap grade determination method, estimation device, learning device, learned model generation method, and program - Google Patents

Description

The present invention relates to a scrap grade determination system, a scrap grade determination method, an estimation device, a learning device, a learned model generation method, and a program.

Generally, at an electric furnace manufacturer's scrap receiving site, two people, an acceptance inspector and a crane operator, visually check the scrap loading trailer and the steel scrap lifted up by a lifting magnet, and determine the proportion of each type and grade.

Japanese Patent Application Publication No. 7-286969

However, because inspection personnel visually check steel scrap from a distance for safety reasons, the accuracy of determining grade ratios may not be sufficient. In addition, there may be variations in the judgment criteria depending on the experience of the inspection personnel.

In particular, regarding iron scrap called heavy scrap, which is graded according to size, it is difficult to determine the grade ratio because iron scrap of various sizes are mixed together.

The present invention has been made in view of the above problems, and its main purpose is to provide a scrap grade determination system, a scrap grade determination method, an estimation device, and a learning method that can improve the accuracy of determining the grade ratio of iron scrap. The object of the present invention is to provide a device, a method for generating a trained model, and a program.

In order to solve the above problems, a scrap grade determination system according to one aspect of the present invention includes a camera that photographs a collection of iron scrap in which multiple grades of iron scrap are mixed depending on the size, and a camera that photographs a collection of iron scrap. Estimate the proportion of each grade included in the set of iron scrap from the image generated by the camera using the image as input data and the proportion of each grade as training data using a trained model built in advance by machine learning. The apparatus includes an inference section and an output section that outputs an estimation result by the inference section.

In addition, in a scrap grade determination method according to another aspect of the present invention, a collection of iron scraps in which iron scraps of a plurality of grades are mixed according to size is photographed with a camera, and an image of the collection of iron scraps is used as input data. , using a trained model built in advance by machine learning using the proportion of each grade as training data, estimate the proportion of each grade included in the set of iron scrap from the image generated by the camera, and output the estimation result. do.

Further, an estimation device according to another aspect of the present invention includes an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades depending on the size are mixed; Using an image of the set of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, calculate the ratio of each grade included in the set of iron scrap from the image generated by the camera. and an inference unit that estimates.

Further, a program according to another aspect of the present invention includes an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades depending on the size are mixed; Using an image of the set of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, calculate the ratio of each grade included in the set of iron scrap from the image generated by the camera. The computer functions as an inference unit that estimates .

Further, a learning device according to another aspect of the present invention provides an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades depending on the size are mixed, and an image obtained by photographing a collection of iron scraps with a camera, an acquisition unit that obtains the proportion of each grade contained in a set of iron scrap; and a trained model that uses the image as input data and the proportion of each grade as training data to estimate the proportion of each grade contained in a set of iron scrap from the image. and a learning section that constructs the system using machine learning.

Further, a program according to another aspect of the present invention provides an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades depending on the size are mixed, and images included in the collection of iron scraps. an acquisition unit that acquires the proportion of each grade; and a trained model that uses the image as input data and the proportion of each grade as training data to estimate the proportion of each grade in a set of iron scrap from the image. The computer functions as a learning section that constructs the following using machine learning.

Further, a method for generating a trained model according to another aspect of the present invention includes an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades according to size are mixed, and an image obtained by photographing a collection of iron scraps with a camera, and A trained program to estimate the proportion of each grade contained in a set of iron scrap from the image, using the image as input data and the proportion of each grade as training data. Build a model using machine learning.

According to the present invention, it is possible to improve the accuracy of determining the grade ratio of iron scrap.

It is a diagram showing an example of the configuration of a scrap grade determination system according to an embodiment. It is a figure showing the grade and requirements of heavy waste. FIG. 3 is a plan view of a scrap acceptance site. It is a side view of a scrap acceptance site. FIG. 1 is a block diagram showing a configuration example of a determination device according to a first embodiment. FIG. 3 is a diagram showing an example of association between images and labels. It is a figure which shows the example of an image. FIG. 3 is a flow diagram showing an example of a procedure of a learning phase. FIG. 3 is a diagram illustrating an example of building a trained model. FIG. 3 is a flow diagram showing an example of a procedure of an inference phase. FIG. 3 is a diagram showing an example of outputting an inference result. It is a block diagram showing an example of composition of a judgment device concerning a 2nd embodiment. FIG. 3 is a diagram showing an example of association between images and labels. FIG. 3 is a flow diagram showing an example of a procedure of an inference phase. FIG. 3 is a diagram showing an example of outputting an inference result. FIG. 7 is a diagram showing an example of association between an image and a label according to a third embodiment. FIG. 3 is a diagram showing an example of outputting an inference result. It is a block diagram showing an example of composition of a judgment device concerning a 4th embodiment. FIG. 3 is a diagram showing an example of association between images and labels. It is a figure which shows the example of an image. FIG. 3 is a diagram showing an example of image processing. It is a block diagram showing an example of composition of a judgment device concerning a 5th embodiment. FIG. 3 is a flow diagram showing an example of a procedure of an inference phase. FIG. 3 is a diagram for explaining processing. FIG. 3 is a diagram for explaining processing.

Embodiments of the present invention will be described below with reference to the drawings.

[System overview]
FIG. 1 is a diagram showing a configuration example of a scrap grade determination system 100 according to an embodiment. The scrap grade determination system 100 includes a determination device 1 , a camera 2 , a weight sensor 3 , output units 6 and 7 , and a crane 9 . The determination device 1 is capable of communicating with the camera 2 and the output units 6 and 7 via a communication network such as the Internet or a LAN.

The scrap grade determination system 100 determines the grade ratio of iron scrap S unloaded from a scrap loading vehicle T to a scrap yard Y by a crane 9 at a scrap receiving site.

The camera 2 photographs the collection of iron scraps S loaded on the scrap loading vehicle T and the collection of iron scraps S lifted by the lifting magnet 91, and transmits the generated images to the determination device 1. The arrangement of the camera 2 will be described in detail later.

The determination device 1 estimates the grade ratio of the iron scrap S from the image generated by the camera 2, and transmits the estimation results to the output units 6 and 7.

The output units 6 and 7 output the estimation results from the determination device 1. The output unit 6 is, for example, a terminal such as a tablet computer carried by the inspection inspector A. The output unit 7 is, for example, a printer for issuing an acceptance certificate.

The weight sensor 3 is, for example, a footboard sensor buried at the entrance/exit of a scrap receiving site, and detects the load applied from the wheels of the scrap loading vehicle T.

The iron scraps S loaded on the scrap loading vehicle T include iron scraps S of various sizes. The iron scrap S is, for example, heavy scrap. As shown in FIG. 2, heavy debris is classified into grades HS to H4 according to size (specifically, thickness, width or height, and length) and weight.

[Scrap acceptance site]
3 and 4 are a plan view and a side view schematically showing a scrap receiving site. The scrap acceptance site includes a scrap yard Y and a vehicle entry area P adjacent thereto. The crane 9 installed at the scrap receiving site moves the lifting magnet 91 above the scrap yard Y and the vehicle entry area P.

The crane 9 is, for example, a trolley-type overhead crane, and includes a pair of runway girders 92 , a pair of crane girders 93 placed astride the pair of runway girders 92 , and a club trolley 94 placed astride the pair of crane girders 93 . It is equipped with The runway girder 92 is supported by building pillars 97.

The crane girder 93 is movable along the longitudinal direction of the runway girder 92, and the club trolley 94 is movable along the longitudinal direction of the crane girder 93. A hook block 98 is suspended from the club trolley 94, and a lifting magnet 91 is suspended from the hook block 98.

As shown in FIG. 3, some of the cameras 21 are installed so as to photograph the loading platform of the scrap-loaded vehicle T that has stopped in the vehicle entry area P (hereinafter, the camera 21 that photographs the loading platform of the scrap-loaded vehicle T (referred to as the "vehicle photographing camera 21"). A plurality of vehicle photographing cameras 21 are installed, for example, in a hallway 95, an acceptance inspection room 96, etc. at a scrap receiving site, and photograph the entire loading platform of a scrap-loaded vehicle T from above.

The walkway 95 and the acceptance inspection room 96 are places where acceptance inspectors have conventionally observed the loading platform of scrap-loaded vehicles T, and by installing the vehicle camera 21 in these locations, the field of view of conventional acceptance inspectors can be recreated. be able to.

When the scrap loading vehicle T enters the vehicle entry area P, the vehicle photographing camera 21 photographs the iron scrap S fully piled up on the loading platform of the scrap loading vehicle T. Subsequently, the vehicle photographing camera 21 photographs the iron scraps S left on the loading platform of the scrap loading vehicle T every time the lifting magnet 91 lifts some of the iron scraps S from the iron scraps S piled up on the loading platform of the scrap loading vehicle T. The vehicle photographing camera 21 repeats this photographing until the iron scrap S disappears from the loading platform of the scrap loading vehicle T.

Here, among the iron scraps S left on the loading platform, the iron scrap S in the surface layer photographed by the vehicle photographing camera 21 is almost the same as the iron scrap S that will be lifted next by the lifting magnet 91.

As shown in FIG. 4, the other camera 22 is installed to photograph the iron scrap S lifted by the lifting magnet 91 (hereinafter, the camera 22 for photographing the iron scrap S lifted by the lifting magnet 91). (referred to as the "RiffMag Photography Camera 22"). A plurality of riffmag photographing cameras 22 are installed, for example, in the acceptance inspection room 96, building pillars 97, near the ground of the scrap yard Y, etc., and photograph the iron scrap S lifted by the lifting magnet 91 from the side or below.

The crane 9 moves the lifting magnet 91 from the scrap loading vehicle T to the scrap yard Y so that the lifting magnet 91 passes through a predetermined position. A riffmag photographing camera 22 is installed to take a photograph toward the determined position, and photographs the iron scrap S lifted by the lifting magnet 91 in synchronization with the timing when the lifting magnet 91 passes the determined position.

The acceptance inspection room 96 is also a place where acceptance inspection personnel traditionally observed the steel scrap S lifted by the lifting magnet 91, and by installing the riffmag photography camera 22 in such a place, the field of view of conventional acceptance inspection personnel can be recreated. can do.

Furthermore, the rifmag photographic camera 22 is installed on the opposite side of the acceptance inspection room 96 or near the ground of the scrap yard Y to photograph the entire steel scrap S lifted by the lifting magnet 91 from various angles. This makes it possible to use visual field images that conventional inspection personnel cannot obtain for learning and inference.

The rifmag photographing camera 22 photographs the iron scrap S lifted by the lifting magnet 91 every time the lifting magnet 91 lifts a part of the iron scrap S from the iron scrap S piled up on the loading platform of the scrap loading vehicle T. The riffmag photographing camera 22 repeats this photographing until the iron scrap S disappears from the loading platform of the scrap loading vehicle T.

[First embodiment]
FIG. 5 is a block diagram showing a configuration example of the determination device 1 according to the first embodiment. The determination device 1 is both a learning device and an estimation device. The determination device 1 includes a control unit 10 including a CPU, RAM, ROM, nonvolatile memory, input/output interface, and the like. The determination device 1 is composed of, for example, one or more server computers.

The control unit 10 can access the database 20. The database 20 may be provided inside the determination device 1 or may be provided outside the determination device 1 and accessed, for example, via a communication network such as the Internet or a LAN.

The control unit 10 includes an image acquisition unit 11 , a learning unit 13 , an inference unit 14 , a partial proportion determination unit 15 , and a total proportion determination unit 16 . These functional units are realized by the CPU of the control unit 10 executing information processing according to a program loaded into the RAM from the ROM or nonvolatile memory.

The program may be supplied, for example, via an information storage medium such as an optical disk or a memory card, or may be supplied, for example, via a communication network such as the Internet or a LAN.

[Learning phase]
Among the functional units realized by the control unit 10 shown in FIG. 5, the image acquisition unit 11 and the learning unit 13 are functional units for executing a learning phase, and correspond to a learning device. The learning phase corresponds to a method for generating a trained model. Note that the functional unit for executing the learning phase may be realized by a device different from the determination device 1.

The image acquisition unit 11 acquires an image of the collection of iron scraps S generated by the camera 2. The acquired images are stored in the database 20 in association with labels including grade percentages.

The learning unit 13 reads out images and labels associated therewith from the database 20, and uses the images as input data and the labels as training data to create a trained model for estimating the grade ratio of a set of iron scraps S from the images. Build with machine learning.

FIG. 6 is a diagram showing an example of association between images and labels. The association between images and labels is managed by a table as shown in the figure. “Image ID” is image identification information.

The "label" includes the proportion of each grade of heavy scrap included in the collection of iron scrap S as a grade proportion. The illustrated example includes grades HS, H1, H2, H3, and H4.

For example, a value determined by an inspection person may be assigned to the "label", or a value actually measured may be assigned to the "label".

The ratio is, for example, a weight ratio, but is not limited to this, and may be an area ratio or a volume ratio. As for the iron scrap S, since the materials are common, the area ratio and volume ratio are almost similar to the weight ratio.

FIGS. 7A and 7B are diagrams showing examples of images of a collection of iron scraps S in which iron scraps S of a plurality of grades are mixed.

FIG. 7A is an example of an image generated by the vehicle photographing camera 21 photographing the loading platform of the scrap-loaded vehicle T. This image corresponds to the entry whose image ID is "aaaa" in the table shown in FIG.

When a plurality of vehicle photographing cameras 21 simultaneously photograph the loading platform of a scrap-loaded vehicle T from different directions, the same label is given to the plurality of generated images. Alternatively, a set of a plurality of generated images may be treated as one input data.

FIG. 7(b) is an example of an image generated by photographing the iron scrap S lifted by the lifting magnet 91 by the rifmag photographing camera 22. This image corresponds to the entry with the image ID "bbbb" in the table shown in FIG.

When photographing the iron scrap S lifted by the lifting magnet 91 simultaneously from different directions with a plurality of rifmag photography cameras 22, the same label is given to the plurality of generated images. Alternatively, a set of a plurality of generated images may be treated as one input data.

Images used for machine learning are not necessarily limited to images taken by the vehicle camera 21 or the RIFMAG camera 22, and may be images of iron scrap taken at other locations, for example.

Further, an image of a collection of iron scraps S for each grade (that is, an image containing only iron scraps S of the same grade) as shown in FIG. 7(c) may be used for machine learning. This image corresponds to the entry with the image ID "cccc" in the table shown in FIG.

Further, a partial image cut out from an image of a collection of iron scraps S according to a user's operation input may be used for machine learning. For example, by cutting out parts of an image that are of interest to experienced inspection personnel and using them for machine learning, estimation accuracy can be expected to improve.

FIG. 8 is a flow diagram illustrating an example of a learning phase procedure as a method for generating a trained model. The control unit 10 functions as an image acquisition unit 11 and a learning unit 13 by executing the information processing shown in the figure according to a program.

First, the control unit 10 acquires an image and a label associated with the image (S11, processing as an acquisition unit). In the learning phase, images and labels associated with them are stored in advance in the database 20, and read from the database 20 as needed.

Next, the control unit 10 extracts a portion of the plurality of prepared images and a set of labels associated therewith as training data for machine learning (S12), and executes machine learning using the extracted training data. (S13, processing as the learning section). Machine learning is performed using images as input data and labels as training data. As a result, a learned model for estimating the grade ratio of the set of iron scraps S from the image is constructed.

Next, the control unit 10 extracts a part other than the training data from a plurality of prepared images and a set of labels associated therewith as test data (S14), and uses the extracted test data to complete the learning process. The model is evaluated (S15). After that, the control unit 10 stores the trained models whose evaluation is equal to or higher than a predetermined value in the database 20 (S16), and ends the learning phase.

FIG. 9 is a diagram showing an example of building a trained model. The trained model is, for example, a convolutional neural network, and includes a convolution layer, a pooling layer, a fully connected layer, and an output layer. In particular, a deep neural network in which neurons are combined in multiple stages is suitable.

The output layer is provided with a number of elements corresponding to the labels (see FIG. 6). That is, a plurality of elements are provided corresponding to the proportion of each grade of heavy scrap included in the set of iron scraps S.

For example, a softmax function is used as an element related to the grade ratio of the output layer (an element related to the proportion of each grade of heavy waste). In this case, since the output value is obtained as a real number between 0 and 1, the output value can be interpreted as a percentage.

Note that the illustrated convolutional neural network is just an example, and the layer structure is not limited to this, and the numbers of convolutional layers, pooling layers, and fully connected layers may be different. Further, machine learning other than neural networks such as support vector machine, Gaussian process, decision tree, etc. may be used.

[Inference phase]
Among the functional units realized by the control unit 10 shown in FIG. 5, the image acquisition unit 11, the inference unit 14, the partial ratio determination unit 15, and the whole ratio determination unit 16 are functional units for executing the inference phase, Corresponds to an estimation device. The inference phase corresponds to the scrap grade determination method.

The image acquisition unit 11 acquires an image of the set of iron scraps S generated by the camera 2 and outputs it to the inference unit 14. In the inference phase, the image acquired by the image acquisition unit 11 is directly input to the inference unit 14. The present invention is not limited to this, and the image may be stored in the database 20 once and then read out.

The inference unit 14 uses the image acquired by the image acquisition unit 11 as input data, and estimates the grade ratio of the set of iron scraps S from the image using the learned model constructed in the learning phase. The inference section 14 outputs the estimation result to the partial ratio determination section 15.

Each time the lifting magnet 91 lifts the iron scrap S, the partial proportion determining unit 15 calculates the grade proportion estimated from the image generated by the vehicle photographing camera 21 immediately before lifting and the image generated by the rifmag photographing camera 22 at the time of lifting. The grade proportion of the lifted iron scrap S is determined based on the grade proportion estimated from .

The overall ratio determination unit 16 determines whether the iron scrap S is unloaded from the scrap loading vehicle T to the scrap yard Y based on the grade ratio of the iron scrap S determined by the partial ratio determination unit 15 each time the lifting magnet 91 lifts the iron scrap S. Determine the grade ratio of the entire iron scrap S.

FIG. 10 is a flow diagram illustrating an example of the procedure of the inference phase as a scrap grade determination method. The control unit 10 functions as an image acquisition unit 11, an inference unit 14, a partial ratio determination unit 15, and a whole ratio determination unit 16 by executing the information processing shown in the figure according to a program.

First, the control unit 10 acquires an image photographed by the vehicle photographing camera 21 (see FIG. 3) (S21, processing as an acquisition unit). The image taken by the vehicle photographing camera 21 is an image of the iron scrap S piled up on the loading platform of the scrap loading vehicle T before being lifted by the lifting magnet 91.

Next, the control unit 10 uses the acquired image as input data and uses the trained model created in the learning phase to estimate the grade proportion (proportion of each grade of heavy scrap) contained in the iron scrap S ( S22, processing as an inference unit). Moreover, when acquiring a plurality of images simultaneously photographed by a plurality of vehicle photographing cameras 21, the class ratio estimated from each image is averaged.

Next, the control unit 10 acquires an image photographed by the RIFMag photography camera 22 (see FIG. 4) (S23, processing as an acquisition unit). The image taken by the rifmag photographing camera 22 is an image of a part of the iron scrap S lifted by the lifting magnet 91 from the iron scrap S piled up on the loading platform of the scrap loading vehicle T.

Next, the control unit 10 uses the acquired image as input data and uses the trained model created in the learning phase to estimate the grade proportion (proportion of each grade of heavy scrap) contained in the iron scrap S ( S24, processing as an inference unit). Moreover, when acquiring a plurality of images simultaneously photographed by a plurality of riffmag photography cameras 22, the grade ratio estimated from each image is averaged.

Next, the control unit 10 averages the grade proportion estimated in S22 and the grade proportion estimated in S24, and determines the averaged grade proportion as the grade proportion of the iron scrap S lifted by the lifting magnet 91. (S25, processing as a partial ratio determination unit). For the averaging, an arithmetic average may be used, or a weighted average in which weights are determined such that one is the main and the other is the secondary, for example, may be used.

Here, of the iron scrap S piled up on the loading platform of the scrap loading vehicle T before being lifted by the lifting magnet 91, the surface portion of the iron scrap S photographed by the vehicle photographing camera 21 is photographed by the rifmag photographing camera 22. This is almost the same as the iron scrap S lifted by the lifting magnet 91. That is, it can be said that the image acquired in S21 and the image acquired in S23 represent substantially the same iron scrap S from different viewpoints.

Therefore, in S25, the grade proportion of the iron scrap S lifted by the lifting magnet 91 is determined by averaging the grade proportion estimated in S22 and the grade proportion estimated in S24.

The control unit 10 repeats the processing of S21 to S25 every time the lifting magnet 91 lifts the iron scrap S until all the iron scrap S is unloaded from the scrap loading vehicle T (S26).

When all the iron scraps S are unloaded from the scrap loading vehicle T (S26: YES), the control unit 10 totals the grade ratio determined in S25 every time the lifting magnet 91 lifts the iron scrap S, and calculates the average By calculating the value, the grade ratio of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y is determined (S27, processing as the overall ratio determination unit). For averaging, an arithmetic average may be used, or a weighted average multiplied by a weight according to the weight, for example, may be used.

After that, the control unit 10 transmits the grade ratio of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y, determined in S27, to the output units 6 and 7 as an estimation result (S28). The output units 6 and 7 output the received estimation results by, for example, displaying or printing.

FIG. 11 is a diagram showing an example of output of the inference result. The estimation result includes the grade ratio (ratio of each grade of heavy scrap) of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y.

According to the embodiment described above, the grade ratio of a set of iron scraps S is estimated from an image using a trained model created in advance by machine learning, so it is possible to improve the estimation accuracy. .

Furthermore, according to the embodiment, it is possible to eliminate variations in grade judgment levels among acceptance inspectors and to equalize the grade judgment levels.

Further, according to the embodiment, by using images taken of a collection of iron scraps S from various angles by a plurality of cameras 2, it is possible to further improve the estimation accuracy.

Furthermore, the camera 2 can be installed in places where it has not been possible to place inspection personnel in the past, such as by photographing the iron scrap S lifted by the lifting magnet 91 with the rifmag photography camera 22 so as to look down from below. This makes it possible to use images taken of a collection of iron scraps S from various angles for learning and inference while ensuring the safety of inspection personnel.

Further, according to the embodiment, since the estimation of the grade ratio is automated, the acceptance inspection work can be carried out by a single driver of the crane 9, and it is possible to save the labor of the acceptance inspector. Furthermore, by combining the operation automation of the crane 9, it becomes possible to unmanned the receiving, unloading, and acceptance inspection work of iron scrap.

[Second embodiment]
The second embodiment will be described below. For configurations or procedures that overlap with those of the above embodiment, detailed explanations will be omitted by assigning the same numbers.

FIG. 12 is a block diagram showing a configuration example of the determination device 1 according to the second embodiment. The control unit 10 includes, in addition to an image acquisition unit 11, a learning unit 13, an inference unit 14, a partial proportion determination unit 15, and a whole proportion determination unit 16, a weight measurement unit 12, a weight calculation unit 17, a weight integration unit 18, and It includes a weight correction section 19.

[Learning phase]
The image of the collection of iron scraps S acquired by the image acquisition unit 11 is stored in the database 20 in association with a label including the grade ratio and weight.

The learning unit 13 reads images and labels associated therewith from the database 20, uses the images as input data, uses the labels as training data, and performs learning to estimate the grade ratio and weight of a set of iron scraps S from the images. Build a model using machine learning.

FIG. 13 is a diagram showing an example of association between images and labels. The "label" includes the proportion of each grade of heavy scrap (HS, H1, H2, H3, and H4 in this example) included in the set of iron scrap S as a grade percentage, as well as the Contains the weight of

Note that no weight label is attached to the image generated by the vehicle photographing camera 21 photographing the loading platform of the scrap-loaded vehicle T (see FIG. 7(a)), which corresponds to the entry with the image ID "aaaa". .

[Inference phase]
Among the functional units realized by the control unit 10 shown in FIG. 12, the image acquisition unit 11, the inference unit 14, the partial proportion determination unit 15, the whole proportion determination unit 16, the weight calculation unit 17, the weight accumulation unit 18, and the weight correction unit. 19 is a functional unit for executing the inference phase, and corresponds to an estimation device.

The inference unit 14 uses the image acquired by the image acquisition unit 11 as input data, and estimates the grade ratio and weight of the set of iron scraps S from the image using the learned model constructed in the learning phase. The inference unit 14 outputs the estimation result regarding the grade ratio to the partial ratio determination unit 15 and outputs the estimation result regarding the weight to the weight calculation unit 17.

The weight measurement unit 12 acquires the weight of the scrap-loaded vehicle T at the time of entry and the weight of the scrap-loaded vehicle T at the time of exit from the weight sensor 3 provided at the entrance/exit of the scrap acceptance site, and calculates the difference between them. The weight (measured weight) of the entire iron scrap S unloaded from T to the scrap yard Y is calculated.

The weight calculation unit 17 obtains the weight of the lifted iron scrap S estimated by the inference unit 14 each time the lifting magnet 91 lifts the iron scrap S.

Furthermore, the weight calculation unit 17 calculates the weight of the lifted iron scrap S based on the grade ratio of the lifted iron scrap S determined by the partial ratio determination unit 15 and the weight of the lifted iron scrap S estimated by the inference unit 14. Calculate the weight of each grade contained in the steel scrap S.

The weight accumulation unit 18 accumulates the weight of the iron scrap S estimated by the inference unit 14 each time the lifting magnet 91 lifts the iron scrap S, thereby calculating the weight of the iron scrap unloaded from the scrap loading vehicle T to the scrap yard Y. Calculate the weight (estimated weight) of the entire S.

Moreover, the weight accumulating unit 18 accumulates the weight of each grade included in the lifted iron scrap S, which is calculated by the weight calculating unit 17 each time the lifting magnet 91 lifts the iron scrap S. The weight (estimated weight) of each grade included in the entire iron scrap S unloaded from T to the scrap yard Y is also calculated.

The weight correction unit 19 corrects the estimated weight of the entire iron scrap S calculated by the weight integration unit 18 based on the measured weight of the entire iron scrap S calculated by the weight measurement unit 12.

In addition, the weight correction section 19 calculates the weight of the entire iron scrap S based on the ratio between the measured weight of the entire iron scrap S calculated by the weight measurement section 12 and the estimated weight of the entire iron scrap S calculated by the weight integration section 18. The estimated weight of each grade included in the entire iron scrap S calculated in step 18 is also corrected.

FIG. 14 is a flow diagram illustrating an example of the procedure of the inference phase as a scrap grade determination method. The control unit 10 executes the information processing shown in the figure according to the program, thereby controlling the image acquisition unit 11, the inference unit 14, the partial proportion determination unit 15, the whole proportion determination unit 16, the weight calculation unit 17, the weight integration unit 18, and functions as a weight correction section 19.

The control unit 10 uses the image of the rifmag photographing camera 22 (see FIG. 4) acquired in S23 as input data, and uses the learned model created in the learning phase to attach the iron scrap S lifted by the lifting magnet 91. The weight is estimated together with the grade ratio included (S31, processing as the inference section).

In addition, in S22, estimation is performed using the trained model using the image of the vehicle camera 21 (see FIG. 3) acquired in S21 as input data, but here, the weight of the iron scrap S is estimated by the trained model. Even if it is, don't use it.

Next, the control unit 10 multiplies the grade ratio and the weight of the iron scrap S lifted by the lifting magnet 91 determined or estimated in S25 and S31, and calculates the content of the iron scrap S lifted by the lifting magnet 91 by multiplying the grade ratio and the weight thereof. The weight of each grade is calculated (S32, processing as a weight calculation unit).

Next, the control unit 10 integrates the weight of the iron scrap S lifted by the lifting magnet 91 estimated in S31 (S33, processing as a weight integration unit). Specifically, the control unit 10 adds the weight estimated in S31 to the previous cumulative weight stored in the memory to update the cumulative weight. Furthermore, the control unit 10 similarly adds up the weight of each grade calculated in S32.

Next, the control unit 10 determines the weight accumulated in S33 each time the lifting magnet 91 lifts the iron scrap S as the weight of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y ( S34). Furthermore, the control unit 10 similarly determines the weight of each grade accumulated in S33 as the weight of each grade included in the entire iron scrap S.

Next, if the weight (estimated weight) of the entire iron scrap S determined in S34 is different from the weight (measured weight) of the entire iron scrap S measured by the weight sensor 3, the control unit 10 determines (S35: YES) , the estimated weight is corrected based on the measured weight (S36). For example, the estimated weight is replaced with the measured weight, with the measured weight as the correct value.

Furthermore, the control unit 10 multiplies the weight of each grade included in the entire iron scrap S determined in S34 by the ratio of the measured weight to the estimated weight (measured weight/estimated weight). The weight of each class included is also corrected.

Thereafter, the control unit 10 outputs the grade ratio and weight of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y, determined in S27 and S34 (or S36), as an estimation result to the output unit 6, 7 (S28). The output units 6 and 7 output the received estimation results by, for example, displaying or printing.

FIG. 15 is a diagram showing an example of output of the inference result. The estimation results include the grade ratio of the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y (ratio of each grade of heavy scrap), the weight of the entire iron scrap S, and the amount included in the entire iron scrap S. Includes the weight of each grade.

Note that the weight of each grade included in the entire iron scrap S may be calculated as follows. Conventionally, an acceptance inspector observes the loading platform of a scrap loading vehicle T loaded with the entire steel scrap S that has just entered the scrap receiving site, and subtracts the vehicle weight from the weight of the scrap loading vehicle T at the time of approach. Considering the weight of the entire iron scrap S, the approximate weight of each grade included in the entire iron scrap S is determined.

In order to obtain a similar outline, we used images taken by the vehicle photographing camera 21 of the loading platform of a scrap loading vehicle T loaded with all of the iron scrap S that had just entered the scrap receiving site. By estimating the proportions and multiplying them by the weight of the entire iron scrap S measured by the weight sensor 3, the weight of each grade included in the entire iron scrap S may be calculated.

[Third embodiment]
The third embodiment will be described below. For configurations or procedures that overlap with those of the above embodiment, detailed explanations will be omitted by assigning the same numbers. The configuration example and procedure example of the determination device 1 according to the third embodiment are the same as those of the determination device according to the first embodiment shown in FIGS. 5, 8, and 10 above.

In addition to heavy scraps, the iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y by the crane 9 at the scrap receiving site contains shredder scraps, new cuttings, steel powder, moisture, dust, etc. There are times when I am.

Therefore, in the third embodiment, not only the proportions of each grade of heavy scraps included in the set of iron scraps S, but also the proportions of shredder scraps, new cuttings, steel powder, and moisture/dust are learned and estimated.

That is, as shown in FIG. 16, the "labels" used in the learning phase include, in addition to the proportion of each grade of heavy scrap (HS, H1, H2, H3, and H4) included in the set of iron scrap S, It also includes shredder scraps, new cuttings, steel powder, and the percentage of moisture and dust.

The learning unit 13 (see FIG. 5) uses such labels as training data to determine from the image the proportions of each grade of heavy scraps included in the set of iron scraps S, as well as shredder scraps, new cuttings, steel scrap powder, and water/water. A trained model for estimating the proportion of dust is constructed using machine learning.

Then, the inference unit 14 uses the learned model constructed by the learning unit 13 to determine the ratio of each grade of heavy scrap included in the set of iron scraps S, shredder scrap, new Estimate the proportions of steel powder, moisture, and dust.

The partial proportion determining unit 15 determines the proportions of each grade of heavy scraps included in the collection of iron scraps S lifted by the lifting magnet 91, as well as the proportions of shredder scraps, new cuttings, steel powder, and moisture/dust.

The overall ratio determination unit 16 determines the ratio of each grade of heavy scraps included in the entire iron scrap S unloaded from the scrap loading vehicle T to the scrap yard Y, as well as the ratio of shredder scraps, new cuttings, steel powder, and moisture/dust. Determine the proportion.

FIG. 17 is a diagram showing an example of output of the inference result. The estimation results include the proportion of each grade of heavy scrap included in the entire steel scrap S unloaded from the scrap loading vehicle T to the scrap yard Y, as well as the proportion of shredder scrap, new cutting, steel powder, and moisture/dust. include.

Note that the proportion of each grade may also be learned and estimated for shredder waste, new cuttings, or steel powder.

The third embodiment described above may further be combined with the weight learning/estimation described in the second embodiment. That is, the ratio and weight of each grade of heavy scrap, shredder scrap, new cutting, steel powder, and moisture/dust included in the collection of iron scrap S may be learned and estimated.

[Fourth embodiment]
The fourth embodiment will be described below. For configurations or procedures that overlap with those of the above embodiment, detailed explanations will be omitted by assigning the same numbers.

In addition to heavy scraps, the iron scrap S unloaded by the crane 9 from the scrap loading vehicle T to the scrap yard Y at the scrap receiving site includes non-ferrous metals such as copper or lead, sealed objects, and items exceeding the specified size and weight. , or items that are excluded from acceptance, such as bump-shaped reinforcing bars, may be mixed in.

Therefore, in the fourth embodiment, the proportion of each grade of heavy scrap included in the set of iron scrap S is learned and estimated, as well as the presence or absence of items excluded from acceptance.

FIG. 18 is a block diagram showing a configuration example of the determination device 1 according to the fourth embodiment. The control unit 10 includes an alarm determination unit 81 in addition to an image acquisition unit 11, a learning unit 13, an inference unit 14, a partial ratio determination unit 15, and a total ratio determination unit 16.

The alarm output unit 8 includes, for example, a revolving light and a siren, and outputs an alarm upon receiving an output command from the determination device 1 (alarm determination unit 81).

As shown in Figure 19, the "labels" used in the learning phase include the percentage of each grade of heavy scrap (HS, H1, H2, H3, and H4) included in the collection of iron scrap S, as well as the percentage of acceptance and exclusion. It also includes the presence or absence of products.

However, this is not limited to the above, and the "label" includes items for each type of non-ferrous metal such as copper or lead, sealed objects, items exceeding specified dimensions and weight, and items that are excluded from acceptance, such as bullet-shaped reinforcing bars. It is preferable.

Using such labels as training data, the learning unit 13 uses machine learning to construct a trained model for estimating the proportion of each grade of heavy scrap included in the set of iron scraps S and the presence or absence of items excluded from acceptance from the image. do.

When a convolutional neural network (see FIG. 9) is used as a trained model, it is preferable to use a sigmoid function that outputs a value of 0 or more and 1 or less, for example, as an element related to the presence or absence of accepted/excluded products in the output layer.

Then, using the trained model constructed by the learning unit 13, the inference unit 14 uses the image acquired by the image acquisition unit 11 to determine which iron scraps S to be included in the set of iron scraps S, each time the lifting magnet 91 lifts the iron scrap S. Estimate the proportion of each grade of heavy scrap and the presence or absence of items that are excluded from acceptance. The inference unit 14 outputs the estimation result regarding the presence or absence of acceptance excluded items to the warning determination unit 81.

The alarm determination unit 81 determines the presence or absence of an excluded product based on the estimation result from the inference unit 14, and when it is determined that there is an excluded product, sends an output command to the alarm output unit 8 to issue an alarm. Output. For example, when a value (for example, a value of 0 or more and 1 or less) indicating the presence or absence of a product excluded from acceptance is equal to or greater than a threshold value, it is determined that there is a product excluded from acceptance.

In addition, regarding non-ferrous metals such as copper or lead, there is little difference in appearance from iron scrap S, and there is a risk that the identification accuracy of non-ferrous metals may not be sufficient. A component analyzer or the like may be used supplementarily.

The fourth embodiment described above may further be combined with the weight learning/estimation described in the second embodiment. That is, the proportion and weight of each grade of heavy scrap included in the collection of iron scrap S may be learned and estimated, and the presence or absence of items excluded from acceptance may be learned and estimated.

Further, the fourth embodiment may further combine the learning and estimation of the proportions of shredder waste, new cuttings, steel powder, and moisture/dust described in the third embodiment. In other words, it learns and estimates the proportions of each grade of heavy scrap, shredder scrap, new cutting, steel powder, and moisture/dust contained in the collection of iron scrap S, and also learns and estimates the presence or absence of items that are excluded from acceptance. Good too.

The fourth embodiment also includes learning and estimating the weight as described in the second embodiment, and learning the proportions of shredder waste, new cuttings, steel powder, and moisture/dust as described in the third embodiment. - Both estimations may be combined. In other words, it learns and estimates the proportion and weight of each grade of heavy scrap, shredder scrap, new cutting, steel powder, and moisture/dust included in the collection of iron scrap S, as well as learns and estimates the presence or absence of items that are excluded from acceptance. You may.

[Fifth embodiment]
The fifth embodiment will be described below. For configurations or procedures that overlap with those of the above embodiment, detailed explanations will be omitted by assigning the same numbers.

[Learning phase]
FIG. 20 is a diagram showing an example of the original image L before being processed into a learning image in this embodiment. The original image L is an image obtained by photographing a collection of iron scraps S lifted by the lifting magnet 91 using a camera such as the rifmag photographing camera 22 (see FIG. 4).

The collection of iron scraps S lifted by the lifting magnet 91 usually has a generally inverted conical shape (approximately an inverted triangular shape in side view) that gradually narrows downward from the lifting magnet 91.

Note that the lifting magnet 91 is an example of a hanging tool. The present invention is not limited to this, and a grab bucket or the like may be used as the hanging tool.

FIG. 21 is a diagram showing an example of image processing performed on the original image L. First, as shown in FIG. 21(a), a range N of the lifting magnet 91 is specified in the original image L. The range N of the lifting magnet 91 is determined and specified by the user, for example.

Next, as shown in FIG. 21(b), a partial image LO determined based on the range N of the lifting magnet 91 is cut out from the original image L. The partial image LO is defined to include at least the area below the lifting magnet 91.

Specifically, the left and right ends of the partial image LO are located slightly outside the left and right ends of the lifting magnet 91. The width of the partial image LO is set to be slightly wider than the width of the lifting magnet 91 (for example, about 1.1 to 1.5 times).

Further, the upper end of the partial image LO is the position of the upper end of the lifting magnet 91. The lower end of the partial image LO is located below the lower end of the lifting magnet 91 by a predetermined height.

Specifically, the lower end of the partial image LO has the same width as the partial image LO, a base that overlaps with the lower end of the hanging part 91, and an inverted triangular range (for example, an inverted right triangle) that gradually narrows toward the bottom. This is the position of the lower end (apex) of NN.

Next, as shown in FIG. 21(c), a masking process is performed in which the inverted triangular range NN of the partial image LO is set as a display area and the other range NF is set as a non-display area. For example, the range NF adjacent to the left and right sides of the inverted triangular range NN is filled with black or the like or made transparent.

The partial image LO extracted and processed in this way is used as a learning image in the learning phase. According to this, in the learning image, background parts that do not require attention are excluded, so it is possible to improve the learning effect.

In addition, in the learning phase, in addition to the trained model for estimating the grade ratio of the set of iron scraps S from the image, the trained model for estimating the range of the lifting magnet 91 in the image (hereinafter referred to as range estimation) is used. (referred to as trained model) is further constructed.

For example, the trained model for range estimation is constructed by machine learning using the original image L shown in FIG. 21(a) as input data and using the range N of the lifting magnet 91 identified in the original image L as training data. .

The trained model for range estimation is, for example, an object detection model such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN.

[Inference phase]
FIG. 22 is a block diagram showing a configuration example of the determination device 1 according to the fifth embodiment. The control unit 10 includes an image acquisition unit 11, a learning unit 13, an inference unit 14, a partial proportion determination unit 15, a whole proportion determination unit 16, a weight calculation unit 17, and a weight correction unit 19, as well as a range estimation unit 41 and a processing unit. 42.

FIG. 23 is a flowchart showing an example of the procedure of the inference phase as a scrap grade determination method, which is implemented in the determination device 1. The control unit 10 of the determination device 1 executes the information processing shown in the figure according to a program.

First, when the control unit 10 acquires an image photographed by the lifting magnet photographing camera 22 (hereinafter referred to as a camera image) (S23), the control unit 10 uses the above-mentioned trained model for range estimation to estimate the lifting magnet 91 in the camera image. The range is estimated (S49, processing as the range estimation unit 41).

Next, the control unit 10 processes the camera image based on the estimated range of the lifting magnet 91 (S50, processing by the processing unit 42). The processing is similar to the processing in the learning phase described above (see FIG. 21).

Specifically, first, as shown in FIG. 24(a), the range M of the lifting magnet 91 is estimated in the camera image C acquired from the riffmag photography camera 22.

Next, as shown in FIG. 21(b), a partial image CO determined based on the range M of the lifting magnet 91 is cut out from the camera image C. Partial image CO is defined to include at least the area below lifting magnet 91.

Specifically, the left and right ends of the partial image CO are located slightly outside the left and right ends of the lifting magnet 91. The width of the partial image CO is set to be slightly wider than the width of the lifting magnet 91 (for example, about 1.1 to 1.5 times).

Further, the upper end of the partial image CO is the position of the upper end of the lifting magnet 91. The lower end of the partial image CO is located below the lower end of the lifting magnet 91 by a predetermined height.

Specifically, the lower end of the partial image CO has the same width as the partial image CO and a base that overlaps with the lower end of the hanging part 91, and the range of an inverted triangle (for example, an inverted right triangle) that gradually narrows toward the bottom. This is the position of the lower end (apex) of MN.

Next, as shown in FIG. 24(c), a masking process is performed in which the inverted triangular range MN of the partial image CO is set as a display area and the other range MF is set as a non-display area. For example, the range MF adjacent to the left and right sides of the inverted triangular range MN is filled with black or the like or made transparent.

Thereafter, the control unit 10 uses the partial image CO extracted and processed in this way as input data, and uses the trained model for estimating the grade ratio and weight to calculate the set of iron scrap S shown in the partial image CO. The grade proportion included in is estimated (S31-S33 in FIG. 23).

According to this, in images used as input data when estimating grade proportions and weights, background parts that do not need to be focused on are excluded, similar to training images, so that the accuracy of estimating grade proportions can be improved. It becomes possible to achieve this goal.

Note that the shape of the range MN used as the display area in the masking process is not limited to an inverted triangle. This applies not only to the masking process in the inference phase but also to the masking process in the learning phase.

For example, the shape of the range MN may be an inverted trapezoid shape that gradually narrows downward as shown in FIGS. It may also be rectangular. Further, as shown in FIGS. 25(b), (d), and (f), the hanging tool 91 above the range MN and both sides thereof may also be filled with black or the like as a non-display area.

[others]
In the above embodiment, the weight is included in the "label" used as training data during learning (see Figure 6), and the weight is estimated together with the grade ratio of iron scrap S at the time of inference (see Figure 10). Conversely, a weight measurement unit may be provided to measure the weight of the iron scrap S lifted by the lifting magnet 91, and the measured weight may be used as input data for learning and estimation. That is, the weight may be used together with the image as input data during learning, and the grade ratio may be estimated from the image and weight of the lifted iron scrap S during inference.

Furthermore, a sound measuring unit may be provided to measure the sound when the lifting magnet 91 lifts the iron scrap S, and the measured sound may be used as input data for learning and estimation. That is, sounds may be used together with images as input data during learning, and the grade ratio may be estimated from the images and sounds of the lifted iron scrap S during inference.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and it goes without saying that various modifications can be made by those skilled in the art.

1 determination device (learning device, estimation device), 2, 21, 22 camera, 3 weight sensor, 6, 7 output section, 10 control section, 11 image acquisition section, 12 weight measurement section, 13 learning section, 14 inference section, 15 partial ratio determination unit, 16 overall ratio determination unit, 17 weight calculation unit, 18 weight accumulation unit, 19 weight correction unit, 20 database, 8 alarm output unit, 9 crane, 91 lifting magnet, 92 runway girder, 93 crane girder, 94 Club trolley, 95 Corridor, 96 Acceptance inspection room, 97 Building pillar, 98 Hook block, 100 Scrap grade determination system, A Acceptance inspector, Y Scrap yard, S Scrap steel, T Scrap loading vehicle, P Vehicle entry area

Claims (20)

A camera that photographs a collection of iron scrap, which is a mixture of multiple grades of iron scrap depending on size,
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
an output unit that outputs the estimation result by the inference unit;
Equipped with
The inference unit uses a trained model built in advance by machine learning, with an image of a collection of iron scraps as input data and the ratio and weight of each grade as teacher data, and uses the image generated by the camera to determine the iron scraps. estimating the proportion of each grade included in the collection and the weight of the collection of iron scrap;
Scrap grading system. It is further equipped with a crane that lifts some of the iron scrap from the pile of iron scrap.
Each time the crane lifts a part of the iron scrap, the inference unit determines each grade contained in the part of the iron scrap from an image of the part of the iron scrap lifted by the crane taken by the camera. Estimate the proportion of
The scrap grading system according to claim 1. It is further equipped with a crane that lifts some of the iron scrap from the pile of iron scrap.
Each time the crane lifts a part of the iron scrap, the inference unit determines the part of the iron scrap from the image of the piled up iron scrap taken by the camera before the crane lifts the part of the iron scrap. Estimate the proportion of each grade contained in the scrap,
The scrap grading system according to claim 1 or 2. A proportion of each grade estimated from an image of some of the iron scrap lifted by the crane, and a proportion of each grade estimated from an image of the piled up iron scrap before the crane lifts some of the iron scrap. further comprising a partial proportion determination unit that determines the proportion of each grade contained in the part of the iron scrap based on
The scrap grading system according to claim 2 or 3. Further comprising an overall proportion determination unit that determines the proportion of each grade included in the entire pile of iron scraps based on the proportion of each grade estimated each time the crane lifts a part of the iron scrap.
The scrap grading system according to any one of claims 2 to 4. Further comprising a weight calculation unit that calculates the weight of each grade included in the iron scrap collection based on the proportion of each grade included in the iron scrap collection and the weight of the iron scrap collection.
A scrap grading system according to any one of claims 1 to 5 . It is further equipped with a crane that lifts some of the iron scrap from the pile of iron scrap.
Each time the crane lifts a portion of iron scrap, the reasoning unit calculates the proportion of each grade contained in the portion of iron scrap and the weight of the portion of iron scrap from the image taken by the camera. presume,
The scrap grade determination system according to any one of claims 1 to 6 . a weight accumulating unit that accumulates the estimated weight each time the crane lifts a portion of iron scrap;
a weight measurement unit that measures the weight of the entire pile of iron scrap;
a weight correction unit that corrects the weight integrated by the weight integration unit based on the weight measured by the weight measurement unit;
further comprising,
The scrap grading system according to claim 7 . The camera photographs a collection of iron scrap in which a plurality of grades of iron scrap according to the size and iron scrap of different types are mixed,
The inference unit uses an image of a collection of iron scraps as input data and a learned model pre-built by machine learning with the proportion of each grade and the proportion of different types as teacher data, and calculates the result from the image generated by the camera. estimating the proportion of each grade and the proportion of different types contained in the collection of iron scrap;
A scrap grading system according to any one of claims 1 to 8 . The camera photographs a collection of iron scrap in which a plurality of grades of iron scrap according to the size and excluded items are mixed,
The inference unit uses an image of a collection of iron scrap as input data, and uses a trained model pre-built by machine learning as training data of the proportion of each grade and the presence or absence of excluded items, and calculates the result from the image generated by the camera. Estimating the proportion of each grade included in the collection of iron scrap and the presence or absence of excluded items,
further comprising an alarm output unit that outputs an alarm when it is estimated that there is an excluded item;
A scrap grading system according to any one of claims 1 to 9 . A camera that photographs a collection of iron scrap, which is a mixture of multiple grades of iron scrap depending on size,
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
an output unit that outputs the estimation result by the inference unit;
Equipped with
A crane that lifts some iron scraps from a pile of iron scraps using a hoist;
processing means for extracting a partial image that is defined based on the range of the hanging device and includes at least a range below the hanging device from the image generated by the camera;
Furthermore,
The inference unit estimates the proportion of each grade included in the set of iron scrap from the partial image,
The processing means performs mask processing of the partial image so that a range that gradually narrows downward from the hanger is set as a display area, and the other range is set as a non-display area.
Scrap grading system. A camera that photographs a collection of iron scrap, which is a mixture of multiple grades of iron scrap depending on size,
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
an output unit that outputs the estimation result by the inference unit;
Equipped with
A crane that lifts some iron scraps from a pile of iron scraps using a hoist;
processing means for extracting a partial image that is defined based on the range of the hanging device and includes at least a range below the hanging device from the image generated by the camera;
Furthermore,
The inference unit estimates the proportion of each grade included in the set of iron scrap from the partial image,
further comprising a range estimating unit that estimates a range of the hanging device in an image generated by the camera;
Scrap grading system. A camera that photographs a collection of iron scrap, which is a mixture of multiple grades of iron scrap depending on size,
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
an output unit that outputs the estimation result by the inference unit;
Equipped with
A crane that lifts some iron scraps from a pile of iron scraps using a hoist;
processing means for extracting a partial image that is defined based on the range of the hanging device and includes at least a range below the hanging device from the image generated by the camera;
Furthermore,
The inference unit estimates the proportion of each grade included in the set of iron scrap from the partial image,
The trained model includes a partial image that is extracted from an image of a collection of iron scrap lifted up by a crane hoist, and is defined based on the range of the hoist, and includes at least a range below the hoist. As input data, the data is constructed in advance by machine learning.
Scrap grading system. A camera is used to photograph a collection of iron scraps, in which multiple grades of iron scraps are mixed according to size.
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. Estimate the proportion of
output the estimation results,
A scrap grade determination method , comprising:
The estimation uses an image of a collection of iron scraps as input data, and uses a trained model built in advance by machine learning with the proportion and weight of each grade as teaching data, and calculates the amount of iron scrap from the image generated by the camera. estimating the proportion of each grade included in the collection and the weight of the collection of iron scrap;
Scrap grade determination method. an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of multiple grades depending on size are mixed;
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
Equipped with
The inference unit uses a trained model built in advance by machine learning, with an image of a collection of iron scraps as input data and the ratio and weight of each grade as teacher data, and uses the image generated by the camera to determine the iron scraps. estimating the proportion of each grade included in the collection and the weight of the collection of iron scrap;
Estimation device. an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades depending on the size are mixed;
Using an image of a collection of iron scrap as input data and a trained model built in advance by machine learning using the ratio of each grade as training data, each grade included in the collection of iron scrap is determined from the image generated by the camera. an inference unit that estimates the proportion of
make the computer function as
The inference unit uses a trained model built in advance by machine learning, with an image of a collection of iron scraps as input data and the ratio and weight of each grade as teacher data, and uses the image generated by the camera to determine the iron scraps. estimating the proportion of each grade included in the collection and the weight of the collection of iron scrap;
program. an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades according to size are mixed, and a proportion of each grade included in the collection of iron scraps;
a learning unit that uses machine learning to construct a trained model for estimating the proportion of each grade included in a set of iron scrap from the image, using the image as input data and the proportion of each grade as training data;
Equipped with
The image is a partial image that is extracted from an image of the collection of iron scraps lifted by a hoist of a crane and is defined based on the range of the hoist and includes at least a range below the hoist. ,
The partial image is a partial image that has been subjected to mask processing such that a range that gradually narrows downward from the hanger is a display area, and the other range is a non-display area.
learning device. An image of a collection of iron scrap for each grade is also used as input data for building the trained model;
The learning device according to claim 17 . an acquisition unit that acquires an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of a plurality of grades according to size are mixed, and a proportion of each grade included in the collection of iron scraps;
a learning unit that uses machine learning to construct a trained model for estimating the proportion of each grade included in a set of iron scrap from the image, using the image as input data and the proportion of each grade as training data;
make the computer function as
The image is a partial image that is extracted from an image of the collection of iron scraps lifted by a hoist of a crane and is defined based on the range of the hoist and includes at least a range below the hoist. ,
The partial image is a partial image that has been subjected to mask processing such that a range that gradually narrows downward from the hanger is a display area, and the other range is a non-display area.
program. Obtaining an image obtained by photographing a collection of iron scraps with a camera, in which iron scraps of multiple grades according to size are mixed, and the proportion of each grade included in the collection of iron scraps,
Using the image as input data and the proportion of each grade as training data, constructing a trained model for estimating the proportion of each grade included in a collection of iron scrap from the image by machine learning;
A method for generating a trained model, the method comprising:
The image is a partial image that is extracted from an image of the collection of iron scraps lifted by a hoist of a crane and is defined based on the range of the hoist and includes at least a range below the hoist. ,
The partial image is a partial image that has been subjected to mask processing such that a range that gradually narrows downward from the hanger is a display area, and the other range is a non-display area.
How to generate a trained model.

Images