JP2021117867A - Waste agitation state evaluation device and waste agitation state evaluation method - Google Patents

Abstract

To provide a waste agitation state evaluation device and a waste agitation state evaluation method that can evaluate an agitation state more appropriately.SOLUTION: A waste agitation state evaluation device D of the present invention includes: an acquisition unit 1 that acquires an image of a waste contained in a storage unit; an identification extraction unit 22 that identifies and extracts an occupied area of the waste for each type of the waste in the image; and an evaluation unit 23 that evaluates the agitation state of the waste contained in the storage unit based on each occupied area for each type of the waste extracted by the identification extraction unit 22.SELECTED DRAWING: Figure 1

Description

The present invention relates to a waste agitation state evaluation device for evaluating a waste agitation state and a waste agitation state evaluation method.

Waste that is judged by the owner to be worthless or unnecessary and discarded as garbage is generally burned in an incinerator and incinerated. In the combustion of this waste, the waste is agitated so as to homogenize the waste in order to realize stable combustion in the incinerator. The agitated state of the waste means a state in which the garbage bag is broken and, for example, plants and sludge are homogenized, and the quality of the agitated state is determined by the degree of homogenization. Regarding the evaluation of the agitated state of such waste, for example, there is a technique disclosed in Patent Document 1.

The waste mixing degree evaluation system disclosed in Patent Document 1 includes an imaging unit installed so as to image the waste in the waste pit from above, and a three-dimensional waste height that calculates three-dimensional height information of the waste. Based on the installation information of the calculation unit and the imaging unit, the image conversion unit that converts the image captured by the imaging unit into an sky viewpoint image, and the sky viewpoint based on the three-dimensional height information of the garbage. An image correction unit that obtains a corrected image corrected so that all areas of the image are on the same height plane, and a binary value that gradations the corrected image and binarizes it with a predetermined threshold to obtain a binarized image. It has a conversion processing unit and a mixing degree evaluation unit that divides the binarized image into two or more evaluation areas having a plurality of divided areas and evaluates the mixing degree of dust in each evaluation area.

Japanese Patent No. 6457137 (Japanese Unexamined Patent Publication No. 2019-148409)

By the way, the garbage mixture evaluation system disclosed in Patent Document 1 is in a state of stirring by a binarized image obtained by binarizing a dust correction image with gradation, that is, simply by the presence or absence of a broken bag of the garbage bag. (For example, paragraph [0036] of Patent Document 1). Therefore, in the waste mixing degree evaluation system disclosed in Patent Document 1, there is a possibility that the evaluation accuracy of the agitated state may be lowered.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a waste stirring state evaluation device and a waste stirring state evaluation method capable of more appropriately evaluating the stirring state.

As a result of various studies, the present inventor has found that the above object can be achieved by the following invention. That is, the waste agitation state evaluation device according to one aspect of the present invention has an acquisition unit that acquires an image of the waste contained in the storage unit, and occupies the waste for each type of waste in the image. An evaluation unit that evaluates the stirring state of the waste contained in the storage unit based on the identification extraction unit that identifies and extracts the region and each occupied area for each type of the waste extracted by the identification extraction unit. And.

Since such a waste agitation state evaluation device evaluates the agitation state of the waste based on each occupied area for each type of waste, the agitation state can be evaluated more appropriately. In particular, the waste stirring state evaluation device can more appropriately evaluate the stirring state as compared with an algorithm (direct recognition algorithm) that directly recognizes the stirring region in which the waste is agitated from the image. That is, since there are many types of waste contained in the storage unit, the images of the agitated region in which these are agitated have diversity. Therefore, in the direct recognition algorithm, it is difficult to comprehensively recognize the stirring region having such diversity in one image processing procedure. On the other hand, the waste agitation state evaluation device first identifies the waste that is easier to identify than the identification of the agitation region, and then evaluates the agitation state of the waste, so that the agitation state can be evaluated more appropriately.

In another aspect, in the above-mentioned waste stirring state evaluation device, the evaluation unit obtains a stirred stirring region based on each occupied area for each type of waste, and the size of the obtained stirring region is large. Based on this, the agitated state of the waste is evaluated. Preferably, in the above-mentioned waste agitation state evaluation device, the evaluation unit obtains the total area of each occupied area for each type of waste, and subtracts the obtained total area from the area of the storage area of the storage unit. The subtraction result is taken as the size of the stirring region. Preferably, in the above-mentioned waste agitation state evaluation device, the evaluation unit obtains the total area of each occupied area for each type of waste, and divides the obtained total area by the area of the storage area of the storage unit. By doing so, the occupancy ratio of the waste is obtained, and the subtraction result obtained by subtracting the obtained occupancy ratio from 1 is taken as the size of the stirring region represented by the ratio. Preferably, in the above-mentioned waste agitation state evaluation device, the evaluation unit evaluates the agitation state of the waste by comparing the size of the obtained agitation region with a predetermined threshold value.

According to this, it is possible to provide a waste agitation state evaluation device that obtains a agitated agitated area based on each occupied area for each type of waste and evaluates the agitation state of the waste.

In another aspect, in the above-mentioned waste agitation state evaluation device, the evaluation unit satisfies the above-mentioned predetermined condition as a result of comparing each size of each occupied area for each type of waste with a predetermined condition. The agitated state of the waste is evaluated based on the size of the occupied area in the type of waste. Preferably, in the above-mentioned waste agitation state evaluation device, the predetermined condition is that the type of waste having an occupancy ratio equal to or higher than the predetermined first condition value is equal to or higher than the predetermined second condition value. Preferably, in the above-mentioned waste agitation state evaluation device, the evaluation unit compares the maximum occupancy ratio of the types of waste satisfying the predetermined condition with a predetermined threshold value. Evaluate the agitation of waste.

According to this, it is possible to provide a waste stirring state evaluation device that evaluates the stirring state of waste based on the size of the occupied area in the type of waste satisfying a predetermined condition.

In another aspect, in these above-mentioned waste agitation state evaluation devices, the identification extraction unit includes a machine learning model that identifies an occupied area of the waste for each type of the waste in the image. Preferably, in the above-mentioned waste agitation state evaluation device, the machine learning model further outputs a reliability indicating the degree of reliability of identification for the identified occupied area, and the identification extraction unit further outputs the reliability indicating the degree of reliability of identification, and the identification extraction unit performs the identification. A determination unit for determining whether or not the occupied area is waste based on the reliability output from the machine learning model is further included.

Since such a waste agitation state evaluation device uses a machine learning model for the identification / extraction unit, it is necessary to construct in advance an information processing procedure for extracting an occupied area from an image, such as gradation or binarization. There is no.

In another aspect, in these above-mentioned waste agitation state evaluation devices, the identification extraction unit includes a machine learning model for identifying an occupied area of the waste for each type of the waste in the image. A learning information storage unit that stores the learning data of the above, and a learning unit that machine-learns the machine learning model using a plurality of learning data stored in the learning information storage unit, and the learning data is stored in the storage. It includes an image of the waste housed in the unit and a type of the waste as teacher data associated with each pixel of the image.

Since such a waste agitation state evaluation device includes a learning information storage unit and a learning unit, the identification / extraction unit can be relearned and updated.

The waste agitation state evaluation method according to one aspect of the present invention includes an acquisition step of acquiring an image of the waste contained in the storage unit, and in the image, the occupied area of the waste is determined for each type of waste. An identification extraction step of identifying and extracting, and an evaluation step of evaluating the stirring state of the waste contained in the storage portion based on each occupied area for each type of waste extracted in the identification extraction step. Be prepared.

In such a waste agitation state evaluation method, the agitation state of the waste is evaluated based on each occupied area for each type of waste, so that the agitation state can be evaluated more appropriately. In particular, as compared with the direct recognition algorithm, the waste stirring state evaluation method can more appropriately evaluate the stirring state.

The waste agitation state evaluation device and the waste agitation state evaluation method according to the present invention can more appropriately evaluate the agitation state.

It is a block diagram which shows the structure of the waste agitation state evaluation apparatus in embodiment. As an example, it is a schematic view which viewed the receiving pit from the side. It is a figure for demonstrating the learning data used for the machine learning of the machine learning model in the identification extraction part. It is a figure for demonstrating the occupied area and the evaluation method used in the said waste agitation state evaluation apparatus. It is a figure for demonstrating the relationship between the ratio of a stirring region and the quality of a stirring state used in the waste stirring state evaluation apparatus. It is a flowchart which shows the operation of the said waste stirring state evaluation apparatus concerning the evaluation of a stirring state. As an example, it is a figure which shows the evaluation result of the stirring state. It is a flowchart which shows the operation of the said waste agitation state evaluation apparatus concerning the machine learning of the identification extraction part.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. It should be noted that the configurations with the same reference numerals in the respective drawings indicate that they are the same configurations, and the description thereof will be omitted as appropriate. In the present specification, when generically referred to, a reference code with a subscript omitted is used, and when referring to an individual configuration, a reference code with a subscript is used.

The waste agitation state evaluation device in the present embodiment identifies the acquisition unit that acquires an image of the waste contained in the storage unit and the occupied area of the waste for each type of the waste in the image. It is provided with an identification extraction unit for extraction and an evaluation unit for evaluating the stirring state of the waste contained in the storage unit based on each occupied area for each type of waste extracted by the identification extraction unit. Hereinafter, a more specific description will be given.

FIG. 1 is a block diagram showing a configuration of a waste agitation state evaluation device according to an embodiment. FIG. 2 is a schematic view of the receiving pit as a side view as an example. FIG. 3 is a diagram for explaining learning data used for machine learning of the machine learning model in the identification extraction unit. FIG. 3A shows an example of an image obtained by capturing an image of the receiving pit, and FIG. 3B shows an enlarged image of the learning data obtained by dividing the receiving pit image area of the image shown in FIG. 3A into six so as to partially overlap. FIG. 4 is a diagram for explaining an occupied area and an evaluation method used in the waste agitation state evaluation device. FIG. 4A shows each occupied area for each type of waste, and FIG. 4B shows a stirring area excluding each occupied area for each type of waste shown in FIG. 4A. FIG. 5 is a diagram for explaining the relationship between the ratio of the stirring region and the quality of the stirring state used in the waste stirring state evaluation device. The horizontal axis of FIG. 5 indicates the ratio of the stirring region expressed as a percentage, and the vertical axis thereof indicates the degree of quality of the stirring state.

The waste agitation state evaluation device D in the embodiment is used, for example, to evaluate the agitation state of the waste contained in the receiving pit PT for receiving the waste waste shown in FIG. The receiving pit PT is, for example, a substantially rectangular parallelepiped vacant space (recess) provided in a garbage incinerator facility for incinerating waste garbage, whose bottom surface and wall surface are made of concrete and whose top surface is open. The receiving pit PT is provided with a crane CR supported by the crane garter CG and suspended from the crane garter CG. When the XYZ Cartesian coordinate system is set as shown in FIG. 2, the crane CR is configured to be movable in the Z direction (vertical direction on the paper surface) with respect to the crane garter CG, and is guided by the crane garter CG in the Y direction ( The crane gata CG is configured to be movable along the X direction (the left-right direction of the paper surface), and the crane garter CG is configured to be movable along the X direction (the front-back direction of the paper surface) guided by a runway rail (not shown) extending in the X direction. As a result, the crane CR is configured to be movable in each of the three-dimensional directions of the Z direction, the Y direction, and the X direction. An input hopper HP is provided adjacent to the receiving pit PT. The input hopper HP is connected to the incinerator shown in the figure, and when the waste picked up by the crane CR from the receiving pit PT is input to the input hopper HP, the waste is introduced into the incinerator and incinerated. NS. A console (control console) CL for controlling the operation of such a crane CR is arranged in the operation room (operation room) OR. The receiving pit PT corresponds to an example of a storage unit for storing waste.

The waste agitation state evaluation device D of the embodiment for evaluating the agitation state of waste in such a receiving pit PT is, for example, as shown in FIG. 1, an acquisition unit 1, a control processing unit 2, and a storage unit 6. In the example shown in FIG. 1, an input unit 3, an output unit 4, and an interface unit (IF unit) 5 are further provided.

The acquisition unit 1 is a device connected to the control processing unit 2 and acquires an image of the waste contained in the storage unit. As shown in FIG. 2, the acquisition unit 1 is arranged above the receiving pit PT so that the entire upper surface of the receiving pit can be imaged, and generates an image (image data) of the receiving pit PT as a pit image. It is an image pickup device. As shown in FIG. 2, the acquisition unit 1 of the imaging device may be arranged at the upper end of the wall surface so that the receiving pit PT is viewed from diagonally above the receiving pit PT, but preferably, the entire upper surface of the receiving pit is covered. The acquisition unit 1 of the image pickup apparatus may be arranged above (immediately above) the center position of the receiving pit so that the image can be easily taken. The image pickup device is a digital camera or the like. The acquisition unit 1 is a communication interface for communication, and may be communicably connected to a communicable digital camera such as a Web camera, which is arranged above the receiving pit PT, via a network.

The input unit 3 is connected to the control processing unit 2 and operates, for example, various commands such as a command for instructing the start of evaluation of the stirring state, and a waste stirring state evaluation device D such as teacher data of learning data. It is a device for inputting various data necessary for making the data into the waste agitation state evaluation device D, for example, a plurality of input switches, a keyboard, a mouse, etc. to which a predetermined function is assigned. The output unit 4 is a device connected to the control processing unit 2 and outputs commands and data input from the input unit 3, a pit image, an evaluation result of a stirring state, and the like according to the control of the control processing unit 2, for example, a CRT. Display devices such as displays, liquid crystal displays and organic EL displays, printing devices such as printers, and the like.

A so-called touch panel may be configured from the input unit 3 and the output unit 4. In the case of configuring this touch panel, the input unit 3 is a position input device that detects and inputs an operation position such as a resistance film method or a capacitance method, and the output unit 4 is a display device. In this touch panel, the position input device is provided on the display surface of the display device, candidates for one or a plurality of input contents that can be input to the display device are displayed, and the user displays the input contents that he / she wants to input. When the position is touched, the position is detected by the position input device, and the display content displayed at the detected position is input to the waste stirring state evaluation device D as the operation input content of the user. With such a touch panel, since the user can intuitively understand the input operation, the waste stirring state evaluation device D that is easy for the user to handle is provided.

The IF unit 5 is a circuit that is connected to the control processing unit 2 and inputs / outputs data to / from an external device according to the control of the control processing unit 2. For example, an interface circuit of RS-232C which is a serial communication method. , An interface circuit using the Bluetooth (registered trademark) standard, an interface circuit for performing infrared communication such as the IrDA (Infrared Data Association) standard, and an interface circuit using the USB (Universal Serial Bus) standard. Further, the IF unit 5 is a circuit that communicates with an external device, and may be, for example, a data communication card, a communication interface circuit according to the IEEE802.11 standard, or the like.

The storage unit 6 is a circuit that is connected to the control processing unit 2 and stores various predetermined programs and various predetermined data under the control of the control processing unit 2. The various predetermined programs include, for example, a control processing program, and the control processing program controls each part 1, 3 to 6 of the waste agitation state evaluation device D according to the function of each part. In the control program and the image acquired by the acquisition unit 1, the identification extraction program that identifies and extracts the occupied area of the waste for each type of waste, and the identification extraction program for each type of waste extracted by the identification extraction program. Based on each occupied area, the occupied area of the waste is identified for each type of waste in the evaluation program for evaluating the stirring state of the waste contained in the storage unit and the image acquired by the acquisition unit 1. A learning program or the like in which a machine learning model is machine-learned using a plurality of learning data stored in a learning information storage unit 61, which will be described later, is included. The various predetermined data include a threshold value for determining whether or not the data is waste (determination threshold value), a threshold value for evaluating the stirring state (evaluation threshold value), learning data, and the like. Contains the data needed to run. The determination threshold value may be one value, but in the present embodiment, a plurality of determination threshold values are prepared in advance for each type of waste according to the type of waste. Such a storage unit 6 includes, for example, a ROM (Read Only Memory) which is a non-volatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) which is a rewritable non-volatile storage element, and the like. The storage unit 6 includes a RAM (Random Access Memory) or the like that serves as a working memory of the so-called control processing unit 2 that stores data or the like generated during the execution of the predetermined program. The storage unit 6 may be provided with a hard disk device capable of storing a large capacity in order to store learning data having a relatively large capacity.

The storage unit 6 functionally includes a learning information storage unit 61 that stores the learning data. The learning data includes an image of the waste housed in the storage unit and a type of the waste as teacher data associated with each pixel of the image. More specifically, as shown in FIG. 3A, there are a plurality of regions (accepting pit image regions) RPs in which the receiving pits are imprinted in the image, and in the example shown in FIG. 3A, the number of RPs partially overlaps with six. As described above, each of the six divided images SP-1 to SP-6 is divided into a predetermined size (for example, 500pixel × 500pixel or the like), and each of the six divided images SP-1 to SP-6 is used as an image of one learning data. Then, in each of the images SP-1 to SP-6 of the training data, for example, in the image SP-3 of the training data shown in FIG. Is specified from the input unit 3 in, and each type of waste in the designated areas TD-1 and TD-2 is input from the input unit 3 as teacher data, and the designated areas TD-1, TD- Each input teacher data is assigned to each pixel in 2. In this way, the types of waste are manually associated with the images SP-1 to SP-6 of the learning data as teacher data for each pixel. Then, the images SP-1 to SP-6 and each teacher data associated with each other are stored in the learning information storage unit 61 as one of the learning data.

The control processing unit 2 is a circuit for evaluating the agitation state of waste by controlling each part 1, 3 to 6 of the waste agitation state evaluation device D according to the function of each part. The control processing unit 2 includes, for example, a CPU (Central Processing Unit) and peripheral circuits thereof. The control processing unit 2 is functionally configured with the control unit 21, the identification / extraction unit 22, the evaluation unit 23, and the learning unit 24 by executing the control processing program.

The control unit 21 controls each unit 1, 3 to 6 of the waste agitation state evaluation device D according to the function of each unit, and controls the entire waste agitation state evaluation device D.

The identification / extraction unit 22 identifies and extracts the occupied area of the waste for each type of waste in the image (pit image) acquired by the acquisition unit 1. In the present embodiment, the identification / extraction unit 22 functionally includes a machine learning model unit 221 and a determination unit 222.

The machine learning model unit 221 includes a machine learning model that identifies the occupied area of the waste for each type of waste in the image acquired by the acquisition unit 1. In the present embodiment, in this machine learning model, when a pit image is input, an image area in which the waste is imprinted is identified as an occupied area of the waste for each type of waste from the pit image, and the position thereof. And the size (size, area, number of pixels) is specified, and the reliability indicating the degree of reliability of identification is obtained as a percentage with respect to the occupied area of the identified waste, and the occupied area (position, position, of these wastes) is obtained. The size), the type (type) of the waste, and its reliability are output. The machine learning model is used for machine learning such as linear regression, Boltzmann machine, neural network, support vector machine (SVM), basian network, decision tree, random forest and the like. It is configured with a model using a known method. In the present embodiment, the machine learning model uses a convolutional neural network (CNN), which is one of the neural networks, and the machine learning determines the parameters of the CNN and acquires its discriminating power. NS.

The determination unit 222 finally determines whether or not the occupied area of the waste identified by the machine learning model of the machine learning model unit 221 is waste based on the reliability output from the machine learning model. Is what you do. More specifically, the determination unit 222 compares the reliability output from the machine learning model with the determination threshold value according to the type of waste output from the machine learning model together with the reliability, and the reliability. When the degree is equal to or more than the determination threshold, it is finally determined to be the occupied area of waste, and when the reliability is less than the determination threshold, it is not finally determined to be the occupied area of waste.

By combining the machine learning model and the threshold value determination in this way, the occupied area of waste can be extracted more accurately. Further, in the present embodiment, since the determination threshold value is prepared for each type of waste, the waste can be identified in consideration of the attributes (for example, ease of identification) according to the type of waste. The occupied area of waste can be extracted accurately.

For example, when the pit image shown in FIG. 4A is processed, such an identification extraction unit 22 identifies the occupied area of the waste for each type of waste, and the occupied area (position, size) of the waste and Output the type of waste. In the example shown in FIG. 4A, vegetation waste and bag waste are identified as waste types, and each occupied area of the plant waste and bag waste is displayed as a rectangle on the pit image. The position and size of the occupied area are displayed by the display position and the size of the rectangle.

The evaluation unit 23 evaluates the stirring state of the waste contained in the storage unit based on each occupied area for each type of waste identified by the identification extraction unit 22. More specifically, the evaluation unit 23 obtains a stirred region that has been agitated based on each occupied area for each type of waste, and based on the size of the obtained agitated region, the agitated state of the waste. To evaluate.

In the present embodiment, the residual area excluding the occupied area of the waste is defined as the stirring area. For example, in the example shown in FIG. 4A, as shown in FIG. 4B, the unhatched residual regions NR-1 and NR-2 excluding the occupied region HR of the hatched waste are designated as stirring regions. More specifically, the evaluation unit 23 obtains the total area of each occupied area for each type of waste, and divides the calculated total area by the area of the storage area of the storage unit to obtain the occupancy ratio of the waste. The value obtained by subtracting the obtained occupancy ratio from 1 is taken as the size of the stirring region represented by the ratio. For example, the area of the receiving pit PT (total number of images) is S (pit), the total area of vegetation waste (total number of pixels) is S (vegetation), and the total area of bag waste (total number of pixels) is S (bag). , The size S (stirring) [%] of the stirring region represented by the ratio is represented by the following equation 1.
Equation 1; S (stirring) = (1- (S (vegetation) + S (bag)) / S (pit))
Here, when the rectangles representing the occupied areas are partially overlapped, the total areas S (plants) and S (bags) can be obtained from only one of the overlapped areas. The evaluation unit 23 obtains the total area of each occupied area for each type of waste, and the subtraction result obtained by subtracting the calculated total area from the area of the storage area of the storage unit may be used as the size of the stirring area. Good (S (stirring) = S (pit)-(S (vegetation) + S (bag)).

Then, the evaluation unit 23 evaluates the stirring state of the waste by comparing the size of the obtained stirring region with a predetermined threshold value (evaluation threshold value).

The evaluation threshold value may be one for evaluating the quality of stirring, but in the present embodiment, there are a plurality of evaluation threshold values in order to evaluate the stirring state in a plurality of stages (levels). For example, as shown in FIG. 5, in order to evaluate in four stages of defective (x), normal (Δ), good (◯), and best (⊚), the evaluation thresholds are three first to third. The evaluation thresholds Th1 to Th3 are provided (Th1 <Th2 <Th3 [%]). The defect (x) is a stirring state in which stirring is insufficient, and when the size of the stirring region represented by the ratio is equal to or less than the first evaluation threshold Th1, the evaluation unit 23 describes the defect (x). evaluate. The good (◯) is a stirring state in which the stirring is performed to an extent suitable for the purpose of stirring, and the size of the stirring region represented by the ratio exceeds the second evaluation threshold Th2 and is equal to or less than the third evaluation threshold Th3. , The evaluation unit 23 evaluates this as good (◯). The best (⊚) is a sufficiently stirred stirring state, and when the size of the stirring region expressed by the ratio exceeds the third evaluation threshold Th3, the evaluation unit 23 evaluates it as the best (⊚). do. The normal (Δ) is a stirring state in which stirring is performed but not until the normal level is reached, and it is preferable to stir if possible, and the size of the stirring region represented by the ratio is When the first evaluation threshold Th1 is exceeded and the second evaluation threshold Th2 or less is equal to or lower than the second evaluation threshold Th2, the evaluation unit 23 evaluates this as normal (Δ) (poor <normal <good <best). These three first to third evaluation thresholds Th1 to Th3 are appropriately set with reference to, for example, the experience of an operator who operates the crane CR, and the first evaluation threshold Th1 for discriminating between the defect and the normal. Is set to, for example, 25 [%] or 30 [%], and the second evaluation threshold Th2 for discriminating between the normal and the good is set to, for example, 50 [%], and the good and the best are set. The third evaluation threshold Th3 for discriminating from the above is set to, for example, 70 [%], 75 [%], 80 [%], or the like.

The learning unit 24 machine-learns the machine learning model of the identification / extraction unit 22 using a plurality of learning data stored in the learning information storage unit 61, and the machine learning of the identification / extraction unit 22 using the machine-learned machine learning model. It updates the model. In the present embodiment, the learning unit 24 receives input of teacher data from the input unit 3 with respect to the image of the learning data, associates the image with the teacher data with each other, and uses the learning information storage unit as one of the learning data. Store in 61.

The control processing unit 2, the input unit 3, the output unit 4, the IF unit 5, and the storage unit 6 can be configured by, for example, a computer such as a desktop type or a notebook type. The computer constituting each of these parts 2 to 6 may be arranged in the operation room OR and incorporated in the console CL (may also be used as the console CL), or may be separate from the console CL.

Next, the operation of this embodiment will be described. FIG. 6 is a flowchart showing the operation of the waste stirring state evaluation device regarding the evaluation of the stirring state. FIG. 7 is a diagram showing the evaluation result of the agitated state as an example. FIG. 8 is a flowchart showing the operation of the waste agitation state evaluation device regarding machine learning of the identification / extraction unit.

When the power is turned on, the waste agitation state evaluation device D having such a configuration executes necessary initialization of each part and starts its operation. The control processing unit 2 is functionally configured with the control unit 21, the identification extraction unit 22, the evaluation unit 23, and the learning unit 24 by executing the control processing program, and the identification extraction unit 22 includes the machine learning model unit 221. And the determination unit 222 is functionally configured.

Hereinafter, firstly, the operation of the waste stirring state evaluation device D relating to the evaluation of the stirring state will be described, and then secondly, the operation of the waste stirring state evaluation device D relating to the machine learning of the identification extraction unit will be described. explain. In the explanation of the operation of the waste stirring state evaluation device D regarding the evaluation of the stirring state, the machine learning model of the identification extraction unit 22 has been machine-learned in advance and stored in the storage unit 6, and the storage unit 6 to the control processing unit 2 It is assumed that the machine learning model unit 221 of the identification extraction unit 22 provided with the machine learning model that has been machine-learned is functionally configured by being read into.

First, in the operation of the waste agitation state evaluation device D relating to the evaluation of the agitation state, in FIG. 6, the waste agitation state evaluation device D first acquires a pit image by the acquisition unit 1, and the acquired pit image is obtained. Is stored in the storage unit 6 by the control processing unit 2 (S11). This pit image is also used for learning data as described later.

Next, the waste agitation state evaluation device D identifies the occupied area of the waste for each type of waste in the pit image acquired by the acquisition unit 1 in the processing S11 by the identification extraction unit 22 of the control processing unit 2. And extract (S12). In the present embodiment, the machine learning model unit 221 identifies the image area on which the waste is imprinted as the occupied area of the waste for each type of waste from the pit image, and the occupied area (position) of the waste. , Size), type and reliability are output. The determination unit 222 finally determines whether or not the occupied area of the waste identified by the machine learning model of the machine learning model unit 221 is waste based on the reliability output from the machine learning model. Then, the occupied area of the finally determined waste is output. Then, the identification / extraction unit 22 notifies the evaluation unit 23 of the occupied area (position, size) of the waste finally determined by the determination unit 222 and the type of the waste.

Next, the waste stirring state evaluation device D obtains a stirring region by the evaluation unit 23 of the control processing unit 2 (S14). In the present embodiment, the evaluation unit 23 obtains the size of the stirring region represented by the ratio by the above formula 1.

Next, the waste agitation state evaluation device D evaluates the agitation state of the waste by the evaluation unit 23 of the control processing unit 2 based on the size of the agitation region obtained in the treatment S14 (S15). In the present embodiment, the evaluation unit 23 evaluates the stirring state of the waste by comparing the size of the obtained stirring region with the first to third evaluation thresholds Th1 to Th3.

Then, the waste agitation state evaluation device D outputs the evaluation result by the control unit 21 of the control processing unit 2, and ends this processing. For example, as shown in FIG. 7, the pit image RP and the stirring state RM are displayed on the output unit 4, and the stirring region SP obtained by the evaluation unit 23 is displayed on the pit image RP with a frame line.

When the agitation state is evaluated, the crane CR is operated and agitation is performed as needed. For example, the waste in the region not determined to be the stirring region is picked up by the crane CR, and the waste is agitated by being dropped from the crane CR at that position or at another position.

The operation of the waste agitation state evaluation device D related to the evaluation of the agitation state is executed at an appropriate timing. For example, the evaluation operation is executed for each series of agitation operations by the crane CR such as traveling, unwinding, gripping, hoisting, traveling, and gripping cancellation, or for each a plurality of stirring operations.

Secondly, in the operation of the waste agitation state evaluation device D related to machine learning of the identification and extraction unit, in FIG. 8, first, the waste agitation state evaluation device D uses the learning unit 24 of the control processing unit 2 to obtain learning data. In order to generate the pit image to which the teacher data is not added, which was acquired during the evaluation operation, is acquired from the storage unit 6 (S21).

Next, the waste agitation state evaluation device D outputs one of the acquired pit images to the output unit 4 by the learning unit 24, and accepts the input of the teacher data (S22). As described above, in the pit image output to the output unit 4, the user specifies the image portion on which the waste is imprinted from the input unit 3 in the area, and each type of waste in the designated area is trained data. Is input from the input unit 3.

Next, the waste stirring state evaluation device D associates the teacher data received by the input unit 3 with each pixel in the region designated by the input unit 3 by the learning unit 24, and pits associated with each other. Each of the image and the teacher data is stored in the learning information storage unit 61 as one of the learning data (S23). As a result, the learning data is updated (added) and stored in the learning information storage unit 61.

Next, the waste agitation state evaluation device D determines whether or not the teacher data has been added to all the pit images acquired in the process S21 by the learning unit 24 (S24). As a result of this determination, if there is an unassigned pit image (No), the learning unit 24 returns the process to S22. On the other hand, as a result of the determination, when the teacher data is added to all the pit images (Yes), the learning unit 24 is next. S25 is executed for the process.

In this process S25, the waste agitation state evaluation device D determines whether or not machine learning has started by the learning unit 24. As a result of this determination, if it is not the start of machine learning (No), the learning unit 24 ends this process. On the other hand, if the result of the determination is that machine learning has started (Yes), the learning unit 24 then sequentially executes the process S26 and the process S27 to end the process. For example, when the user receives an instruction to start machine learning from the input unit 3, the learning unit 24 determines that the machine learning has started. Further, for example, when a predetermined period such as 3 months or 6 months has passed since the previous machine learning, the learning unit 24 determines that the machine learning has started.

In this process S26, the waste agitation state evaluation device D machine-learns the machine learning model of the identification / extraction unit 22 by the learning unit 24 using a plurality of learning data stored in the learning information storage unit 61.

Then, in the process S27 following the process S26, the waste agitation state evaluation device D updates the machine learning model of the identification extraction unit 22 with the machine learning model machine-learned in the process S26 by the learning unit 24.

The pit image used for the evaluation of the stirring state is added to the training data in this way, and the machine learning model of the identification / extraction unit 22 is machine-learned and updated using this training data.

As described above, the waste agitation state evaluation device D and the waste agitation state evaluation method mounted on the waste agitation state evaluation device D in the present embodiment determine the agitation state of waste based on each occupied area for each type of waste. Since it is evaluated, the stirring state can be evaluated more appropriately. Since there are many types of waste contained in the receiving pit PT, the images of the agitated region in which these are agitated have a variety. Therefore, it is difficult to comprehensively recognize the stirring region having such diversity in one image processing procedure by the direct recognition algorithm that directly recognizes the stirring region from the image. On the other hand, the waste agitation state evaluation device D and the waste agitation state evaluation method first identify the waste that is easier to identify than the identification of the agitation region, and then evaluate the agitation state of the waste. Compared with the direct recognition algorithm, the stirring state can be evaluated more appropriately.

Since the waste agitation state evaluation device D and the waste agitation state evaluation method use a machine learning model for the identification extraction unit 22, an information processing procedure for extracting an occupied area from an image, such as gradation or binarization, is performed. There is no need to build in advance.

Since the waste agitation state evaluation device D and the waste agitation state evaluation method include the learning information storage unit 61 and the learning unit 24, the identification / extraction unit 22 can be relearned and updated.

Then, according to the present embodiment, the waste agitation state evaluation device D and the waste agitation state for evaluating the agitation state of the waste by obtaining the agitated agitation area based on each occupied area for each type of waste. An evaluation method can be provided.

In the above-described embodiment, the evaluation unit 23 obtains a stirred region that has been stirred based on each occupied area for each type of waste, and the waste is based on the size of the obtained stirring area. Although the agitation state has been evaluated, the evaluation method for evaluating the agitation state of the waste based on each occupied area for each type of waste is not limited to this, and other methods may be used. For example, the evaluation unit 23 compares the size of each occupied area for each type of waste with a predetermined condition, and as a result, based on the size of the occupied area in the type of waste satisfying the predetermined condition, the said The agitation state of the waste may be evaluated. More specifically, the predetermined condition is that the type of waste having an occupancy ratio equal to or higher than the predetermined first condition value is equal to or higher than the predetermined second condition value, and the evaluation unit 23 determines the predetermined condition. The agitated state of the waste is evaluated by comparing the maximum occupancy ratio of the types of waste satisfying the conditions with a predetermined threshold value (evaluation threshold value). According to this, it is possible to provide the waste agitation state evaluation device D and the waste agitation state evaluation method for evaluating the agitation state of waste based on the size of the occupied area in the type of waste satisfying a predetermined condition.

In one example, the predetermined first condition value CV1 is 20 [%] or the like, and the predetermined second condition value CV2 is 3 mag. That is, the predetermined condition is that there are three or more types of waste having an occupancy ratio of 20% or more. The predetermined threshold value (evaluation threshold value) includes two fourth and fifth evaluation threshold values Th4 and Th5 (Th4 <Th5 [%]). The fourth evaluation threshold Th4 is set to, for example, 33 [%], and the fifth evaluation threshold th2 is set to, for example, 48 [%]. The first and second condition values CV1, CV2, and the fourth and fifth evaluation thresholds Th4 and Th5 are appropriately set with reference to, for example, the experience of an operator who operates the crane CR. As a result of the comparison, if the predetermined condition is not satisfied, the evaluation unit 23 evaluates as the defective (x), and if the predetermined condition is satisfied, the evaluation unit 23 evaluates the best (⊚) and the good. It is evaluated as either (◯) or the above-mentioned normal (Δ), and further, it is determined as follows. When the maximum occupancy ratio is equal to or less than the fourth threshold value Th4, the evaluation unit 23 evaluates as the best (⊚). When the maximum occupancy ratio exceeds the fourth threshold value Th4 and is equal to or less than the fifth threshold value Th5, the evaluation unit 23 evaluates it as good (◯). When the maximum occupancy ratio exceeds the fifth threshold value Th5, the evaluation unit 23 evaluates as the normal (Δ).

For example, as a result of the processing of the identification extraction unit 22, the occupancy ratio of vegetation waste is 40 [%], the occupancy ratio of sludge waste is 25 [%], and the occupancy ratio of white bag waste is 25 [%]. When the occupancy ratio of black bag waste is 10 [%], there are three or more types of waste having an occupancy ratio of 20% or more: vegetation waste, sludge waste, and white bag waste. , It is determined that the above-mentioned predetermined conditions are satisfied. Subsequently, the occupancy ratio of 40 [%] of the vegetation waste having the largest occupancy ratio among these vegetation waste, sludge waste and white bag waste is compared with the 4th and 5th evaluation thresholds Th4 and Th5, and the occupancy ratio is the maximum. Since the ratio 40 [%] exceeds the fourth threshold value Th4 and is equal to or less than the fifth threshold value Th5, the evaluation unit 23 evaluates it as good (◯). Further, for example, as a result of the processing of the identification extraction unit 22, the occupancy ratio of plant waste is 50 [%], the occupancy ratio of sludge waste is 15 [%], and the occupancy ratio of white bag waste is 15 [%]. Yes, when the occupancy ratio of black bag waste is 20 [%], there are two types of waste having an occupancy ratio of 20% or more, plant waste and black bag waste. It is determined that the condition of the above is not satisfied, and it is evaluated as the defect (x).

In order to express the present invention, the present invention has been appropriately and sufficiently described through the embodiments with reference to the drawings described above, but those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that it can be done. Therefore, unless the modified or improved form implemented by a person skilled in the art is at a level that deviates from the scope of rights of the claims stated in the claims, the modified form or the improved form is the scope of rights of the claims. It is interpreted as being comprehensively included in.

D Waste agitation state evaluation device 1 Acquisition unit 2 Control processing unit 3 Input unit 4 Output unit 6 Storage unit 21 Control unit 22 Identification and extraction unit 23 Evaluation unit 24 Learning unit 61 Learning information storage unit 221 Machine learning model unit 222 Judgment unit

Claims (6)

The acquisition unit that acquires the image of the waste contained in the storage unit,
In the image, an identification extraction unit that identifies and extracts the occupied area of the waste for each type of waste,
It is provided with an evaluation unit for evaluating the agitation state of the waste contained in the storage unit based on each occupied area for each type of waste extracted by the identification extraction unit.
Waste agitation condition evaluation device. The evaluation unit obtains a stirred region that has been agitated based on each occupied area for each type of waste, and evaluates the agitated state of the waste based on the size of the obtained agitated region.
The waste agitation state evaluation device according to claim 1. As a result of comparing each size of each occupied area for each type of waste with a predetermined condition, the evaluation unit performs the disposal based on the size of the occupied area in the type of waste satisfying the predetermined condition. Evaluate the agitation state of an object,
The waste agitation state evaluation device according to claim 1. The identification extraction unit includes a machine learning model that identifies an occupied area of the waste for each type of the waste in the image.
The waste agitation state evaluation device according to any one of claims 1 to 3. The identification extraction unit includes a machine learning model that identifies an occupied area of the waste for each type of the waste in the image.
A learning information storage unit that stores multiple learning data,
A learning unit that machine-learns the machine learning model using a plurality of learning data stored in the learning information storage unit is further provided.
The learning data includes an image of the waste housed in the storage unit and a type of the waste as teacher data associated with each pixel of the image.
The waste agitation state evaluation device according to any one of claims 1 to 3. The acquisition process to acquire an image of the waste contained in the containment unit,
In the image, an identification extraction step of identifying and extracting the occupied area of the waste for each type of the waste, and
The present invention includes an evaluation step of evaluating the agitation state of the waste contained in the storage portion based on each occupied area for each type of the waste extracted in the identification extraction step.
Waste agitation state evaluation method.

Images