JP2023048385A - Information processing apparatus, determination method, determination program, and control system - Google Patents

Abstract

To detect a target waste by a general purpose method.SOLUTION: An information processing apparatus (1) includes a candidate region extraction unit (103) that extract a region including pixels in a predetermined color scope corresponding to a target waste in a pit image (202) obtained by photographing the inside of a pit that accommodates the waste as a candidate region, and a determination unit (104) that determines whether the waste photographed in the candidate region is the target waste on the basis of a feature amount generated on the basis of a color of each of the pixels constituting the candidate region.SELECTED DRAWING: Figure 1

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus or the like that detects waste to be detected from an image of a pit containing multiple types of waste.

Garbage carried into the garbage incineration facility is stored in a facility called a garbage pit, sent from the garbage pit to an incinerator and incinerated. At this time, it is desirable that the quality of the waste sent into the incinerator, in other words, the more uniform the combustibility of the waste, the more stable the combustion in the incinerator.

For this reason, research and development of technology for homogenizing the quality of waste to be sent to the incinerator have been in progress for some time. For example, Patent Literature 1 below discloses a technique for identifying the type of waste contained in a waste pit using an identification algorithm built by performing machine learning using an image of the inside of the waste pit. is disclosed. By identifying the type of waste at various locations within the waste pit, it is possible to detect that the same type of waste is agglomerated in some areas within the waste pit and to agitate and homogenize it. .

Japanese Patent Application Laid-Open No. 2020-38058

The technique of Patent Literature 1 employs a configuration in which all objects appearing in an image of a dust pit taken from above are learned and identified. However, the appearance of the floor of the waste treatment plant, the sidewalls of the waste pit, the cranes, etc. may differ from one waste treatment plant to another. In addition, the arrangement of cameras that capture images of the inside of the garbage pit may also differ from one waste treatment plant to another. If the arrangement of the cameras differs, the appearance of the wastes and the like in the captured images will also differ. Thus, an identification algorithm for one waste treatment plant may be less accurate or unusable for another waste treatment plant.

An object of one aspect of the present invention is to provide an information processing apparatus or the like capable of detecting target waste by a general-purpose method as compared with the conventional technology as described above.

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention provides a pit image obtained by imaging the inside of a pit containing waste, in a predetermined color range according to the target waste to be detected. A candidate area extracting unit for extracting an area made up of pixels as a candidate area, and based on a feature amount generated based on the color of each pixel constituting the candidate area, the waste appearing in the candidate area is the target waste. and a determination unit that determines whether there is

In order to solve the above-described problems, an information processing apparatus according to an aspect of the present invention sets a waste area in which the waste is captured in a pit image obtained by imaging the inside of the pit that stores the waste as a determination target. a partial determination unit that determines, for each of a plurality of partial areas within the target area, whether the waste reflected in the partial area is the target waste to be detected; and an integrated judgment unit for judging whether the waste reflected in the area is the target waste.

In order to solve the above problems, a determination method according to an aspect of the present invention is a determination method executed by one or more information processing apparatuses, in which a pit image captured inside a pit containing waste includes: a candidate area extracting step of extracting an area composed of pixels in a predetermined color range corresponding to the target waste to be detected as a candidate area; a determination step of determining whether the waste imaged in the candidate area is the target waste based on the above.

In order to solve the above problems, a determination method according to an aspect of the present invention is a determination method executed by one or more information processing apparatuses, wherein the pit image obtained by imaging the inside of the pit containing the waste is the above-mentioned A partial determination step of determining, for each of a plurality of partial areas within a target area set as a determination target within the waste area in which the waste is captured, whether the waste reflected in the partial area is the target waste to be detected. and an integrated judgment step of judging whether the waste imaged in the target area is the target waste based on the judgment results in each of the partial areas.

In order to solve the above problems, a control system according to an aspect of the present invention provides pixels in a predetermined color range corresponding to target waste to be detected in a pit image captured inside a pit containing waste. Information for determining whether the waste reflected in the candidate area is the target waste based on the feature amount generated based on the color of each pixel that constitutes the candidate area while extracting the area consisting of a processing device; and a control device that controls the operation of the crane placed in the pit based on the determination result of the information processing device.

ADVANTAGE OF THE INVENTION According to one aspect of the present invention, it becomes possible to detect target waste by a general-purpose method compared to the conventional technology.

1 is a block diagram showing an example of the main configuration of an information processing apparatus according to an embodiment of the present invention; FIG. It is a figure which shows the structural example of the control system containing the said information processing apparatus. FIG. 10 is a diagram showing a method of deriving a reference point that serves as a reference for extracting a waste area; FIG. 10 is a diagram showing an example of extracting a waste area; FIG. 10 is a diagram showing an example of noise removal when extracting a candidate region; FIG. 10 is a diagram showing a setting example of a target area; FIG. 10 is a diagram showing a setting example of partial areas in a target area; It is a figure which shows the integration example of the determination result of a determination part, and the determination result of an integrated determination part. It is a flow chart which shows an example of processing which the above-mentioned information processor performs. FIG. 11 is a flowchart showing an example of target waste detection processing performed for each candidate area; FIG. 10 is a flow chart showing an example of target waste detection processing in a target area set as a determination target.

〔System configuration〕
A configuration of a control system 5 according to one embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the control system 5. As shown in FIG. As shown, the control system 5 includes an information processor 1 , a controller 2 and a crane 3 . FIG. 2 also shows a cross-section of a pit P containing waste.

The crane 3 is a machine arranged in the pit P and used for transporting the waste in the pit P, for example. Components of crane 3 include girder 31 , traversing truck 32 , wire 33 and bucket 34 . The girder 31 is arranged so as to bridge between rails (extending in the depth direction of the figure) respectively provided on the opposite wall surfaces of the building (not shown) of the waste treatment plant, and along the rails can be moved in the depth direction in the figure. Moreover, the traversing truck 32 is provided on the girder 31, and can be moved on the girder 31 in the left-right direction (the direction perpendicular to the moving direction of the girder 31) in the figure. A wire 33 is attached to the traversing carriage 32, and a bucket 34 is provided at the tip of the wire 33 for catching waste. The bucket 34 can be opened and closed.

The control device 2 controls the operation of the crane 3 . For example, the control device 2 drives the girder 31 and the trolley 32 to move the bucket 34 to a desired position, opens the bucket 34 at that position, lowers it onto the waste in the pit P, and closes the bucket 34. It controls the winding up of the wire 33 . Through these series of controls, the waste at that position can be picked up by the bucket 34 .

The information processing apparatus 1 determines whether the waste contained in the pit P is the target waste to be detected. More specifically, the information processing apparatus 1 extracts, as a candidate area, an area composed of pixels in a predetermined color range corresponding to the target waste to be detected from a pit image captured by an imaging device (not shown). do. Then, the information processing apparatus 1 determines whether the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel forming the candidate area. The control device 2 described above controls the operation of the crane 3 based on this determination result.

As described above, the control system 5 extracts, as a candidate area, an area composed of pixels in a predetermined color range corresponding to the target waste to be detected in the pit image captured inside the pit P containing the waste. together with the information processing device 1 for determining whether the waste reflected in the candidate region is the target waste based on the feature amount generated based on the color of each pixel constituting the candidate region; and the determination result of the information processing device 1. a control device 2 for controlling the operation of the crane 3 located in the pit P based on.

Although the details will be described later, the information processing apparatus 1 can detect the target waste regardless of the target waste treatment plant. Therefore, according to the control system 5, it becomes possible to detect target waste in a more versatile manner than in the prior art.

Furthermore, in the control system 5, the control device 2 controls the operation of the crane 3 based on the result of determining whether the waste reflected in the candidate area is the target waste. Appropriate automatic control of the crane 3 according to the location of the can be realized. For example, by automatically mixing waste that has been determined to be target waste, and avoiding waste that has been determined to be target waste when deciding which waste to put into the hopper connected to the incinerator. etc. becomes possible.

〔Device configuration〕
The configuration of the information processing device 1 will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the main configuration of an information processing apparatus 1. As shown in FIG. As shown in the figure, the information processing apparatus 1 includes a control section 10 that centrally controls each section of the information processing apparatus 1 and a storage section 20 that stores various data used by the information processing apparatus 1 . The information processing apparatus 1 also includes a communication unit 12 for communicating between the information processing apparatus 1 and other apparatuses, an input unit 13 for receiving input of various data to the information processing apparatus 1, and various data output by the information processing apparatus 1. An output unit 14 is provided for doing so.

The control unit 10 includes a waste area extraction unit 101, an image conversion unit 102, a candidate area extraction unit 103, a determination unit 104, a position calculation unit 105, a target area extraction unit 106, a partial area setting unit 107, a partial determination unit. 108, an integrated determination unit 109, a determination result integration unit 110, and a crane control unit 111 are included. The storage unit 20 stores height information 201, a pit image 202, and trained models 203 and 204. FIG.

The waste area extracting unit 101 extracts a waste area in which waste appears from the pit image 202 . Although the details will be described later, the waste region extraction unit 101 extracts correspondence information indicating the correspondence relationship between the piled-up height of waste at various locations in the pit and the position at which the surface of the waste at various locations in the pit appears in the pit image 202, 201 is used to extract the waste area.

The height information 201 is information indicating the piled-up height of the waste at various locations in the pit. The method of determining the piled-up height of the waste is arbitrary. For example, in the example of FIG. 2, it may be obtained from the length of the wire 33 when the bucket 34 of the crane 3 is lowered at various places in the pit P, or it may be obtained by image analysis of the pit image 202, It may be determined using a range sensor or the like. When the piled-up height of waste at various locations in the pit P changes, it is desirable to update the height information 201 accordingly.

The pit image 202 is an image of the inside of the pit that contains the waste. The pit image only needs to include at least the region in the pit where the target waste is to be detected. Since the information processing apparatus 1 performs detection based on the color of the target waste, the pit image 202 is a color image.

The image converting unit 102 converts the image of the waste area extracted by the waste area extracting unit 101 into a predetermined format in which the external features (particularly, color-related features) of the target waste appear. For example, assume that waste region features appear more in HSV (Hue, Saturation, Value) images than in RGB (Red, Green, Blue) images. In this case, if the pit image 202 is an RGB image, the image conversion unit 102 converts this RGB image into an HSV image. Note that the image conversion unit 102 may be omitted if it is possible to extract a candidate area or detect a target waste without converting the pit image 202 .

The candidate region extracting unit 103 extracts a region consisting of pixels in a predetermined color range according to the target waste to be detected in the pit image 202 obtained by imaging the inside of the pit containing the waste as a region where the target waste exists. is extracted as a candidate, that is, a candidate region. Extraction of the candidate area is performed on the pit image 202 after conversion by the image conversion unit 102 . As described above, if it is possible to extract a candidate area or detect a target waste without converting the pit image 202, the image conversion unit 102 may be omitted. The extraction unit 103 extracts a candidate area from the unconverted pit image 202 .

The determination unit 104 determines whether the waste imaged in the candidate area is the target waste based on the feature amount generated based on the color of each pixel forming the candidate area extracted by the candidate area extraction unit 103 .

For example, the determination unit 104 inputs the feature amount generated based on the color of each pixel constituting the candidate area to the trained model 203 that has learned the feature amount generated based on the color of the target waste. It may be determined whether the waste imaged in the candidate area is the target waste based on the output value obtained by the process.

According to the above configuration, it is possible to determine with high accuracy whether the waste imaged in the candidate area is the target waste. In addition, learning of the side walls of the pits, etc., as in Patent Document 1 is unnecessary, and it is sufficient to learn at least the color of the target waste.

The trained model 203 is, as described above, a trained model that has learned the feature amount generated based on the color of the target waste. The trained model 203 has learned the correspondence relationship between the feature quantity generated based on the color of each pixel that constitutes a certain area in the image and whether or not the target waste is shown in that area. may Also, the trained model 203 may be trained so as to classify objects appearing in a certain region from feature values generated based on the color of each pixel that constitutes the region in the image. In the latter case, the target waste should be included in one of the categories. The algorithm of the trained model 203 is not particularly limited. For example, a random forest trained model 203 may be used. Note that the determination unit 104 may determine whether the waste imaged in the candidate area is the target waste from the feature amount generated based on the color of the pixel. is not limited to

The position calculation unit 105 calculates the position in the actual pit of the candidate area determined by the determination unit 104 to contain the target waste. Specifically, the position calculation unit 105 converts the coordinates indicating the position of the candidate area on the image plane of the pit image 202 into three-dimensional coordinates indicating the three-dimensional position on the actual pit. If the correspondence relationship between the coordinates (two-dimensional coordinates) on the pit image 202 and the three-dimensional coordinates of the actual pits is specified in advance, the above conversion can be performed based on the correspondence relationship.

The target area extracting unit 106 extracts a target area set as a determination target within the waste area in which the waste appears in the pit image 202 . How to set the target area is arbitrary. For example, the target region extraction unit 106 may extract a target region of a size desired by the user from a position desired by the user in the pit image 202 according to an input operation by the user via the input unit 13 .

A partial area setting unit 107 sets a plurality of partial areas within the partial area extracted by the target area extracting unit 106 . Each partial area set by the partial area setting unit 107 is judged by the partial judgment unit 108 . A method of setting the partial area will be described later with reference to FIG.

The partial determination unit 108 determines, for each of a plurality of partial areas within the target area set as a determination target within the waste area in which the waste appears in the pit image 202, that the waste reflected in the partial area is the target waste. determine if there is Specifically, the partial determination unit 108 determines whether the waste imaged in the partial area is the target waste based on the output value obtained by inputting the image of the partial area into the trained model 204 .

As described above, the trained model 204 is trained to output an output value indicating whether or not the waste reflected in the image is the target waste when the image of the partial region is input. is a model. The learned model 204 may be one that has learned a correspondence relationship between an image and whether or not the target waste is shown in the image. Also, the trained model 204 may be trained to classify the waste that appears in the image. In the latter case, the target waste should be included in one of the categories. The algorithm of the trained model 204 is not particularly limited. For example, a convolutional neural network trained model 204 may be used.

Based on the determination result of each partial area by the partial determination unit 108, the integrated determination unit 109 determines whether the waste in the target area including these partial areas is the target waste. The details of this determination method will be described later.

The determination result integration unit 110 integrates the determination result of the determination unit 104 and the determination result of the integrated determination unit 109 . Details of the integration method will be described later. The determination result integration unit 110 may be omitted, and the determination result of the determination unit 104 and the determination result of the integrated determination unit 109 may be output.

The crane control unit 111 controls the crane 3 via the control device 2 by communication via the communication unit 12 . For example, the crane control unit 111 notifies the control device 2 of the position of the waste determined by the determination result integration unit 110 to be the target waste, and causes the crane 3 to agitate the waste at that position.

As described above, the information processing apparatus 1 includes the candidate area extracting unit 103 for extracting, as a candidate area, an area made up of pixels in a predetermined color range corresponding to the target waste to be detected from the pit image 202, and the candidate area and a determination unit 104 that determines whether the waste imaged in the candidate area is the target waste based on the feature amount generated based on the color of each pixel constituting the .

According to the above configuration, first, an area composed of pixels in a predetermined color range corresponding to the target waste is extracted as a candidate area in the pit image 202 . As a result, areas in the pit image that are highly likely to contain the target waste are extracted as candidate areas. Since the color of the waste is generally the same even if the target waste treatment plant is changed, this candidate area extraction can be performed without any trouble even if the target waste treatment plant is changed.

However, in the extraction based on the color range, areas in which other waste with colors similar to the target waste are captured are also extracted, so at this stage, it is necessary to determine whether or not the target waste is captured in the candidate area. can't. In addition, since the shape and size of the extracted candidate regions are not always constant, it is possible to determine whether the waste within the candidate region is the target waste by using an identification algorithm using an image as input data, as in Patent Document 1. Hard to judge.

Therefore, according to the above configuration, it is determined whether the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel forming the candidate area. As a result, the target waste can be detected even if the shapes and sizes of the extracted candidate regions are not constant.

Thus, according to the above configuration, the target waste can be detected regardless of the target waste treatment plant. That is, according to the above configuration, it is possible to detect target waste by a general-purpose method as compared with the conventional technology.

In addition, as described above, the information processing apparatus 1 determines that each of the plurality of partial areas within the target area set as a determination target within the waste area in which the waste appears in the pit image 202 is captured in the partial area. A partial determination unit 108 that determines whether the waste is the target waste, and an integrated determination unit 109 that determines whether the waste reflected in the target area is the target waste based on the determination results in each of the partial areas. .

A pit image 202 obtained by imaging the inside of the pit has a different appearance depending on the target waste treatment plant. However, in the pit image 202, when a target area is set within the waste area in which the waste is captured, and a partial area is set within the target area, the appearance of the waste reflected in the partial area is affected by the appearance of the waste treatment plant. The effect of differences is often negligible. However, since the size of each partial area is small, the determination accuracy for each partial area may be lower than when the entire target area is subjected to determination.

Therefore, according to the above configuration, for each of the plurality of partial regions set within the target region of the pit image 202, it is determined whether the waste reflected in the partial region is the target waste, and then the determination is made. Based on the results, it is determined whether the waste imaged in the target area is the target waste.

According to this configuration, a decrease in identification accuracy caused by making judgments not on the target area but on its partial areas is compensated for by obtaining the final judgment results based on the judgment results of each partial area.

Further, as described above, the effect of the difference in waste treatment plants on the appearance of the waste reflected in the partial area is often negligible. can detect target waste with similar accuracy.

Furthermore, since the information processing apparatus 1 detects the target waste for one pit image 202 by the determination unit 104 and the integrated determination unit 109, the possibility of omission or false detection of the target waste is eliminated. can be reduced.

[Method of extracting the waste area]
A method of extracting a waste area by the waste area extraction unit 101 will be described with reference to FIG. FIG. 3 is a diagram showing a method of deriving a reference point that serves as a reference for extracting a waste area. FIG. 3 shows an example of a pit image 202 plotted with reference points, an enlarged view 2021 of a portion of the pit image 202 framed by a broken line, and height information 201 . Here, the horizontal direction of the pit image 202 is defined as the X direction, and the vertical direction thereof is defined as the Y direction.

As described above, the height information 201 indicates the piled-up height of waste at various locations in the pit. The height information 201 shown in FIG. 3 defines grid-like divisions in the pit, and indicates the piled-up height of the waste in each defined division. More specifically, in the height information 201 shown in FIG. 3, the pits are divided so that there are m divisions in the X direction and n divisions in the Y direction (both m and n are natural numbers).

The height information 201 shown in FIG. 3 specifies the height of each section by combining the X-direction position (X1 to Xm) and the Y-direction position (Y1 to Yn). there is For example, the uppermost section in the pit image 202 shown in FIG. 3 has a position X1 in the X direction and a position Y1 in the Y direction. The location of this partition can be represented as (X1, Y1). The piled-up height of waste in this section (X1, Y1) is "h1-1" shown in column X1, row Y1 in the height information 201 shown in FIG.

The waste region extraction unit 101 obtains the position of the reference point corresponding to each section in the pit image 202 from the piled-up height of the waste in each section indicated by the height information 201 . The reference points indicate the position of the surface of the waste deposited in each compartment. For example, the position of the reference point P1-1 corresponding to the section (X1, Y1), that is, the position of the surface of the waste deposited in the section (X1, Y1) is the height of the waste deposited in the section (X1, Y1). It is obtained from "h1-1". The same applies to other sections, for example, the position of the reference point P2-1 corresponding to the section (X2, Y1) is obtained from the waste pile height "h2-1" in the section (X2, Y1). .

Here, in the pit image 202, the higher the waste piled up in the section, the higher the surface of the waste in that section. That is, there is a correlation between the waste pile height in each section and the position of the waste surface in the pit image 202 . For this reason, it is possible to prepare in advance correspondence information indicating the correspondence relationship between the piled-up height of the waste at various locations in the pit and the position at which the surface of the waste at various locations in the pit appears in the pit image 202 . Using this correspondence information, the waste region extracting unit 101 can obtain the position of the reference point corresponding to each section from the waste pile height in each section indicated by the height information 201 . The position of the reference point may be represented by coordinates, for example.

The height of the waste piled up in each section shown in the height information 201 may be measured using various sensors or the like. It may be what you asked for. However, when the latter method is adopted, it is difficult to determine the height of the waste piled up along the walls of the pit.

In this case, the waste region extraction unit 101 may estimate the position of the reference point indicating the surface position of the waste along the wall surface of the pit in the pit image 202 . For example, the waste area extracting unit 101 determines that the height of waste accumulation at the reference point P0-1 along the wall surface shown in FIG. The position of the reference point P0-1 may be estimated by assuming that it is the same as the object pile height. Further, for example, the waste area extraction unit 101 extracts the reference point P0 by extrapolating from the coordinate values of the reference point P1-1 adjacent to the reference point P0-1 and the reference point P2-1 adjacent to the reference point P1-1. The position of -1 may be estimated. The same applies to the reference points P0-0, P1-0, P2-0, etc. along the wall surface.

The waste area extraction unit 101 extracts a waste area by connecting the reference points obtained as described above. For example, when extracting all the waste regions shown in the pit image 202, the waste region extraction unit 101 uses reference points (P0-1, P0-0, P1-0, P2-0, etc.) is used as an outer edge to extract the waste region.

[Extraction example of waste area]
The waste region extraction unit 101 can extract various waste regions by using the reference points set as described above. This will be described with reference to FIG. FIG. 4 is a diagram showing an example of extraction of a waste area by the waste area extraction unit 101. As shown in FIG.

A pit image 202 shown in FIG. 4 shows a pit in which waste is stored, and also a loading door for throwing waste into the pit. In the pit image 202, the area where the waste is accumulated is a, and the band-shaped area b located directly below the carry-in door is the area where the waste is dropped. The area c, which is the area excluding the area b from the area a, is an area for reloading and storing the waste, and for agitating the waste.

As described above, the waste region extracting unit 101 extracts the waste region using the line connecting the reference points (not shown in FIG. 4) indicating the surface position of the waste along the wall surface of the pit as the outer edge. The entire waste area that appears in the pit image 202 can be extracted. 210a in FIG. 4 shows the extraction result of extracting the waste area in this way.

In addition, the waste area extracting unit 101 extracts a waste area corresponding to the area b mainly based on the reference points along the wall surface of the pit entrance door and the reference points located at the boundary between the area b and the area c. can also be extracted. 210b in FIG. 4 shows the extraction result of extracting the waste area in this way. Similarly, the waste region extracting unit 101 extracts the waste based on the reference points mainly indicating the surface position of the waste along the wall facing the entrance door of the pit and the reference points located at the boundary between the regions b and c. , the waste region corresponding to region c can also be extracted. 210c in FIG. 4 shows the extraction result of extracting the waste area in this way.

As described above, the information processing apparatus 1 provides correspondence information indicating the correspondence relationship between the piled-up height of waste at various locations in the pit and the position at which the surface of the waste at various locations in the pit appears in the pit image, and A waste area extracting unit 101 is provided for extracting a waste area in which waste appears from a pit image 202 using height information 201 indicating the piled-up height of waste. Then, the candidate area extraction unit 103 extracts candidate areas within the waste area extracted by the waste area extraction unit 101 .

This extraction method has the advantage that it does not require analysis of the pit image 202 or learning as in Patent Document 1. Further, according to the above configuration, since the candidate area is extracted within the extracted waste area, it is possible to narrow down the extraction range of the candidate area and efficiently extract the candidate area.

[Method for extracting candidate regions]
The candidate area extraction unit 103 extracts an area composed of pixels in a predetermined color range corresponding to the target waste to be detected as a candidate area within the waste area in the pit image 202 extracted as described above. Extract. At this time, the candidate area extraction unit 103 may extract candidate areas based on at least one of the hue, saturation, and brightness of each pixel of the pit image 202 . In this case, if the pit image 202 is an RGB image, the candidate area extraction unit 103 may convert the pit image 202 into an HSV image. Thus, candidate areas can be extracted based on at least one of the hue (H), saturation (S), and brightness (V) of each pixel of the pit image 202 .

Which of the values contained in the HSV image is to be used and the criteria for extracting candidate regions may be determined according to the color of the target waste itself, the appearance of the target waste in the pit image 202, and the like.

For example, the candidate area extracting unit 103 extracts, from the pit image 202, pixels in a predetermined range of hues, pixels in a predetermined range of saturation values, and pixels in a predetermined range of lightness values, and extracts continuous A region may be a candidate region. Further, for example, the candidate area extracting unit 103 may extract, from the pit image 202, pixels whose hue, saturation value, and brightness value are all within a predetermined range, and use a continuous area made up of these pixels as a candidate area. good.

Further, the candidate area extracting unit 103 may, after extracting the pixels satisfying the predetermined condition as described above, extract the outline of the area composed of the extracted pixels. Then, the candidate area extracting unit 103 may calculate the area of each area surrounded by the contour lines, exclude areas whose area is equal to or smaller than the threshold value, and set the remaining areas as candidate areas.

When the candidate area is extracted as described above, the determination unit 104 determines that the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel that constitutes the candidate area. determine if there is At this time, if the candidate area is extracted based on at least one of hue, saturation, and lightness, the feature amount is also at least one of hue, saturation, and lightness of each pixel constituting the candidate area. It is preferred to use those generated based on

For example, the ratio of pixels whose hue values are in a predetermined range in the entire candidate area, the peak value of hue, the peak value of saturation, and the peak value of lightness may be used as feature amounts. The peak value is the peak value (maximum value) in the histogram generated from the HSV image. When obtaining the peak value, the peak position may be specified by taking the moving average of the generated histogram.

When the judgment unit 104 makes a judgment using the random forest trained model 203, it classifies whether or not the waste reflected in the candidate area is the target waste using the branching condition for the feature amount as described above. The trained model 203 that has been trained may be used. In addition, when judgment is performed using a support vector machine or a trained model 203 of a neural network, the above-described feature amount is used as an explanatory variable, and the waste appearing in the candidate area from which the feature amount is obtained is the target waste. A trained model 203 that has been trained using whether or not there is an object as an objective variable may be used.

The candidate regions extracted by the candidate region extracting unit 103 have various sizes and shapes depending on the pit image 202. However, in a general machine learning model for classifying images, they have the same size and shape as the images used for learning. The input data must be an image of the shape. Therefore, the determination unit extracts a feature amount from the candidate area, and performs determination using the feature amount. As a result, it is possible to use the same learned model 203 to determine candidate areas of different sizes and shapes.

As described above, the candidate area extraction unit 103 may extract candidate areas based on at least one of the hue, saturation, and brightness of each pixel of the pit image 202 . In this case, the determination unit 104 may perform determination based on a feature amount generated based on at least one of hue, saturation, and brightness of each pixel forming the candidate area.

It has been found that the appearance of the waste in the pit image 202 is characterized by hue, saturation, and brightness. For this reason, according to the above configuration, it is possible to capture the features of the appearance of the waste and extract candidate regions with high accuracy, and capture the features of the appearance of the waste and capture them in the candidate regions. It becomes possible to determine with high accuracy whether the waste is the target waste.

[Example of noise removal]
As described above, the candidate region extracting unit 103 may extract pixels satisfying a predetermined condition from the waste region and extract candidate regions made up of those pixels. At this time, the candidate region extraction unit 103 may remove noise after extracting the pixels. This will be described with reference to FIG. FIG. 5 is a diagram showing an example of noise removal when extracting candidate regions.

The candidate region extracting unit 103 may binarize the pit image 202 after extracting pixels satisfying a predetermined condition and before performing noise removal. FIG. 5 shows a binarized image 220 binarized by such processing. In the binarized image 220 of FIG. 5, the pixels that satisfy the predetermined conditions are white, and the pixels that do not satisfy the conditions are black.

In noise removal, the candidate region extraction unit 103 first performs contraction processing on the binarized image 220, and then performs expansion processing. This makes it possible to remove minute noise that does not need to be included in the candidate area. An image obtained by removing noise from the binarized image 220 is shown as 221 in FIG.

The erosion process is a process of leaving the target pixel as a white pixel only when all the pixels surrounding the target pixel are white. For example, in the contraction process shown in FIG. 5, only four pixels included in the pixel group PIC1 have white surrounding pixels. Therefore, as illustrated, only four pixels included in the pixel group PIC1 remain as white pixels, and all other pixels are converted to black. This erosion process can remove scattered white pixels.

On the other hand, the dilation process is a process of making the pixel of interest a white pixel if any of the pixels surrounding the pixel of interest is white. For example, in the example of dilation processing shown in FIG. 5, six black pixels included in pixel group PIC2 are adjacent to white pixels. Therefore, as shown, the six pixels included in pixel group PIC2 are converted to white. This dilation process can expand the area made up of white pixels. Note that if the erosion process is performed first and then the expansion process is performed, the original image is basically not restored.

[Setting example of target area]
Any method can be used to set the target area. For example, the user can freely set the target area. Also, the target area may be set with reference to the reference point described above. This will be described with reference to FIG. FIG. 6 is a diagram showing a setting example of the target area.

In the example of FIG. 6, the target area extracting unit 106 extracts an area surrounded by four reference points (P11 to P14) among the reference points set in the pit image 202 as the target area 230. FIG. These four reference points may be set by the user through an input operation via the input unit 13, or the user inputs a desired section, and the target region extracting unit 106 extracts reference points surrounding the input section. may be specified.

[Example of partial area setting]
A partial region is set in the target region 230 extracted as described above. This will be described with reference to FIG. FIG. 7 is a diagram showing a setting example of the partial area 231 in the target area 230. As shown in FIG. Note that FIG. 7 shows only the outer edge of the target area 230 and omits illustration of the wastes and the like appearing in the target area 230 .

Six partial areas 231 are set in the target area 230 shown in FIG. All of the set partial areas 231 are rectangular and have the same size. The partial regions 231 are arranged in 2 rows vertically by 3 columns horizontally, for a total of 6. The interval between the partial regions 231 in the horizontal direction is w1, and the interval between the partial regions 231 in the vertical direction is w2. Also, the distance from the end of the target area to the partial area 231 is w3. It should be noted that the partial areas 231 may be of a size that allows determination of whether or not the waste reflected therein is the target waste, and the number, size, shape, etc. of the partial areas 231 to be set are not particularly limited.

As in the example of FIG. 7, the partial area 231 may be set so as to avoid the edges of the target area. Although there is a possibility that the side walls of the pit will be reflected at the edges of the target area, the possibility that the side walls of the pits and the like will be reflected is reduced by setting the partial area so as to avoid the edges of the target area. can improve the accuracy of determination. In addition, when the possibility that the side wall of the pit is reflected is originally low, or when there is no problem with the side wall of the pit being reflected, the partial area may be set so as to include the edge of the target area. .

Also, as in the example of FIG. 7, the partial regions 231 may be set at intervals within the target region. As a result, the total number of partial areas can be reduced, and the processing load required for determination of each partial area can be reduced compared to the case where partial areas are set within the target area without spacing. If it is not necessary to reduce the total number of partial areas, the partial areas 231 may be set without spacing.

[Method of Integrated Judgment]
The integrated determination unit 109 determines whether the waste in the target area including the partial areas is the target waste based on the determination results of the plurality of partial areas set as described above. This determination method is not particularly limited as long as it is based on the determination result of each partial area.

For example, the integrated determination unit 109 may determine whether the waste in the target area is the target waste based on the majority of the determination results of a plurality of partial areas. For example, when six partial areas are set in the target area as in the example of FIG. 7, four partial areas are determined to be the target waste, and the other two partial areas are not the target waste ( Or it is a type of waste other than the target waste). In this case, by majority vote, the waste in the target area is determined to be the target waste.

In addition, the integrated determination unit 109 weights the determination result of each partial region according to the accuracy of the determination result, and determines whether the waste in the target region is the target waste based on the weighted determination result. good too. For example, if the certainty of the judgment result by the trained model 204 is 95% or more, 3 points, if less than 95%, 1 point, and the judgment result with the highest total score may be adopted. For example, suppose that the judgment results for the three partial areas are target waste (95% confidence), not target waste (75% confidence), and not target waste (72% confidence). . In this case, the score for the judgment result that it is target waste is 3 points, and the score for the judgment result that it is not target waste is 2 points, so the result of the integrated judgment is that it is target waste. become. According to this configuration, it is possible to detect the existence of the target waste in the target area even when the target waste exists in one part of the target area and does not exist in the other part. .

[Example of integration of two judgments and their judgment results]
As described above, in the information processing apparatus 1, the determination unit 104 determines whether the waste appearing in the candidate area extracted by the candidate area extraction unit 103 is the target waste. Also, the partial determination unit 108 determines whether the waste imaged in each partial area set by the partial area setting unit 107 is the target waste, and based on the determination result, the waste imaged in the target area is the target waste. The integrated determination unit 109 determines whether there is. As described above, the information processing apparatus 1 determines whether the waste is the target waste by two determination methods. Then, the determination result integration unit 110 integrates the determination result of the determination unit 104 and the determination result of the integrated determination unit 109 .

FIG. 8 is a diagram showing an example of integration of the determination result of the determination unit 104 and the determination result of the integrated determination unit 109. As shown in FIG. A pit image 202 shown in FIG. 8 is an image to be determined by the information processing apparatus 1, and the pit image 202 shows waste with different colors.

As described above, the waste region extractor 101 extracts the waste region 210 from the pit image 202 . Next, the candidate area extraction unit 103 extracts an area composed of pixels in a predetermined color range in the waste area 210 as a candidate area. In the example of FIG. 8, the candidate area extraction unit 103 determines whether each pixel from the upper left corner position p1 to the lower right corner position p2 of the waste area 210 is a pixel in a predetermined color range. Then, the candidate area extracting unit 103 extracts an area composed of pixels in a predetermined color range as a candidate area. In the example of FIG. 8, two areas A1 and A2 are extracted as candidate areas, and the determination unit 104 determines that the waste reflected in these candidate areas is the target waste.

On the other hand, the target region extraction unit 106 sets the target region within the waste region 210 . In the example of FIG. 8, three target areas 230a to 230c are set. As described above, the partial area setting unit 107 sets a plurality of partial areas within the set target area, and the partial determination unit 108 determines whether the waste reflected in each partial area is the target waste. Then, the integrated determination unit 109 determines whether the waste imaged in the target regions 230a to 230c is the target waste based on the determination results in each of the partial regions. Target areas 230a and 230b have been determined to be target waste, and target area 230c has been determined not to be target waste.

Regarding the target area 230c, neither the determination by the determination unit 104 nor the determination by the integrated determination unit 109 has determined that it is the target waste. Therefore, the determination result integration unit 110 should set the determination result of the target area 230c as no target waste.

On the other hand, the target area 230a includes the area A1 determined by the determination unit 104 to be the target waste. The determination result integration unit 110 may set the determination result to include target waste for such a target region 230a.

Also, the target area 230b does not include the area determined to be the target waste by the determination unit 104, but the integrated determination unit 109 determines that the waste in this area is the target waste. The determination result integration unit 110 may set the determination result to include target waste for such a target region 230a.

In this way, the determination result integration unit 110 may set the determination result as the presence of target waste for an area determined by either or both of the determination unit 104 and the integrated determination unit 109 to have target waste. Then, both the determination unit 104 and the integrated determination unit 109 may determine that there is no target waste for an area determined to have no target waste (or not determined to have target waste).

Alternatively, the determination result integration unit 110 may divide the pit into a grid pattern, set an address in each section, and determine the presence or absence of target waste in units of addresses. In this case, the determination result integration unit 110 obtains the centroid position in the pit image 202 of the candidate area determined to be the target waste by the determination unit 104 and the centroid position of the target area, and finds the address containing the plurality of centroid positions. It is also possible to detect and integrate determination results for such addresses. For example, the determination result integration unit 110 may compare the area of the candidate area and the area of the target area, and use the larger determination result as the determination result of the address.

[Process flow (overall)]
The flow of processing executed by the information processing apparatus 1 will be described with reference to FIG. FIG. 9 is a flow chart showing an example of processing executed by the information processing device 1 .

In S1, the waste region extracting unit 101 acquires a pit image 202 that is an image of the inside of the pit containing waste. Also, in S2, the waste region extraction unit 101 acquires the height information 201. FIG. Note that the processing of S2 may be performed before the processing of S1, or the processing of S1 and S2 may be performed in parallel.

In S3, the waste region extraction unit 101 performs preprocessing on the pit image 202 acquired in S1. The content of the preprocessing is not particularly limited, and for example, in S3, the waste area extraction unit 101 may perform white balance adjustment, contrast adjustment, and the like. Note that the process of S3 may be performed as required, and is not an essential process.

Further, when a crane is shown in the pit image 202, the waste region extracting unit 101 may perform mask processing for masking the region where the crane is shown. The area to be masked may be determined based on the three-dimensional position of the bucket of the crane. For example, the coordinates indicating the three-dimensional position of the center of the bucket of the crane may be converted into coordinates on the image plane of the pit image 202, and the mask processing area may be set based on the coordinates. Alternatively, a region in which the bucket appears may be detected by a learned model that has learned the appearance of a crane bucket, and mask processing may be performed on that region.

In S4, target waste is detected for each candidate area. Then, in S5, target waste is detected for each target area. The processing of S5 may be performed first, or the processing of S4 and S5 may be performed in parallel. The details of the processing of S4 will be described later with reference to FIG. 10, and the details of the processing of S5 will be described later based on FIG.

In S6, the determination result integration unit 110 integrates the target waste detection results in S4 and S5. Then, in S7, the crane control unit 111 notifies the control device 2 of the position of the target waste according to the detection result integrated in S6, and stirs the waste at that position. The stirring may be performed by dropping the waste picked up by the bucket of the crane by opening the bucket. At this time, by opening the bucket while moving the bucket horizontally, the waste may be dispersed on the movement path of the bucket, or the waste picked up at a certain position may be dropped at that position, You may move it to another position and drop it. Also, if the target waste should not be incinerated in the first place, the crane control unit 111 transfers the waste at the notified position to the area where the waste not to be incinerated is accumulated. , may be instructed to the control device 2.

[Process flow (details of S4)]
Details of the processing of S4 in FIG. 9 will be described based on FIG. FIG. 10 is a flowchart showing an example of target waste detection processing (determination method) performed for each candidate region.

In S41, the waste area extraction unit 101 uses the height information 201 acquired in S2 of FIG. 9 to extract a waste area from the pit image 202 acquired in S1. Subsequently, in S42, the image conversion unit 102 converts the image of the waste region extracted in S41 into an HSV image.

In S43 (candidate area extraction step), the candidate area extraction unit 103 extracts, as a candidate area, an area composed of pixels in a predetermined color range corresponding to the target waste in the pit image 202 converted into the HSV image in the process of S42. Extract. At this time, the candidate area extracting unit 103 may remove noise by the method shown in FIG. 5, for example. As described above, the candidate area is a continuous area made up of pixels of a predetermined color range according to the target waste, and sometimes not even one candidate area is extracted, and sometimes a plurality of candidate areas are extracted. .

In S44, the candidate area extraction unit 103 determines whether or not the number of candidate areas extracted in S43 is greater than zero. If it is determined that the number is greater than 0 (YES in S44), the process proceeds to S45. On the other hand, if it is determined to be 0 (NO in S44), the process of FIG. 10 ends.

In S45 (determination step), the determination unit 104 determines whether the waste imaged in the candidate area is the target waste based on the feature amount generated based on the color of each pixel constituting the candidate area extracted in S43. judge. For example, the determination unit 104 generates a feature amount using the hue, saturation, and brightness values of each pixel constituting the candidate area extracted in S43, and inputs the generated feature amount to the trained model 203. The type of waste that appears in the candidate area may be determined from the output value obtained by

In S46, the candidate area extracting unit 103 excludes the candidate area determined in S45 as being of a type other than the target waste from the candidate areas extracted in S43. As a result, among the candidate areas extracted in S43, the candidate areas determined to be the target waste in S45 remain.

In S47, the candidate area extraction unit 103 determines whether or not the number of candidate areas remaining after the process of S46 is greater than zero. If it is determined that the number is greater than 0 (YES in S47), the process proceeds to S48. On the other hand, if it is determined to be 0 (NO in S47), the process of FIG. 10 ends.

In S48, the position calculation unit 105 calculates the position of the center of gravity of the candidate area remaining after the process of S46. Specifically, the position calculation unit 105 calculates the barycentric position coordinates of the candidate area on the image plane of the pit image 202 .

In S49, the position calculation unit 105 converts the coordinates of the center-of-gravity position calculated in S48 into three-dimensional coordinates. The three-dimensional coordinates obtained by this conversion indicate the position of the detected target waste within the pit.

If there are multiple types of target waste, the process of FIG. 10 may be performed for each of the multiple types of target waste. In this case, the color range of pixels to be extracted in S43 changes according to the type of target waste to be detected.

As described above, the determination method executed by the information processing apparatus 1 is based on the pit image 202 obtained by capturing the inside of the pit containing the waste, and the area composed of pixels in a predetermined color range corresponding to the target waste to be detected. is extracted as a candidate area (S43). This determination method also includes a determination step (S45) of determining whether the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel forming the candidate area. According to this determination method, the target waste can be detected even if the shapes and sizes of the extracted candidate regions are not constant.

[Process flow (details of S5)]
Details of the processing of S5 in FIG. 9 will be described based on FIG. FIG. 11 is a flow chart showing an example of the target waste detection process (determination method) in the target area set as the determination target.

In S51, the target region extracting unit 106 sets the value of the counter i of the number of processing times to zero. Subsequently, in S52, the target area extraction unit 106 determines whether or not the value of the counter i is greater than the specified number of addresses. If it is determined in S52 that it is larger (YES in S52), the process of FIG. 11 ends. On the other hand, if it is determined in S52 that i is equal to or less than the specified number of addresses (NO in S52), the process proceeds to S53. The designated number of addresses is the number of sections included in the set target area among the sections set by dividing the pit in a grid pattern.

In S53, the target area extraction unit 106 extracts a portion corresponding to one address from the pit image 202 acquired in S1 using the height information 201 acquired in S2 of FIG. . Then, in S54, the partial area setting unit 107 sets a plurality of partial areas in the target area extracted in S53.

In S55 (partial determination step), the partial determination unit 108 determines each of a plurality of partial areas within the set target area (a portion corresponding to one address within the set area in this flowchart). Determine the type of waste. In other words, the partial determination unit 108 determines whether the waste reflected in the partial area is the target waste.

In S56, the integrated determination unit 109 determines whether the waste imaged in the target area is the target waste, based on the determination result for each of the partial areas. After that, the target region extracting unit 106 increments the value of the processing counter i by 1, and returns to the processing of S52.

In the above example, the determination is made for all the specified addresses, but it is also possible to specify a part of the addresses and make the determination only for the specified addresses. For example, assume that 12 addresses 1 to 12 are set, and 4 of them 2, 3, 7, and 8 are specified by the user. In this case, the number of specified addresses is 4, so in S52, the target area extracting unit 106 determines whether the value of the counter i is greater than 4 or not. If it is determined NO in S52, the target area extracting unit 106 determines an undetermined address among the four designated addresses 2, 3, 7, and 8 as a determination target, and proceeds to the process of S53. .

As described above, the determination method executed by the information processing apparatus 1 is based on a pit image obtained by imaging the inside of a pit containing waste. For each of the plurality of partial areas, a partial determination step (S55) is included for determining whether the waste reflected in the partial area is the target waste to be detected. This determination method also includes an integrated determination step (S56) of determining whether the waste imaged in the target area is the target waste based on the determination results in each of the partial areas. According to this determination method, it is possible to detect the target waste with the same accuracy even if the target waste treatment plant is changed.

[Modification]
The execution subject of each process described in the above embodiments is arbitrary, and is not limited to the above examples. That is, an information processing system having functions similar to those of the information processing apparatus 1 can be constructed by using a plurality of information processing apparatuses that can communicate with each other. For example, S1, 2, 4 to 6 in FIG. 9 may be executed by the information processing apparatus 1, and S3 and S7 may be executed by separate information processing apparatuses. Thus, the processing shown in FIG. 9 can be realized by one or more information processing apparatuses. The determination method shown in FIGS. 10 and 11 is also the same.

[Reference example]
The information processing apparatus 1 may be configured without the candidate area extraction unit 103 and the determination unit 104 . Even in this case, if the information processing apparatus 1 includes the partial determination unit 108 and the integrated determination unit 109, it is possible to detect the target waste by a general-purpose method compared to the conventional technology. be.

[Example of realization by software]
The function of the information processing device 1 (hereinafter referred to as "device") is a program for causing a computer to function as the device, and the computer is used as each control block (especially each part included in the control unit 10) of the device. It can be realized by a program (determination program) for functioning.

In this case, the apparatus comprises a computer having at least one control device (eg processor) and at least one storage device (eg memory) as hardware for executing the program. Each function described in each of the above embodiments is realized by executing the above program using the control device and the storage device.

The program may be recorded on one or more computer-readable recording media, not temporary. The recording medium may or may not be included in the device. In the latter case, the program may be supplied to the device via any transmission medium, wired or wireless.

Also, part or all of the functions of the above control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits functioning as the control blocks described above are formed are also included in the scope of the present invention. In addition, it is also possible to implement the functions of the control blocks described above by, for example, a quantum computer.

The present invention is not limited to the above-described embodiments, but can be modified in various ways within the scope of the claims, and can be obtained by appropriately combining technical means disclosed in different embodiments. is also included in the technical scope of the present invention.

1 information processing device 101 waste region extraction unit 103 candidate region extraction unit 104 determination unit 108 partial determination unit 109 integrated determination unit 202 pit image 2 control device 3 crane 5 control system

Claims (13)

a candidate area extracting unit for extracting, as a candidate area, an area made up of pixels in a predetermined color range corresponding to the target waste to be detected from a pit image obtained by imaging the inside of the pit containing the waste;
and a determination unit that determines whether the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel that constitutes the candidate area. The determination unit inputs the feature amount generated based on the color of each pixel constituting the candidate area to a trained model that has learned the feature amount generated based on the color of the target waste. 2. The information processing apparatus according to claim 1, wherein based on the obtained output value, it is determined whether the waste imaged in the candidate area is the target waste. Correspondence information indicating a correspondence relationship between the piled-up height of the waste at each location of the pit and the position at which the surface of the waste at each location of the pit appears in the pit image, and the piled-up height of the waste at each location of the pit. a waste region extraction unit that extracts a waste region in which the waste is reflected from the pit image using height information indicating the height of the waste region;
3. The information processing apparatus according to claim 1, wherein said candidate area extraction unit extracts said candidate area within said waste area. The candidate area extraction unit extracts the candidate area based on at least one of hue, saturation, and brightness of each pixel of the pit image,
4. The determination unit according to any one of claims 1 to 3, wherein the determination unit performs the determination based on the feature amount generated based on at least one of hue, saturation, and brightness of each pixel forming the candidate area. The information processing device according to item 1. For each of a plurality of partial areas within a target area set as a determination target within the waste area in which the waste appears in the pit image, it is determined whether the waste appearing in the partial area is the target waste. a partial determination unit;
5. The information processing according to any one of claims 1 to 4, further comprising an integrated determination unit that determines whether the waste reflected in the target area is the target waste based on determination results in each of the partial areas. Device. 6. The information processing apparatus according to claim 5, wherein said plurality of partial areas are set so as to avoid edges of said target area. 7. The information processing apparatus according to claim 5, wherein said plurality of partial areas are set at intervals within said target area. For each of a plurality of partial areas within a target area set as a determination target within the waste area in which the waste is captured in a pit image obtained by imaging the inside of the pit containing the waste, the waste reflected in the partial area is determined. a partial determination unit that determines whether the waste is a target waste to be detected;
and an integrated determination unit that determines whether the waste reflected in the target area is the target waste based on determination results in each of the partial areas. A determination method executed by one or more information processing devices,
a candidate region extracting step of extracting, as a candidate region, a region composed of pixels in a predetermined color range corresponding to the target waste to be detected from a pit image obtained by imaging the inside of the pit containing the waste;
and a determination step of determining whether the waste appearing in the candidate area is the target waste based on the feature amount generated based on the color of each pixel forming the candidate area. A determination method executed by one or more information processing devices,
For each of a plurality of partial areas within a target area set as a determination target within the waste area in which the waste is captured in a pit image obtained by imaging the inside of the pit containing the waste, the waste reflected in the partial area is determined. a partial determination step of determining whether the waste is a target waste to be detected;
and an integrated determination step of determining whether the waste reflected in the target area is the target waste based on the determination results in each of the partial areas. 2. A determination program for causing a computer to function as the information processing apparatus according to claim 1, the determination program for causing the computer to function as the candidate region extractor and the determination unit. 9. A determination program for causing a computer to function as the information processing apparatus according to claim 8, the determination program for causing the computer to function as the partial determination section and the integrated determination section. In a pit image obtained by imaging the inside of a pit containing waste, a region composed of pixels in a predetermined color range corresponding to the target waste to be detected is extracted as a candidate region, and each pixel constituting the candidate region is extracted. an information processing device that determines whether the waste reflected in the candidate area is the target waste based on the feature amount generated based on the color;
and a control device that controls the operation of the crane placed in the pit based on the determination result of the information processing device.

Images