JP2024016293A - waste sorting equipment - Google Patents

Abstract

The present invention provides a waste sorting device that can effectively improve sorting accuracy. [Solution] The waste sorting device 1 includes a belt conveyor 3 that conveys the waste 2, a 3D sensor 4 that measures the shape and height of the waste 2, and a visible light that takes a visible light image of the waste 2. It includes a camera 5, a hyperspectral camera 6 that takes an infrared image of the waste 2, and a metal sensor 19 that detects whether the waste 2 is metal. Based on the information from the hyperspectral camera 6 and the metal sensor 19, the first discrimination section of the discrimination device 7 discriminates the material of the waste 2. Further, based on the information from the visible light camera 5 and the metal sensor 19, the material of the waste 2 is determined by the second determining section of the determining device 7. Among the wastes 2 whose materials have been determined by the first discrimination section and the second discrimination section, the wastes 2 whose materials have a correct discrimination rate equal to or higher than a predetermined value are collected from the belt conveyor 3 by the robot arm 9, and are processed according to the material. They are put into a collection box 10 and sorted. [Selection diagram] Figure 1

Description

The present invention relates to a waste sorting device that uses machine learning techniques to determine the material of waste and performs sorting.

Construction waste, which contains a variety of materials such as concrete, metal, wood, and plastic, used to be disposed of in landfills or dumped in the ocean, but with the aim of making more effective use of resources, Attempts to sort waste by material and recycle it as resources are becoming more widespread.

BACKGROUND ART Conventionally, methods using a specific gravity sorter, a swinging sorter, or a wind sorter are known as methods for sorting waste by material. However, these devices apply vibrations and wind to the object to be processed, and tend to have a complicated and large device configuration. In addition, there is a possibility that fine particles attached to the object to be processed may be scattered due to vibrations or wind, which may affect the surrounding environment.

Therefore, recently, a waste sorting device has been proposed that utilizes a material recognition technique based on a photographed image of the object to be processed. Conventionally, this type of waste sorting equipment includes a conveyor that transports the waste, a photographing device that photographs the waste on the conveyor, and a terminal that displays the photographed image of the waste and receives input of information indicating the material. There is also one that includes a control unit that performs machine learning based on image information of waste, and a robot arm that sorts waste based on commands from the control unit (see, for example, Patent Document 1).

The conventional waste sorting device described above uses an imaging device to photograph the waste transported by a conveyor, measures the mass, and displays the photographed image, mass, volume, and specific gravity on a monitor. When the operator identifies the material from the content displayed on the monitor and inputs it into the terminal, material information regarding the input material is linked to image information and mass information and stored as teacher information. The control unit performs machine learning using a convolutional neural network using an optimization method based on error backpropagation based on the teacher information, and determines the material of the waste based on the results of this machine learning. . Once the material has been determined, the waste is grasped by a robot arm, collected at a collection location according to the material, and sorted.

Patent No. 5969685

However, in the conventional waste sorting device described above, even if the control unit repeatedly performs machine learning, the correct discrimination rate, which is the rate at which the material of waste is correctly discriminated, may not improve. Therefore, there is a problem in that the robot arm collects waste made of a material different from the material to be sorted, and the sorting accuracy of the waste sorting device is reduced. If the waste sorting accuracy is low, there is an inconvenience that it becomes difficult to use the sorted waste for recycling.

Therefore, an object of the present invention is to provide a waste sorting device that can effectively improve sorting accuracy.

In order to solve the above problems, the waste sorting device of the present invention has the following features:
a conveyance device having a conveyance surface on which waste is placed and conveyed;
an infrared irradiation device that irradiates infrared rays across the width direction of the conveyance surface of the conveyance device;
an infrared photographing device that outputs an infrared image obtained by photographing at least infrared reflected light of the waste transported by the transport device;
Using a discrimination model that performs machine learning using an infrared image of the waste prepared in advance and information on the material of the waste as training data, the material of the waste is discriminated based on the infrared image taken by the above-mentioned infrared imaging device. A discrimination section;
The present invention is characterized by comprising a sorting section that collects and sorts waste that has been determined to be of a predetermined material by the discriminating section from the conveying device.

According to the above configuration, the waste thrown into the waste sorting device is placed on the conveying surface of the conveying device and conveyed. An infrared ray irradiation device that irradiates infrared rays across the width direction of the conveyance surface of the conveyance device irradiates the waste conveyed by the conveyance device with infrared rays. The waste irradiated with infrared rays is photographed by an infrared photographing device and an infrared image is output. The discrimination section discriminates the material of the waste based on the infrared image. This discrimination unit performs machine learning of a discrimination model using an infrared image of the waste and material information indicating the material of the waste as training data. The discrimination model may perform machine learning by inputting an infrared image generated by photographing the waste transported by the transport device using the infrared imaging device and material information of the waste. Further, the discrimination model may be subjected to machine learning by inputting an infrared image of waste prepared in advance and material information of the waste. Machine learning of these discriminant models may be performed before the sorting section performs the sorting, after the sorting section performs the sorting, or during an interval when the sorting section performs the sorting. The waste whose material has been determined by the determining section is collected from the conveying device and sorted by the sorting section. In this way, since the material is discriminated based on image recognition using a discriminant model based on machine learning of infrared images, the material of waste can be discriminated with a high discrimination rate. Therefore, in this waste sorting device, the purity of the material of the sorted waste becomes higher than before, so that the sorted waste can be effectively recycled.

Further, a waste sorting device according to another aspect of the present invention includes:
a conveyance device having a conveyance surface on which waste is placed and conveyed;
a visible light photographing device that outputs a visible light image obtained by photographing the reflected light of visible light of the waste transported by the transport device;
an infrared irradiation device that irradiates infrared rays across the width direction of the conveyance surface of the conveyance device;
an infrared photographing device that outputs an infrared image obtained by photographing at least infrared reflected light of the waste transported by the transport device;
a first discrimination unit that discriminates the material of the waste based on the infrared spectral characteristics based on the infrared image of the waste;
For waste other than the waste determined to be of a predetermined material by the first discriminator, a discrimination model is created that performs machine learning using a visible light image of the waste prepared in advance and material information of this waste as training data. a second discrimination unit that discriminates the material of the waste based on the visible light image taken by the visible light imaging device;
The present invention is characterized by comprising a sorting section that collects and sorts waste that has been determined to be of a predetermined material by the first discriminating section or the second discriminating section from the conveying device.

According to the above configuration, the waste thrown into the waste sorting device is placed on the conveying surface of the conveying device and conveyed. The waste transported by the transport device is photographed by a visible light photographing device, and a visible light image is output. Further, the waste transported by the transport device is irradiated with infrared rays by an infrared irradiation device that irradiates infrared light across the width direction of the transport surface of the transport device. The waste irradiated with infrared rays is photographed by an infrared photographing device and an infrared image is output. The first discrimination unit discriminates the material of the waste based on the infrared spectral characteristics based on the infrared image. With respect to waste other than the waste determined to be of the predetermined material by the first determining section, the material of the waste is determined by the second determining section based on the visible light image. This second discrimination unit performs machine learning of a discrimination model using a visible light image of the waste and material information indicating the material of the waste as training data. The discrimination model may perform machine learning by inputting a visible light image generated by photographing the waste transported by the transport device using the visible light imaging device and material information of the waste. . Further, the discrimination model may perform machine learning by inputting a visible light image of waste prepared in advance and material information of the waste. Machine learning of these discriminant models may be performed before the sorting section performs the sorting, after the sorting section performs the sorting, or during an interval when the sorting section performs the sorting. The waste whose material has been determined by the first determining section or the second determining section is collected from the conveying device and sorted by the sorting section. In this way, the material is determined based on the spectral characteristics of infrared rays, and the material is determined based on image recognition using a classification model that performs machine learning of visible light images, so it is possible to identify the material of waste with a good classification rate. It can be determined by Here, a first discrimination section discriminates a predetermined material that can accurately discriminate materials based on infrared spectral characteristics, and other materials are discriminated by image recognition using a discrimination model based on machine learning of visible light images. This makes it possible to efficiently and accurately identify the material. Therefore, in this waste sorting device, the purity of the material of the sorted waste becomes higher than before, so that the sorted waste can be effectively recycled.

In one embodiment of the waste sorting device, the infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands.

According to the above embodiment, by using a multispectral image output by photographing waste with a multispectral camera, it is possible to accurately detect the spectral characteristics of waste from the visible light region to the infrared region. Therefore, the material of waste can be accurately determined. Here, the number and value of the wavelength bands are not limited to the multispectral camera as long as it outputs the intensity distribution of reflected light in a plurality of wavelength bands. Therefore, multispectral cameras include hyperspectral cameras. Furthermore, multispectral images include hyperspectral images. Furthermore, the multispectral image includes information that allows creation of an image showing the intensity distribution of reflected light in multiple wavelength bands. Therefore, based on the multispectral image, the spectral characteristics of the waste in the visible and infrared regions at a predetermined position within the imaging range can be obtained.

In one embodiment of the waste sorting device, the infrared imaging device is a line scan type multispectral camera.

According to the above embodiment, a multispectral image can be efficiently acquired by photographing waste with a line scan type multispectral camera. In particular, when a conveyance device carries waste by placing it on a conveyance surface that moves in a fixed direction, such as a belt conveyor, a line scan type multi-spectral camera is used to scan the conveyance surface of the conveyance device in the width direction. By scanning, a continuous multispectral image can be obtained in the direction of movement of the transport surface. Therefore, it is possible to eliminate the trouble of sequentially photographing by wrapping the photographic range and the process of sequentially joining adjacent images, which is required when photographing with an area type camera. In addition, line scan type multi-spectral cameras can specify the image capturing position to a single point in the moving direction of the conveyance surface of the conveyance device, so waste is not photographed as is the case with area type cameras. It is possible to prevent the inconvenience caused by variations depending on the position in the image during the time from when the image is scanned to when the image is sorted by the sorting section. As a result, the material of the waste can be determined with good accuracy based on the multispectral image, and the waste can be sorted with good accuracy.

In the waste sorting device of one embodiment, a plurality of the infrared irradiation devices are arranged such that a plurality of infrared generation sources are located in the width direction of the conveyance surface of the conveyance device.

According to the above embodiment, the plurality of infrared irradiation devices arranged such that the plurality of infrared ray generation sources are located in the width direction of the conveyance surface of the conveyance device cause the waste on the conveyance surface to be distributed with less deviation in the width direction. It can irradiate a large amount of infrared rays. Therefore, accurate spectral characteristics of the waste can be obtained based on an infrared image taken of the waste by an infrared imaging device.

In one embodiment of the waste sorting device, the infrared irradiation device has an infrared generation source extending in the width direction of the conveyance surface of the conveyance device.

According to the above embodiment, the infrared ray irradiation device having the infrared ray generation source extending in the width direction of the transport surface of the transport device can irradiate the waste on the transport surface with an amount of infrared rays that is not biased in the width direction. Therefore, accurate spectral characteristics of the waste can be obtained based on an infrared image taken of the waste by an infrared imaging device.

In the waste sorting device of one embodiment, the second discrimination unit receives metal information indicating whether the waste photographed by the visible light photographing device is metal, and outputs a visible light image of the waste. The material is determined using the above-mentioned discrimination model based on the metal information.

According to the above embodiment, regarding the waste photographed by the visible light photographing device and whose material is to be determined, metal information indicating whether the waste is metal or not is input to the second determination section. The metal information may be created by an operator who visually recognizes the visible light image, or may be created based on the output of a metal detection device installed around the transport device. The second discrimination unit can discriminate the material of the waste at a high discrimination rate by performing discrimination using a discrimination model based on the visible light image of the waste and the metal information of the waste.

The waste sorting device of one embodiment includes a metal detection device that detects whether the waste transported by the transport device is metal.

According to the above embodiment, the metal detection device detects whether or not the metal object transported by the transport device is metal, and the metal information indicating the detection result is inputted to the second discriminator, thereby disposing of the metal object. The material of an object can be determined with a high discrimination rate.

In the waste sorting device of one embodiment, the infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands,
If a plurality of materials determined based on the multispectral image are present in a single waste and the area corresponding to a predetermined material occupies a predetermined ratio, the first discrimination unit determines whether the waste is It is determined that the material is a predetermined material.

According to the above embodiment, when the first discrimination unit discriminates the material of the waste based on the multispectral image output from the multispectral camera as an infrared imaging device, a single waste has a plurality of materials. may be detected. Examples of such waste include glass bottles with paper or resin labels affixed to them, and wood with dirt and sand attached to them. In such waste, when the distribution area of materials determined based on multispectral images is expressed on a plane coordinate system, the material that occupies most of the volume, such as glass or wood, occupies a relatively small area. Therefore, when a predetermined material occupies a predetermined proportion, by determining that the material of this waste is the above-mentioned material, it is possible to accurately specify the main material of this waste. For example, in a material distribution image based on a multispectral image, if regions of glass and multiple materials other than glass, such as paper or resin, are detected, and the proportion of glass among these materials exceeds 40%, The waste is determined to be glass. Thereby, the material of the glass bottle to which the label is attached can be determined to be glass. Here, if the material that occupies the predetermined ratio occupies the largest ratio among the plurality of materials, it may be determined that the material is the material of this waste.

In the waste sorting device of one embodiment, the infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands,
The discrimination section discriminates the material of polyethylene, polypropylene, polyvinyl chloride resin, paper, concrete, stone, glass, wood, PET bottle, and polystyrene foam based on the multispectral image.

According to the above embodiment, the discrimination unit performs infrared spectral analysis and image processing based on a multispectral image taken by a multispectral camera as an infrared imaging device. The material is determined to be paper, concrete, stone, glass, wood, PET bottle, or polystyrene foam. Thereby, the overall correct classification rate of waste can be effectively increased.

In the waste sorting device of one embodiment, the infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands,
The first discrimination unit discriminates the material of polyethylene, polypropylene, polyvinyl chloride resin, and paper based on the multispectral image,
The second discrimination section discriminates the material of concrete, wood, glass, PET bottle, and polystyrene foam based on the visible light image.

According to the above embodiment, the first discrimination section performs an infrared spectral analysis based on a multispectral image taken by a multispectral camera as an infrared imaging device. One of the materials is determined. Further, the second discrimination unit discriminates the material among concrete, wood, glass, PET bottle, and polystyrene foam by image recognition based on the visible light image. Thereby, the overall correct classification rate of waste can be effectively increased.

In the waste sorting device of one embodiment, the infrared imaging device detects infrared rays having a wavelength of 1000 nm or more and 2350 nm or less to take pictures.

According to the above embodiment, the material of the waste can be determined with high accuracy based on the spectral characteristics related to infrared rays having a wavelength of 1000 nm or more and 2350 nm or less detected by the infrared imaging device.

BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic diagram which shows the waste sorting apparatus of 1st Embodiment of this invention. FIG. 2 is a schematic diagram showing the interior of the imaging chamber of the infrared imaging device along the transport direction of the transport device. FIG. 2 is a schematic diagram showing the interior of the imaging chamber of the infrared imaging device along a direction perpendicular to the transport direction of the transport device. FIG. 3 is a schematic diagram showing the inside of a photographing room in which another infrared irradiation device is installed. It is a block diagram showing a discrimination device and a control device of a waste sorting device of a 1st embodiment. It is a flow diagram showing a sorting method performed by the waste sorting device of the first embodiment. It is a schematic diagram showing the waste sorting device of a 2nd embodiment. It is a block diagram showing a discrimination device and a control device of a waste sorting device of a 2nd embodiment. It is a flowchart which shows the sorting method performed by the waste sorting apparatus of 2nd Embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to illustrated embodiments.

FIG. 1 is a schematic diagram showing a waste sorting device according to a first embodiment of the present invention. This waste sorting device sorts waste, such as municipal waste, which is a mixture of various materials such as concrete, glass bottles, metal scraps, plastics, wood chips, and waste paper, according to the material. The waste sorting device of this embodiment can be installed, for example, at a garbage disposal site, and can be used to sort waste collected from households, commercial facilities, etc., collect it by material, and reuse it. .

The waste sorting device 1 of the first embodiment includes a belt conveyor 3 as a conveying device that conveys waste 2, a 3D sensor 4 that measures the shape and height of the waste 2 on the belt conveyor 3, and a belt conveyor 3 that measures the shape and height of the waste 2 on the belt conveyor 3. a visible light camera 5 as a visible light photographing device for photographing the waste 2 on the belt conveyor 3; a hyperspectral camera 6 as an infrared photographing device for photographing the waste 2 on the belt conveyor 3; A metal sensor 19 is provided to detect whether the waste 2 is metal or non-metal. The 3D sensor 4, the visible light camera 5, the hyperspectral camera 6, and the metal sensor 19 are connected to a discrimination device 7 that discriminates the material of the waste 2 based on information input therefrom. The discrimination device 7 is connected to a control device 8, and the control device 8 controls a robot arm 9 as a sorting section and a belt conveyor 3 according to the discrimination result by the discrimination device 7. A robot arm 9 controlled by a control device 8 picks up the waste 2 on the belt conveyor 3 and puts it into a collection box 10 that collects the waste material separately.

In the belt conveyor 3, a conveyor belt wound around pulleys at both ends is driven by a pulley connected to a motor, and the waste 2 is placed on the conveying surface, which is the upward surface of the conveyor belt. configured to be transported. In this belt conveyor 3, a drive device 30 including a motor, a transmission, etc. is controlled by a control device 8, and the operation of conveying the waste 2 is controlled.

The 3D sensor 4 measures the shape and height of the waste 2 on the belt conveyor 3, and the measurement information is used for determining the objects to be sorted and controlling the robot arm 9. The 3D sensor 4 can be configured using a laser scanner. The 3D sensor 4 is placed inside a measurement chamber installed to surround the belt conveyor 3 above. Here, the 3D sensor 4 may be one using an optical method other than a laser scanner, or one using an acoustic method using ultrasound or the like.

The visible light camera 5 photographs the waste 2 on the belt conveyor 3 and outputs a visible light image, and the material of the waste 2 is discriminated by the discriminating device 7 based on this visible light image. The visible light camera 5 is arranged inside a photographing room that surrounds above the belt conveyor 3, and is designed to prevent the influence of external light. The visible light camera 5 is configured to take a still image every time the conveyor belt of the belt conveyor 3 travels a predetermined distance, and to take pictures of all the waste 2 on the conveyor belt. Note that the visible light camera 5 may be a line scan type camera that continuously photographs the conveying surface of the belt conveyor 3 and outputs continuous images in synchronization with the operating status of the belt conveyor 3.

The hyperspectral camera 6 acquires and records the intensity of reflected light from the object to be photographed for each of a plurality of wavelength bands belonging to visible light and near-infrared rays, and records the light intensity of the plurality of wavelength bands in two-dimensional coordinates. A hyperspectral image in which the information shown is superimposed and recorded is output. Thus, the hyperspectral image includes information in the infrared band and corresponds to the infrared image of the present invention. Note that if the hyperspectral camera 6 records light intensity for multiple wavelength bands from the wavelength of visible light to the wavelength of infrared rays, the number and values of the wavelength bands to be recorded are determined. It can be set as appropriate depending on the material. For example, the hyperspectral camera 6 can be one that captures visible light and infrared light with a wavelength of 1000 nm or more and 2350 nm or less with a resolution of about 5 to 15 nm. In particular, it is particularly preferable to use a hyperspectral camera 6 that photographs near-infrared rays of 1700 nm or more and 2000 nm or less because the accuracy of material discrimination improves. However, both the lower limit and upper limit of the wavelength photographed by the hyperspectral camera 6 may have other values. Further, as the infrared camera, a multispectral camera that records a narrower range of wavelengths than a hyperspectral camera and has lower wavelength resolution may be used. Note that hyperspectral cameras are included in multispectral cameras.

Moreover, it is preferable that the hyperspectral camera 6 is of a line scan type. The line scan type hyperspectral camera 6 scans the conveyance surface of the belt conveyor 3 in the width direction and outputs a hyperspectral image along the extending direction of the conveyance surface of the belt conveyor 3. The line scan type hyperspectral camera 6 can eliminate the trouble of sequentially photographing by wrapping the photographic range and the process of sequentially joining adjacent images, which is required when using an area type camera. Further, in the line scan type multispectral camera 6, the image capturing position is substantially specified at one point in the moving direction of the conveying surface of the belt conveyor 3. Therefore, it is possible to prevent the inconvenience of variations depending on the position in the image in the time from when the waste is photographed until it is sorted by the sorting section, which occurs when photographing with an area type camera. As a result, the material of the waste can be determined with good accuracy based on the multispectral image, and the waste can be sorted with good accuracy.

2 and 3 are schematic diagrams showing the inside of the infrared imaging room 21 in which the hyperspectral camera 6 is installed. 2 shows a cross section of the belt conveyor 3 along the conveyance direction, and FIG. 3 shows a cross section of the belt conveyor 3 along the width direction of the conveyance surface in a direction perpendicular to the conveyance direction. In this infrared imaging room 21, a plurality of halogen lamps 22 as infrared irradiation devices are arranged, and a hyperspectral camera 6 is arranged downstream of these halogen lamps 22 in the conveyance direction of the belt conveyor 3. . This infrared imaging room 21 is installed so as to surround the upper part of the conveyor belt conveying surface of the belt conveyor 3, and is designed to prevent the influence of external light on infrared imaging. A plurality of halogen lamps 22 are arranged in the width direction of the conveyor belt of the belt conveyor 3. Thereby, the amount of infrared rays irradiated onto the conveyance surface is made to be less uneven in the width direction of the conveyance surface. As a result, the waste 2 is irradiated with an even, predetermined amount of infrared rays regardless of its position on the conveyance surface, and accurate spectral characteristics can be obtained from the image captured by the hyperspectral camera 6. Although FIG. 3 shows an example in which four halogen lamps 22 are arranged in the width direction of the conveyor belt of the belt conveyor 3, the number of halogen lamps 22 may be any number.

FIG. 4 is a cross-sectional view showing a linear halogen lamp 24 having a linear infrared radiation source installed as another infrared irradiation device in the infrared imaging room 21. Similar to FIG. 3, FIG. 4 shows a cross section along the width direction of the conveying surface. By using the linear halogen lamps 24 extending in the width direction of the conveyance surface of the belt conveyor 3, it is possible to reduce the deviation in the amount of infrared rays irradiated in the width direction of the conveyance surface. Here, the infrared ray irradiation device may use other infrared generation sources in addition to the halogen lamps 22 and 24.

The metal sensor 19 detects whether the waste 2 on the belt conveyor 3 is metal or non-metal, and outputs metal/non-metal information as a detection result. The metal/nonmetal information is input to the discrimination device 7 and used to discriminate the material of the waste 2. The metal sensor 19 can be one that detects metal or non-metal based on various principles, such as one that uses electromagnetic waves or an electromagnetic field, or one that uses X-rays.

FIG. 5 is a block diagram showing the discrimination device 7 and the control device 8. As shown in FIG. The discrimination device 7 includes an information processing unit 11 that processes measurement information from the 3D sensor 4, visible light images captured by the visible light camera 5, and hyperspectral images output from the hyperspectral camera 6; a first discrimination unit 12 that discriminates the material of the waste 2 based on the hyperspectral image obtained by the processing unit 11; and a second discrimination unit 13 that discriminates the material of the waste 2 based on the visible light image processed by the information processing unit 11; , a storage unit 14 that stores information from the 3D sensor 4, visible light camera 5, and hyperspectral camera 6. The control device 8 includes a control section 16 that controls the robot arm 9 and the belt conveyor 3 based on the discrimination results by the discrimination sections 12 and 13, and an operation section 17 that receives operations related to the settings of the waste sorting device 1. .

The discrimination device 7 and the control device 8 both have a CPU (Central Processing Unit) and a storage device in which a computer program is stored, and each computer program is read out from the storage device and executed by the CPU. The functionality of the device is realized. Note that the storage section 14 of the discrimination device 7 can be formed of a magnetic disk, a semiconductor memory, or the like. Further, a part or all of the storage unit 14 may be separated from the discrimination device 7, placed at a position away from the installation position of the waste sorting device 1, and connected to the discrimination device 7 via an information network. In this case, a cloud server or a distributed ledger using blockchain technology can be used as the storage unit 14. Further, the operation unit 17 of the control device 8 can be formed of an input device such as a button, a touch sensor, a touch pad, or a dial, or a touch panel that also has a display function.

The information processing unit 11 of the discrimination device 7 receives measurement information from the 3D sensor 4, a visible light image taken by the visible light camera 5, a hyperspectral image taken by the hyperspectral camera 6, and detection information from the metal sensor 19. be done. Based on the measurement information of the 3D sensor 4, the information processing unit 11 extracts waste 2 that has a predetermined size and is arranged on the belt conveyor 3 in a grippable state as a sorting candidate. Regarding the size, those that are large enough to be gripped by the gripping section of the robot arm 9 are extracted. Moreover, regarding the arrangement state, one is extracted that allows the gripping part of the robot arm 9 to grip the other waste 2 or other members such as the belt conveyor 3 without interfering with them. The information processing unit 11 cuts out an image of the waste 2 extracted as a sorting candidate from the visible light image. When a plurality of waste materials 2 that are candidates for sorting are reflected in the visible light image, images of portions of the plurality of waste materials 2 are cut out. Regarding the waste material 2 as a sorting candidate whose image has been cut out, the plane position is specified based on the coordinates in the visible light image, and the height information among the measurement information of the 3D sensor 4 is specified and linked. Note that the size and arrangement of the waste 2 may be determined based on a visible light image taken by the visible light camera 5.

The information processing unit 11 calculates the center of gravity of the waste 2 extracted as a sorting candidate based on the part of the waste 2 in the visible light image and the height information, and also calculates the posture of the gripping part when gripped by the robot arm 9. Identify. Information on the center of gravity of the waste 2 and information on the posture of the gripping section when gripping the waste 2 are sent to the control section 16 of the control device 8 together with information on the material determined by the determination sections 12 and 13. It will be done.

Further, the information processing section 11 of the discrimination device 7 receives information indicating the amount of movement of the conveyor belt detected by the movement amount detection section installed on the belt conveyor 3 via the control section 16 of the control device 8 . As the movement amount detection section of the belt conveyor 3, an encoder or the like can be used. The information processing unit 11 controls the 3D sensor 4, the visible light camera 5, the hyperspectral camera 6, and the metal sensor 19 in accordance with the amount of movement of the conveyor belt to determine the timing of measurement and photography. By controlling the timing of the operation of the 3D sensor 4, visible light camera 5, and metal sensor 19 according to the amount of movement of the belt conveyor 3, the 3D sensor 4, visible light camera 5, hyperspectral camera 6, and metal sensor 19 acquire images. Based on this information, the robot arm 9 can accurately identify and sort the waste 2.

The information processing unit 11 of the discriminating device 7 cuts out the part of the waste 2 in the hyperspectral image of the waste 2 extracted as a sorting candidate, extracts it as a sorting candidate to be sorted, and displays the hyperspectral image of the cut out waste 2. It is output to the first determining section 12.

The first discrimination unit 12 of the discrimination device 7 analyzes the wavelength distribution of the reflected light based on the hyperspectral image of the waste 2 input from the information processing unit 11, and specifies the spectral characteristics of the waste 2. The first determining unit 12 identifies the material of the waste 2 based on the identified spectral characteristics. The first discrimination unit 12 identifies the material, and if the identified material is a predetermined material, the information processing unit 11 determines the plane position based on the coordinates in the visible light image cut out in the same way as in the first embodiment. is specified, and height information among the measurement information of the 3D sensor 4 is specified and linked. Further, the information processing unit 11 extracts a portion of the waste 2 from the visible light image, and calculates the center of gravity of the waste 2 based on the extracted portion of the visible light image and the height information. At the same time, the posture of the gripping portion when gripped by the robot arm 9 is specified. Information on the center of gravity of the waste 2 and information on the posture of the gripping section are sent to the control section 16 of the control device 8 together with information on the material determined by the first determination section 12. The control unit 16 controls the robot arm 9 based on the information regarding the center of gravity of the waste 2 and the posture of the gripping part of the robot arm 9 received from the information processing unit 11, and picks up the waste 2 from the belt conveyor 3 and moves it to a predetermined position. to the collection box 10.

This discrimination device 7 specifies the material in a first discrimination section 12 based on the hyperspectral image taken by the hyperspectral camera 6, and when the material specified by the first discrimination section 12 is other than a predetermined material, a second discrimination device 7 specifies the material. The discrimination unit 13 specifies the material based on the visible light image using a discrimination model based on machine learning. Therefore, the material of the waste 2 can be determined with high accuracy.

Here, when the first determining unit 12 determines that the material of the waste 2 is one of polyethylene, polypropylene, polyvinyl chloride resin, polystyrene foam, polyethylene terephthalate, and paper, The robot arm 9 can be configured to sort the waste 2 made of these materials. On the other hand, if the material determined by the first discrimination section 12 is neither polyethylene, polypropylene, polyvinyl chloride resin, polystyrene foam, polyethylene terephthalate, nor paper, the second discrimination section 13 discriminates the material of the waste 2. It can be set to Examples of the material to be determined by the second determining unit 13 include iron, aluminum, copper, stainless steel, concrete, wood, and glass. In this way, polyethylene, polypropylene, polyvinyl chloride resin, polystyrene foam, polyethylene terephthalate, and paper are discriminated by the first discrimination section 12, and iron, aluminum, copper, stainless steel, concrete, wood, glass, etc. are discriminated by the second discrimination section. 13, the material of the waste 2 can be determined with high accuracy. Note that wood and/or concrete may be discriminated by the first discrimination unit 12. Further, iron, aluminum, copper, and stainless steel may be detected only by the metal sensor 19 without using the second discrimination section 13.

When the first discrimination unit 12 detects a plurality of materials from a single waste 2, the information processing unit 11 preferably discriminates the material of the waste 2 based on the distribution ratio of each material. . For example, if glass and paper are detected in a single waste 2, there is a high possibility that it is a glass container with a label, and since the majority of the material is glass, the material of waste 2 is glass. It is determined that In this case, paper and glass are detected, and when the ratio of glass exceeds 40%, it can be determined that the object is glass. In addition, when multiple materials are detected in a single waste 2 by the first discrimination unit 12, the types and/or distribution ratios of the detected materials are extracted from the visible light image taken by the visible light camera 5. The type of the waste 2 may be specified by combining the shapes of the waste 2, and the material may be determined. With this, for example, a glass bottle with a paper or plastic label can be identified and the material can be determined to be glass.

On the other hand, the information processing section 11 performs processing for discriminating the material of the waste 2 other than that determined by the first discriminating section 12 as being made of a predetermined material by the second discriminating section 13 . In other words, the information processing unit 11 determines whether the waste 2 whose material was not determined by the first discrimination unit 12 or whose material is predetermined not to be sorted based on the discrimination result by the first discrimination unit 12. A portion of the waste 2 is cut out from the visible light image, and the cut out image of the waste 2 is output to the second discrimination section 13. Further, the information processing unit 11 specifies the plane position based on the coordinates in the cut out visible light image, and also specifies and links the height information of the measurement information of the 3D sensor 4, and 2 is calculated, and the posture of the gripping portion when gripped by the robot arm 9 is specified.

When the second discrimination unit 13 receives the visible light image of the waste 2 from the information processing unit 11, it discriminates the material of the waste 2 using image recognition technology based on the visible light image of the waste 2. The second discriminator 13 is configured using a neural network that grows an algorithm through supervised learning as a discriminator model. The second discriminator 13 receives an image of the waste 2, performs calculations using a neural network, and outputs the material of the waste 2. Note that the second discriminator 13 may use a support vector machine (SVM) or the like based on another algorithm that performs supervised learning as the discriminant model. Further, the neural network of the second discriminator 13 may have a multiple structure and perform deep learning. Further, the second determination unit 13 may execute the processing using a neural network using software or hardware.

The neural network of the second discriminator 13 performs machine learning in a learning mode before performing the work of sorting the waste 2. In the learning mode, an image of waste is input as training data, and information on the material of the image of waste is input as the correct answer. For a plurality of materials to be sorted by the waste sorting device 1, a plurality of combinations of images showing various forms of waste and correct information indicating the material of this waste are prepared, and these plural combinations are prepared. Perform machine learning using combined images and information.

The control unit 16 of the control device 8 controls the robot arm 9 and the belt conveyor 3 based on information from the discrimination device 7. Further, information indicating the amount of movement of the conveyor belt detected by the movement amount detection section of the belt conveyor 3 is output to the discrimination device 7. The control unit 16 also controls the waste 2 identified by the information processing unit 11 as a sorting target based on the information regarding the center of gravity of the waste 2 and the posture of the gripping portion of the robot arm 9 received from the information processing unit 11. , controls the robot arm 9 to carry out an operation of picking up the waste 2 from the belt conveyor 3 and putting it into a collection box 10 according to the material.

The robot arm 9 is constituted by, for example, a vertical articulated robot, and uses a gripper provided at its tip to grasp and pick up the waste 2 on the belt conveyor 3, and selects settings for each material depending on the material of the picked up waste 2. and put it into the collected collection box 10. As the robot arm 9, other types of robots such as parallel link robots may be used in addition to the vertical articulated robot.

The waste sorting device 1 configured as described above operates in a learning mode and a discrimination mode depending on the processing performed by the second discrimination section 13. That is, the second discrimination unit 13 performs processing in a learning mode in which learning is performed to discriminate waste 2 and in a discrimination mode in which discrimination is performed for sorting waste 2.

In the learning mode, the waste 2 is transported by the belt conveyor 3 of the waste sorting device 1, and the visible light image taken by the visible light camera 5 is processed by the information processing unit 11 to extract parts of the waste 2. Information regarding the material of the waste 2 is input as a correct answer by the operator through the operation unit 17 of the control device 17. Supervised learning can be performed by inputting the image of the waste 2 and the correct information regarding the material to the second discriminator 13. Information regarding the material of the waste 2 as the correct answer may be input by connecting an input section to the discriminating device 7 and performing the input through this input section.

Further, in the learning mode, the operator may input information regarding the material of the waste 2 as the correct answer at a position different from the installation position of the waste sorting device 1. In this case, a communication device is connected to the waste sorting device 1, and an information input device having an image display device, an input section, and a communication section is installed at a position where information is input. The communication device of the waste sorting device 1 and the communication section of the information input device are connected via an information network, and images of the waste 2 are transmitted to the information input device. The image of the waste 2 is displayed on the display section of the information input device, and the operator visually checks the image of the waste 2 displayed on the display section, judges the material of the waste 2, and then inputs the information into the input device. Enter information about the material in . The information input to the input section is transmitted from the information input device to the waste sorting device 1, and inputted together with the image of the waste 2 to the second discrimination section 13 as teacher data. In this way, an operator located at a distance from the waste sorting device 1 determines the material of the waste 2 and transmits information regarding the material to the second discriminating section 13 of the waste sorting device 1. Machine learning can be performed. By placing the information input device in, for example, a workplace for the disabled, an occupational therapy facility, etc., it is possible to provide the task of determining and inputting the material of the waste 2.

In addition, in the learning mode, in addition to performing machine learning based on the photographed image of the waste 2 on the belt conveyor 3, an image of the waste and information indicating the material of the waste in this image are prepared in advance. , these images and information may be input to the second discriminator 13 to perform machine learning. In addition, a computer including a neural network similar to that of the second discriminator 13 is installed at a location different from the waste sorting device 1, and this computer performs machine learning based on the image of the waste and information indicating the material. The parameters obtained by machine learning may be read into the second discrimination section 13 of the waste sorting device 1. Further, the parameters obtained by performing machine learning using the computer described above may be shared with a plurality of waste sorting devices 1.

The second discriminator 13, which has performed machine learning in the learning mode, receives the image of the waste 2 output from the information processing section 11 in the discriminator mode, performs calculations according to an algorithm, and outputs the material. Here, the metal/non-metal information output from the metal sensor 19 is input to the second discrimination unit 13 together with the image of the waste 2 output from the information processing unit 11, and processing is performed by the neural network. In this way, by performing the discrimination based on the image of the waste 2 and the metal/non-metal information, the accuracy of the material discrimination result by the second discriminator 13 can be checked based only on the image of the waste 2. You can improve more than you do.

In addition, the second discrimination unit 13 selects waste 2 whose material has reached a predetermined correct discrimination rate in a discrimination test conducted in advance, among the waste 2 whose materials have been discriminated in the discrimination mode. The information necessary for the operation of the robot arm 9 is outputted to the control device 8 with the robot arm 9 as the object of selection. As the information necessary for the operation of the robot arm 9, information regarding the material of the waste 2, information regarding the center of gravity of the waste 2, information regarding the posture of the gripping portion of the robot arm, etc. can be employed. The discrimination test for specifying the correct discrimination rate may be performed after machine learning is performed in the learning mode of the second discrimination unit 13. Further, the information processing unit 11 may output the information necessary for the operation of the robot arm 9 to the control device 8 in response to the determination result by the second determination unit 13. Further, the value of the correct discrimination rate for the waste 2 to be sorted can be set to various values such as 80% or more, 90% or more.

Table 1 shows that in the discrimination mode, the second discriminating section 13 is a table showing the difference in the correct discrimination rate showing the proportion of correct answers in the material discrimination results according to No. 13. Here, the correct classification rate is 1000 pieces of waste containing a mixture of aluminum, copper, iron, SUS (stainless steel), glass, concrete, wood, paper, plastic, polyvinyl chloride resin, PET bottles, and polystyrene foam. The material was determined by the second determining unit 13, and the proportion of the number correctly determined out of the number determined to be of a predetermined material was calculated. In addition, the second discrimination unit 13 determines whether aluminum, copper, iron, SUS, glass, concrete, wood, paper, plastic, PET bottles, polystyrene foam (expanded styrene foam), and PVC (polyvinyl chloride resin) are determined based on teacher data in advance. Performed machine learning. In Table 1, the numerical unit is % and indicates the correct discrimination rate of the discriminated materials. Here, the plastic is a plastic other than polyvinyl chloride resin, polystyrene foam, and polyethylene terephthalate.

As can be seen from Table 1, in the discrimination mode, by inputting the metal/nonmetal information to the second discrimination section 13 together with the image of the waste 2, it is possible to improve the correct discrimination rate of the material.

Note that Table 1 shows that aluminum, copper, iron, and SUS were not judged by the second discrimination unit 13 in response to their being detected as metal by the metal sensor 19; You may do so. Furthermore, among the materials listed in Table 1, plastic, polyvinyl chloride resin, PET bottles, and polystyrene foam may be discriminated by the first discrimination section 12.

FIG. 6 is a flow diagram showing the operation of the waste sorting device 1 of the first embodiment. As shown in FIG. 6, the waste sorting device 1 of this embodiment first uses a visible light image of waste and the material of this waste in order to perform machine learning in the second discrimination section 13 of the discrimination device 7. Prepare teacher data including information indicating (step S11). To prepare the teacher data, the waste 2 is conveyed by the belt conveyor 3 of the waste sorting device 1, and the visible light image obtained by photographing the waste 2 by the visible light camera 5 is sent to the information processing unit 11 of the discriminating device 7. At the same time as the extraction, information indicating the material of the waste 2 may be inputted from the outside. Moreover, images of waste obtained by other methods and information indicating the material of this waste may be prepared as training data. After preparing the teacher data, the second discriminator 13 enters the teacher mode, receives input of the teacher data, and performs machine learning on waste materials of a plurality of materials to be discriminated (step S12). After the second discriminator 13 performs machine learning, the waste 2 is actually put into the waste sorting device 1, and a test is performed in which the second discriminator 13 discriminates the material of the waste 2. This discrimination test is performed on multiple wastes of the material to be discriminated. The results of the discrimination of the waste 2 by the second discriminator 13 are totaled, it is confirmed whether the material can be accurately discriminated, and the correct discrimination rate for each material is calculated. Further, in addition to the discrimination test by the second discrimination section 13, the discrimination test by the first discrimination section 12 is performed. In the discrimination test performed by the first discriminator 12, the waste 2 is put into the waste sorting device 1, the waste 2 is photographed by the hyperspectral camera 6, and the first discriminator 12 determines the waste 2 based on the hyperspectral image. Determine the material. The results of the discrimination of the waste 2 by the first discrimination section 12 are totaled, and it is confirmed whether the material can be accurately discriminated, and the correct discrimination rate for each material is calculated. Based on these, the correct discrimination rate for each material is calculated for all the materials to be discriminated by the discrimination device 7. (Step S13). Thereafter, the second discriminating section 13 enters the discriminating mode, and the waste 2 is thrown into the starting end of the belt conveyor 3 as shown by arrow E. The waste 2 that is fed onto the belt conveyor 3 and transported is determined whether it is metal or non-metal by the metal sensor 19 (step S14), and its shape is measured by the 3D sensor 4 to obtain three-dimensional information ( Step S15), a visible light image is acquired by the visible light camera 5 (step S16), and a hyperspectral image is acquired by the hyperspectral camera 6 (step S17). Based on this hyperspectral image, the first discrimination section 12 discriminates the material of the waste 2 (step S18). When the material of the waste 2 is determined by the first discriminator 12, it is determined whether or not this material is a predetermined material (step S19), and if it is a predetermined material, the waste 2 of this material is If the material is identified as having a correct discrimination rate of 80% or more in the discrimination test, the discrimination device 7 collects information on the material of the waste 2, information on the center of gravity of the waste 2, and the gripping part of the robot arm. information regarding the posture of is output to the control unit 16 of the control device 8. When the control unit 16 receives information regarding the material of the waste 2, information regarding the center of gravity of the waste 2, and information regarding the posture of the gripping portion of the robot arm, the control unit 16 controls the robot arm 9 based on these information, and controls the belt. The waste 2 on the conveyor 3 is picked up and put into a collection box 10 corresponding to the material, and sorted (step 21). In step S19, if the material discriminated by the first discriminator 12 is not the predetermined material, the metal/non-metal information and the visible light image are input to the neural network of the second discriminator 13, and the material of the waste 2 is is determined (step S20). Among the waste 2 whose materials have been discriminated by the neural network of the second discriminator 13, the waste 2 whose material is identified as having a correct discrimination rate of 80% or more in the discrimination test is collected from above the belt conveyor 3 by the robot arm 9. The materials are picked up, put into a collection box 10 corresponding to the material, and sorted (step 21). Of the waste 2 on the belt conveyor 3, the waste 2 that is not picked up by the robot arm 9 is discharged from the terminal end of the belt conveyor 3 as shown by arrow F1, and returned by a conveyor or other conveyor (not shown). Then, as shown by arrow F2, it is again thrown into the starting end of the belt conveyor 3. The discrimination device 7 determines whether waste 2 to be sorted remains on the belt conveyor 3 based on the three-dimensional information from the 3D sensor 4 and/or the visible light image from the visible light camera 5 (step S22), when the waste 2 to be sorted remains on the belt conveyor 3, the control device 8 continues the operation of the belt conveyor 3, so that the waste 2 on the belt conveyor 3 is processed through steps S14 to S21. is repeated. On the other hand, when there is no waste 2 to be sorted on the belt conveyor 3, the operation of the belt conveyor 3 is stopped by the control device 8, and the sorting work of the waste sorting device 1 is completed.

As described above, according to the waste sorting device 1 of the present embodiment, the first discriminator 12 discriminates the predetermined material of the waste 2 based on the infrared spectral characteristics, and the second Since the discrimination unit 13 performs discrimination based on image recognition using a discrimination model that performs machine learning on visible light images, the material of the waste 2 can be accurately discriminated. Furthermore, among the waste materials 2 classified by the first discriminating section 12 and the second discriminating section 13, the waste materials having a correct discrimination rate of 80% or more were sorted, so they were sorted and placed in the collection box 10. The purity of the collected waste 2 can be effectively increased. Therefore, the waste sorted and collected by this waste sorting device 1 can be effectively and efficiently recycled. It should be noted that among the waste 2 of materials discriminated by the first discriminator 12 and the second discriminator 13, the waste 2 of materials with a correct discrimination rate of 80% or more was sorted, but the correct discrimination rate of the material to be sorted was The criterion for may be other values than 80%. In addition, the correct discrimination rate is determined by considering only the material discriminated by the second discrimination unit 13, without considering the correct discrimination rate of the material discriminated by the first discrimination unit 12. All the waste materials 2 made of a predetermined material may be sorted.

In addition, the waste sorting device 1 of the present embodiment can transfer the waste 2 that has not been selected as a sorting target by the first discrimination section 12 and the second discrimination section 13 to the belt conveyor 3 again and circulate it. The arrangement on the conveyor 3 can be changed. As a result, it is possible to correctly identify and sort the waste 2 whose material was not initially determined by changing the arrangement state, so that it is possible to effectively reduce the amount of waste 2 left unsorted.

FIG. 7 is a schematic diagram showing a waste sorting device 101 according to a second embodiment of the present invention. The waste sorting device 101 of the second embodiment is different from the first embodiment in that it includes only the hyperspectral camera 6 as an imaging device, and the discrimination device 34 discriminates the material based only on the image taken by the hyperspectral camera 6. different from. In the second embodiment, parts having the same functions as those in the first embodiment are given the same reference numerals as in the first embodiment, and detailed description thereof will be omitted.

The following will explain the waste sorting device 101 of the second embodiment, focusing on the differences from the waste sorting device 1 of the first embodiment.

The waste sorting device 101 of the second embodiment includes a hyperspectral camera 6 as an infrared imaging device that photographs the waste 2 on the belt conveyor 3, and a hyperspectral camera 6 that measures the shape and height of the waste 2 on the belt conveyor 3. It includes a 3D sensor 4 and a metal sensor 19 that detects whether the waste 2 transported by the belt conveyor 3 is metal or non-metal. The hyperspectral camera 6, the 3D sensor 4, and the metal sensor 19 are connected to a discrimination device 34 that discriminates the material of the waste 2 based on information input therefrom. The discrimination device 34 is connected to a control device 8 that controls the robot arm 9 and the belt conveyor 3.

The hyperspectral camera 6 of the waste sorting device 101 of the second embodiment may be one that photographs infrared rays with a wavelength of 1000 nm or more and 2350 nm or less for each bandwidth of about 5 nm. Here, it is preferable that the hyperspectral camera 6 photographs light included in near-infrared rays. Moreover, both the lower limit and upper limit of the wavelength photographed by the hyperspectral camera 6 may be other values. In particular, it is particularly preferable to use a hyperspectral camera 6 that photographs near-infrared rays of 1700 nm or more and 2000 nm or less because the accuracy of material discrimination improves. Moreover, the bandwidth for imaging, that is, the wavelength resolution, may have any value.

As shown in the block diagram of FIG. 8, the discrimination device 34 includes an information processing section 35 that processes the hyperspectral image taken by the hyperspectral camera 6 and the measurement information of the 3D sensor 4, and the information processing section 35 that processes the hyperspectral image. It has a determination unit 36 that determines the material of the waste 2 based on processed information or images, and a storage unit 14 that stores information from the 3D sensor 4 and hyperspectral camera 6.

When the information processing unit 35 of the discriminating device 34 extracts the waste 2 that is a selection candidate in the same way as the information processing unit 11 of the first embodiment, the information processing unit 35 cuts out the part of the waste 2 in the hyperspectral image of the extracted waste 2. , and outputs a hyperspectral image of the waste 2 extracted and cut out as a selection candidate to the discrimination unit 36.

The discrimination unit 36 of the discrimination device 34 identifies the material of the waste 2 based on the hyperspectral image of the waste 2 through image recognition using machine learning. First, the discrimination unit 36 creates a discrimination image based on the hyperspectral image of the waste 2 input from the information processing unit 35. As the discrimination image, an image representing a reflection intensity distribution in a predetermined wavelength band, an image obtained by converting data in one or more wavelength bands according to a predetermined method, etc. can be used. The discrimination image is an image in which the distribution of materials is expressed on the two-dimensional coordinates of the photographing range using colors set according to the materials. This discrimination image is input to a discrimination model included in the discrimination section 36, and the material of the waste 2 is discriminated by image recognition technology. The discriminator 36 is configured using a neural network that grows an algorithm through supervised learning as a discriminator model. Similar to the first embodiment, this discrimination section 36 performs machine learning in the learning mode, and discriminates the material of the waste 2 in the discrimination mode. Among the materials discriminated in the discrimination mode of the discriminator 36, information necessary for the operation of the robot arm 9 is outputted to the control device 8 for the waste 2 having a correct discrimination rate equal to or higher than a predetermined value. Note that the information necessary for the operation of the robot arm 9 may be outputted to the control device 8 by the information processing unit 35 upon receiving the determination result by the determination unit 36. Further, the value of the correct discrimination rate for the waste 2 to be sorted can be set to various values such as 80% or more, 90% or more.

The information necessary for the operation of the robot arm 9 is determined by the information processing unit 35 based on the plane position specified based on the coordinates of the waste 2 in the hyperspectral image and the height information among the measurement information of the 3D sensor 4. formed on the basis of Specifically, the center of gravity of the waste 2 is calculated by the information processing unit 35 based on the portion of the waste 2 extracted from the hyperspectral image and the height information. At the same time, the posture of the gripping portion when gripped by the robot arm 9 is specified. Information on the center of gravity of the waste 2 and information on the posture of the gripping section are sent to the control section 16 of the control device 8 together with information on the material determined by the determination section 36. The control unit 16 controls the robot arm 9 based on the information regarding the center of gravity of the waste 2 and the posture of the gripping part of the robot arm 9 received from the information processing unit 35, and picks up the waste 2 from the belt conveyor 3 and moves it to a predetermined position. to the collection box 10.

The waste sorting device 101 of the second embodiment identifies the material by a discrimination model that performs machine learning in the discrimination unit 36 based on the hyperspectral image taken by the hyperspectral camera 6, so the material of the waste 2 can be identified with high accuracy. can be determined.

Here, the discrimination unit 36 selects the material of the waste 2 as the above-mentioned predetermined material, such as polyethylene, polypropylene, polyvinyl chloride resin, polystyrene foam, polyethylene terephthalate, paper, concrete, stone, glass, wood, and polystyrene foam. If it is determined that it is one of these materials, the robot arm 9 can be set to sort out the waste 2 made of these materials. Moreover, iron, aluminum, copper, and stainless steel can be determined based on the detection information of the metal sensor 19.

FIG. 9 is a flow diagram showing the operation of the waste sorting device 101 of the second embodiment. As shown in FIG. 9, the waste sorting device 101 of this embodiment first uses a discrimination image created from a hyperspectral image of waste, and a discrimination image created from a hyperspectral image of waste, in order to perform machine learning in the discrimination unit 36 of the discrimination device 34. Teacher data including information indicating the material of the waste is prepared (step S31). To prepare the teacher data, the waste 2 is conveyed by the belt conveyor 3 of the waste sorting device 101, and the hyperspectral image obtained by photographing the waste 2 by the hyperspectral camera 6 is sent to the information processing unit 35 of the discrimination device 34. In addition to processing to create a discrimination image, information indicating the material of the waste 2 may be input from the outside. Further, as training data, an image for discriminating waste obtained by another method and information indicating the material of this waste may be prepared. After preparing the teacher data, the discriminator 36 enters the teacher mode, receives the input of the teacher data, and performs machine learning on the plurality of waste materials to be discriminated (step S32). After the discriminating unit 36 performs machine learning, the waste 2 is actually put into the waste sorting device 101, and a test is performed in which the discriminating unit 36 discriminates the material of the waste 2. This discrimination test is performed on multiple wastes of the material to be discriminated. The results of the discrimination of the waste 2 by the discrimination section 36 are totaled, and it is confirmed whether the material can be accurately discriminated, and the correct discrimination rate for each material is calculated. (Step S33). Thereafter, the discriminating section 36 enters the discriminating mode, and the waste 2 is thrown into the starting end of the belt conveyor 3 as indicated by arrow E. The metal sensor 19 determines whether the waste 2 to be transported by the belt conveyor 3 is metal or non-metal (step S34), and the 3D sensor 4 measures the shape to obtain three-dimensional information ( Step S35), a hyperspectral image is acquired by the hyperspectral camera 6 (step S36). Based on this hyperspectral image, a discrimination image is created by the discrimination section 36, and this discrimination image is input to the neural network of the discrimination section 36 to discriminate the material of the waste 2 (step S37). When the material of the waste 2 is determined by the discriminator 36, if the material of the waste 2 made of this material is identified as having a correct discrimination rate of 80% or more in the discrimination test, the discriminator 34 determines whether the material is discarded. Information regarding the material of the object 2, information regarding the center of gravity of the waste 2, and information regarding the posture of the gripping portion of the robot arm are output to the control unit 16 of the control device 8. When the control unit 16 receives information regarding the material of the waste 2, information regarding the center of gravity of the waste 2, and information regarding the posture of the gripping portion of the robot arm, the control unit 16 controls the robot arm 9 based on these information, and controls the belt. The waste 2 on the conveyor 3 is picked up and put into a collection box 10 corresponding to the material, and sorted (step 38). Of the waste 2 on the belt conveyor 3, the waste 2 that is not picked up by the robot arm 9 is discharged from the terminal end of the belt conveyor 3 as shown by arrow F1, and returned by a conveyor or other conveyor (not shown). Then, as shown by arrow F2, it is again thrown into the starting end of the belt conveyor 3. The discrimination device 34 determines whether waste 2 to be sorted remains on the belt conveyor 3 based on the three-dimensional information from the 3D sensor 4 and/or the hyperspectral image from the hyperspectral camera 6 (step S39), if the waste 2 to be sorted remains on the belt conveyor 3, the control device 8 continues the operation of the belt conveyor 3, so that the waste 2 on the belt conveyor 3 is processed through steps S34 to S38. is repeated. On the other hand, when there is no waste 2 to be sorted on the belt conveyor 3, the operation of the belt conveyor 3 is stopped by the control device 8, and the sorting work of the waste sorting device 101 is completed.

As described above, according to the waste sorting device 101 of the present embodiment, a predetermined material of the waste 2 is determined using a discrimination image created from a hyperspectral image using a discrimination model using a neural network of the discrimination unit 36. Since the discrimination is based on image recognition, the material of the waste 2 can be accurately discriminated. In addition, since the waste 2 having a correct discrimination rate of 80% or more is selected from among the waste 2 having the materials discriminated by the discriminating unit 36, the purity of the waste 2 that is sorted and collected in the collection box 10 is increased. can be effectively increased. Therefore, the waste sorted and collected by this waste sorting device 101 can be effectively and efficiently recycled. Note that among the waste materials 2 that have been determined by the discriminator 36, the waste materials that have a correct classification rate of 80% or more are selected, but the standard for the correct classification rate of the materials to be sorted is that Other values may also be used.

In the first and second embodiments described above, waste 2 made of materials with a correct classification rate of 80% or more by the first discrimination unit 12, second discrimination unit 13, or discrimination unit 36 is sorted, but it is not possible to classify waste 2 based on the correct discrimination rate. The waste 2 may be sorted without any waste. In this case, there is no need to perform a test to determine the correct classification rate.

In addition, in the first and second embodiments, the waste 2 on the belt conveyor 3 that is not picked up by the robot arm 9 is discharged from the terminal end of the belt conveyor 3 as shown by the arrow F1. The waste 2 that was not picked up by the robot arm 9 is returned to the starting end of the belt conveyor 3 as shown by arrow F2, and is returned to the belt conveyor 3 by a transport device such as a conveyor (not shown). You don't have to. That is, among the waste 2 thrown into the belt conveyor 3, the waste 2 that is not picked up by the robot arm 9 is discharged from the end of the belt conveyor 3 as shown by the arrow F1. It may be discarded or subjected to other processing without being reintroduced.

In addition, if the configuration is such that the waste 2 that has not been picked up by the robot arm 9 is re-injected onto the belt conveyor 3, as time passes, the waste 2 of unidentifiable material will be deposited on the belt conveyor 3. Since they accumulate, it is preferable to remove these waste materials 2 periodically. In addition, when new waste 2 is thrown onto the belt conveyor 3 as shown by arrow E, it is not re-injected, while when new waste 2 is not put on the belt conveyor 3, waste is thrown as shown by arrow F2. 2 may be reintroduced.

In the first and second embodiments described above, the case where the waste material 2 is used for sorting municipal waste has been exemplified, but the present invention can also be applied to sorting other wastes such as rubble during disasters. It is also possible.

Although the present invention has been described above through the embodiments, various changes can be made without departing from the gist of the present invention, and the technical scope of the present invention is not limited to the above embodiments.

1,101 Waste sorting device 2 Waste 3 Belt conveyor 4 3D sensor 5 Visible light camera 6 Hyperspectral camera 7,34 Discrimination device 8 Control device 9 Robot arm 11,35 Information processing section 12 First discrimination section 13 Second discrimination Section 14 Storage section 16 Control section 17 Operation section 19 Metal sensor 21 Infrared imaging room 30 Belt conveyor drive device 36 Discrimination section

Claims (10)

a conveyance device having a conveyance surface on which waste is placed and conveyed;
a visible light photographing device that outputs a visible light image obtained by photographing the reflected light of visible light of the waste transported by the transport device;
an infrared irradiation device that irradiates infrared rays across the width direction of the conveyance surface of the conveyance device;
an infrared photographing device that outputs an infrared image obtained by photographing at least infrared reflected light of the waste transported by the transport device;
a first discrimination unit that discriminates the material of the waste based on the infrared spectral characteristics based on the infrared image of the waste;
For waste other than the waste determined to be of a predetermined material by the first discriminator, a discrimination model is created that performs machine learning using a visible light image of the waste prepared in advance and material information of this waste as training data. a second discrimination unit that discriminates the material of the waste based on the visible light image taken by the visible light imaging device;
A waste sorting device comprising: a sorting section that collects and sorts waste that has been determined to be of a predetermined material by the first discriminating section or the second discriminating section from the conveying device. The waste sorting device according to claim 1,
A waste sorting device characterized in that the infrared photographing device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands. The waste sorting device according to claim 2,
A waste sorting device characterized in that the infrared photographing device is a line scan type multispectral camera. The waste sorting device according to claim 1,
A waste sorting device characterized in that a plurality of the infrared ray irradiation devices are arranged such that a plurality of infrared generation sources are located in the width direction of the conveyance surface of the conveyance device. The waste sorting device according to claim 1,
A waste sorting device characterized in that the infrared ray irradiation device has an infrared generation source extending in the width direction of the conveyance surface of the conveyance device. The waste sorting device according to claim 1,
Metal information indicating whether the waste photographed by the visible light photographing device is metal is input, and the second discriminating unit uses the discriminating model to identify the material based on the visible light image of the waste and the metal information. A waste sorting device characterized by discriminating. The waste sorting device according to claim 6,
A waste sorting device comprising a metal detection device that detects whether the waste transported by the transport device is metal. The waste sorting device according to claim 1,
The infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands,
If a plurality of materials determined based on the multispectral image are present in a single waste and the area corresponding to a predetermined material occupies a predetermined ratio, the first discrimination unit determines whether the waste is A waste sorting device characterized by determining that the material is made of a predetermined material. The waste sorting device according to claim 1,
The infrared imaging device is a multispectral camera that outputs a multispectral image showing the intensity distribution of the reflected infrared light for each of a plurality of wavelength bands,
The first discrimination unit discriminates the material of polyethylene, polypropylene, polyvinyl chloride resin, and paper based on the multispectral image,
A waste sorting device characterized in that the second discrimination section discriminates the material of concrete, wood, glass, PET bottle, and polystyrene foam based on the visible light image. The waste sorting device according to any one of claims 1 to 9,
A waste sorting device characterized in that the infrared imaging device detects infrared rays having a wavelength of 1000 nm or more and 2350 nm or less to take pictures.

Images