KR102651149B1 - System for classifying target objects using a multi-class classification table - Google Patents

Abstract

The present invention includes a data generator that generates data from images of wastes obtained from a vision camera that collects images of wastes moving on a conveyor belt; A target object determination unit that uses an artificial intelligence model to determine whether the waste image is a target object to be classified based on a waste classification standard table having multiple classes; and a robot designation unit that designates a robot to classify the target object using the waste classification standard table. It provides a target object classification system using a multi-class classification standard table.

Description

Target object classification system using a multi-class classification standard {SYSTEM FOR CLASSIFYING TARGET OBJECTS USING A MULTI-CLASS CLASSIFICATION TABLE}

The present invention relates to an object classification system for classifying waste, and more specifically, to an object classification system using a multi-class classification standard table for classifying objects using a multi-class classification standard table.

A large amount of waste is discharged from homes, factories, restaurants, etc. every day, and many people separate and dispose of food waste among general waste, recyclable waste, and food waste.

However, recyclable waste is still discharged mixed with general waste, and manpower is directly invested in separating it.

In addition, in companies that use plastic bottles or glass bottles, recycling waste is not properly separated and is discarded as general waste, making it even more problematic to sort recycling waste.

Various inventions are being made to solve these problems, and methods, devices, and systems for separating various wastes are being applied for and registered as patents.

The inventor of the present invention previously attempted to classify waste based on classification information including the type, degree of contamination, color, and presence or absence of a lid and label in the registered patent publication No. 10-2222177, “Recycling Sorting System Using AI.” A method for identifying waste has been disclosed.

However, in this classification standard, the types and conditions of waste were so diverse that it was difficult to specify target objects one by one, and there were limitations in directly labeling and updating classification information for all such waste.

In addition, there was a problem that waste sorting efficiency was low because all waste to be sorted was assigned to the same robot.

Korean Patent No. 10-2222177 (announcement date, 2021.03.03)

The technical task to be achieved by the present invention is to provide a target object classification system using a waste classification standard table that allows users to easily specify target objects and a multi-class standard table that is easy to upgrade.

In addition, another technical task to be achieved by the present invention is to provide an object classification system using a multi-class standard that can improve waste classification efficiency through matching and combining the object to be classified with a plurality of robots.

In order to solve this problem, the target object classification system using a multi-class classification standard table according to an embodiment of the present invention has a data generation unit that generates data from the waste image obtained from a vision camera that collects the image of the waste moving on the conveyor belt. ; A target object determination unit that uses an artificial intelligence model to determine whether the waste image is a target object to be classified based on a waste classification standard table having multiple classes; and a robot designation unit that designates a robot to classify the target object using the waste classification standard table.

The artificial intelligence model can be learned using waste images labeled according to a preset waste classification standard as learning data.

The waste classification standard table may include multiple classes, each class having its own classification standard.

The first class of the multiple classes is a category according to the material composition of the waste, the second class is a category according to the state of the waste (damaged or not), the third class is a category according to the use of the waste, and the fourth class is a category according to the color of the waste. The category on which it is based, and the fifth class, may include waste not included in the first to fourth classes.

The target object determination unit may determine whether the object is the target object according to the class of the waste classification standard table selected by the user.

It may further include a waste classification standard generating unit that adds a new classification attribute to the waste classification standard table based on the new classification attribute.

The waste classification standard generator may derive new classification attributes from the waste images classified into the fifth class using an artificial intelligence model, or may upgrade the waste classification standard table using new classification attributes input from the user.

It may further include an artificial intelligence model learning unit that automatically labels new classification attributes added to the waste image classified into the fifth class and retrains the artificial intelligence model.

When a plurality of robots are used, the robot designator may match the target object and the robot by considering the type and number of classes selected by the user and the number of robots.

The robot designator may change the matching of the target object and the robot when the target object classification failure rate according to the matching of the target object and the robot falls below a preset standard.

In addition to the technical problems of the present invention mentioned above, other features and advantages of the present invention are described below, or can be clearly understood by those skilled in the art from such description and description.

According to the present invention as described above, the following effects are achieved.

In order to upgrade a multi-class classification table, the present invention can automatically derive new classification attributes and upgrade the multi-class classification table without the user having to manually find new classification standards and upgrade them by comparing them with the existing classification table.

The present invention can designate target objects that each robot can classify so as to have optimal classification efficiency, taking into account the number of target objects and robots to be classified.

In addition, other features and advantages of the present invention may be newly understood through embodiments of the present invention.

Figure 1 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which an object classification system using a multi-class standard table is applied according to an embodiment of the present invention.
Figure 2 is a schematic diagram of a target object classification system using a multi-class reference table according to an embodiment of the present invention.
Figure 3 is a diagram illustrating a multi-class reference table according to an embodiment of the present invention.
Figure 4 is a diagram for explaining a user's selection of a target object using a multi-class reference table according to an embodiment of the present invention.
Figure 5 is a diagram illustrating the addition of a new classification attribute by the waste classification standard creation unit according to an embodiment of the present invention.
Figure 6 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which a plurality of robots are applied according to an embodiment of the present invention.

In this specification, it should be noted that when adding reference numbers to components in each drawing, the same components are given the same number as much as possible even if they are shown in different drawings.

Meanwhile, the meaning of the terms described in this specification should be understood as follows.

Singular expressions should be understood to include plural expressions unless the context clearly defines otherwise, and terms such as “first” and “second” are used to distinguish one element from another element. The scope of rights should not be limited by these terms.

Terms such as “include” or “have” should be understood as not precluding the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

The term “at least one” should be understood to include all possible combinations from one or more related items. For example, “at least one of the first, second, and third items” means each of the first, second, or third items, as well as two of the first, second, and third items. It means a combination of all items that can be presented from more than one.

The term “waste” used in this specification includes plastic, PET bottles, glass bottles, glass, paper, Styrofoam, general waste, and industrial waste.

The target object used in this specification refers to an object to be classified among various types of objects such as waste.

Hereinafter, preferred embodiments of the present invention designed to solve the above problems will be described in detail with reference to the accompanying drawings.

Figure 1 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which an object classification system using a multi-class standard table is applied according to an embodiment of the present invention.

Referring to FIG. 1, a waste classification system using an artificial intelligence-based robot may include a target object classification system 1000 using a multi-class standard table according to the present invention, a vision camera 2000, and a robot 3000. .

Objects 10, such as waste, moving on a conveyor belt 4000 are collected through a vision camera 2000, and objects to be classified are classified through a target object classification system 1000 using a multi-class standard according to the present invention. After selection, the target object can be classified through the robot 3000.

The vision camera 2000 may collect images of the object 10 and then transmit the image data to the target object classification system 1000 using a multi-class reference table.

The vision camera 2000 may be installed in various locations depending on the embodiment. For example, a first vision camera 2100 installed at the beginning of waste movement on the conveyor belt 4000 and a second vision camera 2200 installed in combination with the robot 3000 may be installed.

According to an embodiment, the target object classification system 1000 using a multi-class reference table for image data according to an embodiment of the present invention classifies the target object through the object 10 information collected through the first vision camera 2100. The robot 3000 on which the second vision camera 2200 is installed can select a target object to be classified through information on the object 10 collected through the second vision camera 2200.

In addition to the vision camera 2000, various sensors may be used as equipment for collecting images of the object 10 used in the present invention. For example, an optical sensor, infrared sensor, ultrasonic sensor, metal detector, x-ray sensor, radar sensor, etc. can be used. there is.

The robot 3000 is a robot for classifying objects moving on a conveyor belt, and gripper robotic arms with various grip shapes as well as suction robotic arms can be used.

Figure 2 is a schematic configuration diagram of a target object classification system using a multi-class standard table according to an embodiment of the present invention, Figure 3 is a diagram explaining a multi-class standard table according to an embodiment of the present invention, and Figure 4 is It is a diagram to explain a user selecting a target object using a multi-class standard table according to an embodiment of the present invention, and Figure 5 shows how the waste classification standard generator adds a new classification attribute according to an embodiment of the present invention. This is a drawing to explain this, and Figure 6 is a schematic diagram of a waste classification system using an artificial intelligence-based robot arm to which a plurality of robots are applied according to an embodiment of the present invention.

Referring to Figures 1 and 2, the target object classification system 1000 using a multi-class standard table according to the present invention includes a data generation unit 100, a target object determination unit 200, a robot designation unit 300, and a waste classification unit. It includes a reference generation unit 400 and an artificial intelligence model learning unit 500.

The data generator 100 may generate waste information obtained from the vision camera 2000, which collects waste information moving on the conveyor belt 4000, as data.

The data generator 100 may perform data preprocessing on waste image data through color space correction, resolution correction, bit depth correction, and density and transmittance correction.

The target object determination unit 200 can use an artificial intelligence model to determine whether the waste image is a target object to be classified based on a waste classification standard table having multiple classes.

The artificial intelligence model used to determine whether or not an object is a target object may use deep learning or a neural network. Neural network models include CNN (Convolution Neural Network), R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking), such as GoogleNet, AlexNet, VGG Network, etc. -based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, It may include various types of models such as Classification Network, but is not limited to this.

Referring to Figure 3, the waste classification standard table according to the present invention may include multiple classes, each class having its own classification standard.

As an example, the first class (Class 1) of the multiple classes is a category according to the material composition of the waste, the second class (Class 2) is a category according to the status (damage or not) of the waste, and the third class (Class 3) is a category based on the purpose of the waste, and the fourth class (Class 4) may be a category based on the color of the waste. The fifth class (Class 5) may be a category including waste that is not included at all in the first to fourth classes.

The target object determination unit 200 may determine whether the object is a target object according to the class of the waste classification standard table selected by the user.

Users can freely select (Class 1, Class 2, Class 3, Class 4).

For example, referring to FIG. 4, the user can select (PP, damaged, yogut, white), and the target object determination unit 200 selects among the waste images moving on the conveyor belt according to the multi-class classification criteria selected by the user. The target object can be identified using artificial intelligence.

Users can not only create multiple multi-class classification criteria, but also create various class combinations. For example, a user can create multi-class classification criteria {(PP, damaged, yogut, white), (PET, none, none, black)}. Here, none means that none of the items in the class are selected, which means that the classification criteria for the class are not taken into consideration.

The waste classification standard creation unit 400 may add new classification attributes to the waste classification standard table.

The waste classification standard generator 400 can derive new classification attributes using an artificial intelligence model from waste images classified into the fifth class or upgrade the waste classification standard table using new classification attributes input from the user. .

Previously, in order to upgrade the waste classification standard table, there was a difficulty in requiring the user to individually specify the type and status of waste that was not in the existing waste classification standard table. However, in the present invention, images of waste classified into the 5th class were used to use an artificial intelligence model. New classification properties can be automatically derived and upgraded.

The artificial intelligence model used at this time can analyze waste images classified into the fifth class and derive classification properties corresponding to the new class.

For example, by extracting features from waste images classified into Class 5 through a convolutional neural network (CNN), new classification properties that can classify images based on these features can be derived. .

Here, the new classification attribute may be a new item of an existing class or may be a standard for designating a new class.

The artificial intelligence model learning unit 500 automatically labels new classification attributes added to the waste image classified into the 5th class, and retrains the artificial intelligence model using the labeled waste image as training data to improve the performance of the target object discriminator. can be improved.

Referring to FIG. 5, the waste classification standard generator 400 extracts a whiskey bottle from the waste classified into the fifth class (Class 3) because it is a glass bottle that is not included in the third class (Class 3). This can be added as a new third class item (new classification attribute), and the artificial intelligence model learning unit 500 automatically labels the new third class item on the whiskey bottle and creates an artificial intelligence model based on the labeled data. can be relearned.

In this way, the target object classification system using a multi-class classification standard table according to an embodiment of the present invention automatically does not require the user to find new classification standards one by one and compare them with the existing classification standard table to upgrade the multi-class classification standard table. Multi-class classification rubrics can be upgraded by deriving new classification properties.

The robot designation unit 300 can designate a robot to classify the target object using a multi-class waste classification standard table.

When a plurality of robots are used, the robot designator 300 can match the target object and the robot by considering the type and number of classes selected by the user and the number of robots.

For example, the user can classify the target object to be classified using multi-class classification criteria: {(PP, damaged, yogut, white), (PET, none, none, black)}, (PE, damaged, Drink, black)} , (Glass, original, soju, clear)}, (PET, original, Drink, clear)}, (PS, original, Sauce, white)} can be created.

Referring to FIG. 6, the robot designator 300 may match a robot to classify each target object by considering the type and number of classes and the number of robots among the target objects selected by the user. For example, based on class 1, (PET, none, none, black)}, (PET, original, Drink, clear)} are designated to be classified by the first robot 2100, and {(PP, damaged, yogut , white), (PE, damaged, Drink, black)}, (Glass, original, soju, clear)}, (PS, original, Sauce, white)} can be designated to be classified by the second robot 2200.

The robot designator 300 may change the matching of the target object and the robot when the failure rate of classifying the target object according to the matching of the target object and the robot falls below a preset standard.

For example, based on class 1 described above, (PET, none, none, black)}, (PET, original, Drink, clear)} matches with the first robot 2100, and {(PP, damaged, yogut , white), (PE, damaged, Drink, black)}, (Glass, original, soju, clear)}, (PS, original, Sauce, white)} is the target object after matching with the second robot (2200) If the classification failure rate falls below a preset standard, matching can be changed based on class 2. In another embodiment, if there are six target object categories selected by the user and there are two robots, each robot can be divided into three target object categories and matched.

In this way, the target object classification system using the multi-class classification standard table according to the present invention can specify target objects that each robot can classify so as to have optimal classification efficiency, considering the number of target objects and robots that the user wants to classify. there is.

The system or device described above may be implemented with hardware components or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. It can be embodied in . Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. The medium may continuously store a computer-executable program, or may temporarily store it for execution or download. In addition, the medium may be a variety of recording or storage means in the form of a single or several pieces of hardware combined. It is not limited to a medium directly connected to a computer system and may be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be something configured to store program instructions, including ROM, RAM, flash memory, etc. Additionally, examples of other media include recording or storage media managed by app stores that distribute applications, sites or servers that supply or distribute various other software, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible without departing from the technical spirit of the present invention. It will be obvious to anyone with ordinary knowledge.

100: data generation unit
200: Target object determination unit
300: Robot designation unit
400: Waste classification standard generation unit
500: Artificial intelligence model learning department
1000: Target object classification system using multi-class standard table
2000: Vision Camera
3000: Robot
4000: Conveyor belt

Claims (10)

a data generator that generates data from waste images obtained from a vision camera that collects images of waste moving on a conveyor belt;
A target object determination unit that uses an artificial intelligence model to determine whether the waste image is a target object to be classified based on a waste classification standard table having multiple classes; and
It includes a robot designation unit that designates a robot to classify the target object using the waste classification standard table,
The waste classification standard table includes multiple classes with each class having its own classification standard, the first class of the multiple classes is a category according to the material composition of the waste, and the second class is a category according to the state (whether damaged or not) of the waste. , the third class is a category based on the use of the waste, the fourth class is a category based on the color of the waste, and the fifth class includes waste not included in the first to fourth classes,
Further comprising a waste classification standard generation unit that adds a new classification attribute to the waste classification standard table based on the new classification attribute,
The waste classification standard generation unit derives new classification attributes using an artificial intelligence model from the waste images classified into the fifth class, or upgrades the waste classification standard table using new classification attributes input from the user. Target object classification system using a multi-class classification standard. According to paragraph 1,
The artificial intelligence model is a target object classification system using a multi-class classification standard, characterized in that it is learned using waste images labeled according to a preset waste classification standard as learning data. delete delete According to paragraph 1,
A target object classification system using a multi-class classification table, characterized in that the target object determination unit determines whether the target object is the target object according to the class of the waste classification table selected by the user. delete delete According to paragraph 1,
A target object classification system using a multi-class classification standard further comprising an artificial intelligence model learning unit that automatically labels new classification attributes added to the waste image classified into the fifth class and retrains the artificial intelligence model. According to paragraph 1,
When a plurality of robots are used, the robot designation unit uses a multi-class classification standard table, characterized in that it matches the target object and the robot by considering the type and number of classes selected by the user and the number of robots. Target object classification system. According to clause 9,
The robot designator changes the matching of the target object and the robot when the classification failure rate of the target object according to the matching of the target object and the robot falls below a preset standard. A target object using a multi-class classification standard table. Classification system.

Images