CN111017429B

CN111017429B - Community garbage classification method and system based on multi-factor fusion

Info

Publication number: CN111017429B
Application number: CN201911144325.6A
Authority: CN
Inventors: 鲍敏; 谢超
Original assignee: Chongqing Terminus Technology Co Ltd
Current assignee: Chongqing Terminus Technology Co Ltd
Priority date: 2019-11-20
Filing date: 2019-11-20
Publication date: 2021-05-25
Anticipated expiration: 2039-11-20
Also published as: CN111017429A

Abstract

The embodiment of the application provides a community garbage classification method and system based on multi-factor fusion, which comprises the following steps: shooting the garbage from a plurality of angles through an image acquisition device to obtain a garbage form video; identifying at least one characteristic image in the morphological video to form a characteristic image set; generating a three-dimensional image of the garbage according to the characteristic image set, and extracting characteristic points of the garbage; identifying the type of the garbage through a machine learning algorithm according to the characteristic points; based on the characteristic points of the garbage, adding a plurality of random sampling points to form a sensing point set; performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm; and performing multi-factor fusion on the first classification result and the second classification result, and performing garbage classification guidance. The method and the device improve the accuracy and efficiency of garbage classification by combining multi-factor fusion and community garbage classification practice.

Description

Community garbage classification method and system based on multi-factor fusion

Technical Field

The application relates to the field of operation and research methods and garbage classification, in particular to a community garbage classification method and system based on multi-factor fusion.

Background

In the existing garbage classification process, community residents need to remember a large amount of classification information, so that the community residents are difficult to classify and often have classification errors. The wrongly classified garbage also needs to be classified secondarily, and some users even refuse to classify the garbage. Therefore, there is a need for an intelligent garbage classification method and system.

At present, in addition to a general image/video recognition technology, an object identification technology can also sense the material composition of the object through a sensor, but the material identification technology is not carried out through a method of combining image/video identification and material sensing, and particularly, a precedent that the image/video identification technology is applied to the field of garbage classification is not used.

Disclosure of Invention

In view of this, the present application aims to provide a community garbage classification method and system based on multi-factor fusion, which combine the practical features of garbage classification and solve the technical problems of low user experience, low efficiency and low degree of user experience in the current garbage classification process.

Based on the above purpose, the present application provides a community garbage classification method based on multi-factor fusion, which includes:

shooting the garbage from a plurality of angles through an image acquisition device to obtain a form video of the garbage;

identifying at least one characteristic image in the morphological video to form a characteristic image set; generating a three-dimensional image of the garbage according to the feature image set, and extracting feature points of the garbage; identifying the type of the garbage through a machine learning algorithm according to the feature points to obtain a first classification result;

based on the characteristic points of the garbage, adding a plurality of random sampling points to form a sensing point set; performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm to obtain a second classification result;

and performing multi-factor fusion on the first classification result and the second classification result, broadcasting and/or displaying to a user if the multi-factor fusion result is greater than or equal to a preset confidence coefficient, and performing garbage classification guidance.

In some embodiments, the method further comprises:

and performing multi-factor fusion on the first classification result and the second classification result, and broadcasting and/or displaying the first classification result and the second classification result to a user and performing garbage classification guidance if the multi-factor fusion result is smaller than a preset confidence level.

In some embodiments, shooting the garbage from a plurality of angles to obtain a morphological video of the garbage comprises:

taking the image acquisition device as a first center, and performing at least one annular rotation of different tracks on the garbage around the first center through rotating a hand arm; and/or

And taking the garbage as a second center, and carrying out at least one annular rotation of different tracks around the second center by the image acquisition device.

In some embodiments, sending the shipping information to a support vector machine comprises:

the support vector machine is positioned in a control center and actively and/or passively receives data of each dispatch information acquisition system to form an original data set;

and performing pre-classification according to the types of the sent commodities based on the original data set, and obtaining the receiving time interval of each receiver for each type of commodities in the pre-classification through a support vector machine.

In some embodiments, generating a three-dimensional image of the garbage and extracting feature points of the garbage comprises:

taking a region with a color different from that of an adjacent region as a feature point; and

and taking the area with different concave-convex degrees from the adjacent area as the characteristic point.

In some embodiments, forming a sensing point set based on the feature points of the garbage and adding a plurality of random sampling points includes:

if the garbage is in a pluggable form, randomly sampling a plurality of sampling points on a reference surface by taking one plane of the garbage as the reference surface, and carrying out plug-in type substance sensing analysis;

and if the garbage is in a non-interpenetrable form, taking at least one plane of the garbage as a reference plane set, and randomly sampling a plurality of sampling points for each reference plane in the reference plane set to perform contact type substance sensing analysis.

In some embodiments, multifactorial fusing of the first classification result and the second classification result comprises:

calculating a fused result of the first and second classification results by the formula S.C.P + cov (C, P), wherein S is a feature matrix of the fused result, C is a feature matrix of the first classification result, P is a feature matrix of the second classification result, and cov (C, P) is a gain matrix of the first and second classification results.

Based on above-mentioned purpose, this application has still provided a community waste classification system based on multifactor integration, includes:

the garbage collection device comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for shooting garbage from a plurality of angles through an image acquisition device to obtain a form video of the garbage;

the first classification module is used for identifying at least one characteristic image in the morphological video to form a characteristic image set; generating a three-dimensional image of the garbage according to the feature image set, and extracting feature points of the garbage; identifying the type of the garbage through a machine learning algorithm according to the feature points to obtain a first classification result;

the second classification module is used for adding a plurality of random sampling points based on the characteristic points of the garbage to form a sensing point set; performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm to obtain a second classification result;

and the first fusion module is used for performing multi-factor fusion on the first classification result and the second classification result, broadcasting and/or displaying the multi-factor fusion result to a user if the multi-factor fusion result is greater than or equal to a preset confidence coefficient, and performing garbage classification guidance.

In some embodiments, the system further comprises:

and the second fusion module is used for performing multi-factor fusion on the first classification result and the second classification result, and broadcasting and/or displaying the first classification result and the second classification result to a user and performing garbage classification guidance if the multi-factor fusion result is smaller than a preset confidence level.

In some embodiments, the acquisition module comprises:

the first rotating unit is used for taking the image acquisition device as a first center, and the garbage carries out at least one annular rotation of different tracks around the first center; and/or

And the second rotating unit is used for taking the garbage as a second center, and the image acquisition device performs awakening rotation of different tracks at least once around the second center.

In some embodiments, the second classification module comprises:

the first sensing unit is used for randomly sampling a plurality of sampling points on a reference surface by taking one plane of the garbage as the reference surface to perform plug-in type substance sensing analysis if the garbage is in a plug-in type;

and the second sensing unit is used for taking at least one plane of the garbage as a reference plane set if the garbage is in a non-intrusive form, and randomly sampling a plurality of sampling points for each reference plane in the reference plane set to perform contact type substance sensing analysis.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.

FIG. 1 shows a flowchart of a community garbage classification method based on multi-factor fusion according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a community garbage classification method based on multi-factor fusion according to an embodiment of the present invention.

Fig. 3 illustrates a constitutional diagram showing a community garbage classification system based on multi-factor fusion according to an embodiment of the present invention.

Fig. 4 illustrates a constitutional diagram showing a community garbage classification system based on multi-factor fusion according to an embodiment of the present invention.

Fig. 5 shows a constitutional view of an acquisition module according to an embodiment of the present invention.

Fig. 6 shows a constitutional diagram of the second classification module according to an embodiment of the present invention.

Fig. 7a and 7b show a schematic diagram according to an embodiment of the invention.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

FIG. 1 is a flowchart illustrating a community garbage classification method based on multi-factor fusion according to an embodiment of the present invention. As shown in fig. 1, the community garbage classification method based on multi-factor fusion includes:

and step S11, shooting the garbage from a plurality of angles through an image acquisition device to obtain the form video of the garbage.

Specifically, the garbage is shot from a plurality of angles, and cameras can be arranged at the plurality of angles, so that the garbage is shot statically at the plurality of angles; the garbage can be shot in a rotating mode in a surrounding mode, and the shooting process is dynamic shooting.

In one embodiment, shooting the garbage from multiple angles to obtain a form video of the garbage comprises:

As shown in fig. 7a, with the garbage O as the second center, the image capturing devices C1, C2 perform at least one circular rotation of different tracks around the O point, thereby obtaining multi-angle video/image data of garbage. C1 and C2 may be the same image capture device or different image capture devices.

As shown in fig. 7b, the garbage R performs at least one circular rotation of different tracks around the point O by the operation of the rotating arm with the image capturing device O as the center, so as to obtain multi-angle video/image data of garbage.

In an implementation mode, the garbage images obtained by the two garbage image shooting modes can be integrated to obtain more comprehensive garbage appearance data, so that the accuracy of garbage identification is improved.

Specifically, blind areas may exist in one shooting mode, the shape data of certain parts of the garbage cannot be obtained, the blind areas are positioned, and the shape data of the complementary areas are extracted from the other shooting mode to form a set of complete garbage shape data.

Step S12, identifying at least one characteristic image in the form video to form a characteristic image set; generating a three-dimensional image of the garbage according to the feature image set, and extracting feature points of the garbage; and identifying the type of the garbage through a machine learning algorithm according to the characteristic points to obtain a first classification result.

In particular, the classification of the garbage can be preliminarily obtained through the analysis of the shape of the outer surface of the garbage. Therefore, it is necessary to convert the garbage into a three-dimensional image so as to more accurately analyze the shape of the garbage shape.

For example, the shape of the garbage carton is generally a cuboid, so that the cuboid can be identified, and the garbage carton can be preliminarily judged to belong to according to the length, width and height of the cuboid, and the character and pattern information on the surface.

In one embodiment, generating a three-dimensional image of the garbage and extracting feature points of the garbage comprises:

Specifically, it can be found from practical operation experience of garbage classification that, for a garbage, it is important to identify the contents of an area with a color different from that of the surrounding area, for example, a garbage carton, which is light brown in whole body, but there is an area printed with the character "this face up, do not step on" in red.

In addition, for the garbage with a change in shape, the contents of the area with the concave-convex degree different from the surrounding area need to be recognized when the garbage is classified, for example, a plastic display board is flat on the whole, but the word of "advertisement" is pasted in the center in a convex mode, the garbage recognition key needs to recognize not the whole plane of the plastic display board, but the word of "advertisement" is recognized, so that the flat board is recognized as a plastic advertisement board more accurately.

Step S13, based on the characteristic points of the garbage, adding a plurality of random sampling points to form a sensing point set; and performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm to obtain a second classification result.

Specifically, since the outer surface of the trash is generally deformed by human use, even if a three-dimensional image of the trash is generated, the original state of the trash cannot be recognized, and the trash recognition accuracy may be lowered. For example, many people are used to press the top pop can into a flat shape after drinking, so that the top pop can is convenient to store. Even if a hologram of a flat can bottle is scanned, the flat can bottle cannot be recognized by the shape. Therefore, it is considered that the substance sensing is performed again by the sensor, thereby overcoming the technical problem.

In one embodiment, forming a sensing point set based on the feature points of the garbage and adding a plurality of random sampling points includes:

Specifically, for the form capable of being inserted, for example, a bag of kitchen waste, the kitchen waste bag can be opened, the sensor can be pricked into a plurality of collecting points in the kitchen waste bag by taking the opening plane of the kitchen waste bag as a reference plane, and the type of the substance in the kitchen waste bag can be identified.

In the case of garbage in a non-penetrable form, for example, a metal plate material, the material type of the metal plate material can be identified by randomly sampling a plurality of sampling points on a reference plane with the maximum plane of the plate material as the reference plane, and sensing the metal type of each point on the reference plane by a contact probe sensor.

And S14, performing multi-factor fusion on the first classification result and the second classification result, broadcasting and/or displaying to a user if the multi-factor fusion result is greater than or equal to a preset confidence coefficient, and performing garbage classification guidance.

Specifically, the first classification result identified by the image and the second classification result identified by the sensor are complementary to each other, and the results of the two classifications can be subjected to multi-factor fusion according to the actual classification requirement, so that the optimal identification effect is achieved.

In one embodiment, multifactorial fusion of the first classification result and the second classification result comprises:

In one example, the classification result can be fed back to the user in a voice broadcast mode, so that garbage classification is performed; the classification result can also be fed back to the user in a visual mode through screen display or in an image guidance mode; the garbage classification guidance can be performed on the user in an audio-visual mode by combining the two modes.

FIG. 2 is a flowchart illustrating a community garbage classification method based on multi-factor fusion according to an embodiment of the present invention. As shown in fig. 2, the community garbage classification method based on multi-factor fusion further includes:

and S15, performing multi-factor fusion on the first classification result and the second classification result, and broadcasting and/or displaying the first classification result and the second classification result to a user and performing garbage classification guidance if the multi-factor fusion result is smaller than a preset confidence level.

Specifically, when the two modes are combined, the type of the garbage cannot be accurately predicted, there are two possibilities, one is that the garbage belongs to a category which is difficult to classify, for example, half of the garbage is recyclable, and the other half is not recyclable, and then the garbage can be performed only by performing a secondary operation by a user; another possibility is that the classification is problematic, requiring user intervention. Both of these situations require timely feedback to the user to be handled by the user.

Fig. 3 illustrates a constitutional diagram of a community garbage classification system based on multi-factor fusion according to an embodiment of the present invention. As shown in fig. 3, the community garbage classification system based on multi-factor fusion can be divided into:

the acquisition module 31 is used for shooting the garbage from a plurality of angles through an image acquisition device to obtain a form video of the garbage;

a first classification module 32, configured to identify at least one feature image in the morphological video, and form a feature image set; generating a three-dimensional image of the garbage according to the feature image set, and extracting feature points of the garbage; identifying the type of the garbage through a machine learning algorithm according to the feature points to obtain a first classification result;

the second classification module 33 is configured to add a plurality of random sampling points based on the feature points of the garbage to form a sensing point set; performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm to obtain a second classification result;

and the first fusion module 34 is configured to perform multi-factor fusion on the first classification result and the second classification result, broadcast and/or display the multi-factor fusion result to a user if the multi-factor fusion result is greater than or equal to a preset confidence level, and perform garbage classification guidance.

Fig. 4 shows a constitutional diagram of a community garbage classification system based on multi-factor fusion according to an embodiment of the present invention. As shown in fig. 4, the community garbage classification system based on multi-factor fusion further includes:

and a second fusion module 35, configured to perform multi-factor fusion on the first classification result and the second classification result, and if the multi-factor fusion result is smaller than a preset confidence level, broadcast and/or display both the first classification result and the second classification result to a user, and perform garbage classification guidance.

Fig. 5 shows a configuration diagram of an acquisition module according to an embodiment of the present invention. As can be seen from fig. 5, the acquisition module 31 includes:

a first rotating unit 311, configured to take the image capturing device as a first center, around which the garbage performs at least one circular rotation with different orbits;

a second rotation unit 312, configured to use the garbage as a second center, around which the image capturing apparatus performs at least one awakening rotation of different tracks.

Fig. 6 shows a constitutional diagram of the second classification module according to an embodiment of the present invention. As can be seen from fig. 6, the second classification module 33 includes:

the first sensing unit 331 is configured to, if the garbage is in a form that can be inserted, randomly sample a plurality of sampling points on a reference plane by using one plane of the garbage as the reference plane to perform insertion type substance sensing analysis;

and a second sensing unit 332, configured to, if the garbage is in a non-intrusive form, use at least one plane of the garbage as a reference plane set, and randomly sample a plurality of sampling points for each reference plane in the reference plane set to perform contact substance sensing analysis.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any system that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic system) having one or more wires, a portable computer diskette (magnetic system), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber system, and a portable read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present invention, and these should be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. A community garbage classification method based on multi-factor fusion is characterized by comprising the following steps:

through image acquisition device, shoot rubbish from a plurality of angles, obtain the form video of rubbish includes: taking the image acquisition device as a first center, and performing at least one annular rotation of different tracks on the garbage around the first center through rotating a hand arm; and/or taking the garbage as a second center, and carrying out at least one annular rotation of different tracks around the second center by the image acquisition device;

identifying at least one characteristic image in the morphological video to form a characteristic image set; generating a three-dimensional image of the garbage according to the feature image set, and extracting feature points of the garbage, wherein the steps comprise: taking a region with a color different from that of an adjacent region as a feature point; and using the areas with different concave-convex degrees with the adjacent areas as characteristic points; identifying the type of the garbage through a machine learning algorithm according to the feature points to obtain a first classification result;

based on the characteristic points of the garbage, and adding a plurality of random sampling points to form a sensing point set, comprising the following steps: if the garbage is in a pluggable form, randomly sampling a plurality of sampling points on a reference surface by taking one plane of the garbage as the reference surface, and carrying out plug-in type substance sensing analysis; if the garbage is in a non-interpenetrable form, taking at least one plane of the garbage as a reference plane set, and randomly sampling a plurality of sampling points for each reference plane in the reference plane set to perform contact type substance sensing analysis; performing substance analysis on each sensing point in the sensing point set through a sensor to obtain a sampling result set, and predicting data in the sampling result set through a machine learning algorithm to obtain a second classification result;

performing multi-factor fusion on the first classification result and the second classification result, including: calculating a fused result of the first and second classification results by the formula S · P + cov (C, P), where S is a feature matrix of the fused result, C is a feature matrix of the first classification result, P is a feature matrix of the second classification result, cov (C, P) is a gain matrix of the first and second classification results; if the multi-factor fusion result is greater than or equal to the preset confidence coefficient, broadcasting and/or displaying the multi-factor fusion result to a user, and performing garbage classification guidance;