KR20230082836A - System for supporting fruit sort based on artificial intelligence - Google Patents

Abstract

The present invention relates to an artificial intelligence-based fruit sorting support system, which includes at least one fruit sorting device capable of sorting target fruits, and connected to the fruit sorting device through a network so that the fruit sorting device can sort fruit. Includes a fruit sorting support server that supports; the fruit sorting supporting server supports an artificial intelligence-based classification model for classifying the target fruit in the fruit sorting device based on a photographed image of the target fruit; The classification model includes a plurality of type-specific classification models classified according to the type of the target fruit; Each of the classification models for each type includes a plurality of classification models for each class classified according to the classification criterion level of the target fruit.

Description

AI-based fruit sorting support system {SYSTEM FOR SUPPORTING FRUIT SORT BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to an artificial intelligence-based fruit sorting support system, and more particularly, to an artificial intelligence-based fruit sorting support system that supports an artificial intelligence-based classification model in a fruit sorting device that classifies a target fruit based on an image. it's about

Selection of fruits such as apples is a process of commercializing the produced fruits. It has the characteristics of increasing added value by selecting harvested fruits fresh and uniform to improve the commercial value of agricultural products.

In particular, in the sorting of fruits, there are various factors that can determine the quality of each fruit, such as the size, weight, shape, color, and presence of scars. It is required. Therefore, in order to increase marketability, uniform selection and packaging should be carried out.

In general, in the fruit sorting process, a worker sorts target fruits moving along a conveyor by grade. Recently, a fruit sorting device that automatically classifies and sorts fruits by size or weight has been widely used.

Criteria for classifying target fruits by grade include a method of classifying according to size or weight, but there is also a method of classifying the same size or weight into grades of a certain standard.

However, conventional fruit sorting devices based on size or weight are limited to sorting by size or weight, and there is a limit to sorting target fruits of the same size or weight by grade.

Accordingly, a technology for sorting fruits based on images taken of the target fruit has been proposed. Recently, 'CNN-based diseased and damaged fruit classification apparatus and method' disclosed in Korean Patent Publication No. 10-2020-0084940 As described above, based on artificial intelligence technology such as deep learning, a technique of classifying a target fruit using an image taken of the target fruit has been proposed.

In the case of such a fruit sorting device based on a photographed image of a target fruit, the quality of the image affects the accuracy of classification. In the case of fruits, depending on the direction in which they are photographed, places that affect grades, such as scratches, may not be photographed, so accurate classification may not be achieved even if an artificial intelligence-based classification model with high accuracy is used.

In addition, even for the same target fruit, the government policy or the quality of the fruit of the year may change, so there are cases where the top grade standard changes.

Korean Patent Publication No. 10-2020-0084940

The present invention has been made in view of the above points, and an object of the present invention is to provide an artificial intelligence-based fruit sorting support system that can actively respond to various classification criteria.

In addition, another object of the present invention is to provide an artificial intelligence-based fruit sorting support system capable of classifying target fruits by grade.

In addition, another object of the present invention is to provide an artificial intelligence-based fruit sorting support ?????? system capable of actively coping with a change in the classification standard grade for classifying a target fruit, such as the highest grade.

An object of the present invention is to provide an artificial intelligence-based fruit sorting support system according to the present invention, wherein at least one fruit sorting device capable of sorting a target fruit is connected to the fruit sorting device through a network, a fruit sorting support server supporting fruit sorting by the fruit sorting device; the fruit sorting supporting server supports an artificial intelligence-based classification model for classifying the target fruit in the fruit sorting device based on a photographed image of the target fruit; The classification model includes a plurality of type-specific classification models classified according to the type of the target fruit; Each of the classification models by type is achieved by an artificial intelligence-based fruit sorting support system comprising a plurality of classification models classified according to the classification standard level of the target fruit.

Here, the classification criterion level is divided into an N-th level from a first level; The classification models for each class may include from a first class classification model for classifying into a first class and other classes to an N-1 class classification model for classifying into an N-1 class and other classes.

In addition, the first class classification model is generated by learning using images of the first class and other classes as input learning data and images of the first class as output learning data; The N-1 grade classification model may be generated by learning using images of the N-1 grade or higher and the N grade as input learning data, and an image of the N-1 grade or higher as output training data.

And, the fruit sorting device operates in one of a data collection mode for photographing a plurality of target fruits and a fruit classification mode for performing classification through the classification model provided from the fruit sorting support server for a plurality of target fruits; The fruit sorting support server may receive an image of a target fruit collected through the data collection mode from the fruit sorting apparatus and learn or relearn the classification model.

The fruit sorting support server may relearn the classification model by receiving an image of a target fruit captured while operating in the fruit sorting mode from the fruit sorting apparatus, from the fruit sorting apparatus.

In addition, the classification model may be learned by using, as learning data, an image set photographed at least twice or more in at least two or more directions with respect to the target fruit.

According to the present invention, an artificial intelligence-based fruit sorting support system capable of actively responding to various classification criteria is provided.

In addition, according to the present invention, an artificial intelligence-based fruit sorting support system capable of classifying target fruits by grade is provided.

In addition, according to the present invention, an artificial intelligence-based fruit selection support system capable of actively coping with a change in a classification standard grade for classifying a target fruit, such as the highest grade, is provided.

1 is a diagram showing the configuration of an artificial intelligence-based fruit sorting support system according to an embodiment of the present invention;
2 is a diagram showing an example of the configuration of a fruit sorting support server according to an embodiment of the present invention;
3 and 4 are diagrams for explaining the type, learning process, and classification process of a classification model according to an embodiment of the present invention;
5 is a perspective view of a fruit sorting device according to an embodiment of the present invention;
6 is a side view of a fruit sorting device according to an embodiment of the present invention;
7 is a plan view of a fruit sorting device according to an embodiment of the present invention;
8 is a control block diagram of a fruit sorting device according to an embodiment of the present invention;
9 is a diagram showing an example of the configuration of a sorting processing unit of a fruit sorting device according to an embodiment of the present invention;
10 is an enlarged view of area A of FIG. 5;
11 and 12 are partially exploded perspective views of a fruit sorting unit according to an embodiment of the present invention;
13 is a perspective view of a camera assembly according to an embodiment of the present invention;
14 is a bottom view of a camera assembly according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

1 is a diagram showing the configuration of an artificial intelligence-based fruit sorting support system according to an embodiment of the present invention.

Referring to FIG. 1 , an AI-based fruit sorting support system according to an embodiment of the present invention includes at least one fruit sorting device 10 and a fruit sorting support server 30 . Here, the fruit sorting device 10 and the fruit sorting support server 30 are interconnected through the network 50 to exchange data.

The fruit sorting device 10 according to the embodiment of the present invention is provided to be able to classify the target fruit F. In the embodiment of the present invention, it is exemplified that the fruit sorting device 10 classifies the target fruit F using the classification model 34 provided from the fruit sorting support server 30. In the present invention, the target fruit F ) to be classified into two categories. Here, an example of a specific configuration of the fruit sorting device 10 according to an embodiment of the present invention will be described later.

The fruit sorting support server 30 according to an embodiment of the present invention supports the artificial intelligence-based classification model 34 for classifying the target fruit F to the fruit sorting device 10 . Here, the classification model 34 classifies the target fruit F based on a photographed image of the target fruit F.

2 is a diagram showing an example of the configuration of a fruit sorting support server 30 according to an embodiment of the present invention.

Referring to FIG. 2 , the classification model 34 according to the embodiment of the present invention may include a plurality of classification models 34a and 34b classified according to the type of target fruit F. For example, a first fruit classification model 34a for classifying apples and an Lth fruit classification model 34b for classifying pears may be included.

Through this, even if one fruit sorting device 10 is installed, the fruit sorting support server 30 supports the classification model 34 for various types of target fruit F, and determines the type of target fruit F. Accordingly, by downloading and using the corresponding classification model 34 from the fruit sorting support server 30, there is no need to purchase a separate device for sorting according to the type of target fruit F, and costs can be minimized.

In addition, in the embodiment of the present invention, each of the classification models 34a and 34b for each fruit type includes a plurality of class classification models 34a1, 34a2, 34a3, 34b1, 34b2, 34b3).

In FIG. 2 , the first fruit classification model 34a includes the first class classification model 34a1, the second class classification model 34a2, and the N-1 class classification model 34a3 to classify the target fruit F. It is exemplified that N-1 graded classification models 34a1, 34a2, and 34a3 are included according to the reference grade. At this time, the classification standard grade of the target fruit F corresponding to the first fruit classification model 34a is classified as N grade.

In addition, the L-th fruit classification model 34b includes the first-class classification model 34b1, the second-class classification model 34b2, and the M-1-th class classification model 34b3 to classify the target fruit F. It is an example of including M-1 class classification models 34b1, 34b2, and 34b3 according to the class. At this time, the classification standard grade of the target fruit F corresponding to the Lth fruit classification model 34b is classified as M grade.

Referring to FIG. 3, in the example shown in FIG. 3, the target fruit F is classified into four grades, and three graded classification models 34a1, 34a2, and 34a3, that is, the first grade, are classified. For example, it is divided into a model 34a1, a second classification model 34a2, and a third classification model 34a3.

In addition, learning data for learning the classification models 34a1, 34a2, and 34a3 for each grade are classified as follows.

○: Images of the target fruit (F) (F) corresponding to the first grade

◎: Images of the target fruit (F) (F) corresponding to the second grade

⊙ : Images of target fruits (F) (F) corresponding to the 3rd grade

● : Images of target fruits (F) (F) corresponding to the 4th grade

Each of the images corresponding to the above training data is labeled with a corresponding grade.

As described above, when learning data is collected, the first class classification model 34a1 is trained to classify the target fruit F of the first class. As shown in FIG. 3, the image of the target fruit F corresponding to the first grade is, for example, positive (T), and the remaining grades, that is, objects corresponding to the second, third, and fourth grades. An image of fruit (F) is input as input data, for example negative (F), and it is learned to output a first-class image corresponding to positive (T).

Accordingly, when the target fruit F is selected using the first grade classification model 34a1, the target fruit F is classified into the first grade and the remaining grades.

The second class classification model 34a2 is trained to classify target fruits F of first class and second class. As shown in FIG. 3, the image of the target fruit F corresponding to the first and second grades is positive (T), and the remaining grades, that is, the target fruits corresponding to the third and fourth grades ( An image for F) is input as input data to be negative (F), and it is learned to output images of the first and second grades corresponding to positive (T).

Therefore, when the fruit sorting device 10 operates using the second class classification model 34a2, the target fruit F has one first class and a second class, and the remaining third class and fourth class. can be classified as different.

Similarly, the third class classification model 34a3 is trained to classify target fruits F of first class, second class, and third class. As shown in FIG. 3, the image of the target fruit (F) corresponding to the first, second, and third grades is positive (T), and the target fruit (F) corresponding to the fourth grade (the remaining grades) ) is input as input data for negative (F), and it is learned to output images of the first, second, and third grades corresponding to positive (T).

Therefore, when the target fruit F is classified using the third class classification model 34a3, the first class, the second class, and the third class are one, and the remaining fourth class is the target fruit F. can be classified as different.

As described above, the classification model 34 for the target fruit F is divided into first classification models 34a1 and 34b1, second classification models 34a2 and 34b2, and third classification models 34a3. In this situation, various methods of classifying the target fruit F will be described with reference to FIG. 4 .

As an embodiment, when the target fruits F are to be classified according to the first, second, third, and fourth grades, that is, each grade, first, the first grade classification model 34a1 is used. , among all target fruits F, as shown in (a) of FIG. 4, target fruits F corresponding to the first grade may be selected.

Then, when the remaining target fruits F classified as non-first-class through the first-class classification model 34a1 are classified through the second-class classification model 34a2, FIG. 4(b) As shown, the target fruit F corresponding to the second grade may be selected.

And, the remaining target fruits F classified as non-second rank through the second rank classification model 34a2, that is, target fruits F corresponding to the first and second ranks among all target fruits F When the remaining target fruit F already selected is classified through the third class classification model 34, as shown in FIG. 4(c), it is classified into the third class and the fourth class, and finally, Target fruits F can be classified into first, second, third, and fourth grades.

In another embodiment, when trying to classify target fruits (F) of the second grade or higher, that is, when trying to classify only target fruits (F) corresponding to the first and second grades among all target fruits (F) , As shown in (d) of FIG. 4, if all target fruits F are classified through the second class classification model 34a2, this can be implemented.

Through the configuration as described above, the fruit sorting apparatus 10 according to the embodiment of the present invention uses the classification model 34 classified by fruit type/grade provided by the fruit sorting support server 30 to target fruit ( Even if F) has a simple structure for classifying only two types, classification into various grades is possible.

In addition, even if the highest grade standard changes due to government policies or the quality of the fruit in that year changes every year, classification can be performed using one of the models corresponding to the criteria for classifying the highest grade, It is also possible to actively respond to changes.

Referring again to FIG. 2, the fruit sorting support server 30 according to an embodiment of the present invention includes a data collection unit, a learning data storage unit 32, a model learning unit 33, and the classification model 34 described above. , It may be configured to include a support communication unit 35 and a support control unit 36. In addition, the fruit sorting support server 30 according to the embodiment of the present invention may further include an input unit 37.

The support communication unit 35 communicates with the fruit sorting device 10 through the network 50 . Here, the support control unit 36 may transmit at least one of the classification models 34 to the fruit sorting apparatus 10 through the support communication unit 35 according to the request of the fruit sorting apparatus 10 .

The learning data collection unit 31 collects learning data required for learning or re-learning of the classification model 34 . In the embodiment of the present invention, an example is that the learning data collection unit 31 collects learning data from the fruit sorting device 10, and a detailed description thereof will be described later.

Learning data necessary for learning the classification model 34 is stored in the learning data storage unit 32 . Here, the learning data stored in the learning data storage unit 32 may be used for learning the classification model 34 by labeling and storing information about the type of the target fruit F or the grade of the target fruit F. .

The model learning unit 33 trains the classification model 34 for each type of target fruit F and for each grade of the target fruit F. Here, the model learning unit 33 extracts learning data corresponding to the type of the corresponding target fruit F from among the learning data stored in the learning data storage unit 32, and outputs it according to the grade of the corresponding classification model 34. are taught separately.

The input unit 37 receives various inputs, for example, an input signal such as start of learning, according to the manager's manipulation. Here, the input unit 37 may be provided in the form of a mouse or keyboard, and may include a touch input unit 37 input through a screen touch.

Meanwhile, in an embodiment of the present invention, a data collection mode in which the fruit sorting device 10 photographs a plurality of target fruits F, and a classification model provided by the fruit sorting support server 30 for the plurality of target fruits F It can operate in any one of the fruit classification modes that perform classification through (34).

Through this, in the embodiment of the present invention, the learning data to be used in the classification model 34 for classifying the target fruit F, that is, the target fruit F, using the fruit sorting device 10 manufactured for fruit sorting. Recorded images can be collected.

Here, in the embodiment of the present invention, it is exemplified that the classification model 34 is learned by using as learning data a set of images taken at least twice in at least two directions with respect to the target fruit F. For this purpose, fruit selection The device 10 also photographs the target fruit F at least twice in the same direction and inputs the image to the classification model 34, which will be described in detail later.

Hereinafter, the fruit sorting device 10 according to an embodiment of the present invention will be described in detail with reference to FIGS. 5 to 14 .

5 is a perspective view of a fruit sorting device 10 according to an embodiment of the present invention, FIG. 6 is a side view of the fruit sorting device 10 according to an embodiment of the present invention, and FIG. 7 is an embodiment of the present invention. It is a plan view of the fruit sorting device 10 according to the example, and FIG. 8 is a control block diagram of the fruit sorting device 10 according to an embodiment of the present invention.

Referring to FIGS. 5 to 8 , the artificial intelligence-based fruit sorting apparatus 10 according to an embodiment of the present invention may include a fruit sorting unit 100 and a sorting processing unit 200.

The fruit sorting unit 100 according to an embodiment of the present invention selectively classifies the target fruit F in either direction of both sides in the horizontal direction. That is, the fruit sorting unit 100 according to the embodiment of the present invention classifies the target fruit in one of two directions, positive (T) or negative (F), and classifies the target fruit (F) into two categories. .

Here, the fruit sorting unit 100 according to the embodiment of the present invention may include a camera assembly 160 including at least one camera module 161 (see FIGS. 13 and 14) for photographing the target fruit F. can

Configurations of the fruit sorting unit 100 and the camera assembly 160 according to an embodiment of the present invention will be described later.

The sorting processing unit 200 according to an embodiment of the present invention controls the fruit sorting process of the fruit sorting unit 100 .

9 is a diagram showing an example of the configuration of the sorting processing unit 200 of the fruit sorting device 10 according to an embodiment of the present invention.

Referring to FIG. 9 , the selection processing unit 200 according to an embodiment of the present invention may include a classification model 210 and a processing control unit 220 . In addition, the selection processing unit 200 may further include a selection communication unit 240, an image display unit 230, a data storage unit 260, and a user input unit 250.

The user input unit 250 receives various inputs, for example, an input signal such as start of a sorting operation, according to an operator's manipulation. Here, the user input unit 250 may be provided in the form of a mouse or keyboard, and may include an image display unit 230 and a touch input unit that is input through a screen touch.

The data storage unit 260 may store images captured by the camera module 161 . Here, the image of the target fruit F stored by the data storage unit 260 may be used for re-learning of the classification model 210 after a labeling process by an expert or operator later. Here, when the fruit sorting device 100 according to the embodiment of the present invention operates in the data collection mode, images stored in the data storage unit 260 may be transmitted through the communication unit 240 .

The selection communication unit 240 communicates with the outside of the selection processing unit 200 . In an embodiment of the present invention, it is exemplified that the selection communication unit 240 includes an internal communication unit 241 and an external communication unit 242 .

The internal communication unit 241 communicates with components constituting the fruit sorting device 10 according to the embodiment of the present invention. For example, the internal communication unit 241 performs communication with the sorting rotation driving unit 140, the roller driving unit 150, the first conveyor driving unit 312, the second conveyor driving unit 322, and the camera module 161 to be described later. do.

The external communication unit 242 communicates with the fruit selection support server 30 . Through this, learning data may be transmitted to the fruit sorting support server 30 through the external communication unit 242 and the classification model 34 may be received from the fruit sorting support server 30 .

In the embodiment of the present invention, the internal communication unit 241 and the external communication unit 242 are described as being separated, but it can be configured to perform internal and external communication through the same communication method, of course.

The classification model 210 according to an embodiment of the present invention receives an image captured by the camera module 161 and outputs an inference result for classifying the target fruit F.

As described above, in the embodiment of the present invention, the classification model 30 registered in the fruit sorting support server 30 is received through the network and registered as the classification model 210 of the fruit sorting device 10 as an example. do.

FIG. 9 shows a state in which all classification models 210 corresponding to the classification models 34 of the fruit sorting support server 30 shown in FIG. 2 are registered in the fruit sorting device 10 as an example, but only necessary parts are Of course, it may be registered.

Referring to the example shown in FIG. 9, in the fruit sorting device 10, classification models 211a and 211b by type, and classification models by class 211a1, 211a2, 211a3, 211b1, 211b2 and 211b3) are registered examples.

Meanwhile, the processing controller 220 according to an embodiment of the present invention controls the entire operation of the fruit sorting device 10 as described above. In particular, the processing control unit 220 controls the sorting rotation driving unit 140 of the fruit sorting unit 100 to be described later so that the corresponding target fruit F is classified based on the reasoning result of the classification model 210 .

Again, with reference to FIGS. 5 to 8 , an example of the configuration of mechanical parts of the fruit sorting device 10 according to an embodiment of the present invention will be described.

The fruit sorting unit 100 according to an embodiment of the present invention selectively sorts the target fruit F in either direction of both sides of the horizontal direction W. In addition, the fruit sorting device 10 according to the embodiment of the present invention may further include a fruit transfer unit 300 . Here, the fruit transfer unit 300 transfers the target fruit F to the fruit sorting unit 100 and transfers one target fruit F to the fruit sorting unit 100 . A detailed description of the fruit transfer unit 300 according to an embodiment of the present invention will be described later.

FIG. 10 is an enlarged view of area A of FIG. 5 . 11 and 12 are partially exploded perspective views of a fruit sorting unit 100 according to an embodiment of the present invention, FIG. 13 is a perspective view of a camera assembly 160 according to an embodiment of the present invention, and FIG. It is a bottom view of the camera assembly 160 according to one embodiment of the invention.

The fruit sorting unit 100 according to an embodiment of the present invention includes a sorting frame 110, a sorting rotating member 120, a pair of rotating rollers 130, a sorting rotation driving unit 140, a roller driving unit 150, and A camera assembly 160 may be included.

The sorting frame 110 according to an embodiment of the present invention supports components such as the sorting rotation member 120 and the camera assembly 160 constituting the fruit sorting unit 100 . Here, in FIGS. 5 and 6, the selection frame 110 supporting the camera assembly 160 is not shown, but the selection frame 110 for supporting the camera assembly 160 on the upper side of the camera assembly 160 It goes without saying that this can be arranged.

For example, beam-shaped frames are interconnected in the horizontal, vertical, and height directions (H) to form a main frame that forms the entire support structure, and a part of the main frame supports the camera assembly 160. The sorting frame ( 110) can function. Alternatively, the selection frame 110 supporting the camera assembly 160 may be configured to support the camera assembly 160 in a fixed state to the main frame.

In addition, the sorting frame 110 supporting the sorting rotation member 120 is provided in a form that can be installed on the main frame and supports the sorting rotation member 120, as shown in FIGS. 10 to 12 .

The sorting rotation member 120 according to an embodiment of the present invention is rotatably installed in the sorting frame 110 in the vertical direction (L) as an axis.

Then, the pair of rotating rollers 130 are exposed to the upper side of the sorting rotating member 120, and are rotatable in the horizontal direction (W) or the vertical direction (L) of the sorting rotating member 120 as an axis. installed In the embodiment of the present invention, it is shown as an example that a pair of rollers are rotatably installed about an axis in the longitudinal direction (L).

Selection rotation driving unit 140 according to an embodiment of the present invention rotates the selection rotation member 120 forward and backward in the longitudinal direction (L) axis. Through this, when the target fruit F is seated on the pair of rotating rollers 130, based on the inference result of the classification model 210 of the sorting processing unit 200, the sorting rotation member 120 performs the sorting result. By controlling the sorting rotation driving unit 140 to rotate in either direction of both sides of the horizontal direction (W) according to, it is possible to sort the target fruit (F) into both sides.

The pair of roller driving units 150 according to the embodiment of the present invention rotates at least one of the pair of rotating rollers 130 so that the target fruit F seated on the pair of rotating rollers 130 rotates. In the embodiment of the present invention, it is exemplified that the roller driving unit 150 is shaft-connected to one of the pair of rotating rollers 130 to rotate one rotating roller 130 .

The camera assembly 160 according to an embodiment of the present invention may include at least one camera module 161 . In the embodiment of the present invention, three camera modules 161 constitute the camera assembly 160 as an example, and a detailed description thereof will be described later.

In the present invention, the camera module 161 photographs the target fruit F rotating according to the rotation of the pair of rotating rollers 130, thereby enabling the target fruit F to be photographed in various directions, which will be described later. When the processing unit 200 sorts fruits based on the image taken by the camera module 161, more accurate sorting is possible.

According to the configuration described above, as described above, the sorting processing unit 200 controls the sorting rotation driving unit 140 so that the target fruit F is classified based on the image captured by the camera module, Fruits F can be selectively sorted on either side of the horizontal direction W of the sorting rotation member 120 .

Meanwhile, a pair of rotating rollers 130 according to an embodiment of the present invention are arranged spaced apart from each other at a predetermined interval in the horizontal direction (W). Through this, in a state where the target fruit F is stably seated between the pair of rotating rollers 130, it can be photographed by the camera module 161 while rotating according to the rotation of the rotating rollers 130.

In addition, in the present invention, each rotating roller 130 is configured to have a tapered shape from the central portion to both sides of the longitudinal direction L, so that the target fruit F can rotate in a more stable seated state. In addition, according to the tapered shape, when the sorting rotating member 120 rotates in either direction of both sides of the horizontal direction (W), it is possible to more easily separate from the rotating roller 130 in that direction.

Here, in the embodiment of the present invention, it is exemplified that the selection rotation member 120 includes a pair of rotation plates 121 and 122 and a connection plate 123.

The pair of rotation plates 121 and 122 are rotatably coupled to the sorting frame 110 while being spaced apart from each other in the vertical direction L. Also, the connection plate 123 connects the pair of rotation plates 121 and 122 to each other.

Here, in the embodiment of the present invention, the connection plate 123 connects the pair of rotation plates 121 and 122 to the pair of rotation plates 121 and 122 in the middle portion in the height direction H, and the pair of rotation rollers ( 130) is positioned on top of the connection plate 123 between the pair of rotation plates 121 and 122 as an example.

And, for example, the pair of rotation rollers 130 are rotatably coupled to a pair of rotation plates 121 and 122 spaced apart from each other at an upper portion of the connection plate 123 .

Meanwhile, the fruit sorting unit 100 according to the embodiment of the present invention may further include a first class moving unit 171 and a second class moving unit 172.

The first grade moving unit 171 and the second class moving unit 172 according to an embodiment of the present invention are disposed on both sides of the fruit sorting member in the horizontal direction W, respectively. Through this, when the fruit sorting member rotates in either direction of both sides of the vertical direction (L), the target fruit (F) seated on the pair of rotating rollers (130) is separated from the pair of rotating rollers (130). , the separated target fruit F moves along one of the first class moving unit 171 and the second class moving unit 172 .

Here, although not shown in the figure, at the ends of the first class moving unit 171 and the second class moving unit 172, a subject moving through the first class moving unit 171 and the second class moving unit 172 A collection container for collecting the fruit F may be disposed.

Meanwhile, as shown in FIGS. 13 and 14 , the camera assembly 160 according to an embodiment of the present invention may include the plurality of camera modules 161 and the camera bracket 162 described above. In addition, the camera assembly 160 may include a plurality of lighting units 163 and 164 .

In the camera bracket 162 according to an embodiment of the present invention, a plurality of camera modules 161, as shown in FIG. Supports each camera module 161 to be able to shoot.

In addition, as an example, each camera module 161 photographs one target fruit F at least six times while the target fruit F rotates as the pair of rotating rollers 130 rotate. In the present invention, it is taken as an example that each camera module 161 takes three images while the target fruit F rotates and acquires three images each, so that a total of nine images of the target fruit F are captured. .

That is, the classification model 210 according to an embodiment of the present invention may be configured to use a total of 9 image sets of one target fruit F as training data and input data.

In the embodiment of the present invention, the plurality of camera modules 161 are spaced apart from the top of the pair of rotating rollers 130 at a certain angle so that the target fruit F can be photographed at various angles, and the target fruit F is captured. For example, watching and filming.

For example, as shown in FIG. 14 , three camera modules 161 may be installed on the camera bracket 162 so as to be radially arranged with the target fruit F as the center at intervals of 120°.

Here, the plurality of lighting units 163 and 164 include three first camera modules 161 disposed near each camera module 161 and a second camera module 161 located on the upper side of the target fruit F. Including is an example.

The first lighting unit 163 and the second lighting unit 164 according to an embodiment of the present invention diffuse light such as a diffusion plate on the front surface of the sheet so that the light is not concentrated on the target fruit F and is irradiated in the form of indirect light. For example, it is arranged.

Again, referring to FIGS. 5 to 7 , the fruit transfer unit 300 according to the embodiment of the present invention may include a first conveyor unit 320 and a second conveyor unit 310 .

The first conveyor unit 320 according to the embodiment of the present invention transports the target fruit F in the vertical direction L. In addition, the second conveyor unit 310 transfers the target fruits F transferred from the first conveyor unit 320 one by one in the vertical direction L to the side of the fruit sorting unit 100 so that the pair of rotating rollers 130 Place one target fruit (F) on

Here, the second conveyor unit 310, as shown in FIG. 10, is provided in a form in which one target fruit F can be seated and is arranged at a predetermined interval in the vertical direction L. For example, including a fruit tray 311 . In the present invention, as shown in FIG. 10, a semicircular depression is formed so that each pair of adjacent fruit trays 311 move one target fruit F, so that one target fruit F is adjacent to the fruit tray 311. It is erotic to move while seated on a pair of fruit trays 311.

In addition, as shown in FIG. 6 , the second conveyor unit 310 according to the embodiment of the present invention is installed at an angle inclined from the first conveyor unit 320 to move the target fruit F as an example. do.

And, as shown in FIGS. 6 and 10 , the end of the second conveyor unit 310 is formed higher than the pair of rotating rollers 130 as an example. Through this, according to the driving of the second conveyor unit 310, the target fruit F in the fruit tray 311 located at the end of the second conveyor unit 310 falls to the pair of rotating rollers 130, For example, it is seated on a pair of rotating rollers 130 .

Here, the fruit sorting device 10 according to the embodiment of the present invention may further include a movement detection sensor 400 for detecting the movement of the target fruit F by a pair of rotating rollers 130. .

In the embodiment of the present invention, it is exemplified that the movement detection sensor 400 detects the movement of the target fruit F at the end of the second conveyor unit 310 . That is, after the target fruit F is detected by the movement detection sensor 400, it falls onto the pair of rotating rollers 130. In an embodiment of the present invention, it is assumed that an infrared sensor or the like is applied as the movement detection sensor 400 .

As shown in FIG. 8 , the sorting processing unit 200 according to an embodiment of the present invention includes a first conveyor driving unit 312 for rotating the first conveyor unit 320 and the first conveyor unit 320. By controlling the rotating second conveyor driving unit 322, the target fruit F is transferred to the fruit sorting unit 100 through the first conveyor unit 320 and the second conveyor unit 310.

As shown in FIG. 6 , it is taken as an example that the first conveyor unit 320 and the second conveyor unit 310 according to the embodiment of the present invention are positioned above the ground. In this case, a worker may place the target fruit F on the first conveyor unit 320 . As another example, the fruit transfer unit 300 further includes a third conveyor unit, and in a state in which the third conveyor unit is connected to the second conveyor unit 310 at an angle from the ground, the operator removes the target fruit F from a fruit box placed on the ground. ) may be configured to be transferred by putting it on the third conveyor unit.

According to the configuration as described above, the sorting process of the fruit sorting device 10 according to the embodiment of the present invention will be described in detail.

First, in a state where the target fruits F are placed on the first conveyor unit 320, the sorting processing unit 200 uses the first conveyor driving unit 312 and the second conveyor driving unit 322 to classify the target fruit F. ) is controlled so that the target fruits F are transferred to the second conveyor unit 310 via the first conveyor unit 320.

Then, when the target fruit F placed on the pair of fruit trays 311 of the second conveyor unit 310 moves to the end of the second conveyor unit 310 and is detected by the movement detection sensor 400, sorting The processing unit 200 drives the second conveyor driving unit 322 until the target fruit F falls, and then stops the second conveyor driving unit 322 . Here, whether or not the target fruit F falls can be recognized through a set time considering the installation position of the movement detection sensor 400 and the rotational speed of the second conveyor driving unit 322 . Here, the sorting processing unit 200 may also stop driving the first conveyor driving unit 312 when the fall of the target fruit F is recognized.

When the sorting processing unit 200 recognizes that the target fruit F has fallen and is seated on the pair of rotating rollers 130, the roller driving unit 150 is rotated to rotate the object seated on the pair of rotating rollers 130. Let the fruit (F) rotate.

Then, while the target fruit F rotates, the selection processing unit 200 controls the camera module 161 to photograph the target fruit F a preset number of times, for example, three times as described above.

Then, the sorting processing unit 200 receives the images taken by each camera module 161, and based on this, reads and classifies the target fruit F currently seated on the pair of rotating rollers 130. To derive the results.

Then, the sorting processing unit 200 controls the sorting rotation driving unit 140 to rotate in either direction of both sides of the horizontal direction (W) based on the classification result, so that the object seated on the pair of rotating rollers 130 One side of the fruit (F) is classified. Here, the selection processing unit 200 may stop the driving of the roller driving unit 150 after the photographing by the camera module 161 is completed.

When the selection of one target fruit (F) is completed as described above, the process described above is repeatedly performed to complete the selection process for all target fruits (F).

Meanwhile, as described above, the selection processing unit 200 according to an embodiment of the present invention may operate in a data collection mode in addition to the fruit classification mode.

At this time, the sorting processing unit 200 may control the camera module 161 to capture the target fruit in a rotating state in the same way as in the fruit classification mode, and store the corresponding image in the data storage unit 260 . In addition, the sorting processing unit 200 transmits the corresponding image to the fruit sorting support server 20 according to a request from the fruit sorting support server 20 or a separate criterion for learning or re-learning of the classification model 31. can be made available.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: fruit sorting device 100: fruit sorting unit
110: sorting frame 120: sorting rotating member
121,122: rotation plate 123: connection plate
130: rotating roller 140: sorting rotation drive
150: roller driving unit 160: camera assembly
161: camera module 162: camera bracket
163: first lighting unit 164: second lighting unit
171: first class moving unit 172: second class moving unit
200: sorting processing unit 210: classification model
220: processing control unit 230: image display unit
240: selective communication unit 241: internal communication unit
242: external communication unit 250: user input unit
260: data storage unit 300: fruit transfer unit
310: second container unit 311: fruit tray
320: first container unit 400: movement detection sensor
30: Fruit sorting support server

Claims (6)

In the artificial intelligence-based fruit selection support system,
At least one fruit sorting device provided to sort the target fruit;
a fruit sorting support server connected to the fruit sorting device through a network and supporting fruit sorting by the fruit sorting device;
the fruit sorting supporting server supports an artificial intelligence-based classification model for classifying the target fruit in the fruit sorting device based on a photographed image of the target fruit;
The classification model includes a plurality of type-specific classification models classified according to the type of the target fruit;
The artificial intelligence-based fruit sorting support system, characterized in that each of the classification models by type includes a plurality of classification models classified according to the classification criterion level of the corresponding target fruit. According to claim 1,
The classification criterion level is divided into a first level and an N-th level;
The class classification model is
Artificial intelligence-based fruit, characterized in that it includes from the first class classification model for classifying into the first class and other classes to the N-1 class classification model for classifying into the N-1 class and other classes Screening support system. According to claim 2,
the first class classification model is generated by learning using images of the first class and other classes as input training data and images of the first class as output training data;
The N-1 grade classification model is generated by learning using the N-1 grade or higher and N-grade images as input learning data, and the N-1 grade or higher image as output learning data. Artificial intelligence, characterized in that based fruit sorting support system. According to claim 3,
The fruit sorting device operates in one of a data collection mode for photographing a plurality of target fruits and a fruit classification mode for performing classification through the classification model provided from the fruit sorting support server for a plurality of target fruits;
The fruit sorting support server receives an image of the target fruit collected from the fruit sorting device through the data collection mode from the fruit sorting device and learns or relearns the classification model. Fruit sorting support system. According to claim 4,
The fruit sorting support server receives an image of a target fruit captured in the process of operating in the fruit sorting mode from the fruit sorting apparatus, and relearns the classification model. 's fruit sorting support system. According to claim 5,
The classification model is an artificial intelligence-based fruit sorting support system, characterized in that the learning data is a set of images photographed at least twice in at least two or more directions for the target fruit as learning data.

Images