CN111724339A - Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof - Google Patents

Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof Download PDF

Info

Publication number
CN111724339A
CN111724339A CN202010321209.3A CN202010321209A CN111724339A CN 111724339 A CN111724339 A CN 111724339A CN 202010321209 A CN202010321209 A CN 202010321209A CN 111724339 A CN111724339 A CN 111724339A
Authority
CN
China
Prior art keywords
pistachio
head
tail
image
identifying
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010321209.3A
Other languages
Chinese (zh)
Inventor
钟球盛
侯文峰
吴隽
吴瑞祥
林荣墩
王伯飘
庞炯林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hezhong Power Machinery Factory Gulao Town Heshan City
Guangzhou Panyu Polytechnic
Original Assignee
Hezhong Power Machinery Factory Gulao Town Heshan City
Guangzhou Panyu Polytechnic
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hezhong Power Machinery Factory Gulao Town Heshan City, Guangzhou Panyu Polytechnic filed Critical Hezhong Power Machinery Factory Gulao Town Heshan City
Priority to CN202010321209.3A priority Critical patent/CN111724339A/en
Publication of CN111724339A publication Critical patent/CN111724339A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

The invention discloses a device and a method for recognizing the head and the tail of an pistachio nut based on multi-channel information fusion. The imaging device comprises a computer, an industrial camera, a lens, a spherical integral light source, a fixed adjusting bracket and the like. The identification method of the horizontal left and right side imaging devices comprises the following steps: collecting a color image, converting a gray image, filtering a median of the image, binarizing, detecting Sobel edges, transforming image morphology, extracting a pistachio nut target area, constructing a deep learning network structure, training a network, identifying the tail part and the tail part of the pistachio nut and the like. The identification method of the vertical imaging device comprises the following steps: the method comprises the steps of image acquisition, gray level image conversion, image segmentation, BLOB analysis, feature extraction, object screening, object left side area calculation, object right side area calculation, head and tail identification based on area difference and the like. The method can realize multi-channel image information fusion and comprehensive evaluation and improve the accuracy of the head and tail identification of the pistachio nuts.

Description

Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof
Technical Field
The invention belongs to the technical field of automatic optical detection, and particularly relates to a device for identifying the head and the tail of a split-core fruit by multi-channel fusion of horizontal imaging and vertical imaging, and an identification algorithm of the device.
Background
The pistachio nuts are also called pistachio nuts, are rich in vitamins, mineral substances and antioxidant elements, have the remarkable characteristics of low fat, low calorie and high fiber, have medical and health care effects on cardiovascular and cerebrovascular diseases, senile retinopathy, aging prevention and the like, are health care leisure food which is well sold in the nut market in the world, become the fifth nut crop in the world, and are an essential part for the shelling work.
At present, the identification of the head and tail of the pistachio is carried out by using the most common and traditional method of a manual visual detection method, and the head and tail of the pistachio are checked through human eyes. However, the visual method is time-consuming and labor-consuming, high in labor intensity, low in efficiency and large in subjective factor influence, and visual fatigue of an inspector is easily caused along with the lengthening of detection time, so that the discrimination error rate is high. In the process of the shell opening link of the pistachio nuts, only manual identification is needed, so that time and labor are wasted, and errors are easily identified, so that the visual method cannot meet the requirements of high efficiency and high quality of a production line.
Chinese patent publication No. [ CN 107303566] proposes an open-close port sorting apparatus based on the impact sound characteristic of the physical structure of pistachio nuts. The pistachio nut shells are separated through the impact cavity, the acoustic sensor clamping device, the impacted device, the support frame, the control analysis device and the sorting execution device. However, this method does not have accurate positioning and correction, and noise-induced errors should easily occur.
Chinese patent publication No. [ CN 109410209] proposes a method for detecting exogenous foreign matters in nuts based on deep learning classification. Designing a deep learning classification network, finishing detection model training, and storing a trained network connection weight matrix; and (3) sending the region image of the object to be detected into the trained deep learning classification network for image classification detection, and completing the detection of the exogenous foreign matters of the nuts. However, the method needs a large number of training samples, and the training and detection identification computation amount is large; the identification accuracy of the pistachio nuts by using the method is not high, and the phenomenon of overfitting is easy to occur.
In summary, there is a great need in the art for a device and a method for identifying the head and tail of pistachio nuts, which can quickly and effectively identify the head and tail of pistachio nuts, improve the working efficiency, reduce the labor cost, and liberate the labor force. The intellectualization and the automation of the pistachio nut processing production are realized.
Disclosure of Invention
The invention aims to provide a device and a method for recognizing the head and the tail of an pistachio nut by multi-channel fusion of horizontal imaging and vertical imaging, and solves the problems of long time consumption and low efficiency of manual detection.
The invention also aims to provide a detection method for the head and the tail of the pistachio nuts.
The invention adopts a technical scheme that: the device comprises an object stage on a production line, wherein a computer and an industrial camera which are connected through signals, a ball integral light source and a coaxial light source are respectively arranged on the object stage in the vertical direction, and the object stage bracket on the production line is provided with the horizontally placed pistachios: the computer collects color images through an industrial camera, a coaxial light source and a spherical integral light source. The identification algorithm flow of the horizontal left and right imaging units comprises the following steps: collecting a color image, converting a gray image, filtering a median of the image, binarizing, detecting Sobel edges, transforming image morphology, extracting a pistachio nut target area, constructing a deep learning network structure, training a network, identifying the tail part and the tail part of the pistachio nut and the like; the identification algorithm flow of the vertical imaging unit comprises the following steps: the method comprises the steps of image acquisition, gray level image conversion, image segmentation, BLOB analysis, feature extraction, object screening, object left side area calculation, object right side area calculation, head and tail identification based on area difference and the like. The three imaging devices and the recognition algorithm thereof can realize multi-channel image information fusion, comprehensive evaluation and output the optimal recognition decision of the head and the tail of the pistachio nuts.
Further, the industrial camera of the vertical imaging unit is photographing right against the middle portion of the pistachio.
Further, the industrial cameras of the horizontal imaging unit are shooting right towards both ends of the pistachio nut.
Furthermore, the detection system further comprises a light source controller connected with an industrial camera through signals, the light source controller is connected with a spherical integral light source, the spherical integral light source is located above the pistachio nuts, and the curved surface of the shell of the pistachio nuts is uniformly illuminated.
Furthermore, the ball integral light source is arranged on the assembly line type objective table and is vertical to the horizontal plane of the objective table.
The other technical scheme adopted by the invention is as follows: a method for recognizing the head and the tail of an pistachio nut by multi-channel fusion of horizontal imaging and vertical imaging. The method for identifying the head and the tail of the pistachio nuts in the horizontal direction comprises the following steps:
step 1, acquiring a color image of the head of the pistachio nut through an industrial camera 102 and a spherical integral light source 103;
step 2, converting the color image into a gray image in a binaryzation mode, preprocessing the gray image, and calculating by using a Sobel operator to obtain the gradient of the pistachio nuts in the x and y directions;
step 3, smoothing the image by using a low-pass filter median filter;
step 4, filling and removing white noise points except for pistachio nuts in the black vacant part of the background of the image by using image morphological processing;
step 5, intercepting the effective area of the whole pistachio nut by using a minimum rectangle;
step 6, constructing a deep learning network structure, wherein the network structure can be one of (MobileNet, VGG16, LeNet5, RseNet and the like);
step 7, identifying the probability value η of the head of the output pistachio at the left side by using a deep learning networkLAnd probability value η of the head to the rightRWherein, if ηLGreater than 50%, the head is to the left, if ηRGreater than 50%, the head is on the right.
Further, the deep learning framework in step 6 may adopt various image classification networks and transfer learning methods.
A method for recognizing the head and the tail of an pistachio nut by multi-channel fusion of horizontal imaging and vertical imaging. The method for identifying the head and the tail of the pistachio in the vertical direction comprises the following steps:
step 1, acquiring a color image of the head of the pistachio nut through a coaxial light source 106 and a spherical integral light source by an industrial camera 105;
step 2, converting the color image into a gray image and preprocessing the gray image to obtain the color image;
step 3, segmenting the image;
step 4, extracting pistachio nut targets through Blob analysis;
step 5, calculating and obtaining the gradient of the pistachio in the x and y directions by adopting a Sobel operator to obtain an edge image of the pistachio;
step 6, obtaining extracted pistachio nut features through edge contour and gradient analysis of pistachio nuts;
step 7, as shown in fig. 5, obtaining a straight line L with the farthest distance between the two ends, making a perpendicular bisector A of the straight line L, and dividing the pistachio nut image into a left half part and a right half part;
step 8, calculating the area A1 of the left contour of the target based on the vertical bisector A;
step 9, calculating the area A2 of the right contour of the target based on the perpendicular bisector A;
step 10, identifying the head and the tail through the area difference, and outputting a probability value η that the head is positioned on the left sideMSee formula (1);
Figure BDA0002460288070000041
the head of pistachio is used as the judgment standard of probability, since ηMA negative probability will occur if ηMIf greater than 0, it is the head, if ηMLess than 0 is determined to be trailing, ηMHas a distribution interval of ηM∈ (-1, 1), then ηMIf normalization is required, the probability data from three different viewing angles are distributed in (0, 1), specifically in format (2).
Figure BDA0002460288070000051
The pistachio nut head and tail identification method based on multi-channel fusion of horizontal imaging and vertical imaging can realize multi-channel information fusion of the left imaging unit, the middle top imaging unit and the right imaging unit and obtain the probability value η of the optimal decisionEndIn the formula (3)
Figure BDA0002460288070000052
Probability value η if optimal decisionEndGreater than 50%, the head of the pistachio is on the left, otherwise ηEndLess than 50%, the head of the pistachio nut is on the right.
The invention has the beneficial effects that: the invention relates to a device and a method for recognizing the head and the tail of an pistachio nut based on multi-channel information fusion of horizontal imaging and vertical imaging, which introduce machine vision and deep learning into the detection of the pistachio nut, solve the problems of long time consumption and low efficiency of manual detection, realize the quick imaging of the pistachio nut by adopting a spherical integral light source and a multi-view imaging mode, improve the effective information content, accurately extract a target, complete the high-performance recognition of the head and the tail of the pistachio nut by adopting a decision mode with optimal probability, have the accuracy up to 98 percent, effectively replace manpower and greatly improve the production efficiency of processing the pistachio nut.
Drawings
Fig. 1 is a schematic view of a mechanical assembly of a pistachio nut head and tail recognition device based on multi-channel fusion of horizontal imaging and vertical imaging, provided by the invention;
FIG. 2 is a flow chart of an pistachio nut head and tail recognition algorithm for multi-channel information of horizontal imaging and vertical imaging provided by the invention;
FIG. 3 is a schematic view of a pistachio nut head provided by the present invention;
FIG. 4 is a schematic view of the tail of a pistachio nut provided by the present invention;
FIG. 5 is a schematic top view of a pistachio nut according to the present invention.
In fig. 1, 101, computer, 102, X industrial camera, 103, spherical integral light source (3), 104, pistachio, 105, Z industrial camera, 106, coaxial light source, 107, -X industrial camera, 108, stage on the assembly line.
In fig. 2, S100 starts detection of pistachio nuts. The division into three branches is then done in parallel, as follows:
a first branch, for left side imaging, S401 acquiring a pistachio image from the left side for judging whether the image is a head, S402 obtaining a target region of the left pistachio, S403 pre-predicting the left image by using Sobel operator, binarization, image morphology, etcProcessing, S404 constructing a left deep learning network structure (such as MobileNet, VGG16, LeNet5 and RseNet), S405 training the left network structure, S406 identifying a left image of the pistachio nut, and S407 outputting a probability value η for identifying the head and the tail of the pistachio nutL。
The second branch is that for right side imaging, S201 acquires a pistachio nut image from the right side for judging whether the image is a tail or not, S202 obtains a target area of the right-side pistachio nut, S203 preprocesses the right-side image by using Sobel operator, binarization, image morphology and the like, S204 constructs a deep learning network structure (such as one of MobileNet, VGG16, LeNet5 and RseNet), S205 trains the right-side network structure, S206 identifies the right-side image of the pistachio nut, and S207 outputs a probability value η for identifying the head and the tail of the pistachio nutR。
The third branch, for the middle (top) imaging, S301 collects the partial image in the middle of the pistachio, S302 obtains the target area of the middle part, S303 preprocesses the image of the middle part by using Sobel operator, binarization and the like and divides the image into two left and right area parts, S304 extracts the area A1 of the left area, S305 extracts the area A2 of the right area, S306 compares the difference of the left and right areas, and S307 outputs the probability value η for identifying the head and the tail of the pistachioM。
The head of pistachio is used as the judgment standard of probability, since ηMA negative probability will occur if ηMIf greater than 0, it is the head, if ηMLess than 0 is determined to be trailing, ηMHas a distribution interval of ηM∈ (-1, 1), then ηMIf normalization is required, the probability data from three different viewing angles are distributed in (0, 1), specifically in format (2).
Figure BDA0002460288070000071
The pistachio nut head and tail identification method based on multi-channel fusion of horizontal imaging and vertical imaging can realize multi-channel information fusion of the left imaging unit, the middle top imaging unit and the right imaging unit and obtain the probability value η of the optimal decisionEndIn the formula (3)
Figure BDA0002460288070000072
Probability value η if optimal decisionEndGreater than 50%, the head of the pistachio is on the left, otherwise ηEndLess than 50%, the head of the pistachio nut is on the right.
110 in fig. 3, the pistachio nuts are characterized by irregular round shapes at the heads, and the most remarkable characteristic is that a round small interval exists in the middle;
fig. 4, 120, pistachio nut tail. The most remarkable characteristic is that a diamond skeleton extending from the left side to the right side exists;
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention provides a mechanical assembly diagram of a pistachio nut head and tail recognition device with multi-channel fusion of horizontal imaging and vertical imaging, which is shown in figure 1. The method comprises the following steps: the system comprises an object stage on the production line, wherein pistachios to be detected are arranged on the object stage, and an imaging processing and identifying unit for three visual angles. Wherein, the signals of the industrial camera 107 and the industrial camera 102 of the horizontal direction imaging processing unit on the left side and the right side of the pistachio are transmitted to the computer 101 through the TCP/IP Ethernet. Similarly, next to the vertical imaging processing unit at the top of the pistachio, the industrial camera 105 signal of the processing unit is also transmitted to the computer 101 through the TCP/IP ethernet. The three units each employ a spherical integral light source 103, and particularly the top imaging unit also employs a coaxial light source 106, the coaxial light source 106 being located above the spherical integral light source 103 and below the industrial camera 105. All industrial cameras and light sources are fixed on a designed bracket 109.
The key parameters for machine vision system imaging are: 1) the distance between the industrial camera 105 and the surface of the pistachio 104 is 40 cm-50 cm; 2) the distance between the ball point light source and the pistachio 104 is 15 cm-25 cm; 3) the distance between the coaxial light source 106 and the pistachio 104 is 30 cm-40 cm; 4) the object stage is provided with a plurality of grooves with the width of 1.5 cm-2. cm and is matched with the production line to operate quickly.
Referring to fig. 2, another technical solution adopted by the present invention is: a method for recognizing the head and the tail of an pistachio nut by multi-channel fusion of horizontal imaging and vertical imaging. Wherein, the horizontal direction discerns pistachio nut head and afterbody, includes the following step:
step 1, acquiring a color image of the head of the pistachio nut through an industrial camera 102 and a spherical integral light source 103;
step 2, converting the color image into a gray image in a binaryzation mode, preprocessing the gray image, and calculating by using a Sobel operator to obtain the gradient of the pistachio nuts in the x and y directions;
step 3, smoothing the image by using a low-pass filter median filter;
step 4, filling and removing white noise points except for pistachio nuts in the black vacant part of the background of the image by using image morphological processing;
step 5, intercepting the effective area of the whole pistachio nut by using a minimum rectangle;
step 6, constructing a deep learning network structure, wherein the network structure can be one of (MobileNet, VGG16, LeNet5, RseNet and the like);
step 7, identifying the probability value η of the head of the output pistachio at the left side by using a deep learning networkLAnd probability value η of the head to the rightR；
Further, the deep learning framework in step 6 may adopt various image classification networks and transfer learning methods.
In addition, the method for identifying the head and the tail of the pistachio nuts in the vertical direction comprises the following steps:
step 1, acquiring a color image of the head of the pistachio nut through a coaxial light source 106 and a spherical integral light source by an industrial camera 105;
step 2, converting the color image into a gray image and preprocessing the gray image to obtain the color image;
step 3, segmenting the image;
step 4, extracting pistachio nut targets through Blob analysis;
step 5, calculating and obtaining the gradient of the pistachio in the x and y directions by adopting a Sobel operator to obtain an edge image of the pistachio;
step 6, obtaining extracted pistachio nut features through edge contour and gradient analysis of pistachio nuts;
step 7, as shown in fig. 5, obtaining a straight line L with the farthest distance between the two ends, making a perpendicular bisector A of the straight line L, and dividing the pistachio nut image into a left half part and a right half part;
step 8, calculating the area A1 of the left contour of the target based on the vertical bisector A;
step 9, calculating the area A2 of the right contour of the target based on the perpendicular bisector A;
step 10, identifying the head and the tail through the area difference, and outputting a probability value η that the head is positioned on the left sideMSee formula (1);
Figure BDA0002460288070000091
the head of pistachio is used as the judgment standard of probability, since ηMA negative probability will occur if ηMIf greater than 0, it is the head, if ηMLess than 0 is determined to be trailing, ηMHas a distribution interval of ηM∈ (-1, 1), then ηMIf normalization is required, the probability data from three different viewing angles are distributed in (0, 1), specifically in format (2).
Figure BDA0002460288070000101
The pistachio nut head and tail identification method based on multi-channel fusion of horizontal imaging and vertical imaging can realize multi-channel information fusion of the left imaging unit, the middle top imaging unit and the right imaging unit and obtain the probability value η of the optimal decisionEndIn the formula (3)
Figure BDA0002460288070000102
Probability value η if optimal decisionEndMore than 50 percent of the total weight of the coating,the head of the pistachio is on the left, otherwise ηEndLess than 50%, the head of the pistachio nut is on the right.
The above examples of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. The utility model provides a happy fruit head and tail recognition device based on multichannel information fusion which characterized in that, the device includes: the system comprises an object stage arranged on a production line, wherein pistachios to be detected are arranged on the object stage, and an imaging processing and identifying unit with three visual angles; wherein, the signals of the industrial camera 107 and the industrial camera 102 of the horizontal direction imaging processing unit on the left side and the right side of the pistachio are transmitted to the computer 101 through the TCP/IP Ethernet; similarly, next, the vertical imaging processing unit at the top of the pistachio is started, and the industrial camera 105 signal of the processing unit is also transmitted to the computer 101 through the TCP/IP Ethernet; the three units respectively adopt a spherical integral light source 103, and particularly, the vertical imaging processing unit at the top also adopts a coaxial light source 106, and the coaxial light source 106 is positioned above the spherical integral light source 103 and below an industrial camera 105; all industrial cameras and light sources are fixed on a designed bracket 109.
2. The device for identifying the head and the tail of the pistachio nut based on the multi-channel information fusion as claimed in claim 1, wherein the distance between the industrial camera 105 and the surface of the pistachio nut 104 is 40 cm-50 cm, the distance between the spherical integral light source and the pistachio nut 104 is 15 cm-25 cm, the distance between the coaxial light source 106 and the pistachio nut 104 is 30 cm-40 cm, and a plurality of grooves with the width of 1.5 cm-2. cm are arranged on the object stage and cooperate with a production line to operate rapidly.
3. The device for the recognition of the beginning and the end of the pistachio based on the multi-channel information fusion as claimed in claim 1, wherein the device further comprises a computer processing system connected with the industrial camera signal, and the computer processing system comprises an image data analysis unit and an image data acquisition unit.
4. The device for identifying the head and the tail of the pistachio based on multi-channel information fusion as claimed in claim 1, wherein the horizontal direction imaging processing unit and the vertical imaging processing unit further comprise a method for identifying the head and the tail of the pistachio by the horizontal direction imaging processing unit and the vertical imaging processing unit, and specifically:
the horizontal direction imaging processing unit identifies the left side of the pistachio and outputs a probability value, the horizontal analysis imaging processing unit identifies the right side of the pistachio and outputs a probability value, and the vertical imaging processing unit identifies the middle part of the pistachio and outputs a probability value; and carrying out normalization operation on the probability values of the three units and outputting the optimal decision probability of the head and the tail of the pistachio nuts.
5. The device for identifying the heads and the tails of the pistachios based on multi-channel information fusion as claimed in claim 4, further comprising a procedure for identifying the heads and the tails of the pistachios based on the horizontal direction imaging processing unit, specifically:
collecting a color image, converting a gray image, filtering a median of the image, binarizing, detecting Sobel edges, transforming image morphology, extracting a pistachio target area, constructing a deep learning network structure, training a network, identifying the head and the tail of the pistachio, and outputting a probability value of identifying the head and the tail.
6. The device for identifying the head and the tail of the pistachio nut based on the multi-channel information fusion as claimed in claim 4, further comprising a process for identifying the middle part of the pistachio nut based on the vertical imaging processing unit, specifically:
the method comprises the steps of image acquisition, gray level map conversion, image segmentation, BLOB analysis, feature extraction, Sobel operator edge detection, edge contour and gradient analysis, target left side area calculation A1, target right side area calculation A2, head and tail recognition based on area difference and head and tail probability value output.
7. The device for identifying the beginning and the end of the pistachio nut based on multi-channel information fusion as claimed in claim 4, wherein the operation of normalizing the probability values of the three units, in particular taking the head of the pistachio nut as the criterion of the probability, is characterized in that η is used for judging the probabilityMA negative probability will occur if ηMIf greater than 0, it is the head, if ηMIf less than 0, it is determined as tail, i.e., ηMHas a distribution interval of ηM∈ (-1, 1), then ηMNormalization processing is required, wherein the distribution of probability data of three different viewing angles is (0, 1), specifically:
Figure FDA0002460288060000031
8. a method for identifying the head and the tail of an pistachio nut based on multi-channel information fusion is characterized by comprising a process of identifying the head and the tail of the pistachio nut based on a vertical imaging processing unit, and specifically comprises the following steps:
step S11, acquiring a color image of the head of the pistachio nut through the industrial camera 105, the coaxial light source 106 and the spherical integral light source;
step S12, converting the color image into a gray image and preprocessing the gray image to obtain the color image;
step S13, segmenting the image;
step S14, extracting effective targets of pistachio nuts through Blob analysis;
step S15, calculating and acquiring the gradient of the pistachio in the x and y directions by adopting a Sobel operator to obtain an edge image of the pistachio;
step S16, obtaining extracted pistachio nut features through the edge contour and gradient analysis of the pistachio nuts;
s17, obtaining a straight line L with the farthest distance between the two ends, making a vertical bisector A of the straight line L, and dividing the pistachio nut image into an upper half part and a lower half part;
step S18, calculating the area A1 of the left contour of the target based on the vertical bisector A;
step S19, calculating the area A2 of the right outline of the target based on the vertical bisector A;
step S20, identifying head and tail through area difference, and outputting probability value η that head is positioned at left sideMThe concrete formula is as follows:
Figure FDA0002460288060000032
9. the method for pistachio nut head and tail recognition based on multi-channel information fusion as claimed in claim 8, further comprising: the process for identifying the head and the tail of the pistachio nuts based on the horizontal direction imaging processing unit specifically comprises the following steps:
step S21, acquiring a color image of the head of the pistachio nut through the industrial camera 102 and the spherical integral light source 103;
step S22, converting the color image into a gray image in a binaryzation mode and preprocessing the gray image, and calculating and obtaining the gradient of the pistachio nuts in the x and y directions by using a Sobel operator;
step S23, smoothing the image by median filtering with a low-pass filter;
step S24, filling the black vacant part of the background of the image by using image morphological processing, and removing white noise points except for pistachio nuts;
step S25, intercepting the effective area of the whole pistachio by using the minimum rectangle;
step S26, constructing a deep learning network structure;
step S27, recognizing probability value η of head of output pistachio at left side by using deep learning networkLAnd probability value η of the head to the rightR。
10. The method for pistachio nut head and tail recognition based on multi-channel information fusion as claimed in claim 8, further comprising: based on the following three methodsOutputting the head-tail optimal decision of the pistachio by the probability value, and obtaining the probability value η of the optimal decision by adopting the multi-channel information fusion of the left imaging unit, the middle top imaging unit and the right imaging unitEndThe calculation formula is as follows:
Figure FDA0002460288060000041
wherein, ηLProbability value of the pistachio nut head on the left side ηRThe probability value that the head of the head and the tail of the pistachio is positioned on the right side is shown;
Figure FDA0002460288060000051
the normalized probability value of the head part of the head and the tail part of the pistachio on the left side;
ηEndis the probability value of the preferred decision.
CN202010321209.3A 2020-04-21 2020-04-21 Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof Pending CN111724339A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010321209.3A CN111724339A (en) 2020-04-21 2020-04-21 Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010321209.3A CN111724339A (en) 2020-04-21 2020-04-21 Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof

Publications (1)

Publication Number Publication Date
CN111724339A true CN111724339A (en) 2020-09-29

Family

ID=72563657

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010321209.3A Pending CN111724339A (en) 2020-04-21 2020-04-21 Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof

Country Status (1)

Country Link
CN (1) CN111724339A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112150461A (en) * 2020-10-19 2020-12-29 北京百度网讯科技有限公司 Method and device for evaluating head-tail definition of cell image
CN113139581A (en) * 2021-03-23 2021-07-20 广东省科学院智能制造研究所 Image classification method and system based on multi-image fusion

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112150461A (en) * 2020-10-19 2020-12-29 北京百度网讯科技有限公司 Method and device for evaluating head-tail definition of cell image
CN112150461B (en) * 2020-10-19 2024-01-12 北京百度网讯科技有限公司 Method and apparatus for assessing head-to-tail sharpness of a cell image
CN113139581A (en) * 2021-03-23 2021-07-20 广东省科学院智能制造研究所 Image classification method and system based on multi-image fusion
CN113139581B (en) * 2021-03-23 2023-09-01 广东省科学院智能制造研究所 Image classification method and system based on multi-image fusion

Similar Documents

Publication Publication Date Title
CN101944231B (en) Method for extracting wheatear morphological parameters
CN109447945B (en) Quick counting method for basic wheat seedlings based on machine vision and graphic processing
CN111724339A (en) Happy fruit head and tail recognition device based on multi-channel information fusion and recognition method thereof
WO2009045035A1 (en) White and brown rice appearance characteristics measurement system and method
CN113109348B (en) Paddle image transfer printing defect identification method based on machine vision
CN110705655A (en) Tobacco leaf classification method based on coupling of spectrum and machine vision
CN104820985B (en) A kind of cable visual defects detection and character recognition device
CN107256398B (en) Feature fusion based individual milk cow identification method
CN110705468B (en) Eye movement range identification method and system based on image analysis
CN111852792B (en) Fan blade defect self-diagnosis positioning method based on machine vision
CN110705639A (en) Medical sperm image recognition system based on deep learning
CN103076288A (en) Automatic fish flesh grading device and method based on computer vision
CN108960011A (en) The citrusfruit image-recognizing method of partial occlusion
CN106097368A (en) A kind of recognition methods in veneer crack
WO2023168984A1 (en) Area-array camera-based quality inspection method and system for cathode copper
CN111462058A (en) Method for quickly detecting effective ears of rice
CN106872473A (en) A kind of potato defects detection identifying system design based on machine vision
CN111161295A (en) Background stripping method for dish image
CN111667475A (en) Machine vision-based Chinese date grading detection method
CN113222959A (en) Fresh jujube wormhole detection method based on hyperspectral image convolutional neural network
CN116612192A (en) Digital video-based pest and disease damage area target positioning method
CN213042312U (en) Happy fruit head and tail recognition device based on multi-channel information fusion
KR102030768B1 (en) Poultry weight measuring method using image, recording medium and device for performing the method
CN113180640B (en) Size measuring method and system for livestock breeding
CN112907516B (en) Sweet corn seed identification method and device for plug seedling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination