CN109570051B - Chinese chestnut wormhole detection device based on machine vision, laser and acoustics - Google Patents

Chinese chestnut wormhole detection device based on machine vision, laser and acoustics Download PDF

Info

Publication number
CN109570051B
CN109570051B CN201910038085.5A CN201910038085A CN109570051B CN 109570051 B CN109570051 B CN 109570051B CN 201910038085 A CN201910038085 A CN 201910038085A CN 109570051 B CN109570051 B CN 109570051B
Authority
CN
China
Prior art keywords
laser
detection
screw rod
chinese chestnut
guide rail
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910038085.5A
Other languages
Chinese (zh)
Other versions
CN109570051A (en
Inventor
何文斌
明五一
马军
都金光
谢欢
贾豪杰
魏爱云
李晓科
曹阳
刘琨
侯俊剑
楚昌昊
张笑笑
王萌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou University of Light Industry
Original Assignee
Zhengzhou University of Light Industry
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou University of Light Industry filed Critical Zhengzhou University of Light Industry
Priority to CN201910038085.5A priority Critical patent/CN109570051B/en
Publication of CN109570051A publication Critical patent/CN109570051A/en
Application granted granted Critical
Publication of CN109570051B publication Critical patent/CN109570051B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36—Sorting apparatus characterised by the means used for distribution
    • B07C5/361—Processing or control devices therefor, e.g. escort memory
    • B07C5/362—Separating or distributor mechanisms

Landscapes

  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

Chinese chestnut wormhole detection device based on machine vision, laser and acoustics, including the frame, from the left hand right side has set gradually in the frame and has promoted conveyer belt and horizontal conveyer belt, promotes the conveyer belt and is the high slope setting in the low right side in a left side, has set gradually in the frame from the left hand right side and has been located the camera darkroom detection module, laser detection module, acoustics detection module and the sorting unit of horizontal conveyer belt top, and the frame lateral part is provided with automatically controlled module, chinese chestnut positioner and chinese chestnut unlocking device. According to the invention, under the multi-stage combined intelligent information fusion of machine vision, acoustic detection and laser detection, the high-precision Chinese chestnut wormhole detection can be realized, on one hand, the intelligent level of detection can be effectively improved, on the other hand, the sorting accuracy is improved and the device cost is reduced through the multi-stage combined intelligent detection. The method can be used for on-line real-time detection of the wormholes of the Chinese chestnut agricultural products, has important significance for improving the development of deep processing of the Chinese chestnut agricultural products in China, and has good market application prospect.

Description

Chinese chestnut wormhole detection device based on machine vision, laser and acoustics
Technical Field
The invention belongs to the technical field of agricultural product appearance quality detection used in the agricultural field, and particularly relates to a Chinese chestnut wormhole detection device based on machine vision, laser and acoustics.
Background
The chestnut essence has the reputation of 'the king of dried fruits', the chestnuts are rich in nutrition, the fruits contain 70.1 percent of sugar and starch, and 7 percent of protein. In addition, the chestnut cake also contains fat, calcium, phosphorus, iron, various vitamins and trace elements, particularly the contents of vitamin C, B1 and carotene are higher than those of common dry fruits, the chestnut is rich in nutrition, and besides starch, the chestnut cake also contains nutrient substances such as monosaccharide and disaccharide, carotene, thiamine, riboflavin, nicotinic acid, ascorbic acid, protein, fat, inorganic salts and the like. However, in the planting process of the Chinese chestnut, various attacks of plant diseases and insect pests are often encountered, so that the wormhole detection of the Chinese chestnut is very key for improving the grade of the Chinese chestnut.
At present, the detection of the Chinese chestnut moth eyes mostly stays in the stage of identification and judgment by artificial senses, and a lot of inconvenience exists. The mode of choosing by hand is relied on to accomplish the sorting of chinese chestnut wormhole, and firstly a large amount of labours of wasting, along with the harvest season the recruitment is too old and useless, secondly there is the hourglass to examine easily, and the testing result uniformity is poor, efficient, is difficult to satisfy high standard hierarchical requirement, is unfavorable for realizing the automation. With the accelerated progress of the artificial intelligence technology, the accuracy of the insect eye detection can be improved by utilizing the multi-stage combined detection of the multi-source information intelligent fusion.
Although the Chinese chestnut grading method and equipment based on computer vision gradually become hot at present, the method is generally limited to laboratory research or a camera is adopted to take pictures of the Chinese chestnuts, and the adopted technical means is single. Under the conditions that the wormhole is small or the shape and the position are complex, the judgment cannot be accurately carried out, and the risk of missing detection exists; or the hyperspectral imaging technology is adopted to detect the Chinese chestnut to be detected, but the cost and the complexity of the detection device are increased.
In order to overcome the defects of the conventional Chinese chestnut wormhole sorting device, a multistage combined Chinese chestnut wormhole intelligent detection method and device based on machine vision, laser and acoustics are provided.
Some relevant patent documents are found through the search of domestic patent documents, and the following are mainly found:
1. the publication number is CN 108801971A, the name is 'a mold infection Chinese chestnut detection method based on a hyperspectral imaging technology', the invention provides a mold infection Chinese chestnut detection method based on a hyperspectral imaging technology, but the high-precision detection of Chinese chestnut wormholes cannot be realized by adopting a single hyperspectral imaging technical scheme.
2. The invention discloses a Chinese chestnut automatic sorting device and a Chinese chestnut automatic sorting method with the publication number of CN 107350173A, and the name of the invention is 'Chinese chestnut automatic sorting device and method', which adopts a plurality of cameras, computers, PLC, electromagnetic valves and the like, utilizes the algorithm of machine vision to identify and sort Chinese chestnuts, and carries out filtering and contour extraction according to surface color pictures of the Chinese chestnuts to obtain the effective proportion of the area of damaged Chinese chestnuts and realize the sorting of the Chinese chestnuts to be classified, but the classification device can not accurately judge under the conditions of small wormholes or complex shapes and positions.
3. The invention discloses a multi-information fusion electromagnetic crawler type Chinese chestnut detection and screening device with the publication number of CN 108620338A, and is named as a multi-information fusion electromagnetic crawler type Chinese chestnut detection and screening device.
Although the above patent provides a sorting method and a grading device for Chinese chestnuts, some devices can carry out grading according to appearance characteristics, but the detection of the damage and mildew of the Chinese chestnuts can be realized only by adopting a machine vision method or combining with hyperspectral and infrared spectrum technologies. However, the detection of the chestnut wormholes is not comprehensive, and particularly, the probability of missed detection exists under the condition that the wormholes are small or the shapes and the positions are complex.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides the Chinese chestnut wormhole detection device based on machine vision, laser and acoustics, which is used for further improving the accuracy of wormhole detection and reducing the omission factor.
In order to solve the technical problems, the invention adopts the following technical scheme: chinese chestnut wormhole detection device based on machine vision, laser and acoustics, which comprises a frame 1, frame 1 is gone up from the left hand right side and has set gradually promotion conveyer belt 2 and horizontal conveyer belt 4, it is the high slope setting of low right side in a left side to promote conveyer belt 2, frame 1 is gone up from the left hand right side and has set gradually the darkroom detection module 3 of shooing that is located horizontal conveyer belt 4 top, laser detection module 5, acoustics detection module 6 and sorting unit 7, 1 lateral part of frame is provided with automatically controlled module 9, chinese chestnut positioner and chinese chestnut unlocking device, chinese chestnut positioner's working portion is located and promotes conveyer belt 2 top, chinese chestnut unlocking device's working portion is located between acoustics detection module 6 and the sorting unit 7 of horizontal conveyer belt 4 top.
Promote conveyer belt 2 and horizontal conveyer belt 4 and make by metal heat conductor material, promote conveyer belt 2 and horizontal conveyer belt 4 upper surface and evenly offer along left and right sides direction interval 5~8cm and be used for depositing the recess 54 of waiting to carry the chinese chestnut, promote conveyer belt 2 and horizontal conveyer belt 4 combination and become a complete conveyer belt.
The photographing darkroom detection module 3 comprises a cavity box body 10, an annular LED lamp 11 and an industrial camera 12, the bottom of the cavity box body 10 is open and vertically faces downwards to the horizontal conveying belt 4, the left side and the right side of the cavity box body 10 are fixed on the rack 1 through screws, and the left side and the right side of the bottom edge of the cavity box body 10 are provided with horizontal conveying rectangular gaps convenient for Chinese chestnuts; an annular LED lamp 11 and an industrial camera 12 are installed at a top center position within the cavity housing 10.
The laser detection module 5 comprises a first X-direction stepping motor 13, a first X-direction moving screw rod 14, a first X-direction support 15, a first Y-direction stepping motor 16, a first Y-direction moving screw rod 17, a first Y-direction guide rail 18, a laser detection clamp 19 and a laser detection sub-module 20;
the X direction is a left-right horizontal direction, the Y direction is a front-back horizontal direction, a first support guide rail 55 which is arranged along the X direction and is a rectangular frame is respectively arranged on the front side and the back side of the horizontal conveying belt 4 on the machine frame 1, a first X-direction movement screw rod 14 is arranged in each first support guide rail 55, one end of each first X-direction movement screw rod 14 is rotatably connected to the first support guide rail 55, the other end of each first X-direction movement screw rod 14 is in transmission connection with the first X-direction stepping motor 13, the first X-direction movement screw rods 14 are in threaded connection with the first X-direction supports 15 which slide along the first support guide rails 55, the front end and the back end of each first Y-direction guide rail 18 are fixedly connected to the upper ends of the two first X-direction supports 15, each first Y-direction guide rail 18 is of a hollow rod-shaped structure, each first Y-direction movement screw rod 17 is arranged in the first Y-direction guide rail 18, one end of the first Y-direction moving screw rod 17 is rotatably connected to one end inside the first Y-direction guide rail 18, the other end of the first Y-direction moving screw rod 17 is in transmission connection with the first Y-direction stepping motor 16, the laser detection clamp 19 is in threaded connection with the first Y-direction moving screw rod 17 and is arranged in a sliding mode along the first Y-direction guide rail 18, and the laser detection sub-module 20 is installed on the laser detection clamp 19 and irradiates towards the groove 54.
The acoustic detection module 6 comprises a second X-direction stepping motor 21, a second X-direction movement screw rod 22, a second X-direction bracket 23, a second Y-direction stepping motor 24, a second Y-direction movement screw rod 25, a second Y-direction guide rail 26, a sliding frame, a C-axis movement motor 27, a C-axis speed reducer 28, a high-pressure air needle 29, a high-pressure air pipe 30, an acoustic sensor 31 and a first electromagnetic valve 51; the X direction is the left-right horizontal direction, and the Y direction is the front-back horizontal direction;
a second support guide rail 56 which is arranged along the X direction and is a rectangular frame is respectively arranged at the front side and the rear side of the horizontal conveying belt 4 on the frame 1, a second X direction movement screw rod 22 is arranged in each second support guide rail 56, one end of the second X direction movement screw rod 22 is rotatably connected on the second support guide rail 56, the other end of the second X direction movement screw rod 22 is in transmission connection with the second X direction stepping motor 21, the second X direction movement screw rod 22 is in threaded connection with the second X direction support 23 which slides along the second support guide rail 56, the front end and the rear end of the second Y direction guide rail 26 are fixedly connected at the upper ends of the two second X direction support 23, the second Y direction guide rail 26 is a hollow rod-shaped structure, the second Y direction movement screw rod 25 is arranged in the second Y direction guide rail 26 along the front and rear direction, one end of the second Y direction movement screw rod 25 is rotatably connected at one end in the second Y direction guide rail 26, the other end of the second Y-direction moving screw 25 is in transmission connection with the second Y-direction stepping motor 24, the C-axis moving motor 27 and the C-axis reducer 28 are arranged inside the sliding frame, the sliding frame is in threaded connection with the second Y-direction moving screw 25 and is arranged in a sliding manner along the second Y-direction guide rail 26, the power output end of the C-axis moving motor 27 is in transmission connection with the power input end of the C-axis reducer 28, the high-pressure air needle 29 is installed on the power output shaft of the C-axis reducer 28, the power output shaft of the C-axis reducer 28 is horizontally arranged in the left-right direction, the high-pressure air needle 29 is perpendicular to the power output shaft of the C-axis reducer 28, the air inlet of the high-pressure air needle 29 is connected with the air outlet of the high-pressure air pipe 30, the first electromagnetic valve 51 is arranged on the high-pressure air pipe 30, the.
The sorting device 7 comprises a sorting support plate 32, a sorting cylinder 33, a sorting rod 34, a sorting rod head 35 and a slideway 36; the slideway 36 is installed at the right front side of the frame 1 through bolt connection; select separately backup pad 32 and pass through bolted connection and install perpendicularly in 1 right side rear side position of frame, select separately cylinder 33 fixed mounting and select separately backup pad 32 left surface, select separately cylinder 33's telescopic link and select separately 34 coaxial responses of pole and be connected, select separately 34 poles along left right direction level setting.
The Chinese chestnut positioning device is a cold air spray pipe 37 and a water dropping pipe which are arranged above the lifting conveyer belt 2 side by side, the water dropping pipe is positioned at the left side of the cold air spray pipe 37, and a water dropping hole of the water dropping pipe and a nozzle of the cold air spray pipe 37 are both arranged downwards and face towards the passing groove 54; the cold air spray pipe 37 is provided with a second electromagnetic valve 52;
the Chinese chestnut unlocking device is a hot air spray pipe 38 arranged above the horizontal conveying belt 4, the nozzle of the hot air spray pipe 38 is arranged downwards and faces towards the passing groove 54, and a third electromagnetic valve 53 is arranged on the hot air spray pipe 38.
The electric control module 9 comprises an embedded ARM singlechip, an input/output submodule, an industrial Ethernet communication submodule, a display submodule and a light warning submodule; the embedded ARM single chip microcomputer is connected with the input/output submodule through an on-chip bus;
the embedded ARM single chip microcomputer is respectively connected with the industrial camera 12 and the laser detection sub-module 20 through an industrial Ethernet communication sub-module; the industrial Ethernet communication submodule is internally provided with a duplex industrial Ethernet port, the input/output submodule is respectively connected with the first X-direction stepping motor 13, the first Y-direction stepping motor 16, the second X-direction stepping motor 21, the C-axis motion motor 27, the second Y-direction stepping motor 24 and the sorting cylinder 33 through control signal lines, and the positioning and sorting are realized according to the ordered actions of the stepping motors corresponding to the control flow control; the input/output sub-module is also connected with a first electromagnetic valve 51, a second electromagnetic valve 52 and a third electromagnetic valve 53, and the electromagnetic valves are opened or closed according to the action flow to realize the functions of detection, freezing and unfreezing respectively; the input/output sub-module is also connected with and controls the opening and closing of the annular LED lamp 11 and the brightness of the annular LED lamp; the input/output submodule is also connected to the acoustic sensor 31.
By adopting the technical scheme, the detection method of the Chinese chestnut wormhole detection device based on machine vision, laser and acoustics comprises the following steps,
(1) the lifting conveyer belt 2 and the horizontal conveyer belt 4 are driven by a stepping motor to move from left to right, the Chinese chestnuts are placed into the leftmost groove 54 at the top of the lifting conveyer belt 2 one by one, water in the water dropping pipe is dropped into the groove 54 right below, the Chinese chestnuts are placed into the groove 54 with liquid water of the lifting conveyer belt 2 one by one from the left side, and the lifting conveyer belt 2 and the horizontal conveyer belt 4 drive the Chinese chestnuts to move right;
(2) after the chestnuts move to the lower part of the nozzle of the cold air spray pipe 37, the electric control module 9 controls the second electromagnetic valve 52 to be opened, the high-pressure cold air is sprayed to the groove 54 with the liquid water, and the liquid water is frozen to freeze and fix the chestnuts in the groove 54;
(3) promote conveyer belt 2 and horizontal conveyer belt 4 and drive the chinese chestnut and remove right to the darkroom detection module 3 of shooing in, calculate the chinese chestnut position through the machine vision method of electronic control module 9, the industrial camera 12 of darkroom detection module 3 of shooing is under the control of electronic control module 9, treats the pick-up plate chinese chestnut and shoots, sends to electronic control module 9 through industry ethernet communication submodule, handles through embedded ARM singlechip, and the flow is: by calibrating the industrial camera 12, the height H from the lens of the industrial camera 12 to the real object space is a known quantity, a monocular vision method is adopted, the included angle a can be calculated through a similar triangular proportion, L1 is the length of a pixel and can be obtained through an image processing algorithm, so that the L2 numerical value in the real object space is calculated, and the coordinate position of the Chinese chestnut to be detected in a photographing darkroom is obtained; then, acquiring the size of the Chinese chestnut to be detected and the area information of suspected insect eyes through feature extraction and analysis of the image; the calculated position and size information of the Chinese chestnut to be detected provides positioning information for the subsequent laser detection module 5 and acoustic detection module 6;
(4) the horizontal conveying belt 4 drives the Chinese chestnuts to move rightwards to the position below the laser detection module 5, the first X-direction stepping motor 13 drives the first X-direction moving screw rod 14 to rotate, so that the first X-direction support 15 of the first X-direction moving screw rod 14 moves leftwards or rightwards along the first support guide rail 55, and the first Y-direction guide rail 18, the laser detection clamp 19 and the laser detection sub-module 20 also move leftwards or rightwards; the first Y-direction stepping motor 16 drives the first Y-direction moving screw rod 17 to rotate, so that the laser detection clamp 19 connected with the first Y-direction moving screw rod 17 moves forwards or backwards along the first Y-direction guide rail 18, and the laser detection sub-module 20 also moves forwards or backwards; moving the laser detection clamp 19 to the position above the Chinese chestnut to be detected through X-Y plane positioning, and aligning the laser detection sub-module 20 to the position vertically above the suspected Chinese chestnut wormhole area; after the detection of the laser detection submodule 20 is finished, the detected intensity of the laser reflection signal and the position height information are processed and then are sent to the electric control module 9 for analysis;
the specific detection process of the laser detection submodule 20 in the step (4) is that the laser detection submodule 20 scans suspected Chinese chestnut moth eye areas at intervals of 2mm in the X direction and the Y direction of X-Y plane positioning, the scanning length and the scanning width are both 14mm, the total number of times is 8 × 8=64, and the laser detection submodule 20 acquires a laser measurement height data set H64The height is the distance from a laser emission head in the laser detection submodule to the surface of the Chinese chestnut to be detected, and is as follows:
Figure DEST_PATH_IMAGE001
further, for height data set H64Calculates the average height difference in the X and Y directions as shown in the following equation:
Figure DEST_PATH_IMAGE002
in the above equation, when the height data set index (i or j) is 0 or less or 9 or more, the average height difference (
Figure DEST_PATH_IMAGE003
) Performing the following steps; if the average height difference of the current measurement position (
Figure 171581DEST_PATH_IMAGE003
) If the measured value is larger than the threshold value by 0.18mm, marking the current measuring position as the suspected Chinese chestnut moth eye edge; then carrying out curved surface closing operation on the edge positions (point position set) of the detected suspected Chinese chestnut eyes by using a B spline curve, calculating the area of the suspected Chinese chestnut eye area, and if the area is larger than a threshold value of 15 mm2If yes, the laser detection module 5 judges that the Chinese chestnut to be detected has the defect of the wormhole, otherwise, the laser detection module judges that the Chinese chestnut to be detected does not have the defect of the wormhole;
(5) after the horizontal conveying belt 4 drives the Chinese chestnuts to move rightwards to the position below the acoustic detection module 6, the second X-direction stepping motor 21 drives the second X-direction movement screw rod 22 to rotate, so that the second X-direction support 23 of the second X-direction movement screw rod 22 moves leftwards or rightwards along the second support guide rail 56, and the second Y-direction guide rail 26, the sliding frame, the C-axis movement motor 27, the C-axis speed reducer 28 and the high-pressure air needle 29 also move leftwards or rightwards; the second Y-direction stepping motor 24 drives the second Y-direction moving screw 25 to rotate, so that the carriage with the second Y-direction moving screw 25 moves forward or backward along the second Y-direction rail 26, and the C-axis moving motor 27, the C-axis reducer 28 and the high-pressure air needle 29 also move forward or backward; moving a second C-axis speed reducer 28 to the position above the Chinese chestnut to be detected through X-Y plane positioning, and aligning a high-pressure air needle 29 to the position vertically above suspected Chinese chestnut holes through the rotary motion of the second C-axis speed reducer 28; after the high-pressure air needle 29 sprays the air flow, the acoustic sensor 31 collects the air flow echo signal at the suspected Chinese chestnut eye position, and the air flow echo signal is sent to the electronic control module 9 for analysis.
The concrete process of the acoustic detection module 6 and the electric control module 9 in the step (5) is as follows: the acoustic detection module 6 is positioned and moved to the upper part of the Chinese chestnut to be detected through an X-Y plane under the control of the electric control module 9; acquiring acoustic signal data of three positions through the movement of a C-axis speed reducer 28, wherein the position 1 is vertically above a suspected Chinese chestnut eye area, the position 2 is in the vertical direction of the surface of the suspected Chinese chestnut eye area, and the acoustic signal data pass through the HX49And HY49Data are calculated, and the position 3 is a symmetrical position of the position 1 relative to a perpendicular line of the surface of the suspected Chinese chestnut moth eye area; the electronic control module 9 opens the first electromagnetic valve 51, high-pressure gas acts on the suspected Chinese chestnut eye area through the high-pressure gas needle 29, the acoustic sensor 31 sends data to the electronic control module 9 through the input/output submodule after acquiring the data for 2 seconds, and the electronic control module 9 closes the first electromagnetic valve 51 after success; similarly, the electric control module 9 collects acoustic signal data from the position 2 and the position 3; after the acoustic signal data acquisition of all three positions is completed, the embedded ARM single chip microcomputer of the electric control module 9 processes according to the following procedures:
A. respectively processing acoustic signal data acquired from the position 1, the position 2 and the position 3 according to the processes B-E;
B. carrying out pre-emphasis, framing and windowing (12 short time analysis windows) on 4-second acoustic signal data acquired at each position;
C. for each short-time analysis window, obtaining a corresponding frequency spectrum through FFT (fast Fourier transform), and obtaining a Mel (Mel) frequency spectrum through a Mel filter bank;
D. performing cepstrum analysis (taking logarithm, after DCT discrete cosine transform processing, taking coefficients from 2 to 13 after DCT discrete cosine transform as MFCC coefficients) on the Mel frequency spectrum to obtain 12 Mel frequency cepstrum coefficients MFCC;
E. after 256-level graying is carried out on the 12 Mel frequency cepstrum coefficients MFCC of the 12 short-time analysis windows, a Mel Frequency Cepstrum Coefficient (MFCC) feature graph and a 256-level gray image with the size of 12 × 12 are obtained, further, the 256-level gray image with the size of 12 × 12 is sent into a residual error network for identification, and the identification result is a two-dimensional vector [ P0,P1]When P is0Greater than P1When the Chinese chestnut is detected, the acoustic detection module marks the current position as the Chinese chestnut to be detected has the defect of wormhole, otherwise, the acoustic detection module judges that the Chinese chestnut to be detected does not have the defect of wormhole;
F. summarizing and analyzing the primary results of the position 1, the position 2 and the position 3, if any one of the position 1, the position 2 and the position 3 has a wormhole defect mark, judging that the current Chinese chestnut to be detected has the wormhole defect by the acoustic detection module, otherwise, judging that the current Chinese chestnut to be detected does not have the wormhole defect;
the final identification result after the combined detection of the shooting vision, the laser and the acoustics is displayed by a display sub-module; the chestnuts with the wormholes are also displayed in an alarm mode through the lamplight warning submodule to remind workers; the display sub-module can also carry out statistical analysis on the detected data, and display the statistical data according to hours or batches, so that the data can be conveniently checked manually;
(6) when the detected chestnuts are conveyed to the right below the nozzles of the hot air spray pipes 38, the solid ice in the grooves 54 of the horizontal conveying belt 4 of the metal heat conductor is rapidly heated by hot air to be converted into liquid water, so that the sorting device 7 can conveniently sort;
(7) when the chestnuts with the wormholes are conveyed to the right to the positions corresponding to the front and the back of the sorting device 7, the sorting cylinder 33 drives the sorting rod 34 to move, and the sorting rod head 35 is pushed to push the chestnuts with the wormholes out of the inlet of the slideway 36; the sorting cylinder 33 is connected with the electric control module 9, and under the control of the electric control module 9, the sorting rod 34 is pushed to move, so that the Chinese chestnuts with the defect of wormholes fall into defective frames through the slide way 36, and the qualified Chinese chestnuts are transported to the right side direction by the horizontal conveyor belt 4.
The residual error network comprises 2 residual error blocks, wherein an A1 convolutional layer, an A2 Batch Norm layer, an A3 active layer, a B1 convolutional layer, a B2 Batch Norm layer and a B3 active layer form a first residual error block, the A1 convolutional layer, the A2 Batch Norm layer, the A3 active layer, a B1 convolutional layer, the B2 Batch Norm layer and the B3 active layer are sequentially connected end to end, and further, data of an inlet of the A1 convolutional layer can be inserted between the B2 Batch Norm layer and the B3 active layer in a short circuit mode; the C1 convolution layer, the C2 Batch Norm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer form a second residual block, the C1 convolution layer, the C2 Batch Norm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer are sequentially connected end to end, and further, data of an inlet of the C1 convolution layer can be inserted between the D2 Batch Norm layer and the D3 active layer in a short circuit mode; the first residual block and the second residual block are sequentially connected end to end, the MFCC gray-scale image enters an E1 full-connection layer after being input into the 2 residual blocks of the residual network, data flow into an F1 soft regression layer, and finally an identification result is output.
Furthermore, the residual error network is trained offline, and data of 3160 samples are collected for training in total, wherein 1510 data are data without the bug defect, and 1650 data are data with the bug defect, and the trained model is stored in the embedded ARM single chip microcomputer of the electronic control module 9.
Obtaining the area information of suspected insect eyes according to the machine vision detection result; detecting suspected insect eye position information by machine vision, and obtaining a laser detection result through laser detection; detecting the position information of the suspected insect eye and the curved surface information near the suspected insect eye of the chestnut by using a machine vision detector, and obtaining an acoustic detection result through acoustic detection; from the above machine vision results: the suspected insect eye position and size information, the laser detection result and the acoustic detection result are identified through an SVM model, and the final identification result is output: and normal and bug eyes are output by the display submodule and the lamplight warning submodule.
The SVM model is trained offline, the same training samples are shared with the residual error network, namely data of 3160 samples are collected in total, wherein 1510 data are data without wormhole defects, 1650 data are data with wormhole defects, and trained SVM model parameters are stored in the embedded ARM single chip microcomputer of the electronic control module 9.
The invention can also be set as cascade operation, two devices of the invention and a robot with machine vision are combined into a set of detection unit; the machine hand with the machine vision is arranged between the two devices, the machine vision is utilized to carry out auxiliary positioning, the Chinese chestnuts to be detected are moved from one device to the other device, turning of the Chinese chestnuts is realized in the moving process, and then the detection is carried out, so that the detection accuracy can be further improved.
In summary, compared with the prior art, the invention has the following beneficial effects:
according to the invention, under the multi-stage combined intelligent information fusion of machine vision, acoustic detection and laser detection, the high-precision Chinese chestnut wormhole detection can be realized, on one hand, the intelligent level of detection can be effectively improved, on the other hand, the sorting accuracy is improved and the device cost is reduced through the multi-stage combined intelligent detection. The method can be used for on-line real-time detection of the wormholes of the Chinese chestnut agricultural products, has important significance for improving the development of deep processing of the Chinese chestnut agricultural products in China, and has good market application prospect.
Drawings
FIG. 1 is a schematic perspective view of the device of the present invention from one perspective;
FIG. 2 is a schematic perspective view of the device of the present invention from another perspective;
FIG. 3 is a schematic top view of a laser detection module of the apparatus of the present invention;
FIG. 4 is a left side view of FIG. 3;
FIG. 5 is a schematic diagram of the structure of the acoustic detection module of the apparatus of the present invention;
FIG. 6 is a schematic view of the structure of a sorting device in the apparatus of the present invention;
FIG. 7 is a schematic bottom view of the camera chamber detection module;
FIG. 8 is a schematic diagram of machine vision chestnut location calculation;
FIG. 9 is a schematic diagram of the operation of the laser detection module;
FIG. 10 is a schematic diagram of the operation of the acoustic detection module;
FIG. 11 is a diagram of a residual error network (ResNet) architecture for acoustic signal data processing;
FIG. 12 is a flow chart of a multi-level joint intelligent detection of machine vision, laser and acoustics;
fig. 13 is a structural view of an electronic control module.
Detailed Description
As shown in fig. 1-6, the device for detecting the wormhole of the chestnut based on machine vision, laser and acoustics comprises a frame 1, wherein a lifting conveyer belt 2 and a horizontal conveyer belt 4 are sequentially arranged on the frame 1 from left to right, the lifting conveyer belt 2 is arranged in a left-low-right-high inclined manner, a photographing darkroom detection module 3, a laser detection module 5, an acoustics detection module 6 and a sorting device 7 which are positioned above the horizontal conveyer belt 4 are sequentially arranged on the frame 1 from left to right, an electric control module 9, a chestnut positioning device and a chestnut unlocking device are arranged on the side portion of the frame 1, the working portion of the chestnut positioning device is positioned above the lifting conveyer belt 2, and the working portion of the chestnut unlocking device is positioned between the acoustics detection module 6 and the sorting device 7 which are positioned.
The lifting conveying belt 2 and the horizontal conveying belt 4 are made of metal heat conductor materials, grooves 54 used for storing Chinese chestnuts to be conveyed are uniformly formed in the upper surfaces of the lifting conveying belt 2 and the horizontal conveying belt 4 at intervals of 5-8 cm in the left-right direction, the lifting conveying belt 2 and the horizontal conveying belt 4 are combined into a complete conveying belt, the conveying belt grooves 54 are not replaced for the Chinese chestnuts to be detected, the whole process from lifting to grading is achieved, and a small amount of liquid water is stored in the grooves 54 and used for freezing and fixing the Chinese chestnuts.
The photographing darkroom detection module 3 comprises a cavity box body 10, an annular LED lamp 11 and an industrial camera 12, the bottom of the cavity box body 10 is open and vertically faces downwards to the horizontal conveying belt 4, the left side and the right side of the cavity box body 10 are fixed on the rack 1 through screws, and the left side and the right side of the bottom edge of the cavity box body 10 are provided with horizontal conveying rectangular gaps convenient for Chinese chestnuts; an annular LED lamp 11 and an industrial camera 12 are installed at a top center position within the cavity housing 10.
The laser detection module 5 comprises a first X-direction stepping motor 13, a first X-direction moving screw rod 14, a first X-direction support 15, a first Y-direction stepping motor 16, a first Y-direction moving screw rod 17, a first Y-direction guide rail 18, a laser detection clamp 19 and a laser detection sub-module 20;
the X direction is a left-right horizontal direction, the Y direction is a front-back horizontal direction, a first support guide rail 55 which is arranged along the X direction and is a rectangular frame is respectively arranged on the front side and the back side of the horizontal conveying belt 4 on the machine frame 1, a first X-direction movement screw rod 14 is arranged in each first support guide rail 55, one end of each first X-direction movement screw rod 14 is rotatably connected to the first support guide rail 55, the other end of each first X-direction movement screw rod 14 is in transmission connection with the first X-direction stepping motor 13, the first X-direction movement screw rods 14 are in threaded connection with the first X-direction supports 15 which slide along the first support guide rails 55, the front end and the back end of each first Y-direction guide rail 18 are fixedly connected to the upper ends of the two first X-direction supports 15, each first Y-direction guide rail 18 is of a hollow rod-shaped structure, each first Y-direction movement screw rod 17 is arranged in the first Y-direction guide rail 18, one end of the first Y-direction moving screw rod 17 is rotatably connected to one end inside the first Y-direction guide rail 18, the other end of the first Y-direction moving screw rod 17 is in transmission connection with the first Y-direction stepping motor 16, the laser detection clamp 19 is in threaded connection with the first Y-direction moving screw rod 17 and is arranged in a sliding mode along the first Y-direction guide rail 18, and the laser detection sub-module 20 is installed on the laser detection clamp 19 and irradiates towards the groove 54.
The acoustic detection module 6 comprises a second X-direction stepping motor 21, a second X-direction movement screw rod 22, a second X-direction bracket 23, a second Y-direction stepping motor 24, a second Y-direction movement screw rod 25, a second Y-direction guide rail 26, a sliding frame, a C-axis movement motor 27, a C-axis speed reducer 28, a high-pressure air needle 29, a high-pressure air pipe 30, an acoustic sensor 31 and a first electromagnetic valve 51; the X direction is the left-right horizontal direction, and the Y direction is the front-back horizontal direction;
a second support guide rail 56 which is arranged along the X direction and is a rectangular frame is respectively arranged at the front side and the rear side of the horizontal conveying belt 4 on the frame 1, a second X direction movement screw rod 22 is arranged in each second support guide rail 56, one end of the second X direction movement screw rod 22 is rotatably connected on the second support guide rail 56, the other end of the second X direction movement screw rod 22 is in transmission connection with the second X direction stepping motor 21, the second X direction movement screw rod 22 is in threaded connection with the second X direction support 23 which slides along the second support guide rail 56, the front end and the rear end of the second Y direction guide rail 26 are fixedly connected at the upper ends of the two second X direction support 23, the second Y direction guide rail 26 is a hollow rod-shaped structure, the second Y direction movement screw rod 25 is arranged in the second Y direction guide rail 26 along the front and rear direction, one end of the second Y direction movement screw rod 25 is rotatably connected at one end in the second Y direction guide rail 26, the other end of the second Y-direction moving screw 25 is in transmission connection with the second Y-direction stepping motor 24, the C-axis moving motor 27 and the C-axis reducer 28 are arranged inside the sliding frame, the sliding frame is in threaded connection with the second Y-direction moving screw 25 and is arranged in a sliding manner along the second Y-direction guide rail 26, the power output end of the C-axis moving motor 27 is in transmission connection with the power input end of the C-axis reducer 28, the high-pressure air needle 29 is installed on the power output shaft of the C-axis reducer 28, the power output shaft of the C-axis reducer 28 is horizontally arranged in the left-right direction, the high-pressure air needle 29 is perpendicular to the power output shaft of the C-axis reducer 28, the air inlet of the high-pressure air needle 29 is connected with the air outlet of the high-pressure air pipe 30, the first electromagnetic valve 51 is arranged on the high-pressure air pipe 30, the.
The sorting device 7 comprises a sorting support plate 32, a sorting cylinder 33, a sorting rod 34, a sorting rod head 35 and a slideway 36; the slideway 36 is installed at the right front side of the frame 1 through bolt connection; select separately backup pad 32 and pass through bolted connection and install perpendicularly in 1 right side rear side position of frame, select separately cylinder 33 fixed mounting and select separately backup pad 32 left surface, select separately cylinder 33's telescopic link and select separately 34 coaxial responses of pole and be connected, select separately 34 poles along left right direction level setting.
The Chinese chestnut positioning device is a cold air spray pipe 37 and a water dropping pipe which are arranged above the lifting conveyer belt 2 side by side, the water dropping pipe is positioned at the left side of the cold air spray pipe 37, and a water dropping hole of the water dropping pipe and a nozzle of the cold air spray pipe 37 are both arranged downwards and face towards the passing groove 54; the cold air spray pipe 37 is provided with a second electromagnetic valve 52;
the Chinese chestnut unlocking device is a hot air spray pipe 38 arranged above the horizontal conveying belt 4, a nozzle of the hot air spray pipe 38 is arranged downwards and faces towards a passing groove 54, a third electromagnetic valve 53 is arranged on the hot air spray pipe 38, and the second electromagnetic valve 52 and the third electromagnetic valve 53 are connected with and controlled by the electronic control module 9.
As shown in fig. 13, the electronic control module 9 includes an embedded ARM single-chip microcomputer, an input/output sub-module, an industrial ethernet communication sub-module, a display sub-module, and a light warning sub-module; the embedded ARM single chip microcomputer is connected with the input/output submodule through an on-chip bus; the embedded ARM single chip microcomputer is respectively connected with the industrial camera 12 and the laser detection sub-module 20 through an industrial Ethernet communication sub-module; the industrial Ethernet communication submodule is internally provided with a duplex industrial Ethernet port, the input/output submodule is respectively connected with the first X-direction stepping motor 13, the first Y-direction stepping motor 16, the second X-direction stepping motor 21, the C-axis motion motor 27, the second Y-direction stepping motor 24 and the sorting cylinder 33 through control signal lines, and the positioning and sorting are realized according to the ordered actions of the stepping motors corresponding to the control flow control; the input/output sub-module is also connected with a first electromagnetic valve 51, a second electromagnetic valve 52 and a third electromagnetic valve 53, and the electromagnetic valves are opened or closed according to the action flow to realize the functions of detection, freezing and unfreezing respectively; the input/output sub-module is also connected with and controls the opening and closing of the annular LED lamp 11 and the brightness of the annular LED lamp; the input/output submodule is also connected to the acoustic sensor 31.
The detection method of the Chinese chestnut wormhole detection device based on machine vision, laser and acoustics comprises the following steps:
(1) the lifting conveyer belt 2 and the horizontal conveyer belt 4 are driven by a stepping motor to move from left to right, the Chinese chestnuts are placed into the leftmost groove 54 at the top of the lifting conveyer belt 2 one by one, water in the water dropping pipe is dropped into the groove 54 right below, the Chinese chestnuts are placed into the groove 54 with liquid water of the lifting conveyer belt 2 one by one from the left side, and the lifting conveyer belt 2 and the horizontal conveyer belt 4 drive the Chinese chestnuts to move right;
(2) after the chestnuts move to the lower part of the nozzle of the cold air spray pipe 37, the electric control module 9 controls the second electromagnetic valve 52 to be opened, the high-pressure cold air is sprayed to the groove 54 with the liquid water, and the liquid water is frozen to freeze and fix the chestnuts in the groove 54;
(3) the lifting conveying belt 2 and the horizontal conveying belt 4 are driven by conveying stepping motors (conventional technology in the field), the width of each section of the conveying belt is 30-50 mm, the conveying belt is preliminarily positioned by the stepping motors, Chinese chestnuts to be detected are sent into the photographing darkroom detection module 3, the positions of the Chinese chestnuts are further calculated through a machine vision algorithm of the electric control module 9, and the stepping motors are driven to be precisely positioned, so that the Chinese chestnuts to be detected are located in the area below the industrial camera 12;
promote conveyer belt 2 and horizontal conveyer belt 4 and drive the chinese chestnut and remove right and move to in the darkroom detection module 3 of shooing, as shown in fig. 7 and fig. 8, calculate the chinese chestnut position through the machine vision method of electronic control module 9, the industrial camera 12 of darkroom detection module 3 of shooing is under the control of electronic control module 9, treats the detection board chinese chestnut and shoots, sends to electronic control module 9 through industry ethernet communication submodule, handles through embedded ARM singlechip, the flow is: by calibrating the industrial camera 12, the height H from the lens of the industrial camera 12 to the real object space is a known quantity, a monocular vision method is adopted, the included angle a can be calculated through a similar triangular proportion, L1 is the length of a pixel and can be obtained through an image processing algorithm (conventional technology in the field), so that the L2 numerical value in the real object space is calculated, and the coordinate position of the Chinese chestnut to be detected in a photographing darkroom is obtained; then, acquiring the size of the Chinese chestnut to be detected and the area information of suspected insect eyes through feature extraction and analysis of the image; the calculated position and size information of the Chinese chestnut to be detected provides positioning information for the subsequent laser detection module 5 and acoustic detection module 6;
(4) the horizontal conveying belt 4 drives the Chinese chestnuts to move rightwards to the position below the laser detection module 5, the first X-direction stepping motor 13 drives the first X-direction moving screw rod 14 to rotate, so that the first X-direction support 15 of the first X-direction moving screw rod 14 moves leftwards or rightwards along the first support guide rail 55, and the first Y-direction guide rail 18, the laser detection clamp 19 and the laser detection sub-module 20 also move leftwards or rightwards; the first Y-direction stepping motor 16 drives the first Y-direction moving screw rod 17 to rotate, so that the laser detection clamp 19 connected with the first Y-direction moving screw rod 17 moves forwards or backwards along the first Y-direction guide rail 18, and the laser detection sub-module 20 also moves forwards or backwards; moving the laser detection clamp 19 to the position above the Chinese chestnut to be detected through X-Y plane positioning, and aligning the laser detection sub-module 20 to the position vertically above the suspected Chinese chestnut wormhole area; after the detection of the laser detection submodule 20 is finished, the detected intensity of the laser reflection signal and the position height information are processed and then are sent to the electric control module 9 for analysis;
in the step (4), the specific detection process of the laser detection submodule 20 is that, as shown in fig. 9, the laser detection submodule 20 scans suspected Chinese chestnut eye areas at intervals of 2mm in the X direction and the Y direction of X-Y plane positioning, the scanning length and the scanning width are both 14mm, the total number is 8 × 8=64 times, and the laser detection submodule 20 obtains a laser measurement height data set H64The height is the distance from a laser emission head in the laser detection submodule to the surface of the Chinese chestnut to be detected, and is as follows:
Figure DEST_PATH_IMAGE004
further, for height data set H64Calculates the average height difference in the X and Y directions as shown in the following equation:
Figure 26405DEST_PATH_IMAGE002
in the above equation, when the height data set index (i or j) is 0 or less or 9 or more, the average height difference (
Figure 540563DEST_PATH_IMAGE003
) Performing the following steps; if the average height difference of the current measurement position (
Figure 653881DEST_PATH_IMAGE003
) Greater than 0.18mm of threshold value, marking the current measurement positionSetting the suspected Chinese chestnut moth eye edge; then carrying out curved surface closing operation on the edge positions (point position set) of the detected suspected Chinese chestnut eyes by using a B spline curve, calculating the area of the suspected Chinese chestnut eye area, and if the area is larger than a threshold value of 15 mm2If yes, the laser detection module 5 judges that the Chinese chestnut to be detected has the defect of the wormhole, otherwise, the laser detection module judges that the Chinese chestnut to be detected does not have the defect of the wormhole;
(5) the horizontal conveying belt 4 drives the Chinese chestnuts to move rightwards to the position below the acoustic detection module 6, the second X-direction stepping motor 21 drives the second X-direction movement screw rod 22 to rotate, so that the second X-direction support 23 of the second X-direction movement screw rod 22 moves leftwards or rightwards along the second support guide rail 56, and the second Y-direction guide rail 26, the sliding frame, the C-axis movement motor 27, the C-axis speed reducer 28 and the high-pressure air needle 29 also move leftwards or rightwards; the second Y-direction stepping motor 24 drives the second Y-direction moving screw 25 to rotate, so that the carriage with the second Y-direction moving screw 25 moves forward or backward along the second Y-direction rail 26, and the C-axis moving motor 27, the C-axis reducer 28 and the high-pressure air needle 29 also move forward or backward; moving a second C-axis speed reducer 28 to the position above the Chinese chestnut to be detected through X-Y plane positioning, and aligning a high-pressure air needle 29 to the position vertically above suspected Chinese chestnut holes through the rotary motion of the second C-axis speed reducer 28; after the high-pressure air needle 29 sprays the air flow, the acoustic sensor 31 collects the air flow echo signal at the suspected Chinese chestnut eye position, and the air flow echo signal is sent to the electronic control module 9 for analysis.
The concrete process of the acoustic detection module 6 and the electric control module 9 in the step (5) is as follows: as shown in fig. 10, the acoustic detection module 6 moves to the upper side of the Chinese chestnut to be detected through the X-Y plane positioning under the control of the electric control module 9; acquiring acoustic signal data of three positions through the movement of a C-axis speed reducer 28, wherein the position 1 is vertically above a suspected Chinese chestnut eye area, and the position 2 is a position where the vertical direction of the surface of the suspected Chinese chestnut eye area passes through the HX49And HY49Data are calculated, and the position 3 is a symmetrical position of the position 1 relative to a perpendicular line of the surface of the suspected Chinese chestnut moth eye area; the electric control module 9 opens the first electromagnetic valve 51, high-pressure gas acts on the suspected Chinese chestnut eye area through the high-pressure gas needle 29, and the acoustic sensor 31 collects 2 seconds of dataThen, the data is sent to the electronic control module 9 through the input/output submodule, and the electronic control module 9 closes the first electromagnetic valve 51 after success; similarly, the electric control module 9 collects acoustic signal data from the position 2 and the position 3; after the acoustic signal data acquisition of all three positions is completed, the embedded ARM single chip microcomputer of the electric control module 9 processes according to the following procedures:
A. respectively processing acoustic signal data acquired from the position 1, the position 2 and the position 3 according to the processes B-E;
B. carrying out pre-emphasis, framing and windowing (12 short time analysis windows) on 4-second acoustic signal data acquired at each position;
C. for each short-time analysis window, obtaining a corresponding frequency spectrum through FFT (fast Fourier transform), and obtaining a Mel (Mel) frequency spectrum through a Mel filter bank;
D. performing cepstrum analysis (taking logarithm, after DCT discrete cosine transform processing, taking coefficients from 2 to 13 after DCT discrete cosine transform as MFCC coefficients) on the Mel frequency spectrum to obtain 12 Mel frequency cepstrum coefficients MFCC;
E. after 256-level graying is carried out on the 12 Mel frequency cepstrum coefficients MFCC of the 12 short-time analysis windows, a Mel Frequency Cepstrum Coefficient (MFCC) feature graph and a 256-level gray image with the size of 12 × 12 are obtained, further, the 256-level gray image with the size of 12 × 12 is sent into a residual error network for identification, and the identification result is a two-dimensional vector [ P0,P1]When P is0Greater than P1When the Chinese chestnut is detected, the acoustic detection module marks the current position as the Chinese chestnut to be detected has the defect of wormhole, otherwise, the acoustic detection module judges that the Chinese chestnut to be detected does not have the defect of wormhole;
F. summarizing and analyzing the primary results of the position 1, the position 2 and the position 3, if any one of the position 1, the position 2 and the position 3 has a wormhole defect mark, judging that the current Chinese chestnut to be detected has the wormhole defect by the acoustic detection module, otherwise, judging that the current Chinese chestnut to be detected does not have the wormhole defect;
the final identification result after the combined detection of the shooting vision, the laser and the acoustics is displayed by a display sub-module; the chestnuts with the wormholes are also displayed in an alarm mode through the lamplight warning submodule to remind workers; the display sub-module can also carry out statistical analysis on the detected data, and display the statistical data according to hours or batches, so that the data can be conveniently checked manually;
(6) when the detected chestnuts are conveyed to the right below the nozzles of the hot air spray pipes 38, the solid ice in the grooves 54 of the horizontal conveying belt 4 of the metal heat conductor is rapidly heated by hot air to be converted into liquid water, so that the sorting device 7 can conveniently sort;
(7) when the chestnuts with the wormholes are conveyed to the right to the positions corresponding to the front and the back of the sorting device 7, the sorting cylinder 33 drives the sorting rod 34 to move, and the sorting rod head 35 is pushed to push the chestnuts with the wormholes out of the inlet of the slideway 36; the sorting cylinder 33 is connected with the electric control module 9, and under the control of the electric control module 9, the sorting rod 34 is pushed to move, so that the Chinese chestnuts with the defect of wormholes fall into defective frames through the slide way 36, and the qualified Chinese chestnuts are transported to the right side direction by the horizontal conveyor belt 4.
As shown in fig. 11, the residual network comprises 2 residual blocks, wherein the first residual block comprises an a1 convolutional layer, an a2 Batch Norm layer, an A3 active layer, a B1 convolutional layer, a B2 Batch Norm layer and a B3 active layer, the a1 convolutional layer, the a2 Batch Norm layer, the A3 active layer, the B1 convolutional layer, the B2 Batch Norm layer and the B3 active layer are sequentially connected end to end, and further, data at the inlet of the a1 convolutional layer can be inserted between the B2 Batch Norm layer and the B3 active layer in a short circuit manner; the C1 convolution layer, the C2 BatchNorm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer form a second residual block, the C1 convolution layer, the C2 Batch Norm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer are sequentially connected end to end, and further, data at the inlet of the C1 convolution layer can be inserted between the D2 Batch Norm layer and the D3 active layer in a short circuit mode; the first residual block and the second residual block are sequentially connected end to end, the MFCC gray-scale image enters an E1 full-connection layer after being input into the 2 residual blocks of the residual network, data flow into an F1 soft regression layer, and finally an identification result is output.
Furthermore, the residual error network (Res-Net) is trained offline, and data of 3160 samples are collected for training in total, wherein 1510 data are data without wormhole defects, and 1650 data are data with wormhole defects, and the trained model is stored in the embedded ARM single chip microcomputer of the electronic control module 9.
As shown in fig. 12, the area information of suspected moth eyes is obtained from the machine vision detection result; detecting suspected insect eye position information by machine vision, and obtaining a laser detection result through laser detection; detecting the position information of the suspected insect eye and the curved surface information near the suspected insect eye of the chestnut by using a machine vision detector, and obtaining an acoustic detection result through acoustic detection; the machine vision result (suspected insect eye position and size information), the laser detection result and the acoustic detection result are identified through an SVM model, the final identification result (normal and insect eyes) is output, and then the final identification result is output through a display sub-module and a light warning sub-module.
The SVM model is trained offline, the training samples are the same as those of the residual error network (Res-Net), namely data of 3160 samples collected in total are trained (wherein 1510 are data without the bug defect and 1650 are data with the bug defect), and the trained SVM model parameters are stored in the embedded ARM single chip microcomputer of the electronic control module 9.
The present embodiment is not intended to limit the shape, material, structure, etc. of the present invention in any way, and any simple modification, equivalent change and modification made to the above embodiments according to the technical spirit of the present invention are within the scope of the technical solution of the present invention.

Claims (8)

1. Chestnut wormhole detection device based on machine vision, laser and acoustics its characterized in that: the Chinese chestnut sorting machine comprises a rack (1), wherein a lifting conveying belt (2) and a horizontal conveying belt (4) are sequentially arranged on the rack (1) from left to right, the lifting conveying belt (2) is arranged in a left-low-right-high-inclination manner, a photographing darkroom detection module (3), a laser detection module (5), an acoustic detection module (6) and a sorting device (7) which are positioned above the horizontal conveying belt (4) are sequentially arranged on the rack (1) from left to right, an electric control module (9), a Chinese chestnut positioning device and a Chinese chestnut unlocking device are arranged on the side portion of the rack (1), the working portion of the Chinese chestnut positioning device is positioned above the lifting conveying belt (2), and the working portion of the Chinese chestnut unlocking device is positioned between the acoustic detection module (6) and the sorting device (7) which;
the electric control module (9) comprises an embedded ARM singlechip, an input/output submodule, an industrial Ethernet communication submodule, a display submodule and a light warning submodule; the embedded ARM single chip microcomputer is connected with the input/output submodule through an on-chip bus;
the detection device also comprises a residual error network, wherein the residual error network comprises 2 residual error blocks, an A1 convolution layer, an A2 Batch Norm layer, an A3 active layer, a B1 convolution layer, a B2 Batch Norm layer and a B3 active layer form a first residual error block, the A1 convolution layer, the A2 Batch Norm layer, the A3 active layer, the B1 convolution layer, the B2 Batch Norm layer and the B3 active layer are sequentially connected end to end, and further, data at the inlet of the A1 convolution layer can be inserted between the B2 Batch Norm layer and the B3 active layer in a short circuit mode; the C1 convolution layer, the C2 BatchNorm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer form a second residual block, the C1 convolution layer, the C2 Batch Norm layer, the C3 active layer, the D1 convolution layer, the D2 Batch Norm layer and the D3 active layer are sequentially connected end to end, and further, data at the inlet of the C1 convolution layer can be inserted between the D2 Batch Norm layer and the D3 active layer in a short circuit mode; the first residual block and the second residual block are sequentially connected end to end; the residual error network is trained offline, 3160 samples of data are collected for training in total, wherein 1510 are data without the bug defect, 1650 are data with the bug defect, and the trained model is stored in the embedded ARM single chip microcomputer of the electric control module.
2. The chestnut eye detection device based on machine vision, laser and acoustics of claim 1, characterized in that: the lifting conveying belt (2) and the horizontal conveying belt (4) are made of metal heat conductor materials, grooves (54) used for storing Chinese chestnuts to be conveyed are uniformly formed in the upper surfaces of the lifting conveying belt (2) and the horizontal conveying belt (4) at intervals of 5-8 cm in the left-right direction, and the lifting conveying belt (2) and the horizontal conveying belt (4) are combined into a complete conveying belt.
3. The chestnut eye detection device based on machine vision, laser and acoustics of claim 2, characterized in that: the camera darkroom detection module (3) comprises a cavity box body (10), an annular LED lamp (11) and an industrial camera (12), the bottom of the cavity box body (10) is open and vertically faces downwards to the horizontal conveying belt (4), the left side and the right side of the cavity box body (10) are fixed on the rack (1) through screws, and horizontal conveying rectangular gaps convenient for Chinese chestnuts are reserved on the left side and the right side of the edge of the bottom of the cavity box body (10); an annular LED lamp (11) and an industrial camera (12) are mounted at the top center position in the cavity box body (10).
4. The chestnut eye detection device based on machine vision, laser and acoustics of claim 3, characterized in that: the laser detection module (5) comprises a first X-direction stepping motor (13), a first X-direction moving screw rod (14), a first X-direction support (15), a first Y-direction stepping motor (16), a first Y-direction moving screw rod (17), a first Y-direction guide rail (18), a laser detection clamp (19) and a laser detection sub-module (20);
the X direction is a left and right horizontal direction, the Y direction is a front and back horizontal direction, a first support guide rail (55) which is arranged along the X direction and is a rectangular frame is respectively arranged on the front side and the back side of the horizontal conveying belt (4) on the rack (1), a first X direction movement screw rod (14) is arranged in each first support guide rail (55), one end of each first X direction movement screw rod (14) is rotatably connected onto the first support guide rail (55), the other end of each first X direction movement screw rod (14) is in transmission connection with the first X direction stepping motor (13), the first X direction movement screw rods (14) are in threaded connection with the first X direction supports (15) which slide along the first support guide rails (55), the front end and the back end of each first Y direction guide rail (18) are fixedly connected onto the upper ends of the two first X direction supports (15), and the first Y direction guide rails (18) are of a hollow rod-shaped structure, a first Y-direction movement screw rod (17) is arranged in a first Y-direction guide rail (18) along the front-back direction, one end of the first Y-direction movement screw rod (17) is rotatably connected to one end of the inner portion of the first Y-direction guide rail (18), the other end of the first Y-direction movement screw rod (17) is in transmission connection with a first Y-direction stepping motor (16), a laser detection clamp (19) is in threaded connection with the first Y-direction movement screw rod (17) and is arranged in a sliding mode along the first Y-direction guide rail (18), and a laser detection sub-module (20) is installed on the laser detection clamp (19) and irradiates towards a groove (54).
5. The chestnut eye detection device based on machine vision, laser and acoustics of claim 4, characterized in that: the acoustic detection module (6) comprises a second X-direction stepping motor (21), a second X-direction movement screw rod (22), a second X-direction support (23), a second Y-direction stepping motor (24), a second Y-direction movement screw rod (25), a second Y-direction guide rail (26), a sliding frame, a C-axis movement motor (27), a C-axis speed reducer (28), a high-pressure air needle (29), a high-pressure air pipe (30), an acoustic sensor (31) and a first electromagnetic valve (51); the X direction is the left-right horizontal direction, the Y direction is the front-back horizontal direction,
second support guide rails (56) which are arranged along the X direction and are rectangular frames are respectively arranged on the front side and the rear side of the horizontal conveying belt (4) on the rack (1), one second X-direction movement screw rod (22) is arranged in each second support guide rail (56), one end of each second X-direction movement screw rod (22) is rotatably connected onto the second support guide rail (56), the other end of each second X-direction movement screw rod (22) is in transmission connection with the second X-direction stepping motor (21), the second X-direction movement screw rods (22) are in threaded connection with the second X-direction supports (23) which slide along the second support guide rails (56), the front end and the rear end of each second Y-direction guide rail (26) are fixedly connected to the upper ends of the two second X-direction supports (23), each second Y-direction guide rail (26) is of a hollow rod-shaped structure, and each second Y-direction movement screw rod (25) is arranged inside each second Y-direction guide rail (26) along the front-back direction, one end of a second Y-direction motion screw rod (25) is rotatably connected with one end inside a second Y-direction guide rail (26), the other end of the second Y-direction motion screw rod (25) is in transmission connection with a second Y-direction stepping motor (24), a C-axis motion motor (27) and a C-axis speed reducer (28) are arranged inside a sliding frame, the sliding frame is in threaded connection with the second Y-direction motion screw rod (25) and is arranged in a sliding way along the second Y-direction guide rail (26), the power output end of the C-axis motion motor (27) is in transmission connection with the power input end of the C-axis speed reducer (28), a high-pressure air needle (29) is arranged on the power output shaft of the C-axis speed reducer (28), the power output shaft of the C-axis speed reducer (28) is horizontally arranged along the left-right direction, the high-pressure air needle (29) is vertical to the power output shaft of the C-axis speed reducer (28), and the air inlet of the, the first electromagnetic valve (51) is arranged on the high-pressure air pipe (30), an air inlet of the high-pressure air pipe (30) is connected with an air compressor, and the acoustic sensor (31) is arranged at an air jet of the high-pressure air needle (29).
6. The chestnut eye detection device based on machine vision, laser and acoustics of claim 5, characterized in that: the sorting device (7) comprises a sorting support plate (32), a sorting cylinder (33), a sorting rod (34), a sorting rod head (35) and a slideway (36); the slide way (36) is installed at the right front side of the rack (1) through bolt connection; select separately backup pad (32) and pass through bolted connection and install perpendicularly in frame (1) right side rear side position, select separately cylinder (33) fixed mounting in select separately backup pad (32) left surface, the telescopic link of selecting separately cylinder (33) is connected with the axial with selecting pole (34), selects pole (34) along left right direction level setting.
7. The chestnut eye detection device based on machine vision, laser and acoustics of claim 6, characterized in that: the Chinese chestnut positioning device is a cold air spray pipe (37) and a water dropping pipe which are arranged above the lifting conveying belt (2) side by side, the water dropping pipe is positioned at the left side of the cold air spray pipe (37), and a water dropping hole of the water dropping pipe and a nozzle of the cold air spray pipe (37) are both arranged downwards and face towards a passing groove (54); a second electromagnetic valve (52) is arranged on the cold air spray pipe (37);
the Chinese chestnut unlocking device is a hot air spray pipe (38) arranged above the horizontal conveying belt (4), a nozzle of the hot air spray pipe (38) is downwards arranged and faces towards a passing groove (54), and a third electromagnetic valve (53) is arranged on the hot air spray pipe (38).
8. The chestnut eye detection device based on machine vision, laser and acoustics of claim 7, characterized in that: the embedded ARM single chip microcomputer is respectively connected with an industrial camera (12) and a laser detection sub-module (20) through an industrial Ethernet communication sub-module; the industrial Ethernet communication submodule is internally provided with a duplex industrial Ethernet port, the input/output submodule is respectively connected with a first X-direction stepping motor (13), a first Y-direction stepping motor (16), a second X-direction stepping motor (21), a C-axis motion motor (27), a second Y-direction stepping motor (24) and a sorting cylinder (33) through control signal lines, and the positioning and sorting are realized according to the ordered actions of the stepping motors corresponding to the control flow control; the input/output sub-module is also connected with a first electromagnetic valve (51), a second electromagnetic valve (52) and a third electromagnetic valve (53), and the electromagnetic valves are opened or closed according to the action flow to realize the functions of detection, freezing and unfreezing respectively; the input/output sub-module is also connected with and controls the opening and closing of the annular LED lamp (11) and the brightness of the annular LED lamp; the input/output submodule is also connected to an acoustic sensor (31).
CN201910038085.5A 2019-01-16 2019-01-16 Chinese chestnut wormhole detection device based on machine vision, laser and acoustics Active CN109570051B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910038085.5A CN109570051B (en) 2019-01-16 2019-01-16 Chinese chestnut wormhole detection device based on machine vision, laser and acoustics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910038085.5A CN109570051B (en) 2019-01-16 2019-01-16 Chinese chestnut wormhole detection device based on machine vision, laser and acoustics

Publications (2)

Publication Number Publication Date
CN109570051A CN109570051A (en) 2019-04-05
CN109570051B true CN109570051B (en) 2020-09-11

Family

ID=65916581

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910038085.5A Active CN109570051B (en) 2019-01-16 2019-01-16 Chinese chestnut wormhole detection device based on machine vision, laser and acoustics

Country Status (1)

Country Link
CN (1) CN109570051B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110290257B (en) * 2019-07-09 2020-07-07 北京邮电大学 Mobile phone rear cover visual detection production line
CN110838107B (en) * 2019-10-31 2023-02-17 广东华中科技大学工业技术研究院 Method and device for intelligently detecting defects of 3C transparent component by variable-angle optical video
CN111601087A (en) * 2020-05-25 2020-08-28 广东智源机器人科技有限公司 Visual inspection equipment and processing apparatus of tableware
CN114308698B (en) * 2021-12-23 2023-08-11 芜湖万联新能源汽车零部件有限公司 Quick temperature measuring and sorting device for forging workpieces
CN113984770B (en) * 2021-12-30 2022-03-22 山东蒜都农产品物流园有限公司 Garlic storage digital management alarm device

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007301545A (en) * 2006-05-10 2007-11-22 Gana Engineering:Kk Chestnut sorting apparatus
CN101592631A (en) * 2009-07-10 2009-12-02 浙江大学 Automatic detection device for quality of brake pad
CN101603927A (en) * 2009-07-17 2009-12-16 南京农业大学 A kind of device and usage of Non-Destructive Testing abundance of water pears defective
DE19816881B4 (en) * 1998-04-17 2012-01-05 Gunther Krieg Method and device for detecting and distinguishing between contaminations and acceptances as well as between different colors in solid particles
CN202421173U (en) * 2012-01-12 2012-09-05 河南科技大学 Egg sorting and picture-taking device
CN104668199A (en) * 2014-12-02 2015-06-03 浙江大学 Automatic fruit grading device based on machine vision and bio-speckle
CN105251705A (en) * 2015-11-23 2016-01-20 上海电机学院 Red jujube screening device
CN105675720A (en) * 2016-04-13 2016-06-15 浙江大学 Fruit firmness information online collecting system and method
CN107336056A (en) * 2017-08-30 2017-11-10 无锡鑫旭润科技有限公司 A kind of fixture based on frozen section technology
CN107350168A (en) * 2017-08-28 2017-11-17 宁夏大学 A kind of potato Fast nondestructive evaluation self-grading device and method
CN109129255A (en) * 2018-10-25 2019-01-04 天津职业技术师范大学 Semiconductor Refrigerating suction cup

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19816881B4 (en) * 1998-04-17 2012-01-05 Gunther Krieg Method and device for detecting and distinguishing between contaminations and acceptances as well as between different colors in solid particles
JP2007301545A (en) * 2006-05-10 2007-11-22 Gana Engineering:Kk Chestnut sorting apparatus
CN101592631A (en) * 2009-07-10 2009-12-02 浙江大学 Automatic detection device for quality of brake pad
CN101603927A (en) * 2009-07-17 2009-12-16 南京农业大学 A kind of device and usage of Non-Destructive Testing abundance of water pears defective
CN202421173U (en) * 2012-01-12 2012-09-05 河南科技大学 Egg sorting and picture-taking device
CN104668199A (en) * 2014-12-02 2015-06-03 浙江大学 Automatic fruit grading device based on machine vision and bio-speckle
CN105251705A (en) * 2015-11-23 2016-01-20 上海电机学院 Red jujube screening device
CN105675720A (en) * 2016-04-13 2016-06-15 浙江大学 Fruit firmness information online collecting system and method
CN107350168A (en) * 2017-08-28 2017-11-17 宁夏大学 A kind of potato Fast nondestructive evaluation self-grading device and method
CN107336056A (en) * 2017-08-30 2017-11-10 无锡鑫旭润科技有限公司 A kind of fixture based on frozen section technology
CN109129255A (en) * 2018-10-25 2019-01-04 天津职业技术师范大学 Semiconductor Refrigerating suction cup

Also Published As

Publication number Publication date
CN109570051A (en) 2019-04-05

Similar Documents

Publication Publication Date Title
CN109570051B (en) Chinese chestnut wormhole detection device based on machine vision, laser and acoustics
CN109663747B (en) Intelligent detection method for Chinese chestnut wormholes
CN105044122B (en) A kind of copper piece surface defect visible detection method based on semi-supervised learning model
CN110146516B (en) Fruit grading device based on orthogonal binocular machine vision
CN113030108A (en) Coating defect detection system and method based on machine vision
CN210071686U (en) Fruit grading plant based on orthogonal binocular machine vision
CN116686545B (en) Litchi picking robot shade removing method based on machine vision control
CN113077450B (en) Cherry grading detection method and system based on deep convolutional neural network
CN211070921U (en) Instrument appearance detection device based on 3D scanning method
CN112461135B (en) Dendrobium growth parameter nondestructive online measuring device and measuring method thereof
CN107957245A (en) Engine link dimension measuring device and its measuring method based on machine vision
CN108318494B (en) The red online vision detection and classification devices and methods therefor for proposing fruit powder
CN207238542U (en) A kind of thin bamboo strip defect on-line detecting system based on machine vision
CN106706656A (en) Machine vision-based zipper detection device and method
CN109738454A (en) A kind of soft-package battery tab detection device and method
CN110503638A (en) Spiral colloid amount online test method
CN114252452A (en) Online detection device and method for appearance defects and contour dimension of small-sized revolving body
CN115375636A (en) Full-size detection method and equipment for power battery module
CN207081666U (en) A kind of zipper detecting device based on machine vision
CN115719451A (en) Kiwi fruit actinidia arguta maturity detection method and system
CN110741790A (en) Multi-claw transplanting-sorting processing method for plug seedlings based on depth camera
CN106556602A (en) A kind of detection method and its device of fish body freshness
CN206331487U (en) A kind of agricultural product volume rapid measurement device based on machine vision
CN117434064A (en) Casting quality detection device
CN111369497B (en) Walking type tree fruit continuous counting method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant