CN115015008A - Non-contact type large-batch kiwi fruit maturity detection system and method - Google Patents
Non-contact type large-batch kiwi fruit maturity detection system and method Download PDFInfo
- Publication number
- CN115015008A CN115015008A CN202210584884.4A CN202210584884A CN115015008A CN 115015008 A CN115015008 A CN 115015008A CN 202210584884 A CN202210584884 A CN 202210584884A CN 115015008 A CN115015008 A CN 115015008A
- Authority
- CN
- China
- Prior art keywords
- kiwi fruit
- video
- kiwi
- maturity
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 244000298697 Actinidia deliciosa Species 0.000 title claims abstract description 288
- 235000009436 Actinidia deliciosa Nutrition 0.000 title claims abstract description 288
- 238000001514 detection method Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000006073 displacement reaction Methods 0.000 claims abstract description 86
- 230000003321 amplification Effects 0.000 claims abstract description 69
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 69
- 238000012545 processing Methods 0.000 claims abstract description 62
- 230000008859 change Effects 0.000 claims abstract description 31
- 238000001914 filtration Methods 0.000 claims abstract description 21
- 238000004364 calculation method Methods 0.000 claims abstract description 20
- 238000004458 analytical method Methods 0.000 claims abstract description 19
- 235000013399 edible fruits Nutrition 0.000 claims abstract description 13
- 238000005516 engineering process Methods 0.000 claims abstract description 8
- 230000033001 locomotion Effects 0.000 claims description 31
- 238000000354 decomposition reaction Methods 0.000 claims description 15
- 238000010276 construction Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 9
- 235000009434 Actinidia chinensis Nutrition 0.000 claims description 8
- 238000010191 image analysis Methods 0.000 claims description 6
- 230000035772 mutation Effects 0.000 claims description 6
- 239000002344 surface layer Substances 0.000 claims description 6
- 230000004048 modification Effects 0.000 claims description 4
- 238000012986 modification Methods 0.000 claims description 4
- 235000021028 berry Nutrition 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 230000006798 recombination Effects 0.000 claims description 3
- 238000005215 recombination Methods 0.000 claims description 3
- 230000000241 respiratory effect Effects 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 230000002123 temporal effect Effects 0.000 claims 1
- 230000000007 visual effect Effects 0.000 abstract description 5
- 238000005728 strengthening Methods 0.000 abstract 1
- 230000009471 action Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 3
- 239000003205 fragrance Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000003825 pressing Methods 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000019658 bitter taste Nutrition 0.000 description 1
- 235000009508 confectionery Nutrition 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000796 flavoring agent Substances 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 235000019614 sour taste Nutrition 0.000 description 1
- 235000019605 sweet taste sensations Nutrition 0.000 description 1
- 235000013343 vitamin Nutrition 0.000 description 1
- 229940088594 vitamin Drugs 0.000 description 1
- 229930003231 vitamin Natural products 0.000 description 1
- 239000011782 vitamin Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N3/00—Investigating strength properties of solid materials by application of mechanical stress
- G01N3/40—Investigating hardness or rebound hardness
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N3/00—Investigating strength properties of solid materials by application of mechanical stress
- G01N3/02—Details
- G01N3/06—Special adaptations of indicating or recording means
- G01N3/068—Special adaptations of indicating or recording means with optical indicating or recording means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N2021/8466—Investigation of vegetal material, e.g. leaves, plants, fruits
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2203/00—Investigating strength properties of solid materials by application of mechanical stress
- G01N2203/0058—Kind of property studied
- G01N2203/0076—Hardness, compressibility or resistance to crushing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2203/00—Investigating strength properties of solid materials by application of mechanical stress
- G01N2203/02—Details not specific for a particular testing method
- G01N2203/06—Indicating or recording means; Sensing means
- G01N2203/0641—Indicating or recording means; Sensing means using optical, X-ray, ultraviolet, infrared or similar detectors
- G01N2203/0647—Image analysis
Abstract
The invention discloses a non-contact large-batch kiwi fruit maturity detection system and a non-contact large-batch kiwi fruit maturity detection method, wherein the system comprises a high frame rate video acquisition camera, a vibration platform and a processor, video images of kiwi fruits during micro-vibration are acquired and transmitted to the processor for video analysis and calculation, and the position of kiwi fruits is positioned; amplifying the micro displacement of the skin of the kiwi fruit under vibration by using a video micro-motion amplification technology until the micro displacement can be observed; detecting the centroid position of the kiwi fruit through amplified binarization processing and edge filtering, and identifying the displacement change of the kiwi fruit through a video multi-frame image; setting the maturity grade of the kiwi fruit according to the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit according to the calculated displacement change. Compared with the existing mode that the kiwi fruit is pressed by a fruit hardness meter, the method realizes massive and rapid graded nondestructive detection treatment of the kiwi fruit through visual detection of the hardness of the kiwi fruit, achieves the aim of improving the market competitiveness of the kiwi fruit in China, and has important significance for continuously developing and strengthening the kiwi fruit in the future.
Description
Technical Field
The invention belongs to the field of intelligent agriculture, and particularly relates to a non-contact large-batch kiwi fruit maturity detection system and method.
Background
The kiwi fruit is a fruit rich in nutrients such as multiple vitamins, is sour, sweet and delicious, and is deeply popular with the public, however, the quality of the kiwi fruit is inaccurate in grading screening due to the fact that the ripeness of the kiwi fruit cannot be well identified, so that whether the kiwi fruit is ripe or not is difficult to judge through appearance when a consumer purchases the kiwi fruit, and the quality and the value of the kiwi fruit cannot be well corresponded. The ripeness of the kiwi fruit is judged whether the kiwi fruit is ripe or not to be edible according to the hardness of the kiwi fruit. The unripe kiwi fruit is hard, does not accompany with fragrance, and has bitter taste, which indicates that the kiwi fruit is not suitable for eating; the ripe kiwi fruit is soft, has the fragrance of the fruit and sour and sweet taste, and is suitable for eating; the over-ripe kiwi fruit is soft and accompanied by off-flavors, and is over-ripe or bad, indicating an inedible eating.
The kiwi fruit detection technology is imperfect nowadays, methods such as pressing the kiwi fruit by a fruit hardness meter during detection to enable the kiwi fruit to be eaten, detecting the fruit sweetness by a sweetness meter and the like are adopted, the method is not widely applied to production and life, more methods for judging the ripeness of the kiwi fruit by taking manual sorting as a main method, the manpower demand is higher, the damage to the fruit is caused more easily, and the detection efficiency is extremely low.
In consideration of the above factors, the invention provides a non-contact large-batch kiwi fruit maturity detection system and a non-contact large-batch kiwi fruit maturity detection method. The quality index inside the kiwi fruit is identified through visual detection of the hardness of the kiwi fruit, non-contact type rapid grading nondestructive detection processing of a large number of kiwi fruits is realized, the market competitiveness of kiwi fruits in China is improved, the overall value and the export quantity of commodities are improved, and the method has important significance for continuously developing and making future kiwi fruits strong.
Disclosure of Invention
The invention provides a non-contact large-batch kiwi fruit maturity detection system and a non-contact large-batch kiwi fruit maturity detection method. When the visual sensor is used for shooting the kiwi fruit which is subjected to the same micro vibration, the deformation conditions of the surfaces of the kiwi fruit are different in different maturity, and the micro displacement X-t image is amplified by adopting a micro-motion amplification technology, so that the amplification treatment of the micro displacement is realized. And (3) taking the mature kiwi fruits as a standard value, solving the micro displacement under the equivalent condition of the mature kiwi fruits as a threshold value, and distinguishing whether the kiwi fruits are mature or not. The device is reasonably debugged, the high-frame-rate video acquisition camera and the processor are quickly communicated, and a large amount of pixel point information is synchronously transmitted, so that the whole system can orderly judge the maturity level of a large batch of kiwifruits in operation, and nondestructive detection is realized.
10. The technical scheme adopted by the invention for solving the technical problems is as follows: a non-contact type large-batch kiwi fruit maturity detection system comprises a high frame rate video acquisition camera, a vibration platform and a processor;
the vibration platform comprises a vibration module; the vibration module is used for realizing micro-vibration treatment on the kiwi fruits through the vibration platform;
the high frame rate video acquisition camera comprises a video acquisition module; the video acquisition module is used for acquiring a video image of the kiwi fruit during micro-vibration and transmitting the video image to the processor for video analysis and calculation;
the processor is combined with video image information transmitted by the high frame rate video acquisition camera and comprises a kiwi fruit identification module, a video micro-motion amplification module and a maturity calculation module;
the kiwi fruit identification module is used for identifying the surface layer boundary of the kiwi fruit by the gray value mutation of adjacent pixel points of the image through the image analysis acquired by the camera, and positioning the position of the kiwi fruit;
the video micro-motion amplification module is used for amplifying the skin micro-displacement of the kiwi fruit under vibration to be observable through a video image micro-motion amplification technology, and comprises video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing;
the maturity calculation module comprises a maturity detection module and a maturity calibration module;
the maturity detection module is used for identifying and calibrating the position of a platform where the kiwi fruit is located by the amplified gray level image, detecting the centroid position of the kiwi fruit by edge filtering, and identifying the displacement change of the kiwi fruit by a video multi-frame image;
the maturity calibration module is used for setting the maturity grade of the kiwi fruit to be immature, mature and over mature through setting the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit according to the calculated displacement change.
A control method of a non-contact large-batch kiwi fruit maturity detection system is characterized by comprising the following steps:
collecting kiwi fruit microvibration video data: the vibration module is used for realizing micro-vibration treatment on the kiwi fruits through the vibration platform; the video acquisition module is used for acquiring video images of the kiwi fruits during micro-vibration and transmitting the video images to the processor for video analysis and calculation;
and (3) identifying and positioning kiwi fruits: through image analysis acquired by a camera, identifying the surface boundary of the kiwi fruit by the mutation of the gray value of the adjacent pixel points of the image, calibrating the position of the kiwi fruit by a rectangular block diagram, and intercepting the image for processing;
video data decomposition processing: and performing spatial domain decomposition on the acquired and intercepted kiwi fruit image data frame by using a complex value controllable pyramid to obtain images in different directions and different scales, and separating amplitude and phase analysis of the kiwi fruit video image by using a Fourier transform method.
Video data acquisition and processing: and processing the phase signals by adopting time domain band-pass filtering to obtain the amplified data information of the kiwi fruit micromotion.
Denoising the video signal: and the noise processing is carried out by setting a proper Gaussian filter, so that the signal to noise ratio is improved.
Video sequence amplification processing: and performing motion amplification processing on the micro-motion displacement change signal delta (t) of the kiwi fruit after band-pass filtering to obtain a motion amplification sequence f (x + (1+ alpha) delta (t)).
Video amplification construction treatment: and performing construction analysis on the amplified kiwi fruit displacement video to obtain video micromotion data information after motion amplification.
And (3) detecting and calculating the maturity: identifying and calibrating the position of a platform where the kiwi fruit is located by the amplified gray level image, detecting the centroid position of the kiwi fruit by edge filtering, and identifying the displacement change of the kiwi fruit by a video multi-frame image;
and (3) calibrating maturity: setting the maturity grade of the kiwi fruit to be immature, mature and over mature by setting the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit by the calculated displacement change.
In the above scheme, the specific steps of collecting kiwi fruit microvibration video data are as follows: micro-vibration treatment of the kiwi fruits is realized through the vibration platform, so that the kiwi fruits realize micro-vibration on the kiwi fruits; the high frame rate video acquisition camera is used for acquiring video images of the kiwi fruits during micro-vibration and transmitting the video images to the processor for video analysis and calculation;
in the above scheme, the kiwifruit identification and positioning specifically comprises the following steps:
step S1: accurately storing and fetching kiwi berry microvibration images by adopting a high frame rate video acquisition camera;
step S2: based on the gray value of a pixel point of the image, distinguishing the kiwi fruit from a background color block by using a pixel point threshold value 90, and carrying out image binarization processing to identify the position of the kiwi fruit by using the sudden change of the edge pixel point values of 0 and 1;
step S3: and positioning the position of the boundary point of the kiwi fruit, calibrating the position of the kiwi fruit by using a rectangular block diagram, and intercepting the image of the kiwi fruit for calculation processing.
In the above scheme, the video data decomposition processing specifically includes the steps of: performing spatial domain decomposition on the acquired and intercepted kiwi fruit image data frame by using a Fourier transform and a complex value steerable pyramid to obtain images in different directions and different scales, separating amplitude and phase analysis of a kiwi fruit video image, performing Fourier transform on the amplitude and phase analysis, performing Fourier series decomposition on the displacement image section f (x + delta (t)) as the sum of complex sinusoids under the condition that the section f of a one-dimensional kiwi fruit image is in global translation by using global Fourier transform, and taking the displacement image section f (x + delta (t)) as the sum of complex sinusoids for an amplification factor alpha, wherein each frequency band corresponds to a single frequency omega.
The band of frequency omega being a complex sine wave
S ω (x,t)＝A ω e iω(x+δ(t))
Due to S ω Is a sine wave whose phase ω (x + δ (t)) contains motion information, and the motion can be manipulated by modifying the phase S ω (x, y) is a complex sine wave that moves exactly 1+ alpha times the input. The motion-amplified video can be reconstructed by folding the pyramid, and the motion-amplified sequence f (x + (1+ α) δ (t)) is obtained by summing all sub-bands.
In the above scheme, the video data acquisition and processing specifically comprises the steps of: the collected phase signals are filtered by adopting a proper band-pass filter, and the displacement alpha delta (t) needs to be limited, so that the amplified displacement is well similar to a real displacement signal. And taking a standard deviation of a Gaussian window as a boundary, thereby acquiring the micro-motion amplified data information of the kiwi fruit.
In the above scheme, the video signal denoising process specifically includes the steps of: noise in the input sequence can cause noise in the phase signal itself, thus causing the incorrect motion to be amplified, and amplitude weighting and gaussian blurring are used on the phase to perform noise processing by setting a proper gaussian filter, thereby improving the signal-to-noise ratio of the signal.
In the above scheme, the specific steps of the video sequence amplification process are as follows: the phase of the valid kiwi fruit data is collected through a band-pass filter, and the time band-pass phase corresponds to the motion of different spatial scales and directions. To synthesize the amplification movement, the bandpass phases are multiplied by an amplification factor α. These amplified phase differences are then used to amplify the kiwi minute motion in the sequence, with phase modification for each coefficient of each frame. The displacement change signal delta (t) of the kiwi fruit micromotion after the band-pass filtering is subjected to motion amplification treatment to obtain a result after the action is amplified by alpha times,
to separate out the varying parts, the formulated action is approximated using a first order Taylor series expansion:
let the result of the last step of band-pass filtering be B (x, t), and if the frequency range of all the variation signals delta (t) is exactly within the band range of the band-pass filtering, there are
The approximated action of the formula is amplified by multiplying the changed portion by an amplification factor alpha and adding back to the original signal. Namely:
simultaneous equations can be found:
in the ideal case of such a situation,
the motion amplification sequence f (x + (1+ α) δ (t)) was obtained.
In the above scheme, the video amplification construction processing specifically includes the steps of: when the kiwi fruits shot by the vision sensor are subjected to the same micro vibration, the deformation conditions of the surfaces of the kiwi fruits with different maturity are different, and the micro deformation f (t) -t slice amplified images are subjected to video recombination through the amplification of the micro displacement of the video to realize the amplification treatment of the small displacement.
In the above scheme, the maturity detection and calculation specifically comprises the following steps: as a typical fruit species of a respiratory transition, kiwi fruits gradually soften from hard to edible during their growth. Thus representing a constant softening of the kiwi fruit. In order to detect the maturity of the kiwi fruit and under the condition of no damage to the kiwi fruit, the hardness of the surface layer of the kiwi fruit can be expressed by adopting micro vibration;
step M1: carrying out sobei edge detection on the kiwi fruits;
step M2: performing regionprops function processing on the intercepted image, and detecting the area characteristics and the centroid position of the intercepted image;
step M3: saving the centroid position and calculating the small displacement of each frame of video image to form an array
For example, the displacement change is calculated and stored in an array.
In the above scheme, the steps of calibrating the maturity are as follows: the ripeness of the kiwi fruit is characterized by the hardness of the kiwi fruit, the ripeness grade of the kiwi fruit is set to be immature, mature and over mature by setting the amplified displacement image threshold value, and the ripeness of the kiwi fruit is calibrated by the calculated displacement change;
when the kiwi fruit is immature, the surface hardness of the kiwi fruit is too high, the contact friction force with the vibration table is small, and the displacement amplitude x is small when the kiwi fruit is subjected to micro vibration<ω 1 Indicating that the seed is immature;
when the kiwi fruit is mature, the surface hardness is centered, the contact friction force with the vibration table is centered, and the displacement amplitude omega when the kiwi fruit is subjected to micro vibration 1 <x<ω 2 Lower, indicating maturity;
when the kiwi fruit is over-mature, the surface hardness of the kiwi fruit is too low, the contact friction force with the vibration table is large, and the displacement amplitude x is small when the kiwi fruit is subjected to small vibration>ω 2 Indicating over-maturation.
Compared with the prior art, the invention has the beneficial effects that:
provides a non-contact type large-batch kiwi fruit maturity detection system and a non-contact type large-batch kiwi fruit maturity detection method. The kiwi fruit detection technology is imperfect nowadays, methods such as pressing the kiwi fruit by a fruit hardness meter during detection to enable the kiwi fruit to be eaten, detecting the fruit sweetness by a sweetness meter and the like are adopted, the method is not widely applied to production and life, more methods for judging the ripeness of the kiwi fruit by taking manual sorting as a main method, the manpower demand is higher, the damage to the fruit is caused more easily, and the detection efficiency is extremely low.
According to the method, the internal quality indexes of the kiwi fruits are identified through visual detection of the hardness of the kiwi fruits, non-contact type rapid grading nondestructive detection treatment of a large number of kiwi fruits is realized, the market competitiveness of kiwi fruits in China is improved, the overall value and the export quantity of commodities are improved, and the method has important significance for continuous development and invigoration of future kiwi fruits.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a hardware set-up diagram of the present invention.
Fig. 2 is a flow chart of the operation of the present invention.
Fig. 3 is a comparison image of kiwi fruit based on video micro-motion amplification according to the present invention.
FIG. 4 is a Kiwi fruit jiggle processing image based on video jiggle amplification according to the present invention.
Fig. 5 is a kiwi fruit displacement acquisition image based on video micro-motion amplification according to the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "axial," "radial," "vertical," "horizontal," "inner," "outer," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the present invention and for simplicity in description, and are not intended to indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "secured," and the like are to be construed broadly and can, for example, be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
The invention realizes a non-contact large-batch kiwi fruit maturity detection system and a non-contact large-batch kiwi fruit maturity detection method. When the visual sensor is used for shooting the kiwi fruit which is subjected to the same micro vibration, the deformation conditions of the surfaces of the kiwi fruit are different in different maturity, and the micro displacement X-t image is amplified by adopting a micro-motion amplification technology, so that the amplification treatment of the micro displacement is realized. And (3) taking the mature kiwi fruits as a standard value, solving the micro displacement under the equivalent condition of the mature kiwi fruits as a threshold value, and distinguishing whether the kiwi fruits are mature or not. The device is reasonably debugged, the high-frame-rate video acquisition camera and the processor are quickly communicated, a large amount of pixel point information is synchronously transmitted, so that the whole system can orderly judge the maturity level of a large batch of kiwifruits in operation, and nondestructive detection is realized.
FIG. 1 is a hardware construction diagram of a non-contact high-batch kiwi fruit maturity detection system according to the present invention, including a high frame rate video capture camera, a vibration platform, and a processor;
the vibration platform comprises a vibration module; the vibration module is used for realizing micro-vibration treatment on the kiwi fruits through the vibration platform;
the high frame rate video acquisition camera comprises a video acquisition module; the video acquisition module is used for acquiring a video image of the kiwi fruit during micro-vibration and transmitting the video image to the processor for video analysis and calculation;
the processor is combined with video image information transmitted by the high frame rate video acquisition camera and comprises a kiwi fruit identification module, a video micro-motion amplification module and a maturity calculation module;
the kiwi fruit identification module is used for identifying the surface layer boundary of the kiwi fruit by the gray value mutation of adjacent pixel points of the image through the image analysis acquired by the camera, and positioning the position of the kiwi fruit;
the video micro-motion amplification module is used for amplifying the skin micro-displacement of the kiwi fruit under vibration to be observable through a video image micro-motion amplification technology, and comprises video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing;
the maturity detection module is used for identifying and calibrating the position of a platform where the kiwi fruit is located by the amplified gray level image, detecting the centroid position of the kiwi fruit by edge filtering, and identifying the displacement change of the kiwi fruit by a video multi-frame image;
the maturity calibration module is used for setting the maturity grade of the kiwi fruit to be immature, mature and over mature through setting the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit according to the calculated displacement change.
Fig. 2 is a work flow chart of a control method of the kiwi fruit maturity detection system based on video micro-motion amplification, and is characterized by comprising the following steps:
collecting kiwi fruit microvibration video data: the vibration module is used for realizing micro-vibration treatment on the kiwi fruits through the vibration platform; the video acquisition module is used for acquiring video images of the kiwi fruits during micro-vibration and transmitting the video images to the processor for video analysis and calculation;
and (3) identifying and positioning kiwi fruits: through image analysis acquired by a camera, identifying the surface boundary of the kiwi fruit according to the mutation of the gray value of adjacent pixel points of the image, calibrating the position of the kiwi fruit according to a rectangular block diagram, and intercepting the image for processing;
video data decomposition processing: and performing spatial domain decomposition on the acquired and intercepted kiwi fruit image data frame by using a complex value controllable pyramid to obtain images in different directions and different scales, and separating amplitude and phase analysis of the kiwi fruit video image by using a Fourier transform method.
Video data acquisition and processing: and processing the phase signals by adopting time domain band-pass filtering to obtain the amplified data information of the kiwi fruit micromotion.
Denoising the video signal: and the noise processing is carried out by setting a proper Gaussian filter, so that the signal to noise ratio is improved.
Video sequence amplification treatment: and performing motion amplification processing on the micro-motion displacement change signal delta (t) of the kiwi fruit after band-pass filtering to obtain a motion amplification sequence f (x + (1+ alpha) delta (t)).
Video amplification construction treatment: and performing construction analysis on the amplified kiwi fruit displacement video to obtain video micromotion data information after motion amplification.
And (3) detecting and calculating the maturity: identifying and calibrating the position of a platform where the kiwi fruit is located by the amplified gray level image, detecting the centroid position of the kiwi fruit by edge filtering, and identifying the displacement change of the kiwi fruit by a video multi-frame image;
and (3) calibrating the maturity: setting the maturity grade of the kiwi fruit to be immature, mature and over mature by setting the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit by the calculated displacement change.
In the above scheme, the specific steps of collecting kiwi fruit microvibration video data are as follows: micro-vibration treatment of the kiwi fruits is realized through the vibration platform, so that the kiwi fruits realize micro-vibration on the kiwi fruits; the high frame rate video acquisition camera is used for acquiring video images of the kiwi fruits during micro-vibration and transmitting the video images to the processor for video analysis and calculation;
in the above scheme, the kiwifruit identification and positioning specifically comprises the following steps:
step S1: accurately storing and fetching kiwi berry microvibration images by adopting a high frame rate video acquisition camera;
step S2: based on the gray value of a pixel point of the image, distinguishing the kiwi fruit from a background color block by using a pixel point threshold value 90, carrying out image binarization processing, and identifying the position of the kiwi fruit by using the sudden changes of the edge pixel point values of 0 and 1;
step S3: and positioning the position of the boundary point of the kiwi fruit, calibrating the position of the kiwi fruit by using a rectangular block diagram, and intercepting the image of the kiwi fruit for calculation processing.
In the above scheme, the video data decomposition processing specifically includes the steps of: performing spatial domain decomposition on the acquired and intercepted kiwi fruit image data frame by using a Fourier transform and a complex value steerable pyramid to obtain images in different directions and different scales, separating amplitude and phase analysis of a kiwi fruit video image, performing Fourier transform on the amplitude and phase analysis, performing Fourier series decomposition on the displacement image section f (x + delta (t)) as the sum of complex sinusoids under the condition that the section f of a one-dimensional kiwi fruit image is in global translation by using global Fourier transform, and taking the displacement image section f (x + delta (t)) as the sum of complex sinusoids for an amplification factor alpha, wherein each frequency band corresponds to a single frequency omega.
The band of frequencies omega is a complex sine wave,
S ω (x,t)＝A ω e iω(x+δ(t))
due to S ω Is a sine wave whose phase ω (x + δ (t)) contains motion information, and the motion can be manipulated by modifying the phase S ω (x, y) is a complex sine wave with exactly 1+ α times the input. The motion-amplified video can be reconstructed by folding the pyramid, and the motion-amplified sequence f (x + (1+ α) δ (t)) is obtained by summing all sub-bands.
In the above scheme, the video data acquisition and processing specifically comprises the following steps: the collected phase signals are filtered by adopting a proper band-pass filter, and the displacement alpha delta (t) needs to be limited, so that the amplified displacement is well similar to a real displacement signal. And taking a standard deviation of a Gaussian window as a boundary, thereby acquiring the micro-motion amplified data information of the kiwi fruit.
In the above scheme, the video signal denoising process specifically includes the steps of: noise in the input sequence can cause noise in the phase signal itself, thus causing the incorrect motion to be amplified, and amplitude weighting and gaussian blurring are used on the phase to perform noise processing by setting a proper gaussian filter, thereby improving the signal-to-noise ratio of the signal.
In the above scheme, the video sequence amplification processing specifically includes the steps of: the phase of the valid kiwi fruit data is collected by a band-pass filter, and the time band-pass phase corresponds to the motion of different spatial scales and directions. To synthesize the amplification movement, the bandpass phases are multiplied by an amplification factor α. These amplified phase differences are then used to amplify the kiwi minute motion in the sequence, with phase modification for each coefficient of each frame. The displacement change signal delta (t) of the kiwi fruit micromotion after the band-pass filtering is subjected to motion amplification treatment to obtain a result after the action is amplified by alpha times,
to separate out the varying parts, the formulated action is approximated using a first order Taylor series expansion:
let the result of the last step of band-pass filtering be B (x, t), and if the frequency range of all the variation signals delta (t) is exactly within the band range of the band-pass filtering, there are
The approximated action of the formula is amplified by multiplying the changed portion by an amplification factor alpha and adding back to the original signal. Thus obtaining:
simultaneous equations can be found:
in the ideal case of this kind of situation,
the motion amplification sequence f (x + (1+ α) δ (t)) was obtained.
In the above scheme, the video amplification construction processing specifically includes the steps of: when the kiwi fruits shot by the vision sensor are subjected to the same micro vibration, the deformation conditions of the surfaces of the kiwi fruits with different maturity are different, and the micro deformation f (t) -t slice amplified images are subjected to video recombination through the amplification of the micro displacement of the video to realize the amplification treatment of the small displacement.
In the above scheme, the maturity detection and calculation specifically comprises the following steps: as a typical fruit species of a respiratory transition, kiwi fruits gradually soften from hard to edible during their growth. Thus representing a constant softening of the kiwi fruit. In order to detect the maturity of the kiwi fruit and under the condition of no damage to the kiwi fruit, the hardness of the surface layer of the kiwi fruit can be expressed by adopting micro vibration;
step M1: carrying out sobei edge detection on the kiwi fruits;
step M2: performing regionprops function processing on the intercepted image, detecting the area characteristics and the position of the mass center of the intercepted image, and obtaining a kiwi fruit micro-motion processing image based on video micro-motion amplification in the invention in a figure 4;
step M3: saving the centroid position and calculating the small displacement of each frame of video image to form an array
For example, the displacement change is calculated and stored in an array, and fig. 5 is a diagram of the kiwi displacement acquisition image based on video micro-motion amplification according to the present invention.
In the above scheme, the specific steps of the maturity calibration are as follows: the ripeness of the kiwi fruit is characterized by the hardness of the kiwi fruit, the ripeness grade of the kiwi fruit is set to be immature, mature and over mature by setting the amplified displacement image threshold value, and the ripeness of the kiwi fruit is calibrated by the calculated displacement change;
calculating and calibrating immature kiwi fruit and mature kiwi fruit in large quantity, and setting the threshold value of immature kiwi fruit and mature kiwi fruit as omega 1 The threshold value of mature and over mature of the kiwi fruit is omega 2 Detecting the mean value of the displacement change of the kiwi fruits as x;
when the kiwi fruit is immature, the surface hardness of the kiwi fruit is too high, the contact friction force with the vibration table is small, and the displacement amplitude x is small when the kiwi fruit is subjected to micro vibration<ω 1 Indicating that the seed is immature;
when the kiwi fruit is mature, the surface hardness is centered, the contact friction force with the vibration table is centered, and the displacement amplitude omega is small in vibration 1 <x<ω 2 Lower, indicating maturity;
when the kiwi fruit is over-mature, the surface hardness of the kiwi fruit is too low, the contact friction force with the vibration table is large, and the displacement amplitude x is small when the kiwi fruit is subjected to small vibration>ω 2 Indicating over-maturation.
It should be understood that although the present description has been described in terms of various embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and those skilled in the art will recognize that the embodiments described herein may be combined as suitable to form other embodiments, as will be appreciated by those skilled in the art.
The above-listed detailed description is only a specific description of a possible embodiment of the present invention, and they are not intended to limit the scope of the present invention, and equivalent embodiments or modifications made without departing from the technical spirit of the present invention should be included in the scope of the present invention.
Claims (9)
1. A non-contact type large-batch kiwi fruit maturity detection system is characterized by comprising a high frame rate video acquisition camera, a vibration platform and a processor;
the vibration platform comprises a vibration module; the vibration module is used for realizing micro-vibration treatment on the kiwi fruits through the vibration platform;
the high frame rate video acquisition camera comprises a video acquisition module; the video acquisition module is used for acquiring a video image of the kiwi fruit during micro-vibration and transmitting the video image to the processor for video analysis and calculation;
the processor is combined with video image information transmitted by the high frame rate video acquisition camera and comprises a kiwi fruit identification module, a video micro-motion amplification module and a maturity calculation module;
the kiwi fruit identification module is used for identifying the surface layer boundary of the kiwi fruit by the gray value mutation of adjacent pixel points of the image through the image analysis acquired by the camera, and positioning the position of the kiwi fruit;
the video micro-motion amplification module is used for amplifying the skin micro-displacement of the kiwi fruit under vibration to be observable through a video image micro-motion amplification technology, and comprises video data decomposition processing, video data acquisition processing, video signal denoising processing, video sequence amplification processing and video amplification construction processing;
the maturity calculation module comprises a maturity detection module and a maturity calibration module;
the maturity detection module is used for identifying and calibrating the position of a platform where the kiwi fruit is located by the amplified gray level image, detecting the centroid position of the kiwi fruit by edge filtering, and identifying the displacement change of the kiwi fruit by a video multi-frame image;
the maturity calibration module is used for setting the maturity grade of the kiwi fruit to be immature, mature and over mature through the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit according to the calculated displacement change.
2. The control method of the non-contact high-batch kiwi fruit maturity detection system according to claim 1, comprising the following steps:
collecting kiwi fruit microvibration video data: micro-vibration treatment of the kiwi fruits is realized through a vibration module; collecting a video image of the kiwi fruit during micro-vibration through a video collection module, and transmitting the video image to a processor for video analysis and calculation;
and (3) identifying and positioning kiwi fruits: through image analysis acquired by a camera, identifying the surface boundary of the kiwi fruit by the mutation of the gray value of the adjacent pixel points of the image, calibrating the position of the kiwi fruit by a rectangular block diagram, and intercepting the image for processing;
video data decomposition processing: performing spatial domain decomposition on the acquired and intercepted kiwi fruit image data frame by using a complex value controllable pyramid to obtain images in different directions and different scales, and separating amplitude and phase analysis of the kiwi fruit video image by using a Fourier transform method;
video data acquisition and processing: processing the phase signals by adopting time domain band-pass filtering to obtain the micro-motion amplified data information of the kiwi fruits;
denoising the video signal: a proper Gaussian filter is arranged for noise processing, so that the signal-to-noise ratio of a signal is improved;
video sequence amplification processing: carrying out motion amplification treatment on the displacement change signal delta (t) of the kiwi fruit micromotion after band-pass filtering to obtain a motion amplification sequence f (x + (1+ alpha) delta (t));
video amplification construction treatment: performing construction analysis on the amplified kiwi fruit displacement video to obtain video micromotion data information after motion amplification;
and (3) detecting and calculating the maturity: identifying and calibrating the platform position of the kiwi fruit through the amplified gray image, detecting the centroid position of the kiwi fruit through edge filtering, and identifying the displacement change of the kiwi fruit through a video multi-frame image;
and (3) calibrating maturity: setting the maturity grade of the kiwi fruit to be immature, mature and over mature by setting the amplified displacement image threshold value, and calibrating the maturity of the kiwi fruit by the calculated displacement change.
3. The control method of the non-contact large-batch kiwi fruit maturity detection system according to claim 2, wherein the kiwi fruit identification and positioning specifically comprises the following steps:
step S1: accurately storing and fetching kiwi berry microvibration images by adopting a high frame rate video acquisition camera;
step S2: based on the gray value of a pixel point of the image, distinguishing the kiwi fruit from a background color block by using a pixel point threshold value 90, and carrying out image binarization processing to identify the position of the kiwi fruit by using the sudden change of the edge pixel point values of 0 and 1;
step S3: and positioning the position of the boundary point of the kiwi fruit, calibrating the position of the kiwi fruit by using a rectangular block diagram, and intercepting the image of the kiwi fruit for calculation processing.
4. The control method of the non-contact large-batch kiwi fruit maturity detection system according to claim 2, wherein the video data acquisition and processing comprises the following specific steps: and filtering the acquired phase signals by adopting a proper band-pass filter, wherein the displacement alpha delta (t) needs to be limited, so that the amplified displacement is well similar to a real displacement signal, and a standard deviation of a Gaussian window is used as a boundary, thereby acquiring the micro-motion amplified data information of the kiwi fruit.
5. The control method of the non-contact high-batch kiwi fruit maturity detection system according to claim 2, wherein the video signal denoising processing comprises the following specific steps: noise in the input sequence can cause noise in the phase signal itself, thus causing the incorrect motion to be amplified, and amplitude weighting and gaussian blurring are used on the phase to perform noise processing by setting a proper gaussian filter, thereby improving the signal-to-noise ratio of the signal.
6. The control method of the non-contact high-batch kiwi fruit maturity detection system according to claim 2, wherein the video sequence amplification processing specifically comprises the following steps: the phase of the band-pass valid kiwi data, the temporal band-pass phase corresponds to the motion of different spatial scales and directions. In order to synthesize amplification movement, the band-pass phase is multiplied by an amplification factor alpha, then the amplified phase difference is used for amplifying the tiny movement of the kiwi fruit in the sequence, and the displacement change signal delta (t) of the kiwi fruit subjected to band-pass filtering is subjected to movement amplification processing through phase modification of each coefficient of each frame, so that a movement amplification sequence f (x + (1+ alpha) delta (t)) is obtained.
7. The control method of the non-contact large-batch kiwi fruit maturity detection system according to claim 2, wherein the video amplification construction processing comprises the following specific steps: when the kiwi fruits shot by the vision sensor are subjected to the same micro vibration, the deformation conditions of the surfaces of the kiwi fruits with different maturity are different, and the micro deformation f (t) -t slice amplified images are subjected to video recombination through the amplification of the micro displacement of the video to realize the amplification treatment of the small displacement.
8. The control method of the non-contact high-batch kiwi maturity detection system according to claim 2, wherein the maturity detection calculation comprises the following specific steps: as a typical fruit type of respiratory transition type, during the growth period of kiwi fruit, the fruit is gradually softened from hard to edible, thus showing the continuous softening of kiwi fruit, and in order to detect the ripeness of kiwi fruit and without damaging the kiwi fruit, the hardness of the surface layer can be shown by using small vibration;
step M1: carrying out sobei edge detection on the kiwi fruits;
step M2: performing regionprops function processing on the intercepted image, and detecting the area characteristics and the centroid position of the intercepted image;
step M3: saving the centroid position and calculating the small displacement of each frame of video image to form an array
Calculating displacement change and storing the displacement change in an array;
9. the control method of the non-contact large-batch kiwi fruit maturity detection system according to claim 2, wherein the maturity is calibrated by the following specific steps: the ripeness of the kiwi fruit is characterized by the hardness of the kiwi fruit, the ripeness grade of the kiwi fruit is set to be immature, mature and over mature by setting the amplified displacement image threshold value, and the ripeness of the kiwi fruit is calibrated by the calculated displacement change; calculating and calibrating immature kiwi fruit and mature kiwi fruit in large quantity, and setting the threshold value of immature kiwi fruit and mature kiwi fruit as omega 1 The threshold value of mature and over mature of the kiwi fruit is omega 2 Detecting the mean value of the displacement change of the kiwi fruits as x;
when the kiwi fruit is immature, the surface hardness of the kiwi fruit is too high, the contact friction force with the vibration table is small, and the displacement amplitude x is small when the kiwi fruit is subjected to micro vibration<ω 1 Indicating that the seed is immature;
when the kiwi fruit is mature, the surface hardness is centered, the contact friction force with the vibration table is centered, and the displacement amplitude omega is small in vibration 1 <x<ω 2 Lower, indicating maturity;
when the kiwi fruit is over-mature, the surface hardness of the kiwi fruit is too low, the contact friction force with the vibration table is large, and the displacement amplitude x is small when the kiwi fruit is subjected to small vibration>ω 2 Indicating over-maturation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210584884.4A CN115015008A (en) | 2022-05-27 | 2022-05-27 | Non-contact type large-batch kiwi fruit maturity detection system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210584884.4A CN115015008A (en) | 2022-05-27 | 2022-05-27 | Non-contact type large-batch kiwi fruit maturity detection system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115015008A true CN115015008A (en) | 2022-09-06 |
Family
ID=83071822
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210584884.4A Pending CN115015008A (en) | 2022-05-27 | 2022-05-27 | Non-contact type large-batch kiwi fruit maturity detection system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115015008A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116849183A (en) * | 2023-07-07 | 2023-10-10 | 深圳市微米生物技术有限公司 | Intelligent breeding box for efficient and environment-friendly maggot breeding |
-
2022
- 2022-05-27 CN CN202210584884.4A patent/CN115015008A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116849183A (en) * | 2023-07-07 | 2023-10-10 | 深圳市微米生物技术有限公司 | Intelligent breeding box for efficient and environment-friendly maggot breeding |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103065149B (en) | Muskmelon Fruit phenotype is extracted and quantization method | |
| Khoje et al. | Automated skin defect identification system for fruit grading based on discrete curvelet transform | |
| CN106373123B (en) | Industrial component detection method of surface flaw based on k_tSL central cluster algorithm | |
| CN109100323B (en) | Nondestructive quantitative evaluation method for transmission spectrum of apple water core disease | |
| CN104458766B (en) | A kind of cloth surface flaw detection method based on structural texture method | |
| CN115015008A (en) | Non-contact type large-batch kiwi fruit maturity detection system and method | |
| CN108387887A (en) | A kind of mixing noise-reduction method of underwater sound signal | |
| CN116012346B (en) | Power transmission line monitoring method and system | |
| CN105158186A (en) | Method for detecting black core of white radish based on hyperspectral image | |
| CN109238182A (en) | A kind of objective ranking method of fabric flatness based on Fourier spectrum feature | |
| CN107505325A (en) | The comprehensive quality detecting method of winter-jujube fruit | |
| CN110715918B (en) | Single-kernel corn starch content Raman hyperspectral classification method | |
| CN107590455A (en) | A kind of high frequency partial discharge adaptive-filtering clustering method and device based on wavelet decomposition | |
| CN115436508A (en) | Method for predicting fruit juice quality based on fruit raw material indexes | |
| CN114186596B (en) | Multi-window identification method and device for spectrogram peaks and electronic equipment | |
| CN112985809A (en) | Rolling bearing fault diagnosis method based on signal multi-dimensional fine image | |
| CN106504214B (en) | The high spectrum image Banded improvement removing method of wavelet transformation and local interpolation fusion | |
| CN105426585B (en) | A kind of potato sprouting method for early warning based on SIN function fitting process | |
| CN117437219B (en) | Magnetic glue inductance irregular appearance defect detection method based on computer vision | |
| CN108596879B (en) | Hilbert-Huang transform-based fMRI time-frequency domain dynamic network construction method | |
| CN114295556A (en) | Hyperspectral physicochemical index fitting method and system based on one-dimensional convolution | |
| CN109613531A (en) | Multi-threshold optimization deformation inversion method and system for micro-variation perception early warning radar | |
| CN110882948B (en) | Grapefruit quality classification method and device | |
| CN116071565A (en) | Apple moldy heart disease identification method and system based on hyperspectral imaging | |
| Selvathi et al. | Detection of retinal blood vessels using curvelet transform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |