CN110927174A - White spirit segmentation detection device and method based on embedded module - Google Patents
White spirit segmentation detection device and method based on embedded module Download PDFInfo
- Publication number
- CN110927174A CN110927174A CN201911271818.6A CN201911271818A CN110927174A CN 110927174 A CN110927174 A CN 110927174A CN 201911271818 A CN201911271818 A CN 201911271818A CN 110927174 A CN110927174 A CN 110927174A
- Authority
- CN
- China
- Prior art keywords
- wine
- image
- segmentation
- liquor
- embedded module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention discloses a white spirit segmentation detection device based on an embedded module. The device consists of a wine receiving device, an industrial camera and an NPU-based embedded module, and the current white wine segmentation is rapidly and stably judged according to a hop image by using a computer vision technology, and abnormal conditions are identified. The wine receiving equipment reduces the interference caused by water flow impact, ambient light and reflected light, strengthens the edge characteristics of hops and avoids the fuzzy problem caused by insufficient depth of field of a camera through special structural design, lateral light supplementing, overflowing and other modes; an NPU-based embedded module is used as an operation unit, and a white spirit segmentation and anomaly detection algorithm based on a computer vision technology is operated. The various steps of the algorithm run on different computational units and concurrently through the pipeline. Compared with large-scale equipment such as a workstation and a server, the system has the advantages of small volume, low cost, low power consumption and easiness in installation and maintenance. The detection device can be connected to a liquor taking system through a communication interface, and automatic quality measurement and liquor taking are achieved.
Description
Technical Field
The invention belongs to the technical field of wine making, and particularly relates to a white spirit segmentation detection device and method based on an embedded module.
Background
In the process of brewing and distilling the white spirit, the distillation sequence is different because the volatility and the boiling point of each component in the fermented grains are different. At the initial stage of distillation, volatile aldehydes with low boiling points are distilled off firstly; then, the lipid, alcohol and acid substances with lower boiling points and easier volatilization are distilled out, and the substances have rich flavor components, are main parts forming the flavor components of the white spirit and are also the parts with optimal vinosity; then, higher fatty acid ester, fusel oil and the like which have high boiling points and are difficult to volatilize are distilled out, and the wine shows sour and astringent taste, and is mixed into the wine in the former stage excessively, so that the product quality is easily influenced. The liquor-picking process needs to pick the part with the optimal liquor quality and control the quality of base liquor.
During distillation, the alcohol content and the aroma substance content of the distilled liquor at different stages are different, so that the surface tension of the liquor is different, and the hop shapes are different. Therefore, the traditional liquor picking method adopts a 'flower-watching liquor picking' method, namely, the shape of the hop formed by liquor impact is observed by human eyes, and the segmentation of the liquor is judged. The invention discloses a method for picking liquor from strong aromatic Chinese spirits (application number 201110089637.9), which explains the morphological characteristics of hops in each stage and provides a specific embodiment of picking liquor from the flowers. However, there are two problems in "picking up flowers and wine": firstly, each liquor picking person has a large difference in judgment standard, and the alternation between three-stage liquor and tail liquor is rapid, so that the quality of products is unstable easily due to manual flower-watching liquor picking; secondly, the judgment is made by depending on the continuous observation of hops by liquor picking personnel, which wastes manpower and hinders the automatic implementation of liquor production.
There are many studies and patents currently focused on automated liquor picking. The invention patent of 'gas phase pressure change on-line quality measurement wine-taking process' (application No. 201210569106.4) and the invention patent of 'gas phase temperature change on-line quality measurement wine-taking process' (application No. 201210569125.7) respectively detect the outlet gas pressure and the gas temperature at the cover of the grain steamer, and carry out quality measurement wine-taking by a threshold value judgment mode. The disadvantages of both methods are that the outlet gas pressure and gas temperature at the cover of the retort are related to the temperature and pressure of the input steam in addition to the distillation stage, and are easy to interfere and the alcohol extraction degree is not high. The invention discloses a liquor picking method based on near infrared spectrum online detection and an online simulation liquor picking device thereof (application number 201910418319.9), which establish a mapping relation between a near infrared spectrum and an alcoholic strength by measuring the near infrared spectrum of liquor, and then pick liquor according to an alcoholic strength threshold value. The method indirectly measures the alcoholic strength, has high reliability, but needs to arrange an additional sampling valve and an emptying valve on the main pipeline for sampling measurement. The invention relates to a flavor substance online quality-measuring liquor-taking process (application number 201210568937.X), which detects the content of flavor substances on a liquor pipeline by combining an online gas chromatograph and a probe, and compares the content of the flavor substances with a preset value to take liquor; the invention discloses an alcohol content online quality-measuring liquor-taking process (application No. 201210569149.2), wherein an alcohol content online detector is arranged on a liquor outlet pipeline, the alcohol content of liquor is measured, and liquor is taken by comparing the alcohol content with a preset value. The two schemes directly measure the flavor substances and the alcoholic strength, have high reliability, use an online detection instrument, but have the defects of high equipment cost and direct contact of measurement equipment with wine. The invention patent 'a plucking wine system for industrial production' (application number 201610979934.3) integrates various information such as temperature, pressure, spectrum, concentration and the like, has higher reliability, but also has higher relative cost and higher maintenance difficulty of various devices.
The automatic wine picking method based on computer vision has the advantages of non-contact measurement, less equipment, relatively low cost and high reliability. The utility model discloses an "automatic wine system of plucking" (application number 201220345826.8) uses the camera to shoot the hops image, sends into microcomputer computer hops size, and electronic controller plucks wine according to the hops size, control solenoid valve. The method has the following problems: the hop trough is open, which is easy to cause the wine to splash and volatilize; the method has insufficient detailed description and comprises the steps of camera model selection, wine receiving equipment design, microcomputer model selection and the like. The document, "automatic liquor picking method research based on image classification algorithm" (the university of Zhejiang university Master academic thesis, Captain, 2019.03) uses a high-speed industrial camera to shoot hop images, sends the images to a high-performance server, runs a liquor segmentation algorithm, sends segmentation discrimination results to a DCS, and picks liquor by a DCS control valve. The method has high liquor segmentation precision and high speed, but also has some defects: the scheme has high equipment cost, high power consumption and difficult installation and deployment; the light supplement lamp and the camera are coaxially arranged, so that the liquid level can reflect light; the wine receiving bowl is opened, so that wine is easy to splash and volatilize; the wine liquid flows into the pipeline from the hole at the bottom of the wine receiving bowl, and the liquid level of the wine liquid fluctuates greatly.
The automatic wine-picking method based on computer vision uses a microcomputer or a server as an arithmetic device. Although such devices are computationally intensive and capable of running high accuracy computer vision based liquor extraction algorithms, they suffer from several disadvantages: 1) the cost of the computing equipment is high, for example, the cost of a server thinkSystem SR650 selected for the research of an automatic wine picking method based on an image classification algorithm is about 15000 yuan; 2) the equipment has large volume and complex installation, needs to be placed in a central control room, and is connected with an industrial camera on a production line by a network cable or an optical fiber; 3) the power consumption of the device is high, and extra electricity charge consumption is brought. When the number of the devices is large, the heat dissipation of a machine room is challenged; 4) the device takes a general CPU/GPU as a main operation unit, hardware optimization is not performed aiming at deep learning, and the energy efficiency ratio is low when a deep learning algorithm is operated.
The embedded equipment has the advantages of low cost, efficient power consumption, small volume and easy installation and maintenance, and can solve the problems. If the utility model discloses a "intelligent automatic wine system of plucking based on MCU" (application number 2016212003888. X) has chooseed embedded chip AT89S52 for use promptly as the operation core, operation white spirit segmentation algorithm. But this method uses an ultrasonic sensor as a hop sensor instead of using a camera to observe the hop for segmentation, and is not a computer vision based detection method. And the MCU used in the rigid method has insufficient computing power and is difficult to operate a complex high-precision algorithm. At present, a scheme for running a high-precision white spirit segmentation detection algorithm based on computer vision and deep learning on an embedded device is not found.
Regarding the research in the white spirit segmentation algorithm based on computer vision, the document, "the research of an automatic wine picking method based on an image classification algorithm" (the university paper of great university in chessmen, behcet, 2019.03) uses an ellipse detection algorithm based on circular arc detection and limited random least square fitting, can detect the foreground within 4.2s, uses a random clipping algorithm based on circular arc segments to extract the hop region, and uses a convolutional neural network to perform white spirit segmentation discrimination. The method has high precision, but the pretreatment time is long, and the abnormal condition is not considered.
In summary, aiming at the problem of automatic liquor picking based on computer vision, a set of liquor segmentation detection device based on an embedded module is necessary to be designed, so that a high-precision liquor segmentation detection algorithm based on computer vision can be operated in real time, and the accuracy and stability of liquor segmentation are ensured; but also has the advantages of small volume, low power consumption and low cost of the embedded system. Meanwhile, a set of liquor segmentation algorithm needs to be designed, liquor segmentation can be performed rapidly and accurately, and various anomalies are detected.
Disclosure of Invention
The invention aims to provide a white spirit segmentation detection device based on an embedded module, which can operate a white spirit segmentation detection algorithm based on computer vision and deep learning to quickly and stably judge white spirit segmentation.
In order to achieve the above purpose, the invention specifically adopts the following technical scheme:
a white spirit segmentation detection device based on an embedded module comprises a white spirit receiving device, an industrial camera and an NPU-based embedded module;
the wine receiving equipment comprises a wine receiving groove, a light supplementing lamp and a glass cover, wherein a wine collecting groove for collecting wine liquid is formed in the wine receiving groove; the glass cover covers the wine receiving groove, the top surface of the glass cover is transparent glass provided with wine flowing holes, and both sides of the glass cover are ground glass; the light supplementing lamps are used for supplementing light through white spirit liquid gathered in the wine grooves in butt joint mode through the ground glass on the two sides; an overflow port is arranged between the glass cover and the wine receiving groove; the outlet of the liquor flowing pipeline is suspended right above the liquor flowing hole, liquor to be detected conveyed by the liquor flowing pipeline is injected into the liquor flowing pipeline through the liquor flowing hole, and the liquor to be detected overflows from the overflow port after impacting residual liquor in the liquor receiving tank to generate hops;
the industrial camera is arranged right above the wine receiving groove and is used for capturing a real-time image in the wine receiving groove;
the embedded module is used for acquiring real-time images captured by the industrial camera and running a liquor segmentation detection algorithm and an anomaly detection algorithm based on a computer vision and deep learning method in real time to obtain liquor segmentation results and anomaly information.
According to the invention, the wine receiving equipment is used for receiving wine left by the wine flowing pipe, the residual wine in the equipment is impacted to generate sufficient hops, and the light supplement lamp is horizontally arranged to avoid light reflection and strengthen the hop characteristics. The liquor receiving groove in the liquor receiving equipment discharges liquor in an overflow mode, so that the falling height of liquor left from the liquor flowing pipe is unchanged, the impact force is unchanged, and the hop form is stable; meanwhile, the height of the camera from the liquid level of the wine is guaranteed to be unchanged, and the situation of image blurring caused by insufficient depth of field of the camera and fluctuation of the liquid level height is avoided. In addition, the industrial camera adopts a high-speed shutter industrial camera, and just opposite to one end connected with the overflow of the wine tank, the shutter speed can be set to 1/4000 seconds for capturing the hop image with fast movement; the embedded module takes an embedded AI chip with NPU as an operation core, runs a liquor segmentation detection algorithm and an abnormity detection algorithm based on computer vision and deep learning in real time, and uploads liquor segmentation results and abnormity information through a communication interface. Compared with a workstation and a server, the embedded module has the characteristics of low cost, small volume, low power consumption and easiness in installation and maintenance.
On the basis of the above scheme, the present invention can further provide the following preferred implementation modes.
Preferably, the number of the light supplementing lamps is two, the two light supplementing lamps are respectively opposite to the two side ground glass of the glass cover, the bottom surfaces of the two side ground glass are flush with the bottom surface of the overflow port, the light direction of the light supplementing lamps is parallel to the liquid level in the overflow state, and the light irradiation direction of the light supplementing lamps is parallel to the liquid level in the overflow state. Under this way, connect the light filling lamp among the wine equipment and place in the both sides that connect the spirit groove in fact, one is placed to one side, and the light filling lamp is placed highly and is kept the same with the wine liquid level height. The placing method can prevent the liquid level from reflecting light in a large area and entering a camera interference algorithm; the brightness of the edge of the hop can be enhanced, so that the hop has the characteristics of bright edge and dark middle, namely, the edge characteristics of the hop are highlighted, and the accuracy of the white spirit segmentation algorithm based on computer vision and deep learning can be improved.
Preferably, in the embedded module, an embedded AI chip RK3399Pro is used as an operation core, and a white spirit segmentation detection algorithm and an abnormality detection algorithm based on a computer vision and deep learning method are operated in the chip.
In the invention, an embedded module of the NPU is used as an operation device of a liquor picking algorithm, and an embedded AI chip RK3399Pro with an on-chip NPU is used as an operation core of the embedded module. The NPU module is a hardware operation unit specially designed aiming at the characteristics of a neural network, has 2.4TOPS theoretical peak value calculation power, can quickly carry out neural network calculation, and has power consumption less than 1.5W. When the neural network operation is carried out, compared with the current general CPU/GPU operation platform, the neural network operation platform has the characteristic of high energy efficiency ratio. Compared with a microcomputer, a workstation and a server, the embedded module has the characteristics of low cost, small volume, low power consumption, high energy efficiency ratio and easiness in installation and maintenance; compared with an embedded module without an NPU (neural network unit), such as an MCU (microprogrammed control unit), the embedded module with the RK3399Pro as the core can efficiently run a deep learning algorithm, can run a deep learning-based liquor segmentation detection algorithm with higher precision and better generalization, and can obtain a more accurate and stable identification result.
Preferably, the bottom of the wine receiving groove is provided with a slope, and wine liquid is reserved in a wine collecting groove formed by the top surface of the slope and the inner side wall of the wine receiving groove; the slope is gradually inclined downwards from one side close to the wine flowing hole to one side close to the overflow port, and the overflow port is higher than the surface of the slope right below the wine flowing hole; the lowest point of the liquor flowing pipeline is provided with a residual liquor outlet. The residual wine outlet is used for ensuring that wine does not remain in the wine receiving groove, and the wine in the groove can flow out after a through hole is required to be drilled at the lower end due to the inclination of the wine receiving groove, and the diameter of the through hole is preferably 5 mm.
Preferably, the wine receiving equipment further comprises a wine receiving funnel for receiving the white wine flowing out from the wine receiving groove. Of course, other liquid collecting devices than the wine receiving funnel may be used as long as the collection of the white spirit can be achieved.
Preferably, the wine receiving groove is internally plated with gold and is subjected to matte treatment. The inner part of the wine receiving groove is plated with gold, and forms color contrast with the white color of the edge of the hop under the irradiation of white light, so that the edge characteristic of the hop is highlighted. The golden region can also be used by a segmentation algorithm for image preprocessing, such as detecting whether the camera is shifted or occluded. The wine receiving groove is internally treated by matte treatment, so that light supplement and reflection interference caused by ambient light can be further reduced.
Preferably, a film for preventing water mist and grease is adhered to the inside of the transparent glass of the glass cover. According to the invention, the glass cover covers the upper part and two sides of the wine receiving groove, so that waste caused by wine liquid splashing can be reduced, volatilization of alcohol and fragrant substances is reduced, but the wine liquid inevitably has splashing and grease impurities in the detection process, and the transparency of the transparent glass above the wine receiving groove can be influenced, so that a camera can conveniently and stably shoot a hop image, and the waterproof fog and grease-proof adhesive film is adhered to the inner part of the transparent glass.
Preferably, in the wine segmentation detection algorithm and the anomaly detection algorithm based on the computer vision and the deep learning method, the steps 1) to 7) are executed in a circulating manner from the start of wine flowing until the wine flowing is finished;
1) image acquisition and decoding: setting the shutter speed of the industrial camera to 1/4000 seconds, acquiring a YUV422 format wine receiving tank overlook image in real time, and decoding the overlook image into an RGB format;
2) image preprocessing: sampling the overlook image of the wine receiving groove by 10 times, converting the color space into HSV space, and obtaining an image of a wine collecting groove area in the wine receiving groove through threshold segmentation; removing noise from the wine collection groove region image by using morphological opening operation, and removing holes by using morphological closing operation to obtain a complete binary segmentation image for filtering noise, wherein the morphological operation uses a5 multiplied by 5 circular kernel; detecting corners on the binary segmentation graph by using a Harris corner detection algorithm, if 4 rectangular corners of the wine collection groove are detected, tiling the 4 corners on the whole image through projection transformation to be used as an ROI (Region of interest) of a subsequent algorithm;
3) an image anomaly detection algorithm: abnormal condition detection including image shift and occlusion, image blurring and no wine flowing is carried out on the binary segmentation image; the method for detecting image shift and occlusion comprises the following steps: when the number of corner points of the wine collecting groove detected in the step 2) is not 4, or the number of the corner points is 4 but the shape is not a rectangle, or the shape is a rectangle but the coordinates of the center of the rectangle and the length and width of the rectangle deviate from a normal interval, the abnormal condition of image deviation or shielding is considered to occur; the image blur detection method comprises the following steps: firstly, the gray histogram variance D of the ROI of the original image is counted by using an image re-blurring-based method1After 5 × 5 Gaussian blur is performed on the image, the variance D of the gray histogram of the ROI of the image is counted2With D2/D1As an index of the image blurring degree, the abnormal condition that the original image is blurred is considered to occur when the index is smaller than a first set threshold; the detection method of the non-flowing wine comprises the following steps: using a method based on inter-frame difference, firstly, obtaining a 3 × 3Sobel edge image of an ROI part of each frame of original image, and then compressing the edge image according to the following formula (1), wherein I (x, y) is a (x, y) position pixel value in the edge image before compression, and I (x, y) is a position pixel value in the edge image before compressionc(xc,yc) For (x) in the compressed edge imagec,yc) Position pixel value, kx、kyCompression ratio in x, y directions:
respectively obtaining compressed edge images of two continuous frame images according to the formula (1), and recording the edge images as Ic1、Ic2If II Ic2-Ic1‖2If the value is less than the second set threshold value, the abnormal condition that the wine does not flow is considered to occur;
in the image abnormity detection process, if any abnormal condition occurs, directly proceeding to step 7);
4) obtaining a hop region: obtaining a 3 × 3Sobel edge image I from the ROI image obtained in the step 2), and obtaining an average value in a 224 × 224 area by sliding with 2 as a step length to obtain an image Ia, wherein the pixel value of the image Ia is positively correlated with the number of hops in the 224 × 224 range with the pixel as the center; then, a K-nms (K Non maximum mapping, K Non maximum suppression) method Is operated on the image Ia to obtain K hop regions Is with the size of 224 × 224;
5) white spirit segmentation algorithm: operating a convolutional neural network, and sequentially inputting k hop region images Is to obtain the segmentation probability that the current white spirit in the wine receiving tank belongs to five categories of first-stage wine, second-stage wine, third-stage wine, tail wine and tail water, wherein the average of the output k-component segment probability Is used as the final segmentation probability;
6) and (3) post-processing algorithm: firstly, carrying out first-order filtering with tau being 0.8 on final segmentation probability of the current white spirit in a spirit tank, which belongs to five categories of first-stage spirit, second-stage spirit, third-stage spirit, spirit tail and tail water, filtering out impossible state transition by using a state machine, and then segmenting the white spirit corresponding to the segmentation probability with the maximum probability to serve as the final segmentation result of the current white spirit;
7) reporting the result and exception: and reporting the segmentation result and the abnormal information through an Ethernet interface.
Further, the k-nms method comprises the following specific steps in sequence:
①, sorting the Ia pixel values from big to small to obtain a pixel list L;
②, selecting the first point Pmax in L, namely the point with the maximum pixel value, and adding the point into the set U;
③ selecting the next point P in L;
④ calculating IOU (Intersection over Unit) of 224 × 224 boxes corresponding to all points in P and U, if IOU is higher than 0.5, returning to step ③, otherwise adding P into set U;
⑤, when the number of points in U reaches k, the algorithm ends by taking 224 × 224 areas centered on the k points to obtain k hop area images Is with the size of 224 × 224.
Furthermore, in the liquor segmentation detection algorithm and the abnormity detection algorithm, each step is respectively operated on three operation components, namely a CPU, a GPU and an NPU, in an RK3399Pro chip in the embedded module according to different calculation amounts and calculation characteristics, and concurrent calculation is carried out in a pipeline mode; the three steps of image acquisition and decoding, post-processing algorithm, reporting result and exception are operated by using two CPUs in RK3399 Pro; the image preprocessing, the image anomaly detection algorithm and the hop region acquisition step are carried out by using a Mali-T860 MP4 GPU in the RK3399 Pro; the liquor segmentation algorithm uses NPU inside RK3399Pro to perform high-speed operation.
Compared with the background technology, the invention has the following beneficial effects:
1) and (3) operating a liquor segmentation algorithm based on deep learning in real time by using an NPU-based embedded module as an operation device. Compared with a workstation and a server, the system has the advantages of low cost, small volume, low power consumption and easy installation and maintenance.
2) The light filling lamp is placed highly and the wine liquid is unanimous, both can strengthen hops edge characteristic, can reduce the influence that the wine liquid reflection of light brought again.
3) By using the liquor overflow mode, the liquor level is kept unchanged, the hop form change caused by liquor flowing height change is avoided, and meanwhile, the image blur caused by insufficient depth of field of the camera is avoided.
4) The inner part of the wine receiving groove is plated with gold which forms contrast with the hops to enhance the characteristics of the hops.
5) The wine receiving groove is internally processed by matte treatment, so that interference is reduced.
6) The upper part and two sides of the wine receiving groove are covered by the glass cover, so that the splashing of wine liquid and the volatilization of effective substances are reduced.
7) According to the difference between the calculated amount and the calculation characteristics, all the steps of the algorithm are operated on different calculation components, and the flow line is used for concurrent operation, so that the calculation efficiency and the hardware resource utilization rate are improved, and meanwhile, the throughput of the algorithm is improved.
Drawings
FIG. 1 is a general block diagram of a detection apparatus;
FIG. 2 is a view showing the construction of the wine receiving well and the glass cover, wherein the left side is a top right view and the right side is a sectional view;
FIG. 3 is a schematic diagram of a fill-in lamp placement, wherein a dashed box represents that the algorithm in the box runs in the RK3399Pro specified computing unit;
FIG. 4 is a diagram of an embedded module architecture;
FIG. 5 is a flow chart of a liquor segmentation detection algorithm and an anomaly detection algorithm;
FIG. 6 is a schematic diagram of a post-processing state machine;
FIG. 7 is a schematic diagram of an algorithm execution pipeline;
the reference numbers in the figures are: the device comprises a USB cable 1, an industrial camera 2, a network cable 3, an embedded module 4, a wine flowing pipeline 5, a glass cover 6, a wine receiving groove 7, a light supplementing lamp 8, a wine receiving funnel 9, transparent glass 6-1, ground glass 6-2, a wine flowing hole 6-3, a residual wine outflow port 7-1, a slope 7-2 and an overflow port 7-3.
Detailed Description
The invention will be further elucidated and described with reference to the drawings and the detailed description. The technical features of the embodiments of the present invention can be combined correspondingly without mutual conflict.
The general structure of the white spirit segmentation detection device based on the embedded module is shown in figure 1. The device as a whole can be divided into several parts, namely a wine receiving device, an industrial camera 2 and an NPU-based embedded module 4.
Wherein, connect wine equipment again to include and connect wine groove 7, light filling lamp 8, glass cover 6. Fig. 2 is a structural view of the wine receiving tank and the glass cover, wherein the left drawing is a top right perspective view and the right drawing is a sectional view. The wine receiving groove 7 is a cover-free groove body, and a wine collecting groove for collecting wine is formed in the wine receiving groove. In the present embodiment, in order to facilitate the discharge of the residual wine after the detection, the bottom of the wine receiving groove 7 is provided with a slope 7-2. The top surface of the slope 7-2 and the inner side wall of the wine receiving groove 7 form a wine collecting groove for temporarily storing wine, and the top view of the wine flowing pipeline 5 is rectangular. The slope 7-2 is gradually inclined downwards from the right side to the left side, the lowest point is positioned at the leftmost end, and the lowest point of the wine flowing pipeline 5 is provided with a residual wine outlet 7-1 with the diameter of 5mm penetrating through the bottom of the wine receiving groove 7. The glass cover 6 covers the wine receiving groove 7, the top surface of the glass cover is transparent glass 6-1 provided with wine flowing holes 6-3, and the two sides of the glass cover are ground glass 6-2. An overflow port 7-3 is arranged between the side surface of the top of the glass cover 6 and the wine receiving groove 7 and is used for overflowing the white wine. In addition, the wine receiving equipment also comprises a wine receiving funnel 9 which is used for receiving the white wine flowing out from the residual wine outlet 7-1 and the overflow outlet 7-3 of the wine receiving groove 7.
In addition, the liquor flowing pipeline 5 is used for conveying liquor to be detected into the liquor receiving equipment, the outlet of the liquor flowing pipeline 5 is suspended right above the liquor flowing hole 6-3, the liquor to be detected conveyed is injected into the liquor flowing pipeline 5 through the liquor flowing hole 6-3, and the liquor to be detected is overflowed from the overflow opening 7-3 after impacting residual liquor in the liquor receiving groove 7 to generate hops. Therefore, in order to ensure the occurrence of hops inside, the overflow opening 7-3 should be higher than the surface of the slope 7-2 directly below the wine flow hole 6-3 so that the white wine falling from the wine flow duct 5 in the overflow state hits the liquid surface, rather than the surface of the slope 7-2.
Fig. 3 is a schematic view of a light supplement lamp placement method. A light supplementing lamp is respectively arranged on two sides of the wine receiving groove 7, the two light supplementing lamps 8 are respectively used for supplementing light to the white wine liquid gathered in the wine receiving groove 7 through the two side ground glasses 6-2, the light emitting directions are respectively opposite to the two side ground glasses 6-2 of the glass cover 6, the bottom surfaces of the two side ground glasses 6-2 are flush with the bottom surface of the overflow port 7-3, the light direction of the light supplementing lamps 8 is towards the wine receiving groove, and the light emitting direction is parallel to the liquid level in the overflow state. The light source height of the light supplement lamp is the same as the height of the lower end of the overflow port, namely the height of the light source is the same as the height of the liquid level of the wine. This light filling mode has two benefits: firstly, the reflected light on the liquid surface can not directly enter the industrial camera positioned right above the liquid surface; and secondly, a circle of light spots can be formed at the edge of the hop, so that the characteristics of bright edge and dark center are presented, and the accuracy of the white spirit segmentation algorithm based on the computer vision algorithm is improved.
In addition, in this embodiment, the inner portion of the wine receiving tank 7 is plated with gold, and under the irradiation of the white light supplement lamp, the white hop edge and the gold background generate obvious color difference, so that the accuracy of the segmentation algorithm is improved. The golden background also facilitates image preprocessing and anomaly detection, such as detecting whether the camera is offset, detecting whether the camera is occluded, and the like. The interior of the wine receiving groove is further subjected to sub-light treatment in a specific mode of sand blasting, so that light supplement and reflection caused by ambient light are reduced.
The industrial camera 2 is arranged right above the wine receiving tank 7 and is used for capturing real-time images in the wine receiving tank 7. The embedded module 4 is used for acquiring real-time images captured by the industrial camera 2, and running a liquor segmentation detection algorithm and an anomaly detection algorithm based on a computer vision and deep learning method in real time to obtain liquor segmentation results and anomaly information.
When the wine receiving tank is used, liquor to be detected flows out of the liquor flowing pipeline 5, flows into the liquor receiving tank 7 through the liquor flowing holes 6-3 above the glass cover 6, and is impacted with the liquor in the liquor receiving tank 7 to generate hops. The hop slowly tends to be stable in shape along with the flow of the liquor towards the overflow opening 7-3, and enters the collection area of the industrial camera 2. Finally, the wine liquid flows into a wine receiving funnel 9 from an overflow opening 7-3 between the wine receiving groove 7 and the glass cover 6 and is converged into the pipeline.
In the invention, the wine receiving groove 7 and the glass cover 6 are connected in a sealing way, and only a gap is arranged at the position 7-3 in figure 2 to form the overflow port 7-3, so when wine in the wine receiving groove 7 reaches the height of the overflow port 7-3, the wine can flow out from the overflow port 7-3, and the wine liquid level in the wine receiving groove 7 is stable. The stable liquor level brings two benefits: firstly, the height of the wine flowing pipeline 5 from the liquid level is not changed, and the impact force generated when the flow is stable is not changed, so that the change of the hop form along with the change of the liquid level is avoided; and secondly, the distance between the camera lens and the liquid level of the wine is not changed, the focus of the camera is not changed, imaging blurring caused by the change of the distance of a shooting object is avoided, and the requirement on the depth of field of the industrial camera 2 is greatly reduced.
Due to the fact that the wine receiving groove 7 is sealed by the glass cover 6, the splashing of wine and the volatilization of effective ingredients of the wine are greatly reduced. In order to facilitate the camera to shoot the hop image, the glass above the glass cover 6 is transparent glass 6-1, and a waterproof fog and grease-proof adhesive film is attached to avoid the glass from being fuzzy. The glass on the two sides of the glass cover 6 is ground glass 6-2, which plays a role in softening light and reduces large-area reflection of wine.
As the inner part of the wine receiving groove 7 is provided with the slope 7-2, and the lowest point of the slope 7-2 is provided with a residual wine outlet 7-1. Therefore, when the liquor-picking process is finished, the liquor remaining in the liquor-receiving trough 7 flows out from the residual liquor outlet 7-1. The slope 7-2 should not have too large a slope, so as to avoid that the wine liquid directly impacts the slope 7-2 when remaining from the wine flowing pipeline 5 and cannot form enough hops.
In this embodiment, the wine receiving groove 7 and the wine receiving funnel 9 are made of food-grade stainless steel. The wine receiving groove 7 and the glass cover 6 are hermetically connected through food-grade sealant, and the wine receiving groove 7 and the wine receiving funnel 9 are connected through welding.
When the wine is left from the wine flowing pipeline 5, a large impact force is generated, the hop shape is unstable, and water bloom is generated more. And in the process of flowing the hop to the overflow port 7-3, the hop disappears, and the hop shape tends to be stable. Therefore, the industrial camera 2 is placed about 50cm directly above the wine receiving tank 7, near the end of the overflow 7-3, in order to take a stable hop image. The axis of the lens of the industrial camera 2 is vertical to the wine receiving groove 7. Since the hop movement speed is fast, the industrial camera 2 shutter speed is set to 1/4000 seconds to capture a clear hop image.
In the embodiment, the focal length of the selected lens is 25mm, the field angle is 19.6 degrees, and when the vertical distance between the lens and the liquid level of the wine is 50cm, the field range is about 17cm, so that the requirement of a visual algorithm can be met.
The industrial camera 2 is connected with the embedded module 4 through the USB cable 1 and transmits control signals and image information. The embedded module 4 is connected to the ethernet via the network cable 3.
Fig. 4 is a structural diagram of the embedded module in the present embodiment. The embedded module takes an AI chip RK3399Pro with an NPU as a calculation core, and also comprises an RAM, a Nand Flash, an RS485 circuit and interface, an Ethernet circuit and interface, a USB circuit and interface, a TF circuit and interface, an HDMI interface and a power supply circuit. All the circuits and the interfaces are positioned on the same circuit board, so that the circuit is small in size and easy to install and maintain.
RK3399Pro has an on-chip NPU. The NPU has 1 NN Engine (neural Network Engine) and 1 VPU (Vector Processing Unit). The NPU is designed by special hardware, can process 1920 multiplication and addition operations in parallel in one clock cycle, and is very suitable for running of a deep learning algorithm. The NPU has a theoretical calculation peak value of 2.4TOPs, the power consumption is only 1.5W, and the energy efficiency ratio is high. In addition, RK3399Pro also has 2 Cortex-A72, 4 Cortex-A53, and 1 Mali-T860 MP4 GPU, computing power was adequate.
There are two types of RAM on the embedded module: one is general RAM, 2 pieces of LPDDR3 memory chips of 2GB specifically, couple to dual-channel DDR controller in RK3399Pro one, as the main memory of CPU/GPU; the other is NPU special RAM, specifically 1 piece of 2GB LPDDR3 memory chip, and connected with the NPU DDR controller in RK3399Pro piece as NPU special memory.
The embedded module is provided with a 16GB Nand Flash chip which is connected with RK3399Pro through an eMMC interface and is used for storing an operating system, an algorithm executable file, an algorithm parameter file and the like. The embedded module is also provided with a TF card interface which is accessed to RK3399Pro through an SD/MMC interface; the inserted TF card is used for storing algorithm intermediate results, liquor segmentation results, abnormal detection results, logs and the like.
The embedded module has 4 kinds of communication interfaces: HDMI, USB, RS485, Ethernet. The HDMI is used for connecting the display equipment during debugging; the RS485 is used for connecting an upper computer to carry out debugging and system configuration; the USB is responsible for connecting an industrial camera to acquire images and can also be used for connecting a keyboard and a mouse during debugging; the ethernet is responsible for transmitting the detection result and abnormal data to the superior system, such as PLC and DCS.
Fig. 5 is a flow chart of a liquor segmentation detection algorithm and an anomaly detection algorithm. The dashed boxes in the figure represent the in-box algorithm running in the RK3399 Pro-specific computing unit. The algorithm comprises 7 steps of image acquisition and decoding, image preprocessing, image anomaly detection algorithm, hop region acquisition, liquor segmentation algorithm, post-processing algorithm and reporting result and anomaly, can quickly and effectively detect the image anomaly and carry out high-precision segmentation on the liquor. The following steps are executed in a circulating mode from the time of starting wine flowing until the wine flowing is finished by the following steps 1) to 7).
1) Image acquisition and decoding: the shutter speed of the industrial camera 2 is set to 1/4000 seconds, and the top view image of the wine receiving tank 7 in YUV422 format is acquired in real time through the driving program of the industrial camera and is decoded into RGB format.
2) Image preprocessing: and (3) down-sampling the overlook image of the wine receiving groove 7 by 10 times, converting the color space into HSV space, and dividing by using a threshold value of H element [20,60] and V element [0.1,0.8] to obtain an image of the wine collecting groove area in the wine receiving groove 7, namely a rectangular golden area. And removing noise from the wine collection groove region image by using morphological opening operation, and removing holes by using closing operation to obtain a complete binary segmentation image with noise filtered, wherein the morphological operation uses a5 multiplied by 5 circular kernel. And detecting the corner points on the two-value segmentation graph by using a Harris corner point detection algorithm, and if 4 rectangular corner points Of the wine collection groove are detected, tiling the 4 corner points on the whole image through projection transformation to be used as an ROI (Region Of Interest) Of a subsequent algorithm.
3) An image anomaly detection algorithm: abnormal condition detection including image shift and occlusion, image blurring and no wine flowing is carried out on the binary segmentation map, wherein:
the method for detecting image shift and occlusion comprises the following steps: and when the number of corner points of the wine collecting groove detected in the step 2) is not 4, or the number of the corner points is 4 but the deviation between the shape and the rectangle is large, or the shape is the rectangle but the center coordinates and the length and the width of the rectangle deviate from the normal interval, the abnormal condition of image deviation or shielding is considered to occur.
The image blur detection method comprises the following steps: firstly, the gray histogram variance D of the ROI of the original image is counted by using an image re-blurring-based method1After 5 × 5 Gaussian blur is performed on the image, the variance D of the gray histogram of the ROI of the image is counted2With D2/D1As an index of the degree of image blur, when the index is smaller than a first set threshold, it is considered that an abnormality of the original image blur has occurred.
The detection method of the non-flowing wine comprises the following steps: using a method based on inter-frame difference, firstly, obtaining a 3 × 3Sobel edge image of an ROI part of each frame of original image, and then compressing the edge image according to the following formula (1), wherein I (x, y) is a (x, y) position pixel value in the edge image before compression, and I (x, y) is a position pixel value in the edge image before compressionc(xc,yc) For (x) in the compressed edge imagec,yc) Position pixel value, kx、kyCompression ratio in x, y directions:
respectively obtaining compressed edge images of two continuous frame images according to the formula (1), and recording the edge images as Ic1、Ic2If II Ic2-Ic1‖2If the value is less than the second set threshold value, the abnormal condition that the wine does not flow is considered to occur;
in the image abnormality detection process, if any abnormality occurs, the process proceeds directly to step 7).
It should be noted that each threshold in this step may be adjusted and optimized according to actual needs.
4) Obtaining a hop region: obtaining a 3 × 3Sobel edge image I from the ROI image obtained in step 2), and obtaining an average value in a region of 224 × 224 by sliding with 2 as a step size to obtain an image Ia, wherein a pixel value of the image Ia is positively correlated with the number of hops in a range of 224 × 224 with the pixel as a center. Then, a K-nms (K Non maximum inhibition) method Is run on the image Ia to obtain K hop regions Is with a size of 224 × 224. Where k is 5.
In this embodiment, the specific steps of the k-nms method are sequentially:
①, sorting the Ia pixel values from big to small to obtain a pixel list L;
②, selecting the first point Pmax in L, namely the point with the maximum pixel value, and adding the point into the set U;
③ selecting the next point P in L;
④ calculating IOU (Intersection over Unit) of 224 × 224 boxes corresponding to all points in P and U, if IOU is higher than 0.5, returning to step ③, otherwise adding P into set U;
⑤, when the number of points in U reaches k, the algorithm ends by taking 224 × 224 areas centered on the k points to obtain k hop area images Is with the size of 224 × 224.
5) White spirit segmentation algorithm: operating a convolutional neural network, and sequentially inputting k hop region images Is to obtain the segmentation probability that the current white spirit in the wine receiving tank 7 belongs to five categories of first-stage wine, second-stage wine, third-stage wine, tail wine and tail water, wherein the average of the output k groups of the segmentation probability Is taken as the final segmentation probability;
6) and (3) post-processing algorithm: firstly, carrying out first-order filtering with tau being 0.8 on final segmentation probability of the current white spirit in the spirit tank 7 belonging to five categories of first-stage spirit, second-stage spirit, third-stage spirit, spirit tail and tail water, filtering out impossible state transition through a state machine, only transferring the next-stage spirit when the probability of the next-stage spirit is more than 0.8, and then segmenting the white spirit corresponding to the segmentation probability with the maximum probability to serve as the final segmentation result of the current white spirit. The state machine of this embodiment is shown in fig. 6.
7) Reporting the result and exception: and reporting the segmentation result and the abnormal information through an Ethernet interface.
The steps are arranged to be executed on different operation units due to different calculation amounts and calculation characteristics. The image acquisition and decoding are mainly time-consuming in data transmission, the calculation amount is not large, the calculation density is low, and therefore the image acquisition and decoding part is completed by one CPU core of RK3399 Pro.
And transmitting the decoded image to a GPU in a memory sharing mode, and carrying out operations of three steps of image preprocessing, an anomaly detection algorithm and hop region acquisition. The three steps use a traditional computer vision method, have large calculation amount, high calculation density and high parallelism, and are suitable for operation on a GPU.
The NPU can efficiently and quickly operate the convolutional neural network, so that the liquor segmentation algorithm is operated on the NPU. The training of the convolutional neural network is not performed on an embedded module.
The final post-processing algorithm, the reported result and the abnormal step are mainly based on logic judgment, and the method is suitable for running of a CPU with a complex control circuit.
FIG. 7 is a schematic diagram of an algorithm execution pipeline. The algorithm steps are sequentially operated on different hardware, and the operation of the different hardware is not interfered with each other, so the algorithm can be concurrently executed in a pipeline mode, the overall throughput of the system is improved, and the purpose of real-time operation is achieved. The present invention uses 4 arithmetic elements, thus dividing the algorithm into 4 pipeline stages. Starting from the 4 th moment after the pipeline is started, 4 arithmetic units are all put into operation, and the utilization rate and the concurrency degree of hardware resources reach the maximum.
On the embedded module based on NPU, the wine segmentation detection algorithm and the abnormity detection algorithm based on computer vision are operated, and the wine segmentation detection method has the advantages that:
1) compared with a workstation and a server, the embedded module has the advantages of low cost, small volume, low power consumption and easy installation and maintenance;
2) compared with the CPU, the NPU has high parallelism and sufficient computational power; compared with a GPU, the power consumption is low, and the energy efficiency ratio is high. The hardware architecture is highly parallel, and is suitable for neural network operation.
3) Compared with an embedded liquor segmentation scheme based on the MCU, the embedded module based on the NPU can run a liquor segmentation algorithm with higher precision based on computer vision and deep learning, and is beneficial to the stability of liquor quality.
The above-described embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the invention. Various changes and modifications may be made by one of ordinary skill in the pertinent art without departing from the spirit and scope of the present invention. Therefore, the technical scheme obtained by adopting the mode of equivalent replacement or equivalent transformation is within the protection scope of the invention.
Claims (10)
1. A white spirit segmentation detection device based on an embedded module is characterized by comprising a white spirit receiving device, an industrial camera (2) and an NPU-based embedded module (4);
the wine receiving equipment comprises a wine receiving groove (7), a light supplementing lamp (8) and a glass cover (6), wherein a wine collecting groove for collecting wine is formed in the wine receiving groove (7); the glass cover (6) covers the wine receiving groove (7), the top surface of the glass cover is transparent glass (6-1) provided with wine flowing holes (6-3), and both sides of the glass cover are ground glass (6-2); the light supplement lamp (8) is used for supplementing light for white spirit liquid gathered in the wine tank (7) through two side ground glasses (6-2) respectively; an overflow port (7-3) is arranged between the glass cover (6) and the wine receiving groove (7); the outlet of the liquor flowing pipeline (5) is suspended right above the liquor flowing hole (6-3), liquor to be detected conveyed by the liquor flowing pipeline (5) is injected into the liquor flowing pipeline (5) through the liquor flowing hole (6-3), and the liquor to be detected overflows from the overflow port (7-3) after impacting residual liquor in the liquor receiving tank (7) to generate hops;
the industrial camera (2) is arranged right above the wine receiving groove (7) and is used for capturing a real-time image in the wine receiving groove (7);
the embedded module (4) is used for acquiring real-time images captured by the industrial camera (2) and running a liquor segmentation detection algorithm and an anomaly detection algorithm based on a computer vision and deep learning method in real time to obtain liquor segmentation results and anomaly information.
2. The segmented white spirit detection device based on the embedded module as claimed in claim 1, wherein two light supplement lamps (8) are provided and respectively face two side ground glasses (6-2) of the glass cover (6), the bottom surfaces of the two side ground glasses (6-2) are flush with the bottom surface of the overflow port (7-3), light of the light supplement lamps (8) faces the spirit receiving groove, and the light irradiation direction is parallel to the liquid level in the overflow state.
3. The segmented liquor detection device based on the embedded module as claimed in claim 1, wherein in the embedded module (4), an embedded AI chip RK3399Pro is used as an operation core, and a liquor segmented detection algorithm and an abnormal detection algorithm based on a computer vision and deep learning method are run in the chip.
4. The wine segmentation detection device based on the embedded module as claimed in claim 1, wherein the bottom of the wine receiving groove (7) is provided with a slope (7-2), and wine is retained in a wine collecting groove formed by the top surface of the slope (7-2) and the inner side wall of the wine receiving groove (7); the slope (7-2) is gradually inclined downwards from one side close to the wine flowing hole (6-3) to one side close to the overflow port (7-3), and the overflow port (7-3) is higher than the surface of the slope (7-2) right below the wine flowing hole (6-3); the lowest point of the liquor flowing pipeline (5) is provided with a residual liquor outlet (7-1).
5. The wine segmentation detection device based on embedded module as claimed in claim 1, characterized in that, connect wine equipment in still include connect wine funnel (9), be used for accepting the white wine that flows out from connecing the wine groove (7), the pipeline of converging.
6. The wine segmentation detection device based on the embedded module as claimed in claim 1, wherein the wine receiving tank is internally plated with gold and is subjected to matte treatment.
7. The segmented white spirit detection device based on the embedded module as claimed in claim 1, wherein a film for preventing water mist and grease is pasted inside the transparent glass (6-1) of the glass cover (6).
8. The segmented white spirit detection device based on the embedded module according to claim 1, wherein in the segmented white spirit detection algorithm and the anomaly detection algorithm based on the computer vision and the deep learning method, the steps 1) to 7) are executed in a circulating manner from the time of starting the flow of the white spirit until the flow of the white spirit is finished;
1) image acquisition and decoding: setting the shutter speed of the industrial camera (2) to 1/4000 seconds, acquiring an overlook image of the wine receiving tank (7) in YUV422 format in real time, and decoding the overlook image into RGB format;
2) image preprocessing: down-sampling the overlook image of the wine receiving groove (7) by 10 times, converting the color space into HSV space, and obtaining a wine collecting groove area image of the wine receiving groove (7) through threshold segmentation; removing noise from the wine collection groove region image by using morphological opening operation, and removing holes by using morphological closing operation to obtain a complete binary segmentation image for filtering noise, wherein the morphological operation uses a5 multiplied by 5 circular kernel; detecting corners on the binary segmentation graph by using a Harris corner detection algorithm, if 4 rectangular corners of the wine collection groove are detected, tiling the 4 corners on the whole image through projection transformation to be used as an ROI (Region of interest) of a subsequent algorithm;
3) an image anomaly detection algorithm: abnormal condition detection including image shift and occlusion, image blurring and no wine flowing is carried out on the binary segmentation image; the method for detecting image shift and occlusion comprises the following steps: when step (ii) is carried out2) When the number of corner points of the detected wine collecting grooves is not 4, or the number of the corner points is 4 but the shape is not rectangular, or the shape is rectangular but the center coordinates and the length and the width of the rectangular deviate from a normal interval, the abnormal condition of image deviation or shielding is considered to occur; the image blur detection method comprises the following steps: firstly, the gray histogram variance D of the ROI of the original image is counted by using an image re-blurring-based method1After 5 × 5 Gaussian blur is performed on the image, the variance D of the gray histogram of the ROI of the image is counted2With D2/D1As an index of the image blurring degree, the abnormal condition that the original image is blurred is considered to occur when the index is smaller than a first set threshold; the detection method of the non-flowing wine comprises the following steps: using a method based on inter-frame difference, firstly, obtaining a 3 × 3Sobel edge image of an ROI part of each frame of original image, and then compressing the edge image according to the following formula (1), wherein I (x, y) is a (x, y) position pixel value in the edge image before compression, and I (x, y) is a position pixel value in the edge image before compressionc(xc,yc) For (x) in the compressed edge imagec,yc) Position pixel value, kx、kyCompression ratio in x, y directions:
respectively obtaining compressed edge images of two continuous frame images according to the formula (1), and recording the edge images as Ic1、Ic2If II Ic2-Ic1‖2If the value is less than the second set threshold value, the abnormal condition that the wine does not flow is considered to occur;
in the image abnormity detection process, if any abnormal condition occurs, directly proceeding to step 7);
4) obtaining a hop region: obtaining a 3 × 3Sobel edge image I from the ROI image obtained in the step 2), and obtaining an average value in a 224 × 224 area by sliding with 2 as a step length to obtain an image Ia, wherein the pixel value of the image Ia is positively correlated with the number of hops in the 224 × 224 range with the pixel as the center; then, a K-nms (K Non Maximum Suppression ) method Is operated on the image Ia to obtain K hop regions Is with the size of 224 multiplied by 224;
5) white spirit segmentation algorithm: operating a convolutional neural network, and sequentially inputting k hop region images Is to obtain the segmentation probability that the current white spirit in the wine receiving tank (7) belongs to five categories of first-stage wine, second-stage wine, third-stage wine, feints and tail water, wherein the average of the output k component segmentation probabilities Is used as the final segmentation probability;
6) and (3) post-processing algorithm: firstly, carrying out first-order filtering with tau being 0.8 on final segmentation probability of the current white spirit in a spirit tank (7) belonging to five categories of first-stage spirit, second-stage spirit, third-stage spirit, spirit tail and tail water, filtering out impossible state transition by using a state machine, and then segmenting the white spirit corresponding to the segmentation probability with the maximum probability to serve as a final segmentation result of the current white spirit;
7) reporting the result and exception: and reporting the segmentation result and the abnormal information through an Ethernet interface.
9. The white spirit segmentation detection device based on the embedded module as claimed in claim 8, wherein the k-nms method comprises the following specific steps in sequence:
①, sorting the Ia pixel values from big to small to obtain a pixel list L;
②, selecting the first point Pmax in L, namely the point with the maximum pixel value, and adding the point into the set U;
③ selecting the next point P in L;
④ calculating IOU (Intersection over Unit) of 224 × 224 boxes corresponding to all points in P and U, if IOU is higher than 0.5, returning to step ③, otherwise adding P into set U;
⑤, when the number of points in U reaches k, the algorithm ends by taking 224 × 224 areas centered on the k points to obtain k hop area images Is with the size of 224 × 224.
10. The wine segmentation detection device based on the embedded module as claimed in claim 8, wherein in the wine segmentation detection algorithm and the anomaly detection algorithm, each step is respectively operated on three operation components of CPU, GPU and NPU in an RK3399Pro chip in the embedded module (4) according to different calculation amount and calculation characteristics, and concurrent calculation is carried out in a pipeline mode; the three steps of image acquisition and decoding, post-processing algorithm, reporting result and exception are operated by using two CPUs in RK3399 Pro; the image preprocessing, the image anomaly detection algorithm and the hop region acquisition step are carried out by using a Mali-T860 MP4 GPU in the RK3399 Pro; the liquor segmentation algorithm uses NPU inside RK3399Pro to perform high-speed operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271818.6A CN110927174B (en) | 2019-12-12 | 2019-12-12 | White spirit segmentation detection device and method based on embedded module |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271818.6A CN110927174B (en) | 2019-12-12 | 2019-12-12 | White spirit segmentation detection device and method based on embedded module |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110927174A true CN110927174A (en) | 2020-03-27 |
| CN110927174B CN110927174B (en) | 2021-03-26 |
Family
ID=69860202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911271818.6A Active CN110927174B (en) | 2019-12-12 | 2019-12-12 | White spirit segmentation detection device and method based on embedded module |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110927174B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111537685A (en) * | 2020-05-19 | 2020-08-14 | 河北工业大学 | Alcohol content online detection wine picking and blending system |
| CN112699968A (en) * | 2021-01-18 | 2021-04-23 | 河北工业大学 | Pluck wine equipment based on white spirit hops image |
| CN113337365A (en) * | 2021-07-13 | 2021-09-03 | 河北工业大学 | Multi-sensor online wine picking system with liquid level control function |
| CN114276891A (en) * | 2022-01-11 | 2022-04-05 | 安徽金种子酒业股份有限公司 | Artifical supplementary hierarchical wine device that connects |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202744526U (en) * | 2012-07-17 | 2013-02-20 | 泸州品创科技有限公司 | Automatic wine picking system |
| CN203112805U (en) * | 2013-03-04 | 2013-08-07 | 泸州品创科技有限公司 | Hops generation device |
| CN106018651A (en) * | 2015-08-27 | 2016-10-12 | 泸州品创科技有限公司 | Analysis method for receiving Baijiu in stages in Baijiu distillation process and application |
| CN207318364U (en) * | 2017-09-11 | 2018-05-04 | 浙江乌程酒业有限公司 | A kind of yellow rice wine observes device |
| CN108580154A (en) * | 2018-04-25 | 2018-09-28 | 未石互动科技股份有限公司 | A kind of fountain simulation system |
| CN208076408U (en) * | 2018-05-07 | 2018-11-09 | 江苏食品药品职业技术学院 | A kind of yellow wine fermentation degree detection device |
-
2019
- 2019-12-12 CN CN201911271818.6A patent/CN110927174B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202744526U (en) * | 2012-07-17 | 2013-02-20 | 泸州品创科技有限公司 | Automatic wine picking system |
| CN203112805U (en) * | 2013-03-04 | 2013-08-07 | 泸州品创科技有限公司 | Hops generation device |
| CN106018651A (en) * | 2015-08-27 | 2016-10-12 | 泸州品创科技有限公司 | Analysis method for receiving Baijiu in stages in Baijiu distillation process and application |
| CN207318364U (en) * | 2017-09-11 | 2018-05-04 | 浙江乌程酒业有限公司 | A kind of yellow rice wine observes device |
| CN108580154A (en) * | 2018-04-25 | 2018-09-28 | 未石互动科技股份有限公司 | A kind of fountain simulation system |
| CN208076408U (en) * | 2018-05-07 | 2018-11-09 | 江苏食品药品职业技术学院 | A kind of yellow wine fermentation degree detection device |
Non-Patent Citations (2)
| Title |
|---|
| QING-YI LU ET AL: "Quantification of bioactive constituents and antioxidant activity of Chinese yellow wine", 《JOURNAL OF FOOD COMPOSITION AND ANALYSIS》 * |
| 刘江生等: "酒花与酒的探讨", 《酿酒》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111537685A (en) * | 2020-05-19 | 2020-08-14 | 河北工业大学 | Alcohol content online detection wine picking and blending system |
| CN112699968A (en) * | 2021-01-18 | 2021-04-23 | 河北工业大学 | Pluck wine equipment based on white spirit hops image |
| CN112699968B (en) * | 2021-01-18 | 2022-05-27 | 河北工业大学 | Liquor picking equipment and method based on liquor hops image |
| CN113337365A (en) * | 2021-07-13 | 2021-09-03 | 河北工业大学 | Multi-sensor online wine picking system with liquid level control function |
| CN114276891A (en) * | 2022-01-11 | 2022-04-05 | 安徽金种子酒业股份有限公司 | Artifical supplementary hierarchical wine device that connects |
| CN114276891B (en) * | 2022-01-11 | 2023-07-11 | 安徽金种子酒业股份有限公司 | Manual auxiliary grading wine receiving device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110927174B (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110927174B (en) | White spirit segmentation detection device and method based on embedded module | |
| CN108742159B (en) | Intelligent control device and method for water dispenser based on RGB-D camera | |
| CN108427940B (en) | Intelligent control device and control method for water dispenser water outlet based on depth camera | |
| KR100903704B1 (en) | The White and Brown Rice Appearance Characteristics Measurement System and the Same Method | |
| CN105403147B (en) | One kind being based on Embedded bottle embryo detection system and detection method | |
| CN105389581B (en) | A kind of rice germ plumule integrity degree intelligent identifying system and its recognition methods | |
| CN109409368A (en) | Mine leather belt is vertical to tear detection device and detection method | |
| CN110146516A (en) | Fruit sorter based on orthogonal binocular machine vision | |
| CN110261391A (en) | A kind of LED chip appearance detection system and method | |
| CN108318494B (en) | The red online vision detection and classification devices and methods therefor for proposing fruit powder | |
| CN110473194A (en) | Fruit surface defect detection method based on more image block Threshold Segmentation Algorithms | |
| CN110930366A (en) | Nut loosening detection method for wind power tower cylinder | |
| CN103440492A (en) | Hand-held cigarette recognizer | |
| CN103913469A (en) | System for detecting and processing contaminants on surfaces of poultry carcasses online | |
| CN102788806B (en) | Fruit peel defect detection method based on spheroidic brightness transformation | |
| CN114445707A (en) | Intelligent visual fine detection method for defects of bottled water labels | |
| CN205721986U (en) | A kind of cable visual defects detection and character recognition device | |
| CN108776143A (en) | A kind of online vision inspection apparatus and method of the small stain of egg eggshell surface | |
| CN109461156A (en) | The threaded closure plug assembly and detection method of view-based access control model | |
| CN111562260B (en) | Lotus root mud hole detection method and device based on machine vision | |
| CN111047598B (en) | Deep learning-based ultraviolet discharge light spot segmentation method and device for power transmission and transformation equipment | |
| CN107576660B (en) | A kind of double yellow duck egg Automatic Visual Inspection method based on apart from contour | |
| CN113298776A (en) | Method for detecting appearance defects of metal closed water pump impeller | |
| CN102495067B (en) | System for identifying impurities of edible funguses on line | |
| CN106093051A (en) | Paper roll tangent plane burr detection method based on machine vision and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |