CN103226814B - A kind of medicine bottle foreign matter detecting method based on medical visual detection robot image rectification - Google Patents

Abstract

The invention discloses a medicine bottle foreign object detection method based on medical vision detection robot image correction. By determining the characteristics of the medicine bottle, the rotation angle and horizontal offset of the medicine bottle can be accurately calculated. Based on the template matching method, the maximum matching The vertical offset is determined by the method of degree, and the precise correction of the image is realized, and then the qualification judgment of the medical foreign body is realized through operations such as difference, binarization, and superposition. The high-precision registration method of sequential images that can be used in an environment with insufficient mechanical anti-shake suppression effectively makes up for the lack of hardware, and is suitable for high-speed and high-precision visual detection robots for medical foreign objects such as ampoules, large infusions, oral liquids, and soft bags. detection.

Description

A drug bottle foreign object detection method based on image correction of medical vision detection robot

technical field

The invention relates to a method for detecting foreign objects in medicine bottles based on image correction of a medical vision detection robot.

Background technique

At present, the existing medical foreign body visual detection robots (ampoules, large infusions, oral liquids, etc.) mostly use the method of sequential frame difference processing to detect foreign bodies. The method is simple, fast and effective for feature extraction and recognition judgment of the target. However, the anti-shake ability of the mechanical body is extremely high. A slight shake will cause the target to shift a few pixels in the image, and the imaging of 50-micron pharmaceutical foreign particles in the image is only a few pixels, resulting in the characteristics of the foreign object being different from the image. The background interference features are relatively close, which greatly increases the difficulty of developing high-speed and high-precision visual inspection robots, and the scale of distinction is not large. Therefore, it is necessary to solve the problem of image rotation and offset during the high-speed operation of the robot in order to achieve high-precision detection of faint foreign objects.

Contents of the invention

The present invention proposes a method for detecting foreign objects in medicine bottles based on the image correction of medical vision detection robots. The detection accuracy is low.

The technical scheme that the present invention adopts is as follows:

A method for detecting foreign matter in medicine bottles based on image correction of a medical vision detection robot, comprising the following steps:

Step 1: Image acquisition;

Continuously collect N frames of medical sequence images Image={Image ₀ , Image ₁ ... Image _N-1 };

Among them, the number of frames N to be collected is determined according to the actual needs in the detection process of medical foreign matter; the acquisition mode is a single-trigger acquisition of multiple frames, and the image information of the high-speed rotation period of the foreign matter is collected intensively.

Step 2: Preprocessing the sequence images to obtain the region of interest of each frame image;

Image processing is performed on each frame of the sequence image Image in turn to obtain the sequence image ImageReduced of the medicine bottle area in all images. The specific processing method for each frame of image is as follows:

1) let i=0, i represents the i-th frame in the sequence image, and the value range is 0～(N-1);

2) Carry out binarization processing to image Image _i , generate binary image BinImage _i , the threshold value in the binarization processing is set to T _i , guarantees that the bottle body object in binary image BinImage _i is completely separated from the background;

3) Perform connected domain search operation on the binary image BinImage _i to generate connected domain Connections _i [m], where m is the number of connected domains in each frame of image;

4) Calculate the area Areas[m] of each connected domain in the image BinImage _i , the area of the connected domain Connections _i [m] is the total number of all pixels P _m in the connected domain, where {P _m |P _m ∈ Connections _i [m]};

5) Select the connected domain Connections _i [Areas _max ] with the largest area in the image BinImage _i as the feature of the medicine bottle to be processed, where Areas _max is the connected domain label m corresponding to the connected domain with the largest area in Connections _i [m];

6) Fill the connected domain Connections _i [Areas _max ] with the largest area to ensure that the connected domain is a solid connected domain, and then expand the filled connected domain with a rectangular expansion factor of 9×9, and the expanded connected domain domain as the region of interest ROI (Region Of Interest) of the medicine bottle in the current image, that is, the image ImageReduced _i of the medicine bottle area in the current image; Step 3: sequentially locate the position of the medicine bottle in each frame of image ImageReduced _i , and calculate each frame of image ImageReduced _i Obtain the data needed to locate the position of the medicine bottle in each frame of image ImageReduced _i by the tilt angle of the traditional Chinese medicine bottle. The steps are as follows:

1) Calculate the center of gravity P(x ₀ ,y ₀ ) of the connected domain Connections _i [Areas _max ] with the largest area in ImageReduced _i of each frame, and the calculation formula is:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \}</span>$

Among them, M is the total number of pixels in the connected domain Connections _i [Areas _max ], 0≤i≤M-1;

2) Through the center of gravity P(x ₀ , y ₀ ), make a line segment that intersects the edges of the two walls of the medicine bottle, and calculate the slope of the line segment, and the angle corresponding to the slope of the shortest line segment is the positioning inclination angle θ _min of the bottle;

3) With the center of gravity P(x ₀ , y ₀ ) as the origin, make a sampling line l ₁ ...l _n parallel and equidistant to the shortest line segment obtained in 2), the distance between two adjacent sampling lines is d, and then, respectively Take the intersection of all sampling lines and the edge of the medicine bottle as the center point, select a sampling window R1...Rn, L1...Ln with a size of 30×30 from the image ImageReduced _i , n is a positive integer, n≥2, and the data of the medicine bottle positioning is the data in the sampling window;

Step 4: For the data in the sampling window of the medicine bottle obtained in step 3, the horizontal displacement and rotation angle of the medicine bottle are determined by using sub-pixel image threshold segmentation and hough transform method, and the specific operation process is as follows;

1) Use the canny operator to detect the edge of the bottle wall in the sampling window, and obtain the edge pixel point set of the bottle wall in the sampling window. The edge pixel point set in the sampling window on the left edge of the bottle wall is P _l1 ^* ... P _ln ^* and the edge pixel point set of the acquisition window on the right edge of the bottle wall is P _r1 ^* ... P _rn ^* ;

2) Perform hough transformation on the pixel point set P _li ^* where the left edge of the bottle wall intersects with the sampling window in turn, and calculate the fitting straight line y=k _li x+b _li on the left edge, where 1<i<n, and according to The accumulator array in the hough transform determines the pixel point set L ^* _i corresponding to the maximum value in the array as the most relevant point set on the left edge;

Perform hough transformation on the pixel point set P _ri ^* where the right edge of the bottle wall intersects with the sampling window in turn, and calculate the right edge fitting straight line y=k _ri x+b _ri , where 1<i<n, and according to the hough transformation The accumulator array in determines the pixel point set R ^* _i corresponding to the maximum value in the array as the most relevant point set on the right edge;

3) Carry out data analysis to the set of relevant points L ^* _i on the left edge and the fitting straight line y=k _li x+b _li , and calculate the best fitting straight line on the left edge;

Respectively sort the slope parameter k _li (0≤i≤n) and the intercept parameter b _li (0≤i≤n) in the fitted straight line sequence, and calculate the mean values Avg _Kl and b li of k _li and b _li in the fitted straight line Avg _bl , variance S _Kl , S _bl , maximum value max _Kl , max _bl and minimum value min _Kl , min _bl , iteratively calculate the best fitting straight line y=k _L ^* x+b _L ^* on the left edge: a) If S _Kl ≤ ε and S _bl ≤ ξ, ε and ξ are both set thresholds, then the equation of the best fitting straight line on the left edge is y=Avg _Kl x+Avg _bl ;

b) If S _Kl >ε or S _bl >ξ, ε, ξ are the set thresholds, then compare {|max _Kl -Avg _Kl |, |min _Kl -Avg _Kl |} or {|max _bl -Avg _bl |, The size of |min _bl -Avg _bl |}, and remove the edge most relevant point set L ^* _i that deviates from the mean value Avg _Kl or Avg _bl maximum from the left side edge related point set L ^* ; then re-execute 3);

If the left edge related point set L ^* only contains 2 sets of acquisition window data sets, stop the iteration, and then perform hough transformation on all left edge point sets P _l1 ^* ... P _ln ^* to generate the best fitting curve y= k _L ^*′ x+b _L ^*′ ; otherwise, return to 3) to obtain the best edge fitting straight line on the left;

4) Carry out data analysis to the right side edge related point set R ^* _i and the fitting straight line y=k _ri ·x+b _ri , calculate the best fitting straight line of the right side edge;

Sorting the slope parameter k _ri (0≤i≤n) and the intercept parameter b _ri (0≤i≤n) in the fitted straight line sequence respectively, and calculating the mean values Avg _Kr and b ri of k _ri and b _ri in the fitted straight line Avg _br , variance S _Kr , S _br , maximum value max _Kr , max _br and minimum value min _Kr , min _br , iteratively calculate the best fitting straight line y=k _R ^* x+b _R ^* on the right edge:

a) If S _Kr ≤ ε and S _br ≤ ξ, ε and ξ are both set thresholds, then the equation of the best fitting straight line on the right edge is y=Avg _Kr x+Avg _br ;

b) If S _Kr >ε or S _br >ξ, ε, ξ are the set thresholds, then compare {|max _Kr -Avg _Kr |, |min _Kr -Avg _Kr |} or {|max _br -Avg _br |, The size of |min _br -Avg _br |}, and remove the edge most relevant point set R ^* _i that deviates from the mean Avg _Kr or Avg _br maximum from the right edge related point set R ^* ; then re-execute 4);

If the right edge related point set R ^* only contains two sets of acquisition window data sets, stop the iteration, and then perform hough transformation on all the right edge point set P _r1 ^* ... P _rn ^* to generate the best fitting curve y= k _R ^*′ x+b _R ^*′ ; otherwise, return to 4) to obtain the best edge fitting straight line on the right;

5) calculate the angle of rotation θ ^* of the medicine bottle and the amount of horizontal translation X ^* ;

By 3) and 4) obtain the fitting straight line of two bottle wall edges of medicine bottle, the angle bisector of two fitting straight lines is used as the inclination standard line that measures medicine bottle inclination degree, then has: $x^{*}=(-\frac{b^{*}}{\mathrm{the\; tan}\left({\mathrm{\&theta;}}^{*}\right)}+\frac{{b_R}^{*}-{b_L}^{*}}{{k_L}^{*}-{k_R}^{*}}),{\mathrm{\&theta;}}^{*}=\frac{\arctan \left({k_R}^{*}\right)+\arctan \left({k_L}^{*}\right)}{2},$ $b^{*}=\frac{{k_L}^{*}\&CenterDot;{b_R}^{*}-{k_R}^{*}\&CenterDot;{b_L}^{*}}{{k_L}^{*}-{k_R}^{*}};$

Step 5: Perform the first correction on each frame of the sequence image Image in turn;

Translate according to the horizontal translation amount X ^* , and perform rotation transformation according to the tilt angle θ ^* of the medicine bottle. During the translation and rotation transformation process, the quadratic interpolation algorithm is used, and the transformation matrix is R _xi :

$R_{\mathrm{xi}}=\left[\begin{array}{ccc}\cos {{\mathrm{\&theta;}}^{*}}_i& \sin {{\mathrm{\&theta;}}^{*}}_i& \frac{\mathrm{image}\mathrm{Width}}{2}-\frac{b^{*}}{\mathrm{the\; tan}\left({{\mathrm{\&theta;}}^{*}}_i\right)}+\frac{b_{\mathrm{Ri}}^{*}-b_{\mathrm{Li}}^{*}}{k_{\mathrm{Li}}^{*}-k_{\mathrm{Ri}}^{*}}\\ -\sin {{\mathrm{\&theta;}}^{*}}_i& \cos {{\mathrm{\&theta;}}^{*}}_i& 0\\ 0& 0& 1\end{array}\right],$ ImageWidth is the image width;

After rotation and horizontal translation, a new sequence image ImageR _i is generated, where i is an integer less than N;

Step 6: Use template matching to obtain the vertical offset of the adjacent frame image, and perform a second correction on the image;

1) Select two adjacent frames of images ImageR _i and ImageR _i+1 , and establish the user coordinate system UCS at the point P′(x ₀ ′, y ₀ ′) in the two frames of images respectively, where x ₀ ′=ImageWidth /2, y ₀ '=ImageHeight/10, ImageHeight is the image height;

2) Set n template pairs [P _j , P _j+1 ] in the user coordinate system UCS, the value range of j is 1~n, n is a positive integer, and the template pairs must meet the following conditions:

(1) Template pairs are distributed symmetrically about the y-axis in the user coordinate system, and the distance between template pairs is 10 to 30 pixels;

The main reason for limiting the distance between the template pair is that when the foreign object rotates to the middle position of the bottle body, the moving speed direction is parallel to the imaging plane of the camera. Therefore, compared with other positions, the displacement between pixels per unit time is the largest, so the probability of foreign objects entering the template matrix minimum;

(2) Template pairs are evenly distributed in the vertical direction in the user coordinate system and cover most of the region of interest; the probability of obvious features entering the template can be increased;

(3) The template window in the template window is a square of 15-25 pixels;

If it is too large, the image processing speed will be reduced; if it is too small, the matching accuracy will be affected. Generally, a square of 20 pixel values is selected, depending on the image resolution, jitter amplitude, and processing speed requirements.

3) Select the optimal matching template for the images ImageR _i and ImageR _i+1 by the gray histogram method;

For the corresponding templates in ImageR _i and ImageR _i+1 respectively Perform grayscale histogram statistics to obtain grayscale histogram matrix i is the image frame sequence number, j is the template matching sequence number;

Calculate separately and Variance and to Sort, select the three template windows corresponding to the variances arranged in the first three as the preliminary matching template windows, and then compare the corresponding size, choose the smallest The corresponding template is set as the optimal matching template window P _pm ;

In the process of preliminary matching template selection, the larger the variance value, the more obvious the feature information in the template image, and the higher the matching accuracy after the matching operation is completed, but in the matching process, it should be selected A relatively small template can avoid template matching failure caused by foreign matter entering the matching template.

4) Template matching is carried out by using the de-average normalized correlation method, and the vertical coordinates of the image are corrected;

Take the center coordinates (x _pm , y _pm ) of the optimal matching template P _pm , and then match P _pm with the image window G _q in the image ImageR _i+1 , where the image window G _q in ImageR _i+1 The center coordinates of G are (x _pm , y _pm +q), q is the longitudinal offset vector, the value range is an integer of (-10,10), and the length and width of G _q are respectively the length and width of the optimal matching template P _pm equal in width;

The template and the image window are matched by calculating the template window matching coefficient ρ(G _q ) by using the mean value normalized correlation method:

$\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}{{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Among them, G _q (a, b) is the gray value at the coordinates (a, b) in the image window G _q , and P _pm (a, b) is the gray value at the coordinates (a, b) in the optimal matching template. grayscale value, are the gray mean values of G _q and P _pm respectively, both the optimal matching template and the image window contain S pixels, and both take the lower left corner as the starting point coordinates;

Right now $\stackrel{\&OverBar;}{G_q}=\underset{a=1}{\overset{S}{\mathrm{\&Sigma;}}}\underset{b=1}{\overset{S}{\mathrm{\&Sigma;}}}{G}_q(a,b),\stackrel{\&OverBar;}{P_{\mathrm{pm}}}=\underset{a=1}{\overset{S}{\mathrm{\&Sigma;}}}\underset{b=1}{\overset{S}{\mathrm{\&Sigma;}}}{P}_{\mathrm{pm}}(a,b),$ The closer ρ(G _q ) is to 1, the higher the degree of matching between the template window and the image window, and vice versa, the less matching between the template window and the image window;

(3) Compare the template matching degree coefficient ρ(G _q ), select the maximum value ρ _max (G _q ) as the optimal matching degree, and the q value in ρ _max (G _q ) as the longitudinal direction of ImageR _i+1 relative to ImageR _i The offset is M _ρmax , and the vertical offset between all the images in the image sequence and the previous frame image is calculated sequentially, which is denoted as M ⁱ .

(4) Perform vertical translation transformation on the image ImageR _i+1 to obtain the image ImageR′ _i+1 , and the translation matrix is R _y(i+1) to complete the vertical offset correction caused by mechanical shaking; the translation matrix R _{y(i+1 )} is:

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]$

Wherein, i is an integer less than N;

Step 7: Absolute difference processing is performed on the sequence images after longitudinal offset correction to complete the detection of medical foreign objects.

In the image after precise jitter removal, the imaging position of a certain stain or glass spot on the bottle wall in the sequence frame images is basically the same, and the deviation will not be large. Therefore, in the process of absolute difference between two adjacent images, there will be It is eliminated, and has little influence on the faint foreign objects in the image.

The concrete implementation steps of described step 7 are:

(1) Perform absolute difference processing on two adjacent frames of images ImageR _i ' and ImageR' _i+1 to obtain a difference image

Sub ImageR _i ＝|Im ageR _i ′-Im ageR′ _i+1 |, i∈[1,N-1] and i∈Z;

The precise registration operation of adjacent frame images can ensure that a differential operation can eliminate the background information of the image (including bottle wall stains, glass bags, bottle cracks, etc.), without affecting the imaging characteristics of foreign objects moving in the liquid.

(2) Binarize the primary difference image with a given threshold T to obtain the binarized image Bin ImageR _i , the value of T is determined according to the characteristics of the detection object, and the imaging gray value of glass shavings is generally larger , the threshold T is also relatively large, and for slender objects such as hair and fibers, the gray value is not much different from the background, and the threshold T is selected to be small;

$\mathrm{<span\; class="notranslate">\; Bin\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{<span\; class="notranslate">\; ,</span>}& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; T</span>}\end{array}\right.$

(3) Perform superposition operation on sequence frame binary image Bin ImageR _i :

Image _max ＝MAX{Bin ImageR ₁ ,...Bin ImageR _N-1 }, N is the total number of sequence frames

(4) Find the connected domain of Image _max , count the number of connected domains Num _total , the sum of the area of the connected domain Area _total and the area is greater than the given The number of connected domains of the value ( is the minimum imaging area of foreign matter);

(5) According to the calculated number of connected domains Num _total , judge whether the quality of the medicine is qualified, and the product quality θ is

α and β are the genuine and defective product quality thresholds, respectively.

In the step 2, the connected domain search operation is performed on the binary image BinImage _i , and the connected domain search operation is performed on the image by using the 8-neighborhood connectivity rule.

Beneficial effect

The invention provides a medicine bottle foreign object detection method based on the image correction of the medical vision detection robot. By determining the characteristics of the medicine bottle, the rotation angle and horizontal offset of the medicine bottle can be accurately calculated. Based on the template matching method, the maximum matching The vertical offset is determined by the method of degree, and the precise correction of the image is realized, and then the qualification judgment of the medical foreign body is realized through operations such as difference, binarization, and superposition. Compared with the existing technology, its detection accuracy is high. This method selects the edge of the bottle wall with the most prominent features and the highest contrast as the positioning reference, and uses sub-pixel edge extraction and multi-point set hough change method to realize the rotation angle and Accurate positioning of the horizontal offset. In the process of positioning the vertical coordinates, the high-discrimination feature demeaning normalization correlation template matching method is used to achieve high-precision positioning of the vertical coordinates. The stability is strong. During the matching process, the A variety of evaluation standards meet the needs of different situations in complex environments, reduce the detection misjudgment caused by image registration failure, and reduce the false detection rate; the calculation speed is fast. On the one hand, the sampling matrix is used in the matching process to replace the global Data, and use the feature point set to replace the pixel operation, the calculation speed is fast, and the effect is remarkable; on the other hand, multiple positioning is used to ensure the narrowing of the positioning range, improve the positioning quality, and high efficiency; The high-precision registration method of sequential images used in insufficient environments can effectively make up for the lack of hardware, and is suitable for high-speed and high-precision detection of medical foreign body vision detection robots such as ampoules, large infusions, oral liquids, and soft bags.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Fig. 2 is a schematic diagram of establishing a user coordinate system UCS in the figure;

Fig. 3 is a schematic diagram of drawing equidistant parallel sampling lines in the figure;

Figure 4 is the bottle wall effect diagram obtained after the image is processed by the CANNY operator;

Figure 5 is an effect diagram after the image is rotated and translated;

Fig. 6 is an effect diagram after the vertical coordinate correction is performed on the image.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

As shown in Figure 1, it is a flow chart of the present invention, a method for detecting foreign matter in a medicine bottle based on image correction of a medical visual inspection robot. In this example, a 1.3 million Baumer Gigabit camera, an 8mm lens, and an object distance of 10cm are used. The manufacturer’s foreign matter detection size requirement is 50um, that is, foreign matter exceeding 50um needs to be detected, including the following steps:

Step 1: Image acquisition;

Continuously collect N frames of medical sequence images Image={Image ₀ , Image ₁ ... Image _N-1 };

Among them, the number of frames N to be collected is determined according to the actual needs in the detection process of medical foreign matter; the acquisition mode is a single-trigger acquisition of multiple frames, and the image information of the high-speed rotation period of the foreign matter is collected intensively. Step 2: Preprocessing the sequence images to obtain the region of interest of each frame image;

Image processing is performed on each frame of the sequence image Image in turn to obtain the sequence image ImageReduced of the medicine bottle area in all images. The specific processing method for each frame of image is as follows:

1) let i=0, i represents the i-th frame in the sequence image, and the value range is 0～(N-1);

2) Carry out binarization processing to image Image _i , generate binary image BinImage _i , the threshold value in the binarization processing is set to T _i , guarantees that the bottle body object in binary image BinImage _i is completely separated from the background;

3) Perform connected domain search operation on the binary image BinImage _i to generate connected domain Connections _i [m], where m is the number of connected domains in each frame of image;

The connected domain search operation is performed on the binary image BinImage _i , and the connected domain search operation is performed on the image by using the 8-neighborhood connectivity rule.

4) Calculate the area Areas[m] of each connected domain in the image BinImage _i , the area of the connected domain Connections _i [m] is the total number of all pixels P _m in the connected domain, where {P _m |P _m ∈ Connections _i [m]};

5) Select the connected domain Connections _i [Areas _max ] with the largest area in the image BinImage _i as the feature of the medicine bottle to be processed, where Areas _max is the connected domain label m corresponding to the connected domain with the largest area in Connections _i [m];

6) Fill the connected domain Connections _i [Areas _max ] with the largest area to ensure that the connected domain is a solid connected domain, and then expand the filled connected domain with a rectangular expansion factor of 9×9, and the expanded connected domain domain as the region of interest ROI (Region Of Interest) of the medicine bottle in the current image, that is, the image ImageReduced _i of the medicine bottle region in the current image;

Step 3: Locate the position of the medicine bottle in each frame of image ImageReduced _i sequentially, calculate the inclination angle of the medicine bottle in each frame of image ImageReduced _i , and obtain the required data for locating the position of the medicine bottle in each frame of image ImageReduced _i , the specific operation steps are as follows:

1) Calculate the center of gravity P(x ₀ ,y ₀ ) of the connected domain Connections _i [Areas _max ] with the largest area in ImageReduced _i of each frame, and the calculation formula is:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \}</span>$

Among them, M is the total number of pixels in the connected domain Connections _i [Areas _max ], 0≤i≤M-1;

2) Through the center of gravity P(x ₀ , y ₀ ), make a line segment that intersects the edges of the two walls of the medicine bottle, and calculate the slope of the line segment, and the angle corresponding to the slope of the shortest line segment is the positioning inclination angle θ _min of the bottle;

Firstly, the user coordinate system UCS is established, as shown in Fig. 2, its origin is P(x ₀ , y ₀ ), the x-axis is the image row direction, and the y-axis is the image column direction. Establish a straight line y=tan(θ)·x passing through the origin P(x ₀ ,y ₀ ), n is an integer, and -450<n<450, then, find the intersection points (X _n1 , Y _n1 ) and (X _n2 , Y _n2 ) of the straight line y and the connected domain, and find the distance between the corresponding two points Finally, sort the distance D _n between the two points, find the minimum value D _min , and find the angle θ _min between the straight line corresponding to the shortest line segment and the x-axis, and the calculated θ _min is the inclination angle of the bottle.

3) With the center of gravity P(x ₀ , y ₀ ) as the origin, draw the sampling lines l ₁ , l ₂ , l ₃ , l ₄ and l ₅ parallel and equidistant to the shortest line obtained in 2), between two adjacent sampling lines The distance between them is d, and then, taking the intersection of all sampling lines and the edge of the medicine bottle as the center point, select a sampling window R1...R5, L1...L5 with a size of 30×30 from the image ImageReduced _i , as shown in Figure 3 , the data used for the positioning of the medicine bottle is the data in the sampling window;

Step 4: For the data in the sampling window of the medicine bottle obtained in step 3, the horizontal displacement and rotation angle of the medicine bottle are determined by using the sub-pixel image threshold segmentation and hough transform method, and the specific operation process is as follows;

1) Use the canny operator to detect the edge of the bottle wall in the sampling window, and obtain the edge pixel point set of the bottle wall in the sampling window. The edge pixel point set in the sampling window on the left edge of the bottle wall is P _l1 ^* , P _l2 ^* , P _l3 ^* , P _l4 ^* and P _l5 ^* , and the edge pixel point set of the collection window on the right edge of the bottle wall is P _r1 ^* , P _r2 ^* , P _r3 ^* , P _r4 ^* and P _r5 ^* ; but in The concentration of boundary points, in addition to the boundary data points, there are other interferences of information points, the main reason is caused by the reflection and refraction of the bottle body, the shape of the bottle body is cylindrical, and after the irregular light source is reflected or refracted by the bottle wall, It will cause non-uniform imaging, resulting in non-uniform gray distribution inside the image. After the canny operator processing, curved edges appear, as shown in Figure 4.

2) Perform hough transformation on the pixel point set P _li ^* where the left edge of the bottle wall intersects with the sampling window in turn, and calculate the fitting straight line y=k _li x+b _li on the left edge, where 1<i<5, and according to The accumulator array in the hough transform determines the pixel point set L ^* _i corresponding to the maximum value in the array as the most relevant point set on the left edge; the same method obtains the right edge fitting straight line y=k _ri x+b _ri , where 1<i<5, determine the most relevant point set R ^* _i on the right edge;

3) Carry out data analysis to the set of relevant points L ^* _i on the left edge and the fitting straight line y=k _li x+b _li , and calculate the best fitting straight line on the left edge;

Respectively sort the slope parameter k _li (0≤i≤5) and the intercept parameter b _li (0≤i≤5) in the fitted straight line sequence, and calculate the mean values Avg _Kl and b li of k _li and b _li in the fitted straight line Avg _bl , variance S _K , S _b , maximum value max _Kl , max _bl and minimum value min _Kl , min _bl , iteratively calculate the best fitting straight line y=k _L ^* x+b _L ^* on the left edge:

a) If S _K ≤ε and S _b ≤ξ, both ε and ξ are set thresholds, then the equation of the left edge fitting straight line is y=Avg _Kl x+Avg _bl ;

b) If S _K > ε or S _b > ξ, ε, ξ are the set thresholds, then compare {|max _Kl -Avg _Kl |, |min _Kl -Avg _Kl |} or {|max _bl -Avg _bl |, The size of |min _bl -Avg _bl |}, and remove the edge most relevant point set L ^* _i that deviates from the mean value Avg _Kl or Avg _bl maximum from the left side edge related point set L ^* ; then re-execute 3);

If the left edge related point set L ^* only contains 2 sets of acquisition frame data sets, stop the iteration, and then for all the left edge point set P _l1 ^* , P _l2 ^* , P _l3 ^* , P _l4 ^* and P _l5 ^* Perform hough transformation to generate the best fitting curve y=k _L ^*' x+b _L ^*' ; otherwise, return to 3) to obtain the left edge fitting straight line;

4) Carry out data analysis to the right side edge related point set R ^* _i and the fitting straight line y=k _ri ·x+b _ri , calculate the best fitting straight line of the right side edge;

Sorting the slope parameter k _ri (0≤i≤n) and the intercept parameter b _ri (0≤i≤n) in the fitted straight line sequence respectively, and calculating the mean values Avg _Kr and b ri of k _ri and b _ri in the fitted straight line Avg _br , variance S _Kr , S _br , maximum value max _Kr , max _br and minimum value min _Kr , min _br , iteratively calculate the best fitting straight line y=k _R ^* x+b _R ^* on the right edge:

a) If S _Kr ≤ ε and S _br ≤ ξ, ε and ξ are both set thresholds, then the equation of the best fitting straight line on the right edge is y=Avg _Kr x+Avg _br ;

b) If S _Kr >ε or S _br >ξ, ε, ξ are the set thresholds, then compare {|max _Kr -Avg _Kr |, |min _Kr -Avg _Kr |} or {|max _br -Avg _br |, The size of |min _br -Avg _br |}, and remove the edge most relevant point set R ^* _i that deviates from the mean Avg _Kr or Avg _br maximum from the right edge related point set R ^* ; then re-execute 4);

If the right edge related point set R ^* only contains two sets of acquisition window data sets, stop the iteration, and then perform hough transformation on all the right edge point set P _r1 ^* ... P _rn ^* to generate the best fitting curve y= k _R ^*′ x+b _R ^*′ ; otherwise, return to 4) to obtain the best edge fitting straight line on the right;

5) calculate the angle of rotation θ ^* of the medicine bottle and the amount of horizontal translation X ^* ;

By 3) and 4) the fitted straight lines of the two bottle wall edges of the medicine bottle are obtained, and the angle bisector of the two fitted straight lines is used as the inclination standard line measuring the inclination degree of the medicine bottle, then the intersection point X of the medicine bottle inclination standard line and the x-axis ^* for

in, ${\mathrm{\&theta;}}^{*}=\frac{\arctan \left({k_R}^{*}\right)+\arctan \left({k_L}^{*}\right)}{2},b^{*}=\frac{{k_L}^{*}\&Center\; Dot;{b_R}^{*}-{k_R}^{*}\&CenterDot;{b_{L1}}^{*}}{{k_L}^{*}-{k_R}^{*}};$

Step 5: Perform the first correction on each frame of the sequence image Image in turn;

Translate according to the horizontal translation amount X ^* , and perform rotation transformation according to the tilt angle θ ^* of the medicine bottle. During the translation and rotation transformation process, the quadratic interpolation algorithm is used, and the transformation matrix is R _xi :

$R_{\mathrm{xi}}=\left[\begin{array}{ccc}\cos {{\mathrm{\&theta;}}^{*}}_i& \sin {{\mathrm{\&theta;}}^{*}}_i& \frac{\mathrm{image}\mathrm{Width}}{2}-\frac{b^{*}}{\mathrm{the\; tan}\left({{\mathrm{\&theta;}}^{*}}_i\right)}+\frac{b_{\mathrm{Ri}}^{*}-b_{\mathrm{Li}}^{*}}{k_{\mathrm{Li}}^{*}-k_{\mathrm{Ri}}^{*}}\\ -\sin {{\mathrm{\&theta;}}^{*}}_i& \cos {{\mathrm{\&theta;}}^{*}}_i& 0\\ 0& 0& 1\end{array}\right],$ ImageWidth is the image width;

After rotation and horizontal translation, a new sequence image ImageR _i is generated, i is an integer less than N, as shown in Figure 5, a new image ImageR _{0 is generated for image Image 0 ;}

Step 6: Use template matching to obtain the vertical offset of the adjacent frame image, and perform a second correction on the image;

1) Select two adjacent frames of images ImageR _i and ImageR _i+1 , respectively establish the user coordinate system UCS at the point P′(x ₀ ′,y ₀ ′) in the two frames of images, where x ₀ ′ =Image Width/2, y ₀ '=Image Height/10, ImageHeight is image height;

2) Set three template pairs [R ₁ , R ₂ ], [R ₃ , R ₄ ] and [R ₅ , R ₆ ] in the user coordinate system UCS, and the template pairs must meet the following conditions:

(1) Template pairs are distributed symmetrically about the y-axis in the user coordinate system, and the distance between template pairs is 10 to 30 pixels;

The main reason for limiting the distance between templates is that when the foreign object rotates to the middle of the bottle body, the direction of motion velocity is parallel to the imaging plane of the camera. Therefore, compared with other positions, the displacement between pixels per unit time is the largest, so the probability of foreign objects entering the template is the smallest. ;

(2) Template pairs are evenly distributed in the vertical direction in the user coordinate system and cover most of the region of interest; the probability of obvious features entering the template can be increased;

(3) The template matrix in the template pair is a square of 15-25 pixels;

If it is too large, the image processing speed will be reduced; if it is too small, the matching accuracy will be affected. Generally, a square of 20 pixel values is selected, depending on the image resolution, jitter amplitude, and processing speed requirements.

3) Select the optimal matching template for images ImageR _i and ImageR _i+1 by the grayscale histogram method;

For the corresponding templates in the images ImageR _i and ImageR _i+1 respectively Perform grayscale histogram calculation to obtain grayscale histogram matrix i is the image frame sequence number, j is the template matching sequence number;

calculate and Variance and to Sorting, select the three templates corresponding to the first three variances as the preliminary matching template matrix, and then compare the corresponding size, choose the smallest The corresponding template matrix is set as the optimal matching template P _pm ;

In the process of preliminary matching template selection, the larger the variance value, the more obvious the feature information in the template image, and the higher the matching accuracy after the matching operation is completed, but in the matching process, it should be selected A relatively small template, which can avoid template matching failure caused by foreign matter entering the matching template;

4) Use the mean value normalization correlation method to carry out template matrix matching, and carry out ordinate correction to the image;

Take the center coordinates (x _pm , y _pm ) of the optimal matching template P _pm , and then match P _pm with the image window G _q in the image ImageR _{i +1, where the image window in ImageR i+1} The center coordinates of G _q are (x _pm , y _pm +q), q is the longitudinal offset vector, the value range is an integer of (-10,10), and the length and width of G _q are respectively the optimal matching template P _pm The length and width are equal; the template and the image window are matched by calculating the template matching degree coefficient ρ(G _q ) by using the mean value normalized correlation method:

$\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}{{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Among them, G _q (a, b) is the gray value at the coordinate (a, b) in the pixel image matrix G _q , and P _pm (a, b) is the coordinate (a, b) in the optimal matching template the gray value of are the gray mean values of G _q and P _pm respectively, both the optimal matching template and the image window contain S pixels, and both take the lower left corner as the starting point coordinates;

now that $\stackrel{\&OverBar;}{G_q}=\underset{a=1}{\overset{S}{\mathrm{\&Sigma;}}}\underset{b=1}{\overset{S}{\mathrm{\&Sigma;}}}{G}_q(a,b),\stackrel{\&OverBar;}{P_{\mathrm{pm}}}=\underset{a=1}{\overset{S}{\mathrm{\&Sigma;}}}\underset{b=1}{\overset{S}{\mathrm{\&Sigma;}}}{P}_{\mathrm{pm}}(a,b),$ The closer ρ(G _q ) is to 1, the higher the degree of matching between the template and the image window; otherwise, the less matching between the template and the image window;

(1) Compare the template matching degree coefficient ρ(G _q ), select the maximum value ρ _max as the optimal matching degree, and the q value in ρ _max (G _q ) as the longitudinal offset of ImageR _i+1 relative to ImageR _i for Sequentially calculate the longitudinal offsets of all images in the image sequence and the previous frame image, denoted as M ⁱ ;

(2) Perform vertical translation transformation on the image ImageR _i+1 to obtain the image ImageR′ _i+1 , and the translation matrix is R _y(i+1) to complete the vertical offset correction caused by mechanical shaking; the translation matrix R _{y(i +1)} for:

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]$

Step 7: Absolute difference processing is performed on the sequence images after longitudinal offset correction to complete the detection of medical foreign objects.

In the image after precise jitter removal, the imaging position of a certain stain or glass spot on the bottle wall in the sequence frame images is basically the same, and the deviation will not be large. Therefore, in the process of absolute difference between two adjacent images, there will be It is eliminated, and has little influence on the faint foreign objects in the image.

The concrete implementation steps of described step 7 are:

(1) Perform absolute difference processing on two adjacent frames of images ImageR _i ′ and ImageR′ _i+1 to obtain the difference image Sub ImageR _i =|Im ageR _i ′-Im ageR′ _i+1 |,i∈[ 1,N-1] and i∈Z;

The precise registration operation of adjacent frame images can ensure that a differential operation can eliminate the background information of the image (including bottle wall stains, glass bags, bottle cracks, etc.), without affecting the imaging characteristics of foreign objects moving in the liquid.

(2) Binarize the primary difference image with a given threshold T to obtain the binarized image Bin ImageR _i , the value of T is determined according to the characteristics of the detection object, and the imaging gray value of glass shavings is generally larger , the threshold T is also relatively large, and for slender objects such as hair and fibers, the gray value is not much different from the background, and the threshold T is selected to be small;

$\mathrm{<span\; class="notranslate">\; Bin\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; T</span>}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{<span\; class="notranslate">\; ,</span>}& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; T</span>}\end{array}\right.$

(3) Perform superposition operation on sequence frame binary image Bin ImageR _i :

Image _max ＝MAX{Bin ImageR ₁ ,...Bin ImageR _N-1 }, N is the total number of sequence frames

(4) Find the connected domain of Image _max , count the number of connected domains Num _total , the sum of the areas of connected domains Area _total and the area greater than a given The number of connected domains of the value ( is the minimum imaging area of foreign matter);

(5) According to the amount of calculated data, it is judged whether the quality of the medicine is qualified, and the product quality θ is

α and β are the genuine and defective product quality thresholds respectively, α is set to 5, and β is set to 20.

Claims (3)

1. A medicine bottle foreign matter detection method based on medical vision detection robot image correction, is characterized in that, comprises the following steps: Step 1: Image acquisition; Continuously collect N frames of medical sequence images Image={Image ₀ , Image ₁ ... Image _N-1 }; Step 2: Preprocessing the sequence images to obtain the region of interest of each frame image; Image processing is performed on each frame of the sequence image Image in turn to obtain the sequence image ImageReduced of the medicine bottle area in all images. The specific processing method for each frame of image is as follows: 1) let i=0, i represents the i-th frame in the sequence image, and the value range is 0～(N-1); 2) Carry out binarization processing to image Image _i , generate binary image BinImage _i , the threshold value in the binarization processing is set to T _i , guarantees that the bottle body object in binary image BinImage _i is completely separated from the background; 3) Perform connected domain search operation on the binary image BinImage _i to generate connected domain Connections _i [m], where m is the number of connected domains in each frame of image; 4) Calculate the area Areas[m] of each connected domain in the image BinImage _i , the area of the connected domain Connections _i [m] is the total number of all pixels P _m in the connected domain, where {P _m |P _m ∈ Connections _i [m]}; 5) Select the connected domain Connections _i [Areas _max ] with the largest area in the image BinImage _i as the feature of the medicine bottle to be processed, where Areas _max is the connected domain label m corresponding to the connected domain with the largest area in Connections _i [m]; 6) Fill the connected domain Connections _i [Areas _max ] with the largest area to ensure that the connected domain is a solid connected domain, and then expand the filled connected domain with a rectangular expansion factor of 9×9, and the expanded connected domain domain as the region of interest ROI (Region Of Interest) of the medicine bottle in the current image, that is, the image ImageReduced _i of the medicine bottle region in the current image; Step 3: Locate the position of the medicine bottle in each frame of image ImageReduced _i sequentially, calculate the inclination angle of the medicine bottle in each frame of image ImageReduced _i , and obtain the required data for locating the position of the medicine bottle in each frame of image ImageReduced _i , the steps are as follows: 1) Calculate the center of gravity P(x ₀ ,y ₀ ) of the connected domain Connections _i [Areas _max ] with the largest area in ImageReduced _i of each frame, and the calculation formula is: $\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \}</span>$ Among them, M is the total number of pixels in the connected domain Connections _i [Areas _max ], 0≤i≤M-1; 2) Through the center of gravity P(x ₀ , y ₀ ), make a line segment that intersects the edges of the two walls of the medicine bottle, and calculate the slope of the line segment, and the angle corresponding to the slope of the shortest line segment is the positioning inclination angle θ _min of the bottle; 3) With the center of gravity P(x ₀ , y ₀ ) as the origin, make a sampling line l ₁ ...l _n parallel and equidistant to the shortest line segment obtained in 2), the distance between two adjacent sampling lines is d, and then, respectively Take the intersection of all sampling lines and the edge of the medicine bottle as the center point, select a sampling window R1...Rn, L1...Ln with a size of 30×30 from the image ImageReduced _i , n is a positive integer, n≥2, and the data of the medicine bottle positioning is the data in the sampling window; Step 4: For the data in the sampling window of the medicine bottle obtained in step 3, the horizontal displacement and rotation angle of the medicine bottle are determined by using sub-pixel image threshold segmentation and hough transform method, and the specific operation process is as follows; 1) Use the canny operator to detect the edge of the bottle wall in the sampling window, and obtain the edge pixel point set of the bottle wall in the sampling window. The edge pixel point set in the sampling window on the left edge of the bottle wall is P _l1 ^* ... P _ln ^* and the edge pixel point set of the acquisition window on the right edge of the bottle wall is P _r1 ^* ... P _rn ^* ; 2) Perform hough transformation on the pixel point set P _li ^* where the left edge of the bottle wall intersects with the sampling window in turn, and calculate the fitting straight line y=k _li x+b _li on the left edge, where 1<i<n, and according to The accumulator array in the hough transform determines the pixel point set L ^* _i corresponding to the maximum value in the array as the most relevant point set on the left edge; Perform hough transformation on the pixel point set P _ri ^* where the right edge of the bottle wall intersects with the sampling window in turn, and calculate the right edge fitting straight line y=k _ri x+b _ri , where 1<i<n, and according to the hough transformation The accumulator array in determines the pixel point set R ^* _i corresponding to the maximum value in the array as the most relevant point set on the right edge; 3) Carry out data analysis to the set of relevant points L ^* _i on the left edge and the fitting straight line y=k _li x+b _li , and calculate the best fitting straight line on the left edge; Respectively sort the slope parameter k _li (0≤i≤n) and the intercept parameter b _li (0≤i≤n) in the fitted straight line sequence, and calculate the mean values Avg _Kl and b li of k _li and b _li in the fitted straight line Avg _bl , variance S _Kl , S _bl , maximum value max _Kl , max _bl and minimum value min _Kl , min _bl , iteratively calculate the best fitting straight line y=k _L ^* x+b _L ^* on the left edge: a) If S _Kl ≤ ε and S _bl ≤ ξ, ε and ξ are both set thresholds, then the equation of the best fitting straight line on the left edge is y=Avg _Kl x+Avg _bl ; b) If S _Kl >ε or S _bl >ξ, ε, ξ are the set thresholds, then compare {|max _Kl -Avg _Kl |, |min _Kl -Avg _Kl |} or {|max _bl -Avg _bl |, The size of |min _bl -Avg _bl |}, and remove the edge most relevant point set L ^* _i that deviates from the mean value Avg _Kl or Avg _bl maximum from the left side edge related point set L ^* ; then re-execute 3); If the left edge related point set L ^* only contains 2 sets of acquisition window data sets, stop the iteration, and then perform hough transformation on all left edge point sets P _l1 ^* ... P _ln ^* to generate the best fitting curve y= k _L ^*′ x+b _L ^*′ ; otherwise, return to 3) to obtain the best edge fitting straight line on the left; 4) Carry out data analysis to the right side edge related point set R ^* _i and the fitting straight line y=k _ri ·x+b _ri , calculate the best fitting straight line of the right side edge; Sorting the slope parameter k _ri (0≤i≤n) and the intercept parameter b _ri (0≤i≤n) in the fitted straight line sequence respectively, and calculating the mean values Avg _Kr and b ri of k _ri and b _ri in the fitted straight line Avg _br , variance S _Kr , S _br , maximum value max _Kr , max _br and minimum value min _Kr , min _br , iteratively calculate the best fitting straight line y=k _R ^* x+b _R ^* on the right edge: a) If S _Kr ≤ ε and S _br ≤ ξ, ε and ξ are both set thresholds, then the equation of the best fitting straight line on the right edge is y=Avg _Kr x+Avg _br ; b) If S _Kr >ε or S _br >ξ, ε, ξ are the set thresholds, then compare {|max _Kr -Avg _Kr |, |min _Kr -Avg _Kr |} or {|max _br -Avg _br |, The size of |min _br -Avg _br |}, and remove the edge most relevant point set R ^* _i that deviates from the mean Avg _Kr or Avg _br maximum from the right edge related point set R ^* ; then re-execute 4); If the right edge related point set R ^* only contains two sets of acquisition window data sets, stop the iteration, and then perform hough transformation on all the right edge point set P _r1 ^* ... P _rn ^* to generate the best fitting curve y = k _R ^*′ x+b _R ^*′ ; otherwise, return to 4) to obtain the best edge fitting straight line on the right; 5) calculate the angle of rotation θ ^* of the medicine bottle and the amount of horizontal translation X ^* ; By 3) and 4) obtain the fitting straight line of two bottle wall edges of medicine bottle, the angle bisector of two fitting straight lines is used as the inclination standard line that measures medicine bottle inclination degree, then has: $x^{*}=(-\frac{b^{*}}{\mathrm{the\; tan}\left({\mathrm{\&theta;}}^{*}\right)}+\frac{{b_R}^{*}-{b_L}^{*}}{{k_L}^{*}-{k_R}^{*}}),$ ${\mathrm{\&theta;}}^{*}=\frac{\arctan \left({k_R}^{*}\right)+\arctan \left({k_L}^{*}\right)}{2},$ $b^{*}=\frac{{k_L}^{*}\&Center\; Dot;{b_R}^{*}-{k_R}^{*}\&Center\; Dot;{b_L}^{*}}{{k_L}^{*}-{k_R}^{*}};$ Step 5: Perform the first correction on each frame of the sequence image Image in turn; Translate according to the horizontal translation amount X ^* , and perform rotation transformation according to the inclination angle θ ^* of the medicine bottle. The quadratic interpolation algorithm is used in the process of translation and rotation transformation, and the transformation matrix is R _i : $R_i=\left[\begin{array}{ccc}\cos {{\mathrm{\&theta;}}^{*}}_i& \sin {{\mathrm{\&theta;}}^{*}}_i& \frac{\mathrm{ImageWidth}}{2}-\frac{b^{*}}{\mathrm{the\; tan}\left({{\mathrm{\&theta;}}^{*}}_i\right)}+\frac{{b_{\mathrm{Ri}}}^{*}-{b_{\mathrm{Li}}}^{*}}{{k_{\mathrm{Li}}}^{*}-{k_{\mathrm{Ri}}}^{*}}\\ -\sin {{\mathrm{\&theta;}}^{*}}_i& \cos {{\mathrm{\&theta;}}^{*}}_i& 0\\ 0& 0& 1\end{array}\right],$ ImageWidth is the image width; After rotation and horizontal translation, a new sequence image ImageR _i is generated, where i is an integer less than N; Step 6: Use template matching to obtain the vertical offset of the adjacent frame image, and perform a second correction on the image; 1) Select two adjacent frames of images ImageR _i and ImageR _i+1 , and establish the user coordinate system UCS at the point P′(x ₀ ′, y ₀ ′) in the two frames of images respectively, where x ₀ ′=ImageWidth /2, y ₀ '=ImageHeight/10, ImageHeight is the image height; 2) Set n template pairs [P _j , P _j+1 ] in the user coordinate system UCS, the value range of j is 1~n, n is a positive integer, and the template pairs must meet the following conditions: (1) Template pairs are distributed symmetrically about the y-axis in the user coordinate system, and the distance between template pairs is 10 to 30 pixels; (2) Template pairs are evenly distributed in the vertical direction in the user coordinate system and cover most of the region of interest; (3) The template in the template is a square of 15-25 pixels; 3) Select the optimal matching template for the images ImageR _i and ImageR _i+1 by the gray histogram method; For the corresponding templates in ImageR _i and ImageR _i+1 respectively Perform grayscale histogram statistics to obtain grayscale histogram matrix i is the image frame sequence number, j is the template matching sequence number; Calculate separately and Variance and to Sort, select the three templates corresponding to the top three variances as the preliminary matching templates, and then compare the corresponding size, choose the smallest The corresponding template is set as the optimal matching template P _pm ; 4) Template matching is carried out by using the de-average normalized correlation method, and the vertical coordinates of the image are corrected; Take the center coordinates (x _pm , y _pm ) of the optimal matching template P _pm , and then match P _pm with the image window G _q in the image ImageR _i+1 , where the image window G _q in ImageR _i+1 The center coordinates of G are (x _pm , y _pm +q), q is the longitudinal offset vector, the value range is an integer of (-10,10), and the length and width of G _q are respectively the length and width of the optimal matching template P _pm equal in width; The template is matched with the image window by calculating the template matching degree coefficient ρ(G _q ) by using the average value normalized correlation method: $\mathrm{<span\; class="notranslate">\; \&rho;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}{{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{<span\; class="notranslate">\; \{</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; S</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; pm</span>}}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span>}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}}}$ Among them, G _q (a, b) is the gray value at the coordinates (a, b) in the image window G _q , and P _pm (a, b) is the gray value at the coordinates (a, b) in the optimal matching template. grayscale value, are the gray mean values of G _q and P _pm respectively, both the optimal matching template and the image window contain S pixels, and both take the lower left corner as the starting point coordinates; (1) Compare the template matching degree coefficient ρ(G _q ), select the maximum value ρ _max (G _q ) as the optimal matching degree, and the q value in ρ _max (G _q ) as the longitudinal direction of ImageR _i+1 relative to ImageR _i offset by Sequentially calculate the longitudinal offsets of all images in the image sequence and the previous frame image, denoted as M ⁱ ; (2) Perform vertical translation transformation on the image ImageR _i+1 to obtain the image ImageR′ _i+1 , the translation matrix is R _y(i+1) , to complete the vertical offset correction caused by mechanical shaking, the translation matrix R _{y(i+1 )} is: ${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; m</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]$ Wherein, i is an integer less than N; Step 7: Absolute difference processing is performed on the sequence images after longitudinal offset correction to complete the detection of medical foreign objects. 2. The medicine bottle foreign matter detection method based on medical vision detection robot image correction according to claim 1, the specific implementation steps of the step 7 are: (1) Perform absolute difference processing on two adjacent frames of images ImageR′ _i and ImageR′ _i+1 to obtain the difference image SubImageR _i =|ImageR′ _i -ImageR′ _i+1 |,i∈[1,N-1] and i ∈ Z; (2) Carry out the binarization processing of given threshold T to primary difference image, obtain binarized image BinImageR _i ; $\mathrm{<span\; class="notranslate">\; Bin\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 255</span>}& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; T</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; Sub\; Image</span>}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; T</span>}\end{array}\right.<span\; class="notranslate">\; ;</span>$ (3) Perform superimposition operation on sequence frame binary image BinImageR _i : Image _max ＝MAX{Bin ImageR ₁ ,...Bin ImageR _N-1 }, N is the total number of sequence frames; (4) Find the connected domain of Image _max , count the number of connected domains Num _total , the sum of the area of the connected domain Area _total and the area is greater than the given The number of connected domains of the value ( is the minimum imaging area of foreign matter); (5) According to the calculated number of connected domains Num _total , judge whether the quality of the medicine is qualified, and the product quality θ is α and β are the genuine and defective product quality thresholds, respectively. 3. The medicine bottle foreign matter detection method based on medical vision inspection robot image correction according to claim 1, characterized in that, in the step 2, the binary image BinImage _i is searched for connected domains, using the 8-neighborhood connected rule Perform a connected domain search operation on an image.