CN107316330B - Underground hydraulic support frame group pose and verticality measuring method based on more image sequences - Google Patents
Underground hydraulic support frame group pose and verticality measuring method based on more image sequences Download PDFInfo
- Publication number
- CN107316330B CN107316330B CN201710442811.0A CN201710442811A CN107316330B CN 107316330 B CN107316330 B CN 107316330B CN 201710442811 A CN201710442811 A CN 201710442811A CN 107316330 B CN107316330 B CN 107316330B
- Authority
- CN
- China
- Prior art keywords
- positioning mark
- hydraulic support
- receptive field
- positioning
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
Abstract
The invention discloses underground hydraulic support frame group poses and verticality measuring method based on more image sequences, include the following steps：In the rectangular positioning mark of each hydraulic support same position placement, and pass through camera acquisition hydraulic support image, obtain target image；The target image collected is pre-processed, the positioning mark after extraction pretreatment in image；Edge lines fitting is carried out to each positioning mark, obtains four apex coordinates of each positioning mark；The normal vector of each positioning mark is calculated, the pose of each hydraulic support is determined by normal vector；The straightness of hydraulic support group is calculated by the relationship positioned between mark and scraper plate again.The present invention measures the pose of hydraulic support using vision technique, the pose of every frame hydraulic support can be detected automatically, necessary technical support is provided for the automatic Modulation of coal mine underground hydraulic support frame and the automation and intelligentification of fully-mechanized mining working, the safe operation in safeguard work face reduces the labor intensity of personnel.
Description
Technical field
The invention belongs to mining equipment monitoring running state field more particularly to a kind of coal mines based on more image sequences
Lower hydraulic support group pose and verticality measuring method.
Background technology
Important equipment of the hydraulic support as coal mine fully-mechanized mining working, pose are very important ginseng with straightness information
Number.Frequently with obliquity sensor, displacement sensor or inertial navigation sensor measurement；Wherein obliquity sensor is easily by acceleration
It influences, displacement sensor is due to being that contact type measurement sensor is easily damaged, and inertial navigation sensor cannot be single due to accumulated error
It solely uses, its systematic comparison complexity cost is higher when use is merged with other sensors.
Invention content
The defects of for existing technology of preparing and deficiency, the object of the present invention is to provide a kind of coals based on more image sequences
Ore deposit underground hydraulic support frame group pose and verticality measuring method solve sensor rapid wear in existing contact measurement method, are
It unites the problem of more complicated.
To achieve these goals, the present invention is realised by adopting the following technical scheme：
Underground hydraulic support frame group pose measuring method based on more image sequences, includes the following steps：
Step 1：Rectangular positioning marking plate on hydraulic support is installed, by camera acquisition hydraulic support image, is obtained
Target image；
Step 2：Noise reduction process is carried out to the target image that collects, extracts the positioning mark in image after noise reduction process
Know；
Step 3：Positioning is identified using receptive field cell model and carries out edge lines fitting, obtains the four of positioning mark
A apex coordinate；
Step 4：Four apex coordinates of positioning mark are obtained according to step 3, calculate the normal vector of positioning mark；Then
The pose of the pose, i.e. hydraulic support of positioning mark is calculated by formula (1)；
Wherein,For the unit normal vector of positioning mark, θkRepresent that k-th of plane that optical center reference axis is formed is marked with positioning
Knowledge plane included angle, k=1,2,3, θ1For the pitch angle of positioning mark, θ2For the deflection angle of positioning mark, θ3Horizontal stroke for positioning mark
Roll angle,
Further, the step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using based on the method for connected component
It takes；
Further, positioning is identified using receptive field cell model in step 3 and carries out edge lines fitting, it is specific to walk
Suddenly it is：
Step 3.1：To positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+1) * (r*
Mask 2+1), r are receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each
Receptive field direction is qualitatively judged, so as to be respectively divided out on the top edge of positioning mark, lower edge, left hand edge and right hand edge
The receptive field cell of distribution；
Step 3.3：Using receptive field model, the response of each receptive field in top edge direction is calculated, according to single impression
The contrast fringes position that pixel is formed in open country and the relationship of receptive field centre distance, determine each receptive field center to receptive field
Then the distance of interior contrast fringes fits the straight line where marking plate top edge using the LEAST SQUARES MODELS FITTING of belt restraining；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is carried out respectively
Fitting；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, positioning mark is obtained
Four apex coordinates known.
Further, the specific steps for calculating the pose of hydraulic support include；
Step 4.1：According to four apex coordinates that the positioning that step 3 obtains identifies, image coordinate is translated into, very
According to image coordinate and the relationship of optical center coordinate, the normal vector of positioning mark is calculated；
Wherein,For positioning mark normal vector, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) for positioning mark vertex i-th of vertex optical center coordinate, (xi,yi) for positioning mark vertex the
The image coordinate on i vertex, C are the effective focal length of video camera；
B1=x2y3+x4y2-x2y4+x3y4-x3y4
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is the length of long sides of rectangle positioning mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support of positioning mark is calculated by formula (1),
Wherein,For the unit normal vector of positioning mark, θkRepresent that k-th of plane that optical center reference axis is formed is marked with positioning
Knowledge plane included angle, k=1,2,3, θ1For the pitch angle of positioning mark, θ2For the deflection angle of positioning mark, θ3Horizontal stroke for positioning mark
Roll angle,
Underground hydraulic support frame group verticality measuring method based on more image sequences, this method are straight to N number of hydraulic support
Dimension measures, and scraper plate, crusing robot to be moved along scraper plate there are one connections on each hydraulic support, which is characterized in that
Include the following steps：
Step 1：Rectangular positioning marking plate is installed on each hydraulic support, the positioning marking plate is located at hydraulic pressure branch
In the same position of frame, when crusing robot walking to every section scraper plate end, often the junction time-division of section scraper plate and hydraulic support
Image is not acquired；
Step 2：Noise reduction process is carried out to the target image that collects, extracts the positioning mark in image after noise reduction process
Know；
Step 3：Edge lines fitting is carried out to each positioning mark using receptive field cell model, obtains each positioning
Four apex coordinates of mark；
Step 4：Four apex coordinates of each positioning mark are obtained according to step 3, calculate each positioning mark
Coordinate in optical center coordinate system；
Step 5：Calculate the straightness of hydraulic support group；
Step 5.1：Calculate coordinate of four vertex of each positioning mark in world coordinate system
βk=θk-γk(7)
Wherein, PjiRepresent P (Xji,Yji,Zji), P (Xji,Yji,Zji) for j-th positioning mark i-th of vertex in optical center
Coordinate in coordinate system,It representsI-th of vertex for j-th of positioning mark is in world coordinate system
Coordinate, i=1,2,3,4, n=2,3,4 ..., N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,
For i-th of apex coordinate of+1 positioning marking plate of kth, i is any one positioned in four vertex of marking plate；LkFor kth
Projector distance between+1 positioning marking plate in a scraper plate end and kth in optical center coordinate system, lkFor+1 scraper plate end of kth
Projector distance between+1 positioning marking plate of kth in optical center coordinate system, l0Length for every section scraper plate；
Step 5.2：Take the vertex of all positioning marksFitting a straight line is carried out to these vertex in XOZ planes, is obtained
Straight line, the straightness H of hydraulic support are：
Wherein, L be video camera first shooting point and the last one shooting point air line distance, djFor vertexWith plan
Distance between conjunction straight line, the maximum distance for taking fitting a straight line both sides is respectively dmax1And dmax2, b=dmax1+dmax2。
Further, the step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using based on the method for connected component
It takes；
Further, edge lines fitting, tool are carried out to each positioning mark using receptive field cell model in step 3
Body step is：
Step 3.1：To j-th of positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+
1) mask of * (r*2+1), r are receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each
Receptive field direction is qualitatively judged, so as to be respectively divided out on the top edge of positioning mark, lower edge, left hand edge and right hand edge
The receptive field cell of distribution；
Step 3.3：Using receptive field model, the response of each receptive field in top edge direction is calculated, according to single impression
The contrast fringes position that pixel is formed in open country and the relationship of receptive field centre distance, determine each receptive field center to receptive field
Then the distance of interior contrast fringes fits the straight line where marking plate top edge using the LEAST SQUARES MODELS FITTING of belt restraining；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is carried out respectively
Fitting；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, obtain j-th
Position four apex coordinates of mark；
Step 3.6：Step 3.1~step 3.5 is repeated, obtains four apex coordinates of all positioning marks.
Further, coordinate P of i-th of the vertex of j-th of positioning mark in optical center coordinate system in the step four
(Xji,Yji,Zji), i=1,2,3,4, it is calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of long sides or the length of side of square positioning mark identified for j-th of rectangle positioning, (xji,yji)
For the image coordinate on positioning mark vertex that step 3 obtains, C is the effective focal length of video camera.
Compared with prior art, the beneficial effects of the invention are as follows：
The present invention measures the pose of hydraulic support using vision technique, can detect the position of every frame hydraulic support automatically
Appearance, while the straightness of hydraulic support in working face can be detected, it is the automatic Modulation of coal mine underground hydraulic support frame and fully mechanized mining work
The automation and intelligentification for making face provides necessary technical support, and the safe operation in safeguard work face reduces the labour of personnel
Intensity.
Description of the drawings
Fig. 1 is the hydraulic support measuring method flow chart of the present invention.
Fig. 2 is that the image of the method for the present invention acquisition and its treated image, (a) original image, (b) binaryzation are located in advance
Image after reason, the positioning mark of (c) extraction.
The fitting a straight line of hydraulic support group that Fig. 3 the method for the present invention measurements obtain.
Explanation is further explained in detail to the particular content of the present invention with reference to embodiments.
Specific embodiment
The present invention is first explained coordinate system used herein：
Optical center coordinate system (camera coordinates system)：Using the optical center of camera as coordinate origin, X-axis, Y-axis are respectively parallel to CCD and put down
The optical axis coincidence of two vertical edges in face, Z axis and camera；Image coordinate system：Coordinate origin is in the center of ccd image plane, X
Axis, Y-axis are respectively parallel to two vertical edges of CCD planes；Pixel coordinate system：Coordinate origin in the upper left corner of ccd image plane,
U axis, V axis are respectively parallel to the X-axis of image coordinate, Y-axis；World coordinate system：With the junction of first hydraulic support and scraper plate
For coordinate origin, X-axis is parallel to first segment scraper plate, and Z axis is directed toward first hydraulic support, Y-axis is vertical perpendicular to first segment scraper plate
Downwards.
The underground hydraulic support frame group pose measuring method based on more image sequences of the present invention, includes the following steps：
Step 1：It is identified in the rectangular positioning of hydraulic support placement, in the inspection machine that edge is moved with working face parallel direction
Video camera, and the camera acquisition hydraulic support image by being fixedly mounted on crusing robot on people are installed, obtain target
Image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark after extraction pretreatment in image；
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using based on the method for connected component
It takes；
Step 3：Positioning is identified and carries out edge lines fitting, obtains four apex coordinates of positioning mark；
Step 3.1：To positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+1) * (r*
Mask 2+1), r are receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator；
Each receptive field direction is qualitatively judged by mask gradient direction, so as to which positioning mark be respectively divided out
The receptive field cell being distributed on top edge, lower edge, left hand edge and right hand edge；
Step 3.3：Using receptive field model, the response S of each receptive field in top edge direction is calculated,
S=S1-S2(2)
Wherein, σD=rD/ 4, σS=rS/4；rDAnd rSThe center of expression receptive field and perimeter region are (also comprising center respectively
The great circle in area) radius, h (u, v, η) represents comparison stimulation coverage rate when being η, and the pixel stimulation positioned at pixel (u, v) is strong
Degree, the point value for being 255 for pixel value in bianry image is 1, and the point value that pixel value is 0 is 0；
S1For receptive field center response, S2For the response of receptive field neighboring area, DlTo be located at receptive field center
Pixel, SeTo be located at the pixel of receptive field neighboring area, l=0,1,2 ..., f, e=1,2 ..., m, f be positioned at sense
By the pixel of Yezhong heart district, m is the pixel number positioned at receptive field neighboring area.
According to the contrast fringes position that pixel in single receptive field is formed and the relationship of receptive field centre distance, determine every
The distance d of fitting contrast fringes in a receptive field center to receptive fieldh, h=1,2 ..., n, wherein n represent receptive field cell
Number.
Assuming that linear equation (a, b) (u, v) where edgeT+ c=0, according to the range formula of point to straight line：
Wherein, (uh, vh) it is the coordinate points that receptive field center is fastened in pixel coordinate.
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Wherein, L (a, b, c, λ) represents Lagrangian, and λ is parameter；A, b, c are linear equation where the edge of fitting
Coefficient, U, V represent the n*1 matrixes that the pixel coordinate by receptive field cell centre forms, and D represents dhThe n*1 matrixes of composition；
In constraints a2+b2Under=1, a, b, c optimal solutions (a are found*,b*,c*)；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is carried out respectively
Fitting；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, positioning mark is obtained
Four apex coordinates known.
Step 4：Calculate the pose of the pose, i.e. hydraulic support of positioning mark；
Step 4.1：According to four apex coordinates that the positioning that step 3 obtains identifies, it is translated into image coordinate, root
According to image coordinate and the relationship of optical center coordinate, the normal vector of positioning mark is calculated；In calculating method vector process, regioselective mark
Know wherein three apex coordinates, the vector formed by three vertex can determine space plane where positioning mark, so as to ask
Obtain its unit normal vector
Wherein,For positioning mark normal vector, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) for positioning mark vertex i-th of vertex optical center coordinate, (xi,yi) for positioning mark vertex the
The image coordinate on i vertex, C are the effective focal length of video camera；
B1=x2y3-x4y3+x4y2-x2y4+x3y4-x3y2
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is the length of long sides of rectangle positioning mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support of each positioning mark is calculated by formula (1), wherein, liquid
The pose of pressure stent is represented by deflection angle, roll angle and pitch angle；
Wherein,For the unit normal vector of positioning mark, θkRepresent that k-th of plane that optical center reference axis is formed is marked with positioning
Knowledge plane included angle, k=1,2,3, θ1For the pitch angle of positioning mark, the i.e. angle around the rotation of the X-axis of optical center coordinate system, θ2It is fixed
The deflection angle of bit identification, the i.e. angle around the rotation of the Y-axis of optical center coordinate system, θ3For the roll angle of positioning mark, i.e., sat around optical center
The angle of the Z axis rotation of mark system,
The invention also includes the straightness computational methods of hydraulic support group, this method includes N number of hydraulic support, each liquid
Scraper plate there are one being connected on stent is pressed, the scraper plate is end to end, and crusing robot is moved along scraper plate, specifically includes：
Step 1：Rectangular positioning marking plate is installed on each hydraulic support, the positioning marking plate is located at hydraulic pressure branch
In the same position of frame, video camera is being installed along the crusing robot moved with working face parallel direction, is working as crusing robot
It walks to shooting figure picture is distinguished when often saving scraper plate end, the often junction of section scraper plate and hydraulic support, obtains target image；
Step 2：Noise reduction process is carried out to the target image that collects, extracts the positioning mark in image after noise reduction process
Know；
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using based on the method for connected component
It takes；
Step 3：Edge lines fitting is carried out to each positioning mark, obtains four apex coordinates of each positioning mark；
Step 3.1：To j-th of positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+
1) mask of * (r*2+1), j=1,2 ..., N, N be position marking plate number, r be receptive field cell radius, mask center with
Receptive field cell centre overlaps；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator；
Each receptive field direction is qualitatively judged by mask gradient direction, so as to which positioning mark be respectively divided out
The receptive field cell being distributed on top edge, lower edge, left hand edge and right hand edge；
Step 3.3：Using receptive field model, the response S of each receptive field in top edge direction is calculated,
S=S1-S2 (2)
Wherein, σD=rD/ 4, σS=rS/4；rDAnd rSThe center of expression receptive field and perimeter region are (also comprising center respectively
The great circle in area) radius, h (u, v, η) represents comparison stimulation coverage rate when being η, and the pixel stimulation positioned at pixel (u, v) is strong
Degree, the point value for being 255 for pixel value in bianry image is 1, and the point value that pixel value is 0 is 0；
S1For receptive field center response, S2For the response of receptive field neighboring area, DlTo be located at receptive field center
Pixel, SeTo be located at the pixel of receptive field neighboring area, l=0,1,2 ..., f, e=1,2 ..., m, f be positioned at
The pixel of receptive field center, m are the pixel number positioned at receptive field neighboring area.
According to the contrast fringes position that pixel in single receptive field is formed and the relationship of receptive field centre distance, determine every
The distance d of fitting contrast fringes in a receptive field center to receptive fieldh, h=1,2 ..., n, wherein n represent receptive field cell
Number.
Assuming that linear equation (a, b) (u, v) where edgeT+ c=0, according to the range formula of point to straight line：
Wherein, (uh, vh) it is the coordinate points that receptive field center is fastened in pixel coordinate.
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Wherein, L (a, b, c, λ) represents Lagrangian, and λ is parameter；A, b, c are linear equation where the edge of fitting
Coefficient, U, V represent the n*1 matrixes that the pixel coordinate by receptive field cell centre forms, and D represents dhThe n*1 matrixes of composition；
In constraints a2+b2Under=1, a, b, c optimal solutions (a are found*,b*,c*)；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is carried out respectively
Fitting；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, obtain j-th
Position four apex coordinates of mark；
Step 3.6：Step 3.1~step 3.5 is repeated, obtains four apex coordinates of all positioning marks；
Step 4：Four apex coordinates of each positioning mark are obtained according to step 3, calculate each positioning mark
Coordinate in optical center coordinate system；Coordinate P (X of i-th of the vertex of j-th of positioning mark in optical center coordinate systemji,Yji,Zji), i
=1,2,3,4, it is calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of long sides or the length of side of square positioning mark identified for j-th of rectangle positioning, (xji,yji)
For the image coordinate on positioning mark vertex that step 3 obtains, C is the effective focal length of video camera.
Step 5：Calculate the straightness of hydraulic support group；
Step 5.1：It is the coordinate under world coordinate system by optical center coordinate transformation obtained above
βk=θk-γk (7)
Wherein, PjiRepresent P (Xji,Yji,Zji), seat of i-th of vertex in optical center coordinate system for j-th of positioning mark
Mark,It representsCoordinate of i-th of vertex in world coordinate system for j-th of positioning mark, i=1,2,
3,4, n=2,3,4 ..., N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,
For i-th of apex coordinate of+1 marking plate of kth, i is any one positioned in four vertex of marking plate；LkIt is scraped for k-th
Projector distance between+1 positioning marking plate in plate end and kth in optical center coordinate system, lkFor+1 scraper plate end of kth and the
Projector distance between k+1 positioning marking plate in optical center coordinate system, l0Length for every section scraper plate；
Step 5.2：Take the vertex of all positioning marksIt is straight that least square method is carried out to these vertex in XOZ planes
Line is fitted, and obtains straight line,Fitting a straight line both sides are distributed in, ask vertexThe distance d between the straight linej, take fitting a straight line
Top maximum distance is dmax1, fitting a straight line lower section maximum distance is dmax2, b=dmax1+dmax2, then the straightness H of hydraulic support
For：
Wherein, L is first shooting point of video camera and the air line distance of the last one shooting point.
Specific embodiments of the present invention are given below, it should be noted that the invention is not limited in implement in detail below
Example, all equivalents done on the basis of technical scheme each fall within protection scope of the present invention.
Embodiment 1
A kind of underground hydraulic support frame group pose measuring method based on more image sequences of the present embodiment, including：At one
Square positioning mark is disposed on hydraulic support, is taken the photograph being fixedly mounted along the crusing robot moved with working face parallel direction
Camera, along polling path, to underground coal mine, a certain range of hydraulic support image is acquired, and obtains several target image sequences
Row.
A collected sub-picture is illustrated, Fig. 2 is the reality of collected piece image, at this time its hydraulic support
Border pose is：Deflection angle is 7 °, and roll angle is 2 °, and pitch angle is 10 °；
Self-adaption binaryzation pretreatment is carried out to the every frame target image collected, shown in handling result such as Fig. 2 (a)；
Extract the connected component in the pretreated hydraulic support image of binaryzation；Utilize the axial ratio and area information of connected component
Extraction is identified with the positioning in segmentation hydraulic support image, image such as Fig. 2 (b) after extraction；
Each receptive field mask gradient direction is calculated using gradient operator, by gradient direction, to each receptive field direction
It is qualitatively judged, so as to which the receptive field cell being distributed on the upper any edge of positioning mark be respectively divided out, as shown in Fig. 2 (c)；
Using receptive field model, the response of the positioning mark each receptive field of edge direction is calculated, according to pixel in single receptive field
The contrast fringes position of formation and the relationship of receptive field centre distance determine fitting comparison in each receptive field center to receptive field
The distance at edge；
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；According to the top edge of fitting,
Four lower edge, left hand edge and right hand edge linear equations, the pixel for calculating four vertex that mark is positioned in development machine image are sat
Mark.
Using imaging model and the pixel coordinate on four vertex, four vertex of positioning mark are calculated in optical center coordinate system
Coordinate；The normal vector of positioning mark is determined by optical center coordinate；The pose for finally obtaining the hydraulic support is that deflection angle is
6.7879 °, roll angle is 2.0412 °, and pitch angle is 10.1120 °, and the pose of the angle and hydraulic support reality relatively connects
Closely.
Embodiment 2
The present embodiment provides a kind of underground hydraulic support frame group verticality measuring method based on more image sequences, this method pair
5 hydraulic supports measure, and the straightness of this practical 5 hydraulic support groups is 4.022mm/m.
It is calculated below by way of the method for the present invention：Square positioning mark is disposed in each hydraulic support same position, on edge
With video camera is fixedly mounted on the crusing robot of working face parallel direction movement, when crusing robot moves to each hydraulic pressure branch
Frame acquires image with scraper plate junction, obtains several target image sequences.
A wherein collected sub-picture is illustrated, Fig. 2 is collected piece image；It is every to what is collected
Frame target image carries out self-adaption binaryzation pretreatment, shown in handling result such as Fig. 2 (a)；Extract the pretreated liquid of binaryzation
Press the connected component in stent images；Utilize the axial ratio of connected component and area information extraction and segmentation hydraulic support image
In positioning mark, image such as Fig. 2 (b) after extraction；
Each receptive field mask gradient direction is calculated using gradient operator, by gradient direction, to each receptive field direction
It is qualitatively judged, so as to which the receptive field cell being distributed on the upper any edge of positioning mark be respectively divided out, as shown in Fig. 2 (c)；
Using receptive field model, the response of the positioning mark each receptive field of edge direction is calculated, according to pixel in single receptive field
The contrast fringes position of formation and the relationship of receptive field centre distance determine fitting comparison in each receptive field center to receptive field
The distance at edge；Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；According to the top of fitting
Four edge, lower edge, left hand edge and right hand edge linear equations calculate the pixel on four vertex that mark is positioned in development machine image
Coordinate.
Using imaging model and the pixel coordinate on four vertex, four vertex of positioning mark are calculated in optical center coordinate system
Coordinate；It converts it under world coordinate system, takes coordinate of first vertex of each positioning mark under world coordinate system.
Using obtained coordinate, least square fitting method is carried out, fits O in straight line such as Fig. 3, Fig. 3 in XOZ planes1~O5Institute
Curve represent scraper plate track, i.e. crusing robot movement locus, dotted portion for hydraulic support group in XOZ planes
Coordinate q1~q5It is fitted obtained straight line；Each first vertex for positioning mark is calculated to the distance of the straight line, maximum distance
The as straightness of hydraulic support group, the straightness for finally obtaining first five hydraulic support are 4.326mm/m.As can be seen that this
The straightness that inventive method is measured and the straightness of hydraulic support group reality are closer to.
Claims (6)
1. the underground hydraulic support frame group pose measuring method based on more image sequences, which is characterized in that include the following steps：
Step 1：Rectangular positioning marking plate on hydraulic support is installed, by camera acquisition hydraulic support image, obtains target
Image；
Step 2：To the target image progress noise reduction process collected, the positioning after extraction noise reduction process in image identifies；
Step 3：Positioning is identified using receptive field cell model and carries out edge lines fitting, obtains four tops of positioning mark
Point coordinates, the specific steps are：
Step 3.1：To positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+1) * (r*2+1)
Mask, r is receptive field cell radius, and mask center overlaps with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each impression
Wild direction is qualitatively judged, and is distributed on the top edge of positioning mark, lower edge, left hand edge and right hand edge so as to be respectively divided out
Receptive field cell；
Step 3.3：Using receptive field model, the response of each receptive field in top edge direction is calculated, according in single receptive field
The contrast fringes position that pixel is formed and the relationship of receptive field centre distance, it is right in each receptive field center to receptive field to determine
Than the distance at edge, the straight line where marking plate top edge is then fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is fitted respectively；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, positioning mark is obtained
Four apex coordinates；
Step 4：According to four apex coordinates that the positioning that step 3 obtains identifies, the normal vector of positioning mark is calculated；Then lead to
Cross the pose that the pose, i.e. hydraulic support of positioning mark is calculated in formula (1)；
Wherein,For the unit normal vector of positioning mark, θkRepresent k-th of plane of optical center reference axis formation and positioning identity planar
Angle, k=1,2,3, θ1For the pitch angle of positioning mark, θ2For the deflection angle of positioning mark, θ3Roll for positioning mark
Angle,
2. the underground hydraulic support frame group pose measuring method based on more image sequences as described in claim 1, it is characterised in that：
The step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is extracted using based on the method for connected component.
3. the underground hydraulic support frame group pose measuring method based on more image sequences as described in claim 1, it is characterised in that：
The specific steps for calculating the pose of hydraulic support include；
Step 4.1：According to four apex coordinates that the positioning that step 3 obtains identifies, image coordinate is translated into, very according to figure
As the relationship of coordinate and optical center coordinate, the unit normal vector of positioning mark is calculated；
Wherein,For positioning mark unit normal vector, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) for positioning mark vertex i-th of vertex optical center coordinate, (xi,yi) it is position mark vertex i-th
The image coordinate on vertex, C are the effective focal length of video camera；
B1=x2y3-x4y3+x4y2-x2y4+x3y4-x3y2
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is the length of long sides of rectangle positioning mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support of positioning mark is calculated by formula (1).
4. the underground hydraulic support frame group verticality measuring method based on more image sequences, this method is to the straight line of N number of hydraulic support
Degree measures, and scraper plate, crusing robot to be moved along scraper plate there are one connections on each hydraulic support, it is characterised in that：Packet
Include following steps：
Step 1：Rectangular positioning marking plate is installed on each hydraulic support, the positioning marking plate is located at hydraulic support
In same position, adopted respectively when crusing robot walking is to every section scraper plate end, the often junction of section scraper plate and hydraulic support
Collect image；
Step 2：To the target image progress noise reduction process collected, the positioning after extraction noise reduction process in image identifies；
Step 3：Edge lines fitting is carried out to each positioning mark using receptive field cell model, obtains each positioning mark
Four apex coordinates, the specific steps are：
Step 3.1：To j-th of positioning mark according to the center spacing of setting and size distribution receptive field, using (r*2+1) * (r*
Mask 2+1), r are receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each impression
Wild direction is qualitatively judged, and is distributed on the top edge of positioning mark, lower edge, left hand edge and right hand edge so as to be respectively divided out
Receptive field cell；
Step 3.3：Using receptive field model, the response of each receptive field in top edge direction is calculated, according in single receptive field
The contrast fringes position that pixel is formed and the relationship of receptive field centre distance, it is right in each receptive field center to receptive field to determine
Than the distance at edge, the straight line where marking plate top edge is then fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Step 3.4：Step 3.3 is repeated, straight line where the lower edge, left hand edge, right hand edge of positioning mark is fitted respectively；
Step 3.5：According to four top edge of fitting, lower edge, left hand edge and right hand edge linear equations, j-th of positioning is obtained
Four apex coordinates of mark；
Step 3.6：Step 3.1~step 3.5 is repeated, obtains four apex coordinates of all positioning marks；
Step 4：Four apex coordinates of each positioning mark obtained according to step 3, calculate each positioning mark in light
Coordinate in heart coordinate system；
Step 5：Calculate the straightness of hydraulic support group；
Step 5.1：Calculate coordinate of four vertex of each positioning mark in world coordinate system
βk=θk-γk (7)
Wherein, PjiRepresent P (Xji,Yji,Zji), coordinate of i-th of vertex in optical center coordinate system for j-th of positioning mark,
It representsCoordinate of i-th of vertex in world coordinate system for j-th of positioning mark, i=1,2,3,4, n=
2,3,4…,N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,It is
I-th of apex coordinate of k+1 positioning marking plate, i are any one positioned in four vertex of marking plate；LkIt is scraped for k-th
Projector distance between+1 positioning marking plate in plate end and kth in optical center coordinate system, lkFor+1 scraper plate end of kth and the
Projector distance between k+1 positioning marking plate in optical center coordinate system, l0Length for every section scraper plate；
Step 5.2：Take the vertex of all positioning marksFitting a straight line is carried out to these vertex in XOZ planes, obtains one
Straight line, the straightness H of hydraulic support are：
Wherein, L be video camera first shooting point and the last one shooting point air line distance, djFor vertexIt is straight with being fitted
Distance between line, the maximum distance for taking fitting a straight line both sides are respectively dmax1And dmax2, b=dmax1+dmax2。
5. the underground hydraulic support frame group verticality measuring method based on more image sequences, feature exist as claimed in claim 4
In：The step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is extracted using based on the method for connected component.
6. the underground hydraulic support frame group verticality measuring method based on more image sequences, feature exist as claimed in claim 4
In：Coordinate P (X of i-th of the vertex of j-th of positioning mark in optical center coordinate system in the step fourji,Yji,Zji), i=
1,2,3,4, it is calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of long sides or the length of side of square positioning mark identified for j-th of rectangle positioning, (xji,yji) to walk
The rapid three obtained image coordinates on positioning mark vertex, C is the effective focal length of video camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710442811.0A CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710442811.0A CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107316330A CN107316330A (en) | 2017-11-03 |
| CN107316330B true CN107316330B (en) | 2018-06-29 |
Family
ID=60183254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710442811.0A Active CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107316330B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108033203B (en) * | 2017-11-28 | 2019-01-25 | 天地科技股份有限公司 | Fully mechanized coal face rear portion drag conveyor straightness determining device draws shifting system |
| CN109470149B (en) * | 2018-12-12 | 2020-09-29 | 北京理工大学 | Method and device for measuring position and posture of pipeline |
| CN111127406B (en) * | 2019-12-10 | 2022-09-27 | 创维集团智能装备有限公司 | Back plate machining position adjusting method, terminal, system and storage medium |
| CN111442759B (en) * | 2020-03-05 | 2023-10-31 | 天地科技股份有限公司 | Unified monitoring system for pose of fully-mechanized coal mining face equipment |
| CN111798476B (en) * | 2020-06-08 | 2023-10-20 | 国网江西省电力有限公司电力科学研究院 | Extraction method for conductive arm axis of high-voltage isolating switch |
| CN112324483B (en) * | 2020-11-03 | 2022-03-25 | 山东科技大学 | Automatic hydraulic support straightening system, follow-up straightening system of scraper conveyor and application of follow-up straightening system |
| CN112880561A (en) * | 2021-01-19 | 2021-06-01 | 华中科技大学 | Parachute rope loop pose detection method and system based on machine vision |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006026891A1 (en) * | 2006-06-09 | 2008-06-05 | Trw Automotive Gmbh | Servo steering gear for use in hydraulic steering support of servo-steering mechanism of motor vehicle, has butterfly valve assigned to outlet, where butterfly valve is closable for purpose of increasing servo-steering mechanism pressure |
| CN101915573A (en) * | 2010-08-04 | 2010-12-15 | 中国科学院自动化研究所 | Positioning measurement method based on key point detection of marker |
| CN103912294A (en) * | 2014-03-19 | 2014-07-09 | 北京天地玛珂电液控制系统有限公司 | Method using optical fibers for controlling straightness of working surface hydraulic support group |
-
2017
- 2017-06-13 CN CN201710442811.0A patent/CN107316330B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006026891A1 (en) * | 2006-06-09 | 2008-06-05 | Trw Automotive Gmbh | Servo steering gear for use in hydraulic steering support of servo-steering mechanism of motor vehicle, has butterfly valve assigned to outlet, where butterfly valve is closable for purpose of increasing servo-steering mechanism pressure |
| CN101915573A (en) * | 2010-08-04 | 2010-12-15 | 中国科学院自动化研究所 | Positioning measurement method based on key point detection of marker |
| CN103912294A (en) * | 2014-03-19 | 2014-07-09 | 北京天地玛珂电液控制系统有限公司 | Method using optical fibers for controlling straightness of working surface hydraulic support group |
Non-Patent Citations (2)
| Title |
|---|
| 利用鲍尔点设计液压支架直线导向机构;钱卫香;《机床与液压》;20130831;第41卷(第15期);第140-143页 * |
| 综采工作面直线度控制系统研究;牛剑峰;《工矿自动化》;20150531;第41卷(第5期);第5-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107316330A (en) | 2017-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107316330B (en) | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences | |
| CN107292926B (en) | Crusing robot movement locus verticality measuring method based on more image sequences | |
| CN105447853B (en) | Flight instruments, flight control system and method | |
| CN104930985B (en) | Binocular vision 3 D topography measurement method based on space-time restriction | |
| CN102654902B (en) | Contour vector feature-based embedded real-time image matching method | |
| CN107316288B (en) | A kind of boom-type roadheader cutterhead pose vision measuring method | |
| CN106123772B (en) | A kind of nuclear fuel rod pose automatic identification equipment and method | |
| CN105346706A (en) | Flight device, and flight control system and method | |
| CN105113403A (en) | Intelligent detecting equipment and method for bottom of bridge | |
| CN106546263A (en) | A kind of laser leveler shoot laser line detecting method based on machine vision | |
| CN210573937U (en) | Online three-dimensional imaging device of roll surface of roll squeezer | |
| CN104700395A (en) | Method and system for detecting appearance crack of structure | |
| CN112050732B (en) | Method and system for automatically detecting spatial pose of cantilever type heading machine | |
| CN106996748A (en) | A kind of wheel footpath measuring method based on binocular vision | |
| CN110570422B (en) | Capsule defect visual detection method based on matrix analysis | |
| CN103913149B (en) | A kind of binocular range-measurement system and distance-finding method thereof based on STM32 single-chip microcomputer | |
| CN108489383A (en) | A kind of measuring device and method of H-type cross dimensions | |
| CN206863817U (en) | Camera review automatic acquisition device and camera calibration systems | |
| CN115619738A (en) | Detection method for module side seam welding after welding | |
| CN112785654A (en) | Calibration method and device for track geometry detection system | |
| CN110243293A (en) | Section of jurisdiction faulting of slab ends device for fast detecting and method based on structure light and machine vision | |
| CN111383174A (en) | Pile bursting data acquisition method for photogrammetry | |
| CN115578315A (en) | Bridge strain close-range photogrammetry method based on unmanned aerial vehicle image | |
| CN105303564A (en) | Tower type crane load stereo pendulum angle vision detection method | |
| CN114964007A (en) | Visual measurement and surface defect detection method for weld size |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |