CN107316330A - Underground hydraulic support frame group pose and verticality measuring method based on many image sequences - Google Patents
Underground hydraulic support frame group pose and verticality measuring method based on many image sequences Download PDFInfo
- Publication number
- CN107316330A CN107316330A CN201710442811.0A CN201710442811A CN107316330A CN 107316330 A CN107316330 A CN 107316330A CN 201710442811 A CN201710442811 A CN 201710442811A CN 107316330 A CN107316330 A CN 107316330A
- Authority
- CN
- China
- Prior art keywords
- positioning mark
- hydraulic support
- receptive field
- positioning
- mark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
Abstract
The invention discloses the underground hydraulic support frame group pose and verticality measuring method based on many image sequences, comprise the following steps：In the square positioning mark of each hydraulic support same position placement, and by camera acquisition hydraulic support image, obtain target image；The target image collected is pre-processed, the positioning mark after pretreatment in image is extracted；Edge lines fitting is carried out to each positioning mark, four apex coordinates of each positioning mark are obtained；The normal vector of each positioning mark is calculated, the pose of each hydraulic support is determined by normal vector；The linearity of hydraulic support group is calculated by the relation positioned between mark and scraper plate again.The present invention measures the pose of hydraulic support using vision technique, can automatic detection go out the pose of every frame hydraulic support, the automation and intelligentification of automatic Modulation and fully-mechanized mining working for coal mine underground hydraulic support frame provides necessary technical support, the safe operation in safeguard work face, reduces the labor intensity of personnel.
Description
Technical field
The invention belongs to mining equipment monitoring running state field, more particularly to a kind of coal mine based on many image sequences
Lower hydraulic support group pose and verticality measuring method.
Background technology
Hydraulic support is as the visual plant of coal mine fully-mechanized mining working, and its pose and linearity information are very important ginsengs
Number.Frequently with the measurement of obliquity sensor, displacement transducer or inertial navigation sensor；Wherein obliquity sensor is to easily by acceleration
Influence, displacement transducer is due to being that contact type measurement sensor is easily damaged, and inertial navigation sensor can not due to accumulated error
It is used alone, its systematic comparison complexity cost when using is merged with other sensors higher.
The content of the invention
For the defect and deficiency of existing technology of preparing, it is an object of the invention to provide a kind of coal based on many image sequences
Ore deposit underground hydraulic support frame group pose and verticality measuring method, solve sensor rapid wear in existing contact measurement method, are
The problem of uniting more complicated.
To achieve these goals, the present invention, which is adopted the following technical scheme that, is achieved：
Underground hydraulic support frame group pose measuring method based on many image sequences, comprises the following steps：
Step one：Square positioning marking plate is installed on hydraulic support, by camera acquisition hydraulic support image, obtained
Target image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark in image after noise reduction process is extracted
Know；
Step 3：Edge lines fitting is carried out to positioning mark using receptive field cell model, the four of positioning mark are obtained
Individual apex coordinate；
Step 4：Four apex coordinates of positioning mark are obtained according to step 3, the normal vector of positioning mark is calculated；Then
The pose of the pose, i.e. hydraulic support that obtain positioning mark is calculated by formula (1)；
Wherein,For the unit normal vector of positioning mark, θkK-th of the plane and positioning for representing the formation of photocentre reference axis are marked
Know plane included angle, k=1,2,3, θ1For the angle of pitch of positioning mark, θ2For the deflection angle of positioning mark, θ3For the horizontal stroke of positioning mark
Roll angle,
Further, described step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using the method based on connected component
Take；
Further, edge lines fitting, specific step are carried out to positioning mark using receptive field cell model in step 3
Suddenly it is：
Step 3.1：To center spacing and size distribution receptive field of the positioning mark according to setting, using (r*2+1) * (r*2
+ 1) mask, r is receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each
Receptive field direction is qualitatively judged, so as to be respectively divided out on top edge, lower edge, left hand edge and the right hand edge of positioning mark
The receptive field cell of distribution；
Step 3.3：Using receptive field model, the response of each receptive field of top edge direction is calculated, according to single impression
The contrast fringes position of pixel formation and the relation of receptive field centre distance in wild, it is determined that each receptive field center is to receptive field
Then interior contrast fringes fit marking plate top edge to the distance at receptive field center using the LEAST SQUARES MODELS FITTING of belt restraining
Place straight line；
Step 3.4：Repeat step 3.3, is carried out to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively
Fitting；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, positioning mark is obtained
Four apex coordinates known.
Further, calculating the specific steps of the pose of hydraulic support includes；
Step 4.1：Four apex coordinates of positioning mark are obtained according to step 3, image coordinate, very evidence is translated into
Image coordinate and the relation of photocentre coordinate, calculate the normal vector of positioning mark；
Wherein,For the normal vector of positioning mark, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) it is the photocentre coordinate for positioning i-th of summit for identifying summit, (xi,yi) for the of positioning mark summit
The image coordinate on i summit, C is the effective focal length of video camera；
B1=x2y3-x4y3+x4y2-x2y4+x3y4-x3y2
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is that rectangle positions the length of long sides of mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support that obtain positioning mark is calculated by formula (1),
Wherein,For the unit normal vector of positioning mark, θkK-th of the plane and positioning for representing the formation of photocentre reference axis are marked
Know plane included angle, k=1,2,3, θ1For the angle of pitch of positioning mark, θ2For the deflection angle of positioning mark, θ3For the horizontal stroke of positioning mark
Roll angle,
Underground hydraulic support frame group verticality measuring method based on many image sequences, this method is straight to N number of hydraulic support
Dimension is measured, and a scraper plate is connected with each hydraulic support, and crusing robot is moved along scraper plate, it is characterised in that
Comprise the following steps：
Step one：Square positioning marking plate is installed on each hydraulic support, described positioning marking plate is located at hydraulic pressure branch
In the same position of frame, when crusing robot was walked to the junction time-division often saved scraper plate end, often save scraper plate and hydraulic support
Image is not gathered；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark in image after noise reduction process is extracted
Know；
Step 3：Edge lines fitting is carried out to each positioning mark using receptive field cell model, each positioning is obtained
Four apex coordinates of mark；
Step 4：Four apex coordinates of each positioning mark are obtained according to step 3, calculate each positioning mark
Coordinate in photocentre coordinate system；
Step 5：Calculate the linearity of hydraulic support group；
Step 5.1：Calculate coordinate of four summits of each positioning mark in world coordinate system
βk=θk-γk (7)
Wherein, PjiRepresent P (Xji,Yji,Zji), P (Xji,Yji,Zji) i-th of summit of mark is positioned in photocentre for j-th
Coordinate in coordinate system,RepresentSeat of i-th of the summit of mark in world coordinate system is positioned for j-th
Mark, i=1,2,3,4, n=2,3,4 ..., N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,For
I-th of apex coordinate of+1 positioning marking plate of kth, i is any one in four summits of positioning marking plate；LkFor k-th
Projector distance between+1 positioning marking plate in scraper plate end and kth in photocentre coordinate system, lkFor+1 scraper plate end of kth with
Projector distance between+1 positioning marking plate of kth in photocentre coordinate system, l0For the length of every section scraper plate；
Step 5.2：Take the summit of all positioning marksFitting a straight line is carried out to these summits in XOZ planes, obtained
Straight line, the linearity H of hydraulic support is：
Wherein, L is first shooting point of video camera and the air line distance of last shooting point, djFor summitWith plan
Distance between straight line is closed, the ultimate range respectively d of fitting a straight line both sides is takenmax1And dmax2, b=dmax1+dmax2。
Further, described step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using the method based on connected component
Take；
Further, edge lines fitting, tool are carried out to each positioning mark using receptive field cell model in step 3
Body step is：
Step 3.1：To center spacing and size distribution receptive field of j-th of positioning mark according to setting, using (r*2+
1) * (r*2+1) mask, r is receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each
Receptive field direction is qualitatively judged, so as to be respectively divided out on top edge, lower edge, left hand edge and the right hand edge of positioning mark
The receptive field cell of distribution；
Step 3.3：Using receptive field model, the response of each receptive field of top edge direction is calculated, according to single impression
The contrast fringes position of pixel formation and the relation of receptive field centre distance in wild, it is determined that each receptive field center is to receptive field
Then interior contrast fringes fit marking plate top edge to the distance at receptive field center using the LEAST SQUARES MODELS FITTING of belt restraining
Place straight line；
Step 3.4：Repeat step 3.3, is carried out to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively
Fitting；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, obtain j-th
Position four apex coordinates of mark；
Step 3.6：3.1~step 3.5 of repeat step, obtains four apex coordinates of all positioning marks.
Further, coordinate P of i-th of the summit of j-th of positioning mark in photocentre coordinate system in described step four
(Xji,Yji,Zji), i=1,2,3,4, calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of side of mark, (x are positioned for the length of long sides or square of j-th of rectangle positioning markji,yji)
The image coordinate on the positioning mark summit obtained for step 3, C is the effective focal length of video camera.
Compared with prior art, the beneficial effects of the invention are as follows：
The present invention measures the pose of hydraulic support using vision technique, can automatic detection go out the position of every frame hydraulic support
Appearance, is the automatic Modulation and fully mechanized mining work of coal mine underground hydraulic support frame while the linearity of hydraulic support in working face can be detected
The automation and intelligentification for making face provides necessary technical support, and the safe operation in safeguard work face reduces the work of personnel
Intensity.
Brief description of the drawings
Fig. 1 is hydraulic support measuring method flow chart of the invention.
Fig. 2 is the image after the image that the inventive method is gathered and its processing, (a) original image, the pretreatment of (b) binaryzation
Image afterwards, the positioning mark that (c) is extracted.
The fitting a straight line for the hydraulic support group that Fig. 3 the inventive method measurements are obtained.
Explanation is further explained in detail to the particular content of the present invention with reference to embodiments.
Embodiment
The present invention is explained to coordinate system used herein first：
Photocentre coordinate system (camera coordinates system)：Using the photocentre of camera as the origin of coordinates, X-axis, Y-axis are respectively parallel to CCD and put down
The optical axis coincidence of two vertical edges in face, Z axis and camera；Image coordinate system：The origin of coordinates is at the center of ccd image plane, X
Axle, Y-axis are respectively parallel to two vertical edges of CCD planes；Pixel coordinate system：The origin of coordinates in the upper left corner of ccd image plane,
U axles, V axles are respectively parallel to the X-axis of image coordinate, Y-axis；World coordinate system：With the junction of first hydraulic support and scraper plate
For the origin of coordinates, X-axis is parallel to first segment scraper plate, and Z axis points to first hydraulic support perpendicular to first segment scraper plate, and Y-axis is vertical
Downwards.
The underground hydraulic support frame group pose measuring method based on many image sequences of the present invention, comprises the following steps：
Step one：In the square positioning mark of hydraulic support placement, the inspection machine moved on edge with working face parallel direction
Video camera, and the camera acquisition hydraulic support image by being fixedly mounted on crusing robot are installed on people, target is obtained
Image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark after pretreatment in image is extracted；
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using the method based on connected component
Take；
Step 3：Edge lines fitting is carried out to positioning mark, four apex coordinates of positioning mark are obtained；
Step 3.1：To center spacing and size distribution receptive field of the positioning mark according to setting, using (r*2+1) * (r*2
+ 1) mask, r is receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator；
Each receptive field direction is qualitatively judged by mask gradient direction, so that positioning mark is respectively divided out
The receptive field cell being distributed on top edge, lower edge, left hand edge and right hand edge；
Step 3.3：Using receptive field model, the response S of each receptive field of top edge direction is calculated,
S=S1-S2 (2)
Wherein, σD=rD/ 4, σS=rS/4；rDAnd rSRepresent the center and perimeter region of receptive field (also comprising center respectively
The great circle in area) radius, when h (u, v, η) represents contrast to stimulate coverage rate be η, stimulate strong positioned at the pixel of pixel (u, v)
Degree, is 1 for the point value that pixel value in bianry image is 255, the point value that pixel value is 0 is 0；
S1For receptive field center response, S2For the response of receptive field neighboring area, DlFor positioned at receptive field center
Pixel, SeFor positioned at the pixel of receptive field neighboring area, l=0,1,2 ..., f, e=1,2 ..., m, f be positioned at sense
By the pixel of Yezhong heart district, m is the pixel number positioned at receptive field neighboring area.
According to the contrast fringes position of pixel formation and the relation of receptive field centre distance in single receptive field, it is determined that often
Individual receptive field center is to the interior fitting contrast fringes of receptive field apart from dh, h=1, wherein 2 ..., n, n represent receptive field cell
Number.
Assuming that linear equation (a, b) (u, v) where edgeT+ c=0, according to the range formula of point to straight line：
Wherein, (uh,vh) it is the coordinate points that receptive field center is fastened in pixel coordinate.
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Wherein, L (a, b, c, λ) represents Lagrangian, and λ is parameter；A, b, the c linear equation where the edge of fitting
Coefficient, U, V represent the n*1 matrixes being made up of the pixel coordinate of receptive field cell centre, and D represents dhThe n*1 matrixes of composition；
In constraints a2+b2Under=1, a, b, c optimal solutions (a are found*,b*,c*)；
Step 3.4：Repeat step 3.3, is carried out to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively
Fitting；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, positioning mark is obtained
Four apex coordinates known.
Step 4：Calculate the pose of the pose, i.e. hydraulic support of positioning mark；
Step 4.1：Four apex coordinates of positioning mark are obtained according to step 3, image coordinate is translated into, according to
Image coordinate and the relation of photocentre coordinate, calculate the normal vector of positioning mark；In calculating method vector process, regioselective mark
Wherein three apex coordinates, the vector formed by three summits is that can determine that space plane where positioning mark, so as to try to achieve
Its normal vector
Wherein,For the normal vector of positioning mark, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) it is the photocentre coordinate for positioning i-th of summit for identifying summit, (xi,yi) for the of positioning mark summit
The image coordinate on i summit, C is the effective focal length of video camera；
B1=x2y3-x4y3+x4y2-x2y4+x3y4-x3y2
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is that rectangle positions the length of long sides of mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support that obtain each positioning mark is calculated by formula (1), wherein, liquid
The pose of pressure support is represented by deflection angle, roll angle and pitch angle；
Wherein,For the unit normal vector of positioning mark, θkK-th of the plane and positioning for representing the formation of photocentre reference axis are marked
Know plane included angle, k=1,2,3, θ1For the angle of pitch of positioning mark, i.e., the angle rotated around the X-axis of photocentre coordinate system, θ2It is fixed
The deflection angle of bit identification, i.e., the angle rotated around the Y-axis of photocentre coordinate system, θ3For the roll angle of positioning mark, i.e., sat around photocentre
The angle of the Z axis rotation of mark system,
Present invention additionally comprises the linearity computational methods of hydraulic support group, this method includes N number of hydraulic support, each liquid
A scraper plate is connected with pressure support, described scraper plate is end to end, and crusing robot is moved along scraper plate, is specifically included：
Step one：Square positioning marking plate is installed on each hydraulic support, described positioning marking plate is located at hydraulic pressure branch
In the same position of frame, video camera is being installed along along the crusing robot moved with working face parallel direction, is working as crusing robot
Walk to shooting image is distinguished when often saving scraper plate end, the often junction of section scraper plate and hydraulic support, obtain target image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark in image after noise reduction process is extracted
Know；
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is carried using the method based on connected component
Take；
Step 3：Edge lines fitting is carried out to each positioning mark, four apex coordinates of each positioning mark are obtained；
Step 3.1：To center spacing and size distribution receptive field of j-th of positioning mark according to setting, using (r*2+
1) * (r*2+1) mask, j=1,2 ..., N, N for positioning marking plate number, r be receptive field cell radius, mask center with
Receptive field cell centre is overlapped；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator；
Each receptive field direction is qualitatively judged by mask gradient direction, so that positioning mark is respectively divided out
The receptive field cell being distributed on top edge, lower edge, left hand edge and right hand edge；
Step 3.3：Using receptive field model, the response S of each receptive field of top edge direction is calculated,
S=S1-S2 (2)
Wherein, σD=rD/ 4, σS=rS/4；rDAnd rSRepresent the center and perimeter region of receptive field (also comprising center respectively
The great circle in area) radius, when h (u, v, η) represents contrast to stimulate coverage rate be η, stimulate strong positioned at the pixel of pixel (u, v)
Degree, is 1 for the point value that pixel value in bianry image is 255, the point value that pixel value is 0 is 0；
S1For receptive field center response, S2For the response of receptive field neighboring area, DlFor positioned at receptive field center
Pixel, SeFor positioned at the pixel of receptive field neighboring area, l=0,1,2 ..., f, e=1,2 ..., m, f be positioned at sense
By the pixel of Yezhong heart district, m is the pixel number positioned at receptive field neighboring area.
According to the contrast fringes position of pixel formation and the relation of receptive field centre distance in single receptive field, it is determined that often
Individual receptive field center is to the interior fitting contrast fringes of receptive field apart from dh, h=1, wherein 2 ..., n, n represent receptive field cell
Number.
Assuming that linear equation (a, b) (u, v) where edgeT+ c=0, according to the range formula of point to straight line：
Wherein, (uh, vh) it is the coordinate points that receptive field center is fastened in pixel coordinate.
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；
Wherein, L (a, b, c, λ) represents Lagrangian, and λ is parameter；A, b, the c linear equation where the edge of fitting
Coefficient, U, V represent the n*1 matrixes being made up of the pixel coordinate of receptive field cell centre, and D represents dhThe n*1 matrixes of composition；
In constraints a2+b2Under=1, a, b, c optimal solutions (a are found*,b*,c*)；
Step 3.4：Repeat step 3.3, is carried out to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively
Fitting；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, obtain j-th
Position four apex coordinates of mark；
Step 3.6：3.1~step 3.5 of repeat step, obtains four apex coordinates of all positioning marks；
Step 4：Four apex coordinates of each positioning mark are obtained according to step 3, calculate each positioning mark
Coordinate in photocentre coordinate system；Coordinate P (X of i-th of the summit of j-th of positioning mark in photocentre coordinate systemji,Yji,Zji), i
=1,2,3,4, calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of side of mark, (x are positioned for the length of long sides or square of j-th of rectangle positioning markji,yji)
The image coordinate on the positioning mark summit obtained for step 3, C is the effective focal length of video camera.
Step 5：Calculate the linearity of hydraulic support group；
Step 5.1：It is the coordinate under world coordinate system by photocentre coordinate transformation obtained above
βk=θk-γk (7)
Wherein, PjiRepresent P (Xji,Yji,Zji), it is seat of i-th of the summit of j-th of positioning mark in photocentre coordinate system
Mark,RepresentFor coordinate of i-th of summit in world coordinate system of j-th of positioning mark, i=1,2,3,
4, n=2,3,4 ..., N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,For
I-th of apex coordinate of+1 marking plate of kth, i is any one in four summits of positioning marking plate；LkFor k-th of scraper plate
Projector distance between+1 positioning marking plate in end and kth in photocentre coordinate system, lkFor+1 scraper plate end of kth and kth+
Projector distance between 1 positioning marking plate in photocentre coordinate system, l0For the length of every section scraper plate；
Step 5.2：Take the summit of all positioning marksLeast square method is carried out to these summits in XOZ planes straight
Line is fitted, and obtains straight line,Fitting a straight line both sides are distributed in, summit is askedApart from d between the straight linej, take fitting a straight line
Top ultimate range is dmax1, fitting a straight line lower section ultimate range is dmax2, b=dmax1+dmax2, then the linearity H of hydraulic support
For：
Wherein, L is first shooting point of video camera and the air line distance of last shooting point.
Specific embodiment of the invention given below is, it is necessary to which explanation is that the invention is not limited in implement in detail below
Example, all equivalents done on the basis of technical scheme each fall within protection scope of the present invention.
Embodiment 1
A kind of underground hydraulic support frame group pose measuring method based on many image sequences of the present embodiment, including：At one
Square positioning mark is disposed on hydraulic support, is taken the photograph being fixedly mounted along along the crusing robot moved with working face parallel direction
Camera, is acquired to a range of hydraulic support image of underground coal mine along polling path, obtains several target image sequences
Row.
The sub-picture collected is illustrated, Fig. 2 is the reality of the piece image collected, now its hydraulic support
Border pose is：Deflection angle is 7 °, and roll angle is 2 °, and luffing angle is 10 °；
Self-adaption binaryzation pretreatment is carried out to the every frame target image collected, shown in result such as Fig. 2 (a)；
Extract the connected component in the pretreated hydraulic support image of binaryzation；Utilize the axial ratio and area information of connected component
Extract and identified with the positioning in segmentation hydraulic support image, image such as Fig. 2 (b) after extraction；
Each receptive field mask gradient direction is calculated using gradient operator, by gradient direction, to each receptive field direction
Qualitatively judged, so that the receptive field cell being distributed on the upper any edge of positioning mark is respectively divided out, shown in such as Fig. 2 (c)；
Using receptive field model, the response of each receptive field of positioning mark edge direction is calculated, according to pixel in single receptive field
The contrast fringes position of formation and the relation of receptive field centre distance, it is determined that each receptive field center is contrasted to fitting in receptive field
The distance at edge；
Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；According to the top edge of fitting,
Lower edge, four linear equations of left hand edge and right hand edge, the pixel for calculating four summits that mark is positioned in development machine image are sat
Mark.
Using imaging model and the pixel coordinate on four summits, four summits of positioning mark are calculated in photocentre coordinate system
Coordinate；The normal vector of positioning mark is determined by photocentre coordinate；The pose for finally obtaining the hydraulic support is that deflection angle is
6.7879 °, roll angle is 2.0412 °, and luffing angle is 10.1120 °, and the angle pose actual with hydraulic support, which is compared, to be connect
Closely.
Embodiment 2
The present embodiment provides a kind of underground hydraulic support frame group verticality measuring method based on many image sequences, this method pair
5 hydraulic supports are measured, and the linearity of this actual 5 hydraulic support groups is 4.022mm/m.
Calculated below by way of the inventive method：In each hydraulic support same position placement square positioning mark, on edge
Video camera is fixedly mounted on the crusing robot moved with working face parallel direction, when crusing robot moves to each hydraulic pressure branch
Frame gathers image with scraper plate junction, obtains several target image sequences.
The sub-picture wherein collected is illustrated, Fig. 2 is the piece image collected；It is every to what is collected
Frame target image carries out self-adaption binaryzation pretreatment, shown in result such as Fig. 2 (a)；Extract the pretreated liquid of binaryzation
Press the connected component in stent images；Extracted and segmentation hydraulic support image using the axial ratio and area information of connected component
In positioning mark, image such as Fig. 2 (b) after extraction；
Each receptive field mask gradient direction is calculated using gradient operator, by gradient direction, to each receptive field direction
Qualitatively judged, so that the receptive field cell being distributed on the upper any edge of positioning mark is respectively divided out, shown in such as Fig. 2 (c)；
Using receptive field model, the response of each receptive field of positioning mark edge direction is calculated, according to pixel in single receptive field
The contrast fringes position of formation and the relation of receptive field centre distance, it is determined that each receptive field center is contrasted to fitting in receptive field
The distance at edge；Straight line where the edge is fitted using the LEAST SQUARES MODELS FITTING of belt restraining；According to the top of fitting
Edge, lower edge, four linear equations of left hand edge and right hand edge, calculate the pixel on four summits that mark is positioned in development machine image
Coordinate.
Using imaging model and the pixel coordinate on four summits, four summits of positioning mark are calculated in photocentre coordinate system
Coordinate；Convert it under world coordinate system, take coordinate of first summit of each positioning mark under world coordinate system.
Using obtained coordinate, least square fitting method is carried out, O in straight line such as Fig. 3, Fig. 3 in XOZ planes is fitted1~O5Place
Curve represent scraper plate track, i.e. crusing robot movement locus, dotted portion be hydraulic support group sat in XOZ planes
Mark q1~q5It is fitted obtained straight line；First summit of each positioning mark is calculated to the distance of the straight line, ultimate range is
For the linearity of hydraulic support group, the linearity for finally obtaining first five hydraulic support is 4.326mm/m.As can be seen that this hair
The linearity that bright method the is measured linearity actual with hydraulic support group is closer to.
Claims (8)
1. the underground hydraulic support frame group pose measuring method based on many image sequences, it is characterised in that comprise the following steps：
Step one：Square positioning marking plate is installed on hydraulic support, by camera acquisition hydraulic support image, target is obtained
Image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark after noise reduction process in image is extracted；
Step 3：Edge lines fitting is carried out to positioning mark using receptive field cell model, four tops of positioning mark are obtained
Point coordinates；
Step 4：Four apex coordinates of positioning mark are obtained according to step 3, the normal vector of positioning mark is calculated；Then pass through
Formula (1) calculates the pose of the pose, i.e. hydraulic support that obtain positioning mark；
Wherein,For the unit normal vector of positioning mark, θkRepresent that k-th of the plane and positioning mark of the formation of photocentre reference axis are flat
Face angle, k=1,2,3, θ1For the angle of pitch of positioning mark, θ2For the deflection angle of positioning mark, θ3For the roll of positioning mark
Angle,
2. the underground hydraulic support frame group pose measuring method as claimed in claim 1 based on many image sequences, it is characterised in that：
Described step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the target image collected；
Step 2.2：The positioning mark in pretreated target image is extracted using the method based on connected component.
3. the underground hydraulic support frame group pose measuring method as claimed in claim 1 based on many image sequences, it is characterised in that：
Edge lines fitting is carried out to positioning mark using receptive field cell model in step 3, concretely comprised the following steps：
Step 3.1：To center spacing and size distribution receptive field of the positioning mark according to setting, using (r*2+1) * (r*2+1)
Mask, r is receptive field cell radius, and mask center overlaps with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each impression
Wild direction is qualitatively judged, and is distributed so as to be respectively divided out on top edge, lower edge, left hand edge and the right hand edge of positioning mark
Receptive field cell；
Step 3.3：Using receptive field model, the response of each receptive field of top edge direction is calculated, according in single receptive field
The relation of contrast fringes position and the receptive field centre distance of pixel formation, it is determined that each receptive field center is to right in receptive field
Than the distance at edge to receptive field center, then marking plate top edge place is fitted using the LEAST SQUARES MODELS FITTING of belt restraining
Straight line；
Step 3.4：Repeat step 3.3, is fitted to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, positioning mark is obtained
Four apex coordinates.
4. the underground hydraulic support frame group pose measuring method as claimed in claim 1 based on many image sequences, it is characterised in that：
Calculating the specific steps of the pose of hydraulic support includes；
Step 4.1：Four apex coordinates of positioning mark are obtained according to step 3, image coordinate are translated into, very according to image
The relation of coordinate and photocentre coordinate, calculates the normal vector of positioning mark；
Wherein,For the normal vector of positioning mark, a ∈ i, b ∈ i, c ∈ i, a ≠ b ≠ c, i=1,2,3,4；
(Xi,Yi,Zi) it is the photocentre coordinate for positioning i-th of summit for identifying summit, (xi,yi) for i-th of positioning mark summit
The image coordinate on summit, C is the effective focal length of video camera；
B1=x2y3-x4y3+x4y2-x2y4+x3y4-x3y2
B2=x3y4-x4y3-x1y4+x4y1+x1y3-x3y1
B3=x1y2-x4y2-x2y1-x1y4+x4y1+x2y4
B4=x3y2-x2y3+x1y3+x2y1-x1y2-x3y1
Wherein, l is that rectangle positions the length of long sides of mark or the length of side of square positioning mark；
Step 4.2：The pose of the pose, i.e. hydraulic support that obtain positioning mark is calculated by formula (1),
Wherein,For the unit normal vector of positioning mark, θkRepresent k-th of the plane and positioning identity planar of photocentre reference axis formation
Angle, k=1,2,3, θ1For the angle of pitch of positioning mark, θ2For the deflection angle of positioning mark, θ3The roll angle identified for positioning,
5. the underground hydraulic support frame group verticality measuring method based on many image sequences, straight line of this method to N number of hydraulic support
Degree is measured, and a scraper plate is connected with each hydraulic support, and crusing robot is moved along scraper plate, it is characterised in that：Bag
Include following steps：
Step one：Square positioning marking plate is installed on each hydraulic support, described positioning marking plate is located at hydraulic support
In same position, adopted respectively when crusing robot is walked to often section scraper plate end, the often junction of section scraper plate and hydraulic support
Collect image；
Step 2：Noise reduction process is carried out to the target image collected, the positioning mark after noise reduction process in image is extracted；
Step 3：Edge lines fitting is carried out to each positioning mark using receptive field cell model, each positioning mark is obtained
Four apex coordinates；
Step 4：Four apex coordinates of each positioning mark are obtained according to step 3, calculate each positioning mark in photocentre
Coordinate in coordinate system；
Step 5：Calculate the linearity of hydraulic support group；
Step 5.1：Calculate coordinate of four summits of each positioning mark in world coordinate system
βk=θk-γk (7)
Wherein, PjiRepresent P (Xji,Yji,Zji), it is coordinate of i-th of the summit of j-th of positioning mark in photocentre coordinate system,
RepresentCoordinate of i-th of the summit of mark in world coordinate system, i=1,2,3,4, n=are positioned for j-th
2,3,4…,N；
βkFor the angle between adjacent scraper plate, k=1,2 ..., number that N-1, N are marking plate,For kth+
I-th of apex coordinate of 1 positioning marking plate, i is any one in four summits of positioning marking plate；LkFor k-th of scraper plate
Projector distance between+1 positioning marking plate in end and kth in photocentre coordinate system, lkFor+1 scraper plate end of kth and kth+
Projector distance between 1 positioning marking plate in photocentre coordinate system, l0For the length of every section scraper plate；
Step 5.2：Take the summit of all positioning marksFitting a straight line is carried out to these summits in XOZ planes, one is obtained
Straight line, the linearity H of hydraulic support is：
Wherein, L is first shooting point of video camera and the air line distance of last shooting point, djFor summitIt is straight with fitting
Distance between line, takes the ultimate range respectively d of fitting a straight line both sidesmax1And dmax2, b=dmax1+dmax2。
6. the measuring method of the underground hydraulic support frame group linearity as claimed in claim 5 based on many image sequences, its feature
It is：Described step two includes：
Step 2.1：Self-adaption binaryzation pretreatment is carried out to the every frame target image collected；
Step 2.2：The positioning mark in pretreated target image is extracted using the method based on connected component.
7. the measuring method of the underground hydraulic support frame group linearity as claimed in claim 5 based on many image sequences, its feature
It is：Edge lines fitting is carried out to each positioning mark using receptive field cell model in step 3, concretely comprised the following steps：
Step 3.1：To center spacing and size distribution receptive field of j-th of positioning mark according to setting, using (r*2+1) * (r*
Mask 2+1), r is receptive field cell radius, and mask center is overlapped with receptive field cell centre；
Step 3.2：Each receptive field mask gradient direction is calculated using gradient operator, by mask gradient direction to each impression
Wild direction is qualitatively judged, and is distributed so as to be respectively divided out on top edge, lower edge, left hand edge and the right hand edge of positioning mark
Receptive field cell；
Step 3.3：Using receptive field model, the response of each receptive field of top edge direction is calculated, according in single receptive field
The relation of contrast fringes position and the receptive field centre distance of pixel formation, it is determined that each receptive field center is to right in receptive field
Than the distance at edge to receptive field center, then marking plate top edge place is fitted using the LEAST SQUARES MODELS FITTING of belt restraining
Straight line；
Step 3.4：Repeat step 3.3, is fitted to straight line where the lower edge, left hand edge, right hand edge of positioning mark respectively；
Step 3.5：According to the top edge of fitting, lower edge, four linear equations of left hand edge and right hand edge, j-th of positioning is obtained
Four apex coordinates of mark；
Step 3.6：3.1~step 3.5 of repeat step, obtains four apex coordinates of all positioning marks.
8. the measuring method of the underground hydraulic support frame group linearity as claimed in claim 5 based on many image sequences, its feature
It is：Coordinate P (X of i-th of the summit of j-th of positioning mark in photocentre coordinate system in described step fourji,Yji,Zji), i
=1,2,3,4, calculated by formula (2)：
Bj1=xj2yj3-xj4yj3+xj4yj2-xj2yj4+xj3yj4-xj3yj2
Bj2=xj3yj4-xj4yj3-xj1yj4-xj4yj1+xj1yj3-xj3yj1
Bj3=xj1yj2-xj4yj2-xj2yj1-xj1yj4+xj4yj1-xj2yj4
Bj4=xj3yj2-xj2yj3+xj1yj3-xj2yj1-xj1yj2-xj3yj1
Wherein, ljThe length of side of mark, (x are positioned for the length of long sides or square of j-th of rectangle positioning markji,yji) it is step
The image coordinate on rapid three obtained positioning mark summits, C is the effective focal length of video camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710442811.0A CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710442811.0A CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107316330A true CN107316330A (en) | 2017-11-03 |
| CN107316330B CN107316330B (en) | 2018-06-29 |
Family
ID=60183254
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710442811.0A Active CN107316330B (en) | 2017-06-13 | 2017-06-13 | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107316330B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108033203A (en) * | 2017-11-28 | 2018-05-15 | 天地科技股份有限公司 | Fully mechanized coal face rear portion drag conveyor straightness determining device, draw shifting system |
| CN109470149A (en) * | 2018-12-12 | 2019-03-15 | 北京理工大学 | A kind of measurement method and device of pipeline pose |
| CN111127406A (en) * | 2019-12-10 | 2020-05-08 | 创维集团智能装备有限公司 | Back plate machining position adjusting method, terminal, system and storage medium |
| CN111798476A (en) * | 2020-06-08 | 2020-10-20 | 国网江西省电力有限公司电力科学研究院 | Method for extracting axis of conductive arm of high-voltage isolating switch |
| CN112324483A (en) * | 2020-11-03 | 2021-02-05 | 山东科技大学 | Automatic hydraulic support straightening system, follow-up straightening system of scraper conveyor and application of follow-up straightening system |
| CN112880561A (en) * | 2021-01-19 | 2021-06-01 | 华中科技大学 | Parachute rope loop pose detection method and system based on machine vision |
| WO2021174744A1 (en) * | 2020-03-05 | 2021-09-10 | 天地科技股份有限公司 | Unified position and posture monitoring system for fully mechanized coal mining face devices |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006026891A1 (en) * | 2006-06-09 | 2008-06-05 | Trw Automotive Gmbh | Servo steering gear for use in hydraulic steering support of servo-steering mechanism of motor vehicle, has butterfly valve assigned to outlet, where butterfly valve is closable for purpose of increasing servo-steering mechanism pressure |
| CN101915573A (en) * | 2010-08-04 | 2010-12-15 | 中国科学院自动化研究所 | Positioning measurement method based on key point detection of marker |
| CN103912294A (en) * | 2014-03-19 | 2014-07-09 | 北京天地玛珂电液控制系统有限公司 | Method using optical fibers for controlling straightness of working surface hydraulic support group |
-
2017
- 2017-06-13 CN CN201710442811.0A patent/CN107316330B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102006026891A1 (en) * | 2006-06-09 | 2008-06-05 | Trw Automotive Gmbh | Servo steering gear for use in hydraulic steering support of servo-steering mechanism of motor vehicle, has butterfly valve assigned to outlet, where butterfly valve is closable for purpose of increasing servo-steering mechanism pressure |
| CN101915573A (en) * | 2010-08-04 | 2010-12-15 | 中国科学院自动化研究所 | Positioning measurement method based on key point detection of marker |
| CN103912294A (en) * | 2014-03-19 | 2014-07-09 | 北京天地玛珂电液控制系统有限公司 | Method using optical fibers for controlling straightness of working surface hydraulic support group |
Non-Patent Citations (2)
| Title |
|---|
| 牛剑峰: "综采工作面直线度控制系统研究", 《工矿自动化》 * |
| 钱卫香: "利用鲍尔点设计液压支架直线导向机构", 《机床与液压》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108033203A (en) * | 2017-11-28 | 2018-05-15 | 天地科技股份有限公司 | Fully mechanized coal face rear portion drag conveyor straightness determining device, draw shifting system |
| CN108033203B (en) * | 2017-11-28 | 2019-01-25 | 天地科技股份有限公司 | Fully mechanized coal face rear portion drag conveyor straightness determining device draws shifting system |
| CN109470149A (en) * | 2018-12-12 | 2019-03-15 | 北京理工大学 | A kind of measurement method and device of pipeline pose |
| CN111127406A (en) * | 2019-12-10 | 2020-05-08 | 创维集团智能装备有限公司 | Back plate machining position adjusting method, terminal, system and storage medium |
| WO2021174744A1 (en) * | 2020-03-05 | 2021-09-10 | 天地科技股份有限公司 | Unified position and posture monitoring system for fully mechanized coal mining face devices |
| CN111798476A (en) * | 2020-06-08 | 2020-10-20 | 国网江西省电力有限公司电力科学研究院 | Method for extracting axis of conductive arm of high-voltage isolating switch |
| CN111798476B (en) * | 2020-06-08 | 2023-10-20 | 国网江西省电力有限公司电力科学研究院 | Extraction method for conductive arm axis of high-voltage isolating switch |
| CN112324483A (en) * | 2020-11-03 | 2021-02-05 | 山东科技大学 | Automatic hydraulic support straightening system, follow-up straightening system of scraper conveyor and application of follow-up straightening system |
| CN112324483B (en) * | 2020-11-03 | 2022-03-25 | 山东科技大学 | Automatic hydraulic support straightening system, follow-up straightening system of scraper conveyor and application of follow-up straightening system |
| CN112880561A (en) * | 2021-01-19 | 2021-06-01 | 华中科技大学 | Parachute rope loop pose detection method and system based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107316330B (en) | 2018-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107316330B (en) | Underground hydraulic support frame group pose and verticality measuring method based on more image sequences | |
| CN107292926B (en) | Crusing robot movement locus verticality measuring method based on more image sequences | |
| CN104637073B (en) | It is a kind of based on the banding underground structure detection method for shining upon shadow compensation | |
| CN105447853B (en) | Flight instruments, flight control system and method | |
| CN106969706A (en) | Workpiece sensing and three-dimension measuring system and detection method based on binocular stereo vision | |
| CN107316288B (en) | A kind of boom-type roadheader cutterhead pose vision measuring method | |
| CN103051872B (en) | Based on the belt deflection detection method of Edge extraction | |
| CN106643545A (en) | Calibration method for steel rail profile measured by adopting laser displacement technology | |
| CN105113403A (en) | Intelligent detecting equipment and method for bottom of bridge | |
| CN109978940B (en) | Visual measurement method for SAB safety airbag size | |
| CN210573937U (en) | Online three-dimensional imaging device of roll surface of roll squeezer | |
| CN105891220B (en) | A kind of pavement marker line defect detection device and its detection method | |
| CN104700395A (en) | Method and system for detecting appearance crack of structure | |
| CN104050446A (en) | Meter pointer image identification method based on pointer width character | |
| CN112050732B (en) | Method and system for automatically detecting spatial pose of cantilever type heading machine | |
| CN106996748A (en) | A kind of wheel footpath measuring method based on binocular vision | |
| CN113063795B (en) | Expressway tunnel defect position determining method and system | |
| CN102496232A (en) | Transmission facility monitoring method and system | |
| CN111452840B (en) | Railway steel rail crawling displacement detection method based on monocular vision measurement technology | |
| CN110243293A (en) | Section of jurisdiction faulting of slab ends device for fast detecting and method based on structure light and machine vision | |
| CN112785654A (en) | Calibration method and device for track geometry detection system | |
| CN105303564A (en) | Tower type crane load stereo pendulum angle vision detection method | |
| CN102175172A (en) | Image-identification-based system for detecting contour dimension of motor vehicle | |
| CN115619738A (en) | Detection method for module side seam welding after welding | |
| CN115578315A (en) | Bridge strain close-range photogrammetry method based on unmanned aerial vehicle image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |