CN105466523B - The measurement method and device of heap grain height based on single camera image - Google Patents
The measurement method and device of heap grain height based on single camera image Download PDFInfo
- Publication number
- CN105466523B CN105466523B CN201410466772.4A CN201410466772A CN105466523B CN 105466523 B CN105466523 B CN 105466523B CN 201410466772 A CN201410466772 A CN 201410466772A CN 105466523 B CN105466523 B CN 105466523B
- Authority
- CN
- China
- Prior art keywords
- camera
- pixel
- image
- coordinate system
- grain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The measurement method and device for the heap grain height based on single camera image that the present invention provides a kind of.This method comprises: shooting image by the video camera installed in silo, according to the variation characteristic of the pixel value of pixel in image, two regions are divided the image into using image Segmentation Technology, choose multiple boundary points on the boundary in two regions according to the pixel selection rule of setting；The pixel coordinate of each boundary point is converted to camera coordinates according to the mapping relations between pixel coordinate system and camera coordinate system by the pixel coordinate for obtaining each boundary point；According to the mapping relations between camera coordinate system and world coordinate system, the camera coordinates of each boundary point are converted into world coordinates；The heap grain altitude line in silo is obtained according to the world coordinates of each boundary point.The embodiment of the present invention can automatically, quickly and intuitively measure the height of grain heap coboundary, thus the heap grain height in effectively measuring Present Grain Bin out.
Description
Technical field
The present invention relates to grain storage management domain more particularly to a kind of surveys of the heap grain height based on single camera image
Measure method and apparatus.
Background technique
The automatization level of grain storage management still has to be hoisted at present.Heap grain elevation carrection, grain go out put in storage information note
Record etc., which is dependent on, to be accomplished manually, and the development of Grain Logistics systematism level is hindered.Heap grain altitude line, also referred to as stock line, are heaps
Grain elevation carrection and control, grain go out the important references markings of detection etc. of putting in storage, and the detection of position is to silo intelligent monitoring
It plays an important role with supervision.
It takes an inventory of warehouses in work in grain, common bulk density method estimates roughly the weight of grain when buying securities with all one's capital.Tool
The method of body is eye estimate first or manual measurement grain face at a distance from heap grain altitude line, then high by reference to heap grain in silo
The height for spending line, calculates heap grain height, then according to the length and width of silo or diameter, calculates grain using cubature formula
Volume, is finally multiplied by density, that is, calculates grain weight.The accuracy for the heap grain height that this mode is calculated and it is artificial because
Element is in close relations, and generally there was only the altitude information of a small amount of discrete point, so the accuracy of subsequent calculated result is also dropped therewith
It is low.
In addition, also being obtained using laser distance measuring principle by more three-dimensional laser scanners of fusion or separate unit Multiple-Scan
The three-dimensional information in silo is reconstructed in the point cloud data arrived.The selection of three-dimensional laser scanner and restructing algorithm all can be straight
Connect influence reconstruction result.This method can not be limited by grain heap shap, it require that expensive equipment is supported, and be calculated
Method complexity is very high, is difficult to apply in the daily operation management of silo.
Summary of the invention
The measurement method and device for the heap grain height based on single camera image that the embodiment provides a kind of, with
Realize the heap grain height in effectively measuring Present Grain Bin out.
The present invention provides following schemes:
A kind of measurement method of the heap grain height based on single camera image, installs video camera in silo, described in setting
Mapping relations in the image of video camera shooting between the pixel coordinate system and camera coordinate system of pixel, setting video camera are sat
Mapping relations between mark system and world coordinate system, comprising:
Image is shot by the video camera installed in silo, described image at least covers bottom edge and the heap grain height of silo wall
Spend line；
According to the variation characteristic of the pixel value of pixel in described image, described image is divided using image Segmentation Technology
At two regions, multiple boundary points are chosen on the boundary in described two regions according to the pixel selection rule of setting；
The pixel coordinate (u, v) for obtaining each boundary point, according between the pixel coordinate system and camera coordinate system
The pixel coordinate (u, v) of each boundary point is converted to camera coordinates (x/z, y/z) by mapping relations；
According to the mapping relations between the camera coordinate system and world coordinate system, the video camera of each boundary point is sat
Mark (x/z, y/z) is converted to world coordinates (X, Y)；
The heap grain altitude line in the silo is obtained according to the world coordinates (X, Y) of each boundary point.
In the image of the setting video camera shooting between the pixel coordinate system and camera coordinate system of pixel
Mapping relations, comprising:
The pixel coordinate that pixel in image is indicated with (u, v) indicates camera coordinates with (x, y, z), arrives from (x, y, z)
(u's, v) is mapped as
Formula 1
Formula 2
Wherein, r is distance of the pixel away from pixel coordinate origin in image, andfxAnd fyIt is respectively
The focal length of camera horizon and vertical direction, cxAnd cyIt is the basic point coordinate of camera horizon and vertical direction, k respectively1,k2,k3,
k4,k5,k6,p1,p2It is the parameter that the video camera is used to correct lens distortion.
Mapping relations between the setting camera coordinate system and world coordinate system, comprising:
In image in the hole capital after selling all securities of video camera shooting, from the lower left corner of wall and the lower right corner and heap grain height
6 index points are selected in 4 angles of line, obtain the pixel coordinate (u, v) of each index point；
According to the mapping relations between the pixel coordinate system and camera coordinate system, the video camera of each index point is obtained
Coordinate (x/z, y/z)；
It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point, if institute on wall
World coordinates Z=0 a little, the positive direction of X-axis are that horizontally to the right, the positive direction of Y-axis is vertically upward, according to each index point
The each index point of position acquisition world coordinates；
If the mapping relations between the world coordinate system and camera coordinate system of index point are as follows:
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1,
To each index point, have
Formula 3
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient；
The corresponding relationship of (X, the Y) and (x/z, y/z) of 6 index point constitutes 6 linear restrictions, this 6 it is linear about
Beam constitutes nonhomogeneous linear equation, solves the nonhomogeneous linear equation, acquires the least square solution of A.
The variation characteristic according to the pixel value of pixel in described image, using image Segmentation Technology by the figure
As being divided into two regions, comprising:
It, will be entire described using image segmentation algorithm by position between pixel in described image and similarity relationship
Image segmentation is indicated at several zonules, each zonule with a pixel value, with classifier trained in advance to each zonule
Classify, each zonule is classified as grain heap and wall, all zonules for being classified as grain heap are merged to form grain heap area
All zonules for being classified as wall and heap grain altitude line are merged to form wall area by domain.
The world coordinates (X, Y) according to each boundary point obtains the heap grain altitude line in the silo, comprising:
The size that the world coordinates (X, Y) of all boundary points presses X is subjected to ascending sort, obtains the point set S of (X, Y).
If the world coordinates in the upper right corner of grain bulk height line is (X0, Y0), in a height of X0, width is in the two-dimensional rectangle of Y0, by side each in S
Point is drawn at the ordinate of boundary's point, and sequence line, the line are the upper border line of grain heap in silo, are obtained according to the upper border line
To the corresponding heap grain height of each level point.
A kind of measuring device of the heap grain height based on single camera image, comprising:
Coordinate mapping relations setup module, for the picture of pixel in the image that the video camera installed in silo is shot to be arranged
Mapping relations between plain coordinate system and camera coordinate system, the mapping being arranged between camera coordinate system and world coordinate system are closed
System；
Image segmentation module, for obtaining the image for the shot by camera installed in silo, described image is at least covered
The bottom edge of silo wall and heap grain altitude line；According to the variation characteristic of the pixel value of pixel in described image, image point is utilized
It cuts technology and described image is divided into two regions；
Coordinate transferring is chosen on the boundary in described two regions more for the pixel selection rule according to setting
A boundary point obtains the pixel coordinate (u, v) of each boundary point, according between the pixel coordinate system and camera coordinate system
The pixel coordinate (u, v) of each boundary point is converted to camera coordinates (x/z, y/z) by mapping relations；According to the video camera
The camera coordinates (x/z, y/z) of each boundary point are converted to the world by the mapping relations between coordinate system and world coordinate system
Coordinate (X, Y)；
Heap grain altitude line obtains module, for being obtained in the silo according to the world coordinates (X, Y) of each boundary point
Heap grain altitude line.
The coordinate mapping relations setup module, for the pixel of pixel in the image that the video camera is shot to be arranged
Mapping relations between coordinate system and camera coordinate system indicate the pixel coordinate of pixel in image with (u, v), with (x, y,
Z) camera coordinates are indicated, the mapping of (u, v) is arrived from (x, y, z) are as follows:
Formula 1
Formula 2
Wherein, r is distance of the pixel away from pixel coordinate origin in image, andfxAnd fyIt is respectively
The focal length of camera horizon and vertical direction, cxAnd cyIt is the basic point coordinate of camera horizon and vertical direction, k respectively1,k2,k3,
k4,k5,k6,p1,p2It is the parameter that the video camera is used to correct lens distortion.
The coordinate mapping relations setup module, the mapping for being arranged between camera coordinate system and world coordinate system
Relationship, in the image in the hole capital after selling all securities of video camera shooting, from the lower left corner of wall and the lower right corner and heap grain altitude line
6 index points are selected in 4 angles, obtain the pixel coordinate (u, v) of each index point；
According to the mapping relations between the pixel coordinate system and camera coordinate system, the video camera of each index point is obtained
Coordinate (x/z, y/z)；
It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point, if institute on wall
World coordinates Z=0 a little, the positive direction of X-axis are that horizontally to the right, the positive direction of Y-axis is vertically upward, according to each index point
The each index point of position acquisition world coordinates；
If the mapping relations between the world coordinate system and camera coordinate system of index point are as follows:
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1,
To each index point, have
Formula 3
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient；
The corresponding relationship of (X, the Y) and (x/z, y/z) of 6 index point constitutes 6 linear restrictions, this 6 it is linear about
Beam constitutes nonhomogeneous linear equation, solves the nonhomogeneous linear equation, acquires the least square solution of A.
The image segmentation module, specifically for passing through the position and similarity pass between pixel in described image
System, is divided into several zonules for entire described image using image segmentation algorithm, and each zonule is indicated with a pixel value,
Classified with classifier trained in advance to each zonule, each zonule is classified as grain heap and wall, grain will be classified as
All zonules of heap merge to form grain heap region, and all zonules for being classified as wall and heap grain altitude line are merged to be formed
Wall area.
The heap grain altitude line obtains module, specifically for the world coordinates (X, Y) of all boundary points is pressed the big of X
Small carry out ascending sort, obtains the point set S of (X, Y).If the world coordinates in the upper right corner of grain bulk height line is (X0, Y0), in height
For X0, width is point will to be drawn at the ordinate of boundary point each in S, and sequence line, the line are silo in the two-dimensional rectangle of Y0
The upper border line of middle grain heap obtains the corresponding heap grain height of each level point according to the upper border line.
As can be seen from the technical scheme provided by the above-mentioned embodiment of the present invention, the embodiment of the present invention is by utilizing calibration point
Pixel coordinate and world coordinates that boundary point is completed without manual intervention may be implemented in the inside and outside parameter for precomputing video camera
Between conversion, the height of grain heap coboundary can automatically, be quickly and intuitively measured, and algorithm is simple and practical, to have
The heap grain height in Present Grain Bin is measured to effect, is effectively applied in the daily operation management of silo.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, required use in being described below to embodiment
Attached drawing be briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for this
For the those of ordinary skill of field, without any creative labor, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is a kind of place of the measurement method for heap grain height based on single camera image that the embodiment of the present invention one provides
Manage flow chart；
Fig. 2 is that a kind of single camera that the embodiment of the present invention one provides takes pictures to the metope of setting heap grain altitude line
Schematic diagram；
Fig. 3 is in a kind of image in hole capital after selling all securities that the embodiment of the present invention one provides, and artificial point selects the lower left corner of wall
And the schematic diagram of the lower right corner and 4 angles of heap grain altitude line totally 6 index points；
Fig. 4 is a kind of tool of the measuring device of the heap grain height based on single camera image provided by Embodiment 2 of the present invention
Body structure chart, in figure, coordinate mapping relations setup module 41, image segmentation module 42, coordinate transferring 43, heap grain altitude line
Obtain module 44.
Specific embodiment
In order to facilitate understanding of embodiments of the present invention, it is done by taking several specific embodiments as an example below in conjunction with attached drawing further
Explanation, and each embodiment does not constitute the restriction to the embodiment of the present invention.
Embodiment one
A kind of process flow of the measurement method of heap grain height based on single camera image provided in an embodiment of the present invention
As shown in Figure 1, including following processing step:
Step S110, a video camera is installed in silo, it is desirable that video camera is fixed always, and makes to shoot at the top of a wall
The image arrived at least covers bottom edge and heap grain altitude line to sidewalls.A kind of single camera that the embodiment provides is to setting heap
The schematic diagram that the metope of grain altitude line is taken pictures, the region of video camera covering is as shown in shadow region in Fig. 2.
Video camera is connect with computer, enables a computer to the image for obtaining video camera shooting, the meter of heap grain altitude line
Calculation is completed by computer.The calculation process that the embodiment of the present invention uses includes two modules, i.e. demarcating module and measurement module.?
Camera interior and exterior parameter is initialized using demarcating module when hole capital after selling all securities, when buying securities with all one's capital using measurement module to heap grain height into
Row measurement.
Step S120, camera interior and exterior parameter is initialized using demarcating module.
Step 1: solving camera intrinsic parameter
The camera intrinsic parameter solved in this step reflects the pass of the mapping between pixel coordinate system and camera coordinate system
System.
Pixel coordinate is indicated with (u, v)；Camera coordinates are indicated with (x, y, z).From (x, y's, z) to (u, v) is mapped as
Formula 1
Formula 2
Wherein, r is the distance away from pixel coordinate origin, andfxAnd fyBe respectively camera horizon and
The focal length of vertical direction, cxAnd cyIt is the basic point coordinate (generally picture centre) of camera horizon and vertical direction, k respectively1,
k2,k3,k4,k5,k6,p1,p2It is the parameter for correcting lens distortion.
fx,fy,cx,cy,k1,k2,k3,k4,k5,k6,p1,p2It is the camera intrinsic parameter for needing to solve, preferably chessboard method pair
These intrinsic parameters are solved, and step is to make a chequered with black and white chessboard shaped like chessboard first, and chessboard is big
Then chessboard is placed in front of video camera, detects 8 × 5=40 angle point, record all angle points and scheming by small preferably 9 × 6 lattice
Position as in changes chessboard position and angle, repeats to detect and record 5 times, the image angle point finally acquired according to this 5 times with
The position corresponding relationship of practical chessboard angle point solves camera intrinsic parameter.
Step 2: solving external parameters of cameras according to the index point clicked.
Heap grain altitude line has specific color and shape, the generally horizontal peony for brushing the fixed width on wall
Strip, heap grain altitude line are the upper limits of heap grain height.The embodiment of the present invention is real-time heap grain height in silo to be measured, and is not
Measure heap grain altitude line.
As shown in Fig. 2, in image in hole capital after selling all securities, from 4 of the lower left corner of wall and the lower right corner and heap grain altitude line
Select 6 index points in angle.And it will be by the wall lower left corner, the wall lower right corner, the heap grain altitude line upper left corner, heap grain altitude line upper right
The quadrilateral image area that angle surrounds will be used as area-of-interest, area-of-interest in measurement module.
Each index point is reversely reflected according to the camera intrinsic parameter solved in its pixel coordinate (u, v) and step 1
It penetrates to obtain (x/z, y/z).
Step 3: solving external parameters of cameras
The external parameters of cameras solved in this step reflects the pass of the mapping between world coordinate system and camera coordinate system
System.
World coordinates is indicated with (X, Y, Z).From (X, Y, Z's) to (x, y, z) is mapped as
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1, and [R, t] is also referred to as outside video camera
Parameter.It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point.If all the points on wall
World coordinates Z=0, the positive direction of X-axis is that horizontally to the right, the positive direction of Y-axis is vertically upward.
According to the design requirement of silo, the width of wall, the height of heap grain altitude line and line width are fixed and known.Cause
This, the corresponding world coordinates (X, Y) of each index point is known in above-mentioned steps two, be index point real standard position and
Upright position.
Since Z=0 has each index point
Formula 3
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient.In step 26 index points (X,
Y 12 groups of linear restrictions) are constituted with the corresponding relationship of (x/z, y/z), which constitutes overdetermination Nonhomogeneous Linear
Equation solves the overdetermination nonhomogeneous linear equation, acquires the least square solution of A, the external parameters of cameras for as needing to solve.Due to
The above-mentioned world coordinates Z=0 for setting all the points on wall, the mapping of from (X, Y, Z) to (x, y, z) have been simplified to from (X, Y) to (x/
Z, y/z) mapping, so [R, t] has been simplified as A, r3 is not needed.
Step S130, according to the variation characteristic of the pixel value of pixel in described image, using image Segmentation Technology by institute
Image segmentation is stated into two regions, chooses multiple sides on the boundary in described two regions according to the pixel selection rule of setting
Boundary's point.
Image is shot with above-mentioned video camera, the image taken at least covers bottom edge and heap grain altitude line to sidewalls.
According to the variation characteristic of the pixel value of pixel in described image, the area-of-interest in image is divided into two
Region, the boundary in above-mentioned two region are the coboundary of grain heap and the boundary of wall.One classifier of training in advance, preferably schemes
As pixel color as feature, support vector machines as classifier, for wall of classifying, the image of heap grain altitude line and grain heap
Pixel.Specific dividing method is to carry out over-segmentation, above-mentioned image segmentation algorithm to image using image segmentation algorithm first
It is preferred that mean shift algorithm, that is, the position between pixel and similarity relationship will be passed through, if whole image is divided into
Dry zonule, each zonule is indicated with a pixel value, is equivalent to and has been done a smoothing processing to image, then with instruction in advance
Experienced classifier classifies to each zonule, is classified as grain heap and wall.All zonules of grain heap will be finally classified as
Merging forms grain heap region, and all zonules for being classified as wall and heap grain altitude line are merged to form wall area.Note that
May there was only wall area when hole capital after selling all securities, there is no grain heap region, may there was only grain heap region when buying securities with all one's capital, without wall area.
Multiple boundary points are chosen on the handover boundary in above-mentioned two region according to the pixel selection rule of setting, than
Such as, a boundary point is chosen every fixed range on the handover boundary in above-mentioned two region.
Step S140, the pixel coordinate (u, v) for each boundary point chosen on the handover boundary in above-mentioned two region is obtained,
The camera coordinates (x/z, y/z) of each boundary point are obtained according to above-mentioned formula 1 and 2 back mapping of formula.Further according to above-mentioned public affairs
Formula 3 obtains the world coordinates (X, Y) of each boundary point.
Step S150, the heap grain altitude line in the silo is obtained according to the world coordinates (X, Y) of each boundary point.
The size that the world coordinates (X, Y) of all boundary points presses X is subjected to ascending sort, obtains the point set S of (X, Y).
If the world coordinates in the upper right corner of grain bulk height line is (X0, Y0), in a height of X0, width is in the two-dimensional rectangle of Y0, by side each in S
Point is drawn at the ordinate of boundary's point, and sequence line, the line are the upper border line of grain heap in silo, it can according to the upper border line
To obtain the corresponding heap grain height of each level point.
Embodiment two
A kind of measuring device for heap grain height that the embodiment provides, specific structure is as shown in figure 4, include following mould
Block:
Coordinate mapping relations setup module 41, for pixel in the image that the video camera installed in silo is shot to be arranged
The mapping between camera coordinate system and world coordinate system is arranged in mapping relations between pixel coordinate system and camera coordinate system
Relationship；
Image segmentation module 42, for obtaining the image for the shot by camera installed in silo, described image is at least covered
The bottom edge of lid silo wall and heap grain altitude line；According to the variation characteristic of the pixel value of pixel in described image, image is utilized
Described image is divided into two regions by cutting techniques；
Coordinate transferring 43, for being chosen on the boundary in described two regions according to the pixel selection rule of setting
Multiple boundary points obtain the pixel coordinate (u, v) of each boundary point, according between the pixel coordinate system and camera coordinate system
Mapping relations, the pixel coordinate (u, v) of each boundary point is converted into camera coordinates (x/z, y/z)；According to the camera shooting
The camera coordinates (x/z, y/z) of each boundary point are converted to generation by the mapping relations between machine coordinate system and world coordinate system
Boundary's coordinate (X, Y)；
Heap grain altitude line obtains module 44, for being obtained in the silo according to the world coordinates (X, Y) of each boundary point
Heap grain altitude line.
Further, the coordinate mapping relations setup module 41, for being arranged in the image of the video camera shooting
Mapping relations between the pixel coordinate system and camera coordinate system of pixel indicate the pixel of pixel in image with (u, v)
Coordinate indicates camera coordinates with (x, y, z), the mapping of (u, v) is arrived from (x, y, z) are as follows:
Formula 1
Formula 2
Wherein, r is distance of the pixel away from pixel coordinate origin in image, andfxAnd fyIt is respectively
The focal length of camera horizon and vertical direction, cxAnd cyIt is the basic point coordinate of camera horizon and vertical direction, k respectively1,k2,k3,
k4,k5,k6,p1,p2It is the parameter that the video camera is used to correct lens distortion.
Further, the coordinate mapping relations setup module 41, for camera coordinate system and world coordinates to be arranged
Mapping relations between system, the video camera shooting hole capital after selling all securities when image in, from the lower left corner and the lower right corner of wall, and
6 index points are selected in 4 angles of heap grain altitude line, obtain the pixel coordinate (u, v) of each index point；
According to the mapping relations between the pixel coordinate system and camera coordinate system, the video camera of each index point is obtained
Coordinate (x/z, y/z)；
It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point, if institute on wall
World coordinates Z=0 a little, the positive direction of X-axis are that horizontally to the right, the positive direction of Y-axis is vertically upward, according to each index point
The each index point of position acquisition world coordinates；
If the mapping relations between the world coordinate system and camera coordinate system of index point are as follows:
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1,
To each index point, have
Formula 3
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient；
The corresponding relationship of (X, the Y) and (x/z, y/z) of 6 index point constitutes 6 linear restrictions, this 6 it is linear about
Beam constitutes nonhomogeneous linear equation, solves the nonhomogeneous linear equation, acquires the least square solution of A.
Further, the image segmentation module 42, specifically for passing through the position between pixel in described image
With similarity relationship, entire described image is divided into several zonules using image segmentation algorithm, each zonule is with one
Pixel value indicates, is classified with classifier trained in advance to each zonule, each zonule is classified as grain heap and wall, will
All zonules for being classified as grain heap merge to form grain heap region, will be classified as all cells of wall and heap grain altitude line
Domain merges to form wall area.
Further, the heap grain altitude line obtains module 44, specifically for by the world coordinates of all boundary points (X,
Y the size for) pressing X carries out ascending sort, obtains the point set S of (X, Y).If the world coordinates in the upper right corner of grain bulk height line is
(X0, Y0), in a height of X0, width is that point, and sequence line will be drawn at the ordinate of boundary point each in S in the two-dimensional rectangle of Y0, should
Line is the upper border line of grain heap in silo, obtains the corresponding heap grain height of each level point according to the upper border line.
The detailed process for carrying out the detection of heap grain altitude line with the device of the embodiment of the present invention is similar to the previous method embodiment,
Details are not described herein again.
In conclusion the embodiment of the present invention is by precomputing the inside and outside parameter of video camera, Ke Yishi using calibration point
The conversion between the pixel coordinate and world coordinates of boundary point is now completed without manual intervention, can automatically, quickly and intuitively be surveyed
The height of grain heap coboundary is measured, and algorithm is simple and practical, so that the heap grain height in effectively measuring Present Grain Bin out, has
Effect ground is applied in the daily operation management of silo.
The embodiment of the present invention is respectively adopted different types of calibration and clicks through according to the heterogeneity of the inside and outside parameter of video camera
Calibration is gone, so that calibration process is more easy.Cost is relatively low for the equipment of the embodiment of the present invention, it is only necessary to a video camera connection
One computer, so that it may complete the measurement function of heap grain height.
Those of ordinary skill in the art will appreciate that: attached drawing is the schematic diagram of one embodiment, module in attached drawing or
Process is not necessarily implemented necessary to the present invention.
As seen through the above description of the embodiments, those skilled in the art can be understood that the present invention can
It realizes by means of software and necessary general hardware platform.Based on this understanding, technical solution of the present invention essence
On in other words the part that contributes to existing technology can be embodied in the form of software products, the computer software product
It can store in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that a computer equipment
(can be personal computer, server or the network equipment etc.) executes the certain of each embodiment or embodiment of the invention
Method described in part.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device or
For system embodiment, since it is substantially similar to the method embodiment, so describing fairly simple, related place is referring to method
The part of embodiment illustrates.Apparatus and system embodiment described above is only schematical, wherein the conduct
The unit of separate part description may or may not be physically separated, component shown as a unit can be or
Person may not be physical unit, it can and it is in one place, or may be distributed over multiple network units.It can root
According to actual need that some or all of the modules therein is selected to achieve the purpose of the solution of this embodiment.Ordinary skill
Personnel can understand and implement without creative efforts.
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
In the technical scope disclosed by the present invention, any changes or substitutions that can be easily thought of by anyone skilled in the art,
It should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with scope of protection of the claims
Subject to.
Claims (6)
1. a kind of measurement method of the heap grain height based on single camera image, which is characterized in that video camera is installed in silo,
Mapping relations in the image of the video camera shooting between the pixel coordinate system and camera coordinate system of pixel, setting are set
Mapping relations between camera coordinate system and world coordinate system, comprising:
Image is shot by the video camera installed in silo, described image at least covers bottom edge and the heap grain height of silo wall
Line；
According to the variation characteristic of the pixel value of pixel in described image, described image is divided into two using image Segmentation Technology
Multiple boundary points are chosen according to the pixel selection rule of setting in a region on the boundary in described two regions；
The pixel coordinate (u, v) for obtaining each boundary point, according to the mapping between the pixel coordinate system and camera coordinate system
The pixel coordinate (u, v) of each boundary point is converted to camera coordinates (x/z, y/z) by relationship；
According to the mapping relations between the camera coordinate system and world coordinate system, by the camera coordinates of each boundary point
(x/z, y/z) is converted to world coordinates (X, Y)；
The heap grain altitude line in the silo is obtained according to the world coordinates (X, Y) of each boundary point；
Wherein, in the image of the setting video camera shooting between the pixel coordinate system and camera coordinate system of pixel
Mapping relations, comprising:
The pixel coordinate that pixel in image is indicated with (u, v) indicates camera coordinates with (x, y, z), arrived from (x, y, z) (u,
Being mapped as v)
Wherein, r is distance of the pixel away from pixel coordinate origin in image, andfxAnd fyIt is camera shooting respectively
The focal length in the horizontal and vertical direction of machine, cxAnd cyIt is the basic point coordinate of camera horizon and vertical direction, k respectively1,k2,k3,k4,
k5,k6,p1,p2It is the parameter that the video camera is used to correct lens distortion；
Mapping relations between the setting camera coordinate system and world coordinate system, comprising:
In image in the hole capital after selling all securities of video camera shooting, from the 4 of the lower left corner of wall and the lower right corner and heap grain altitude line
6 index points are selected in a angle, obtain the pixel coordinate (u, v) of each index point；
According to the mapping relations between the pixel coordinate system and camera coordinate system, the camera coordinates of each index point are obtained
(x/z,y/z)；
It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point, if all the points on wall
World coordinates Z=0, the positive direction of X-axis is that horizontally to the right, the positive direction of Y-axis is vertically upward, according to the position of each index point
Set the world coordinates for obtaining each index point；
If the mapping relations between the world coordinate system and camera coordinate system of index point are as follows:
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1,
To each index point, have
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient；
The corresponding relationship of (X, the Y) and (x/z, y/z) of 6 index points constitutes 6 linear restrictions, 6 linear restrictions
Nonhomogeneous linear equation is constituted, the nonhomogeneous linear equation is solved, acquires the least square solution of A.
2. the measurement method of the heap grain height according to claim 1 based on single camera image, which is characterized in that described
The variation characteristic according to the pixel value of pixel in described image, described image is divided into two using image Segmentation Technology
Region, comprising:
By position between pixel in described image and similarity relationship, using image segmentation algorithm by entire described image
Several zonules are divided into, each zonule is indicated with a pixel value, is carried out with classifier trained in advance to each zonule
Classification, is classified as grain heap and wall for each zonule, all zonules for being classified as grain heap is merged to form grain heap region, will
All zonules for being classified as wall and heap grain altitude line merge to form wall area.
3. the measurement method of the heap grain height according to claim 1 or 2 based on single camera image, which is characterized in that
The world coordinates (X, Y) according to each boundary point obtains the heap grain altitude line in the silo, comprising:
The size that the world coordinates (X, Y) of all boundary points presses X is subjected to ascending sort, obtains the point set S of (X, Y)；If grain
The world coordinates in the upper right corner of stack height line is (X0, Y0), and in a height of X0, width is in the two-dimensional rectangle of Y0, by boundary point each in S
Ordinate at draw point, and sequence line, the line are the upper border line of grain heap in silo, are obtained respectively according to the upper border line
The corresponding heap grain height of a level point.
4. a kind of measuring device of the heap grain height based on single camera image characterized by comprising
Coordinate mapping relations setup module, the pixel for pixel in the image that the video camera installed in silo is shot to be arranged are sat
Mapping relations between mark system and camera coordinate system, are arranged the mapping relations between camera coordinate system and world coordinate system；
Image segmentation module, for obtaining the image for the shot by camera installed in silo, described image at least covers silo
The bottom edge of wall and heap grain altitude line；According to the variation characteristic of the pixel value of pixel in described image, image segmentation skill is utilized
Described image is divided into two regions by art；
Coordinate transferring, for choosing multiple sides on the boundary in described two regions according to the pixel selection rule of setting
Boundary's point obtains the pixel coordinate (u, v) of each boundary point, according to the mapping between the pixel coordinate system and camera coordinate system
The pixel coordinate (u, v) of each boundary point is converted to camera coordinates (x/z, y/z) by relationship；According to the camera coordinates
The camera coordinates (x/z, y/z) of each boundary point are converted to world coordinates by the mapping relations between system and world coordinate system
(X,Y)；
Heap grain altitude line obtains module, for obtaining the heap grain in the silo according to the world coordinates (X, Y) of each boundary point
Altitude line；
The coordinate mapping relations setup module, for the pixel coordinate of pixel in the image that the video camera is shot to be arranged
Mapping relations between system and camera coordinate system indicate the pixel coordinate of pixel in image with (u, v), with (x, y, z) table
Show camera coordinates, the mapping of (u, v) arrived from (x, y, z) are as follows:
Wherein, r is distance of the pixel away from pixel coordinate origin in image, andfxAnd fyIt is camera shooting respectively
The focal length in the horizontal and vertical direction of machine, cxAnd cyIt is the basic point coordinate of camera horizon and vertical direction, k respectively1,k2,k3,k4,
k5,k6,p1,p2It is the parameter that the video camera is used to correct lens distortion；
The coordinate mapping relations setup module, the mapping for being arranged between camera coordinate system and world coordinate system are closed
System, in the image in the hole capital after selling all securities of video camera shooting, from the 4 of the lower left corner of wall and the lower right corner and heap grain altitude line
6 index points are selected in a angle, obtain the pixel coordinate (u, v) of each index point；
According to the mapping relations between the pixel coordinate system and camera coordinate system, the camera coordinates of each index point are obtained
(x/z,y/z)；
It is X=0, Y=0, Z=0 by the coordinate that the lower left corner of wall is set as world coordinates origin O, O point, if all the points on wall
World coordinates Z=0, the positive direction of X-axis is that horizontally to the right, the positive direction of Y-axis is vertically upward, according to the position of each index point
Set the world coordinates for obtaining each index point；
If the mapping relations between the world coordinate system and camera coordinate system of index point are as follows:
Wherein, R=[r1, r2, r3] is the spin matrix of 3*3, and t is the translation vector of 3*1,
To each index point, have
Wherein, A=k [r1, r2, t] is the matrix of a 3*3, and k is proportionality coefficient；
The corresponding relationship of (X, the Y) and (x/z, y/z) of 6 index points constitutes 6 linear restrictions, 6 linear restrictions
Nonhomogeneous linear equation is constituted, the nonhomogeneous linear equation is solved, acquires the least square solution of A.
5. the measuring device of the heap grain height according to claim 4 based on single camera image, it is characterised in that:
The image segmentation module makes specifically for passing through position and similarity relationship in described image between pixel
Entire described image is divided into several zonules with image segmentation algorithm, each zonule is indicated with a pixel value, with pre-
First trained classifier classifies to each zonule, and each zonule is classified as grain heap and wall, will be classified as grain heap
All zonules merge to form grain heap region, and all zonules for being classified as wall and heap grain altitude line are merged to form wall
Region.
6. the measuring device of the heap grain height according to claim 4 or 5 based on single camera image, it is characterised in that:
The heap grain altitude line obtains module, specifically for the world coordinates (X, Y) of all boundary points is pressed the size of X into
Row ascending sort obtains the point set S of (X, Y)；If the world coordinates in the upper right corner of grain bulk height line is (X0, Y0), in a height of X0,
Width is point will to be drawn at the ordinate of boundary point each in S, and sequence line, the line are silo China Oil and Food Import and Export Corporation in the two-dimensional rectangle of Y0
The upper border line of heap obtains the corresponding heap grain height of each level point according to the upper border line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410466772.4A CN105466523B (en) | 2014-09-12 | 2014-09-12 | The measurement method and device of heap grain height based on single camera image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410466772.4A CN105466523B (en) | 2014-09-12 | 2014-09-12 | The measurement method and device of heap grain height based on single camera image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105466523A CN105466523A (en) | 2016-04-06 |
| CN105466523B true CN105466523B (en) | 2019-09-17 |
Family
ID=55604473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410466772.4A Active CN105466523B (en) | 2014-09-12 | 2014-09-12 | The measurement method and device of heap grain height based on single camera image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105466523B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107588732B (en) * | 2016-07-07 | 2024-03-26 | 苏州华兴致远电子科技有限公司 | Rail side train part height measurement method and system |
| CN109472261B (en) * | 2018-06-15 | 2020-09-18 | 河南工业大学 | Computer vision-based automatic monitoring method for grain storage quantity change of granary |
| CN109443476B (en) * | 2018-10-17 | 2021-07-09 | 水利部交通运输部国家能源局南京水利科学研究院 | Non-contact measuring device and method for water level fluctuation process |
| CN109459121A (en) * | 2018-12-29 | 2019-03-12 | 中航电测仪器股份有限公司 | A kind of bird weight center of gravity measurement method |
| CN109800688B (en) * | 2019-01-03 | 2023-04-07 | 沈阳天骄科技有限公司 | Identification method for calculating grain pile height based on gyroscope and depth value |
| CN112734826B (en) * | 2020-12-29 | 2022-05-31 | 华信咨询设计研究院有限公司 | Grain quantity estimation method based on deep learning and LSD (least squares-based) linear detection algorithm |
| CN117268498B (en) * | 2023-11-20 | 2024-01-23 | 中国航空工业集团公司金城南京机电液压工程研究中心 | Oil mass measurement method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266131A (en) * | 2008-04-08 | 2008-09-17 | 长安大学 | Volume measurement device based on image and its measurement method |
| CN101936761A (en) * | 2009-06-30 | 2011-01-05 | 宝山钢铁股份有限公司 | Visual measuring method of stockpile in large-scale stock ground |
| CN102012213A (en) * | 2010-08-31 | 2011-04-13 | 吉林大学 | Method for measuring foreground height through single image |
| CN103033132A (en) * | 2012-12-20 | 2013-04-10 | 中国科学院自动化研究所 | Plane measuring method and plane measuring device based on monocular vision |
| CN103177439A (en) * | 2012-11-26 | 2013-06-26 | 惠州华阳通用电子有限公司 | Automatically calibration method based on black and white grid corner matching |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| BRPI0820626A2 (en) * | 2007-11-14 | 2015-06-16 | Intergraph Software Tech Co | Method and apparatus for aerial surveys |
-
2014
- 2014-09-12 CN CN201410466772.4A patent/CN105466523B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266131A (en) * | 2008-04-08 | 2008-09-17 | 长安大学 | Volume measurement device based on image and its measurement method |
| CN101936761A (en) * | 2009-06-30 | 2011-01-05 | 宝山钢铁股份有限公司 | Visual measuring method of stockpile in large-scale stock ground |
| CN102012213A (en) * | 2010-08-31 | 2011-04-13 | 吉林大学 | Method for measuring foreground height through single image |
| CN103177439A (en) * | 2012-11-26 | 2013-06-26 | 惠州华阳通用电子有限公司 | Automatically calibration method based on black and white grid corner matching |
| CN103033132A (en) * | 2012-12-20 | 2013-04-10 | 中国科学院自动化研究所 | Plane measuring method and plane measuring device based on monocular vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105466523A (en) | 2016-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105466523B (en) | The measurement method and device of heap grain height based on single camera image | |
| Alidoost et al. | Comparison of UAS-based photogrammetry software for 3D point cloud generation: a survey over a historical site | |
| Jay et al. | In-field crop row phenotyping from 3D modeling performed using Structure from Motion | |
| US11443133B2 (en) | Computer vision system for industrial equipment gauge digitization and alarms | |
| Lati et al. | Estimating plant growth parameters using an energy minimization-based stereovision model | |
| Persson et al. | Tree species classification of individual trees in Sweden by combining high resolution laser data with high resolution near-infrared digital images | |
| KR20150079576A (en) | Depth map generation from a monoscopic image based on combined depth cues | |
| Lou et al. | Accurate multi-view stereo 3D reconstruction for cost-effective plant phenotyping | |
| Nguyen et al. | Comparison of structure-from-motion and stereo vision techniques for full in-field 3d reconstruction and phenotyping of plants: An investigation in sunflower | |
| CN113435282B (en) | Unmanned aerial vehicle image ear recognition method based on deep learning | |
| CN110199317A (en) | Automatic detection, counting and measurement using handheld device to plank | |
| CN114241031A (en) | Fish body ruler measurement and weight prediction method and device based on double-view fusion | |
| CN112561983A (en) | Device and method for measuring and calculating surface weak texture and irregular stacking volume | |
| CN116071424A (en) | Fruit space coordinate positioning method based on monocular vision | |
| CN115471542A (en) | Packaging object binocular recognition and positioning method based on YOLO v5 | |
| CN113947770B (en) | Method for identifying object placed in different areas of intelligent cabinet | |
| CN115082777A (en) | Binocular vision-based underwater dynamic fish form measuring method and device | |
| CN115861409B (en) | Soybean leaf area measuring and calculating method, system, computer equipment and storage medium | |
| Xiang et al. | PhenoStereo: a high-throughput stereo vision system for field-based plant phenotyping-with an application in sorghum stem diameter estimation | |
| CN111985472A (en) | Trough hay temperature image processing method based on artificial intelligence and active ball machine | |
| Chen et al. | Fusion of LIDAR data and high resolution images for forest canopy modelling | |
| Albrecht et al. | Seeing the unseen: Simple reconstruction of transparent objects from point cloud data | |
| CN109978937A (en) | A kind of ox body measurement system detected based on deep learning and characteristic portion | |
| CN111860035A (en) | Book cover detection method and device, storage medium and electronic equipment | |
| CN109641351B (en) | Object feature identification method, visual identification device and robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |