CN114299283B - Image-based displacement measurement method and system - Google Patents
Image-based displacement measurement method and system Download PDFInfo
- Publication number
- CN114299283B CN114299283B CN202210213628.4A CN202210213628A CN114299283B CN 114299283 B CN114299283 B CN 114299283B CN 202210213628 A CN202210213628 A CN 202210213628A CN 114299283 B CN114299283 B CN 114299283B
- Authority
- CN
- China
- Prior art keywords
- anchor point
- marker
- central anchor
- values
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to a displacement measurement method and system based on an image, and belongs to the field of image processing. The method comprises the following steps: constructing a detection network based on a central anchor point; the basic network of the detection network is Faster-RCNN, and an anchor frame generation strategy based on a central anchor point, an area intersection ratio calculation mode based on the central anchor point and a regression loss function based on the central anchor point are adopted; the central anchor point is the centroid coordinate of the target, the anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame; training the detection network based on the regression loss function to obtain a marker detection model; and inputting the marker images acquired at two moments into a marker detection model to obtain a displacement measurement result of the marker. The invention can improve the precision of displacement measurement.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a displacement measurement method and system based on an image.
Background
The high-precision displacement measurement can play a role in monitoring and forecasting, is beneficial to finding out potential quality safety hazards as early as possible so as to take remedial measures in time, and has important application value in various fields such as manufacturing industry, civil engineering and the like. In recent years, with the improvement of industrialization level and the development of computer science, the displacement measurement technology based on computer vision becomes a large research hotspot in the displacement measurement field by virtue of the advantages of non-contact, high precision, multipoint monitoring and the like, and is practically applied. The target detection technology based on deep learning can automatically capture the position information of the target of interest in the field of view, and can provide a reliable technical approach for displacement measurement.
However, in the current stage, the target detection algorithm based on deep learning mostly adopts a frame regression mode to perform target positioning, so that the improvement of positioning precision is limited, and further the improvement of displacement measurement precision is limited.
Disclosure of Invention
The invention aims to provide a displacement measurement method and system based on an image so as to improve the precision of displacement measurement.
In order to achieve the purpose, the invention provides the following scheme:
an image-based displacement measurement method, comprising:
constructing a detection network based on a central anchor point; the basic network of the detection network is fast-RCNN, the detection network adopts an anchor frame generation strategy based on a central anchor point, the detection network adopts an area intersection ratio calculation mode based on the central anchor point, and the detection network adopts a regression loss function based on the central anchor point; the central anchor point is the centroid coordinate of a target, an anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame;
training the detection network based on the regression loss function to obtain a marker detection model;
and inputting the marker images acquired at two moments into the marker detection model to obtain the displacement measurement result of the marker.
Optionally, the constructing a detection network based on a central anchor point specifically includes:
an anchor frame generation strategy based on a central anchor point is constructed, and the process is as follows: adjusting the proportion of the anchor frames adopted in the generation stage of the anchor frames of the RPN in the Faster-RCNN to be 1:1, and adjusting the sizes of the anchor frames to be 16 multiplied by 16, 32 multiplied by 32, 64 multiplied by 64, 128 multiplied by 128, 256 multiplied by 256 and 512 multiplied by 512;
constructing an area intersection ratio calculation mode based on a central anchor point, wherein the area intersection ratio calculation formula is as follows:
(ii) a Wherein the content of the first and second substances,IoUin terms of area cross ratio (x，y) For predicting the coordinates of the central anchor point of the frame, the length of one half of the side of the frame is usedcRepresenting; (G x ，G y ) The coordinates of the central anchor point of the real bounding box,G c is half of the side length of the real bounding box;
constructing a regression loss function based on the central anchor point, wherein the process is as follows:
modifying the dimension of a predicted value output by the last full-connection layer in the regression branch in the Faster-RCNN network into 3 dimensions, corresponding to [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities;
determining the regression loss function, wherein the formula is as follows: CPR Loss = α (cL x +L y )+βL c (ii) a Wherein α and β are regulatory factors;L x is a function of the regression loss for the x coordinate,
，d x representing the x value of the prediction box and the true bounding boxG x The coordinate between the values transforms the predicted value,d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values transform the true values,
；L y is a function of the regression loss in the y-coordinate,
，d y representing the y-value of the prediction box and the true bounding boxG y The coordinate between the values transforms the predicted value,d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values transform the true values,
；L c is a function of the side length regression loss,
，d c representing the c-value of the prediction box and the true bounding boxG c The coordinate between the values transforms the predicted value,d' c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values transform the true values,
。
optionally, the training the detection network based on the regression loss function to obtain a marker detection model specifically includes:
shooting a marker image for displacement monitoring by using an industrial camera to obtain an image set;
marking the image in the image set one by adopting a central anchor point marking method to obtain a marking file;
and training the detection network based on the regression loss function by adopting the label file to obtain a trained marker detection model.
Optionally, the inputting the marker images acquired at two moments into the marker detection model to obtain the displacement measurement result of the marker specifically includes:
inputting the marker images acquired at two moments into the marker detection model, and outputting the position information of the markers in the marker images acquired at two moments;
using a formula
Determining the displacement of the marker; wherein D is the displacement of the marker, and the position information output by the marker image acquired at the first moment is [ xA,yA,cA]And the position information output by the marker image acquired at the second moment is [ x ]B,yB,cB]。
The invention also provides an image-based displacement measurement system, comprising:
the detection network construction module is used for constructing a detection network based on the central anchor point; the basic network of the detection network is fast-RCNN, the detection network adopts an anchor frame generation strategy based on a central anchor point, the detection network adopts an area intersection ratio calculation mode based on the central anchor point, and the detection network adopts a regression loss function based on the central anchor point; the central anchor point is the centroid coordinate of a target, an anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame;
the training module is used for training the detection network based on the regression loss function to obtain a marker detection model;
and the displacement measurement module is used for inputting the marker images acquired at two moments into the marker detection model to obtain the displacement measurement result of the marker.
Optionally, the detection network constructing module specifically includes:
the anchor frame generation strategy construction unit is used for constructing an anchor frame generation strategy based on the central anchor point, and the process is as follows: adjusting the proportion of the anchor frames adopted in the generation stage of the anchor frames of the RPN in the Faster-RCNN to be 1:1, and adjusting the sizes of the anchor frames to be 16 multiplied by 16, 32 multiplied by 32, 64 multiplied by 64, 128 multiplied by 128, 256 multiplied by 256 and 512 multiplied by 512;
the area intersection ratio calculation mode construction unit is used for constructing an area intersection ratio calculation mode based on the central anchor point, and the area intersection ratio calculation formula is as follows:
(ii) a Wherein the content of the first and second substances,IoUin terms of area cross ratio (x，y) For predicting the coordinates of the central anchor point of the frame, the length of one half of the side of the frame is usedcRepresents; (G x ，G y ) The coordinates of the central anchor point of the real bounding box,G c as a real bounding boxOne half of the side length of (1);
the regression loss function construction unit is used for constructing a regression loss function based on the central anchor point, and the process is as follows:
modifying the dimension of a predicted value output by the last full-connection layer in the regression branch in the Faster-RCNN network into 3 dimensions, corresponding to [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities;
determining the regression loss function, wherein the formula is as follows: CPR Loss = α (cL x +L y )+βL c (ii) a Wherein α and β are regulatory factors;L x is a function of the regression loss for the x coordinate,
，d x representing the x value of the prediction box and the true bounding boxG x The coordinate between the values transforms the predicted value,d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values transform the true values,
；L y is a function of the regression loss in the y-coordinate,
，d y representing the y-value of the prediction box and the true bounding boxG y The coordinate between the values transforms the predicted value,d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values transform the true values,
；L c is a function of the side length regression loss,
，d c representing the c-value of the prediction box and the true bounding boxG c The coordinate between the values transforms the predicted value,d' c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values transform the true values,
。
optionally, the training module specifically includes:
the image set acquisition unit is used for shooting a marker image for displacement monitoring by adopting an industrial camera to obtain an image set;
the marking unit is used for marking the markers of the images in the image set one by adopting a central anchor point marking method to obtain marking files;
and the training unit is used for training the detection network based on the regression loss function by adopting the labeled file to obtain a trained marker detection model.
Optionally, the displacement measurement module specifically includes:
the position information output unit is used for inputting the marker images acquired at two moments into the marker detection model and outputting the position information of the markers in the marker images acquired at two moments;
a displacement determination unit for utilizing the formula
Determining the displacement of the marker; wherein D is the displacement of the marker, and the position information output by the marker image acquired at the first moment is [ xA,yA,cA]And the position information output by the marker image acquired at the second moment is [ x ]B,yB,cB]。
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
aiming at the displacement measurement task, the invention adopts a central anchor point detection mode, obtains the position information of the target through less parameter quantity, and effectively reduces the difficulty of network training and prediction; and the center anchor point detection directly represents the specific position of the target in the image through the center anchor point coordinates, and compared with the traditional marking method, the method can realize more accurate marker position prediction and further improve the precision of displacement measurement.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart of the image-based displacement measurement method of the present invention;
FIG. 2 is a schematic diagram illustrating a comparison between a conventional labeling method and the labeling method of the present invention;
FIG. 3 is a schematic illustration of area cross-over ratio calculation according to the present invention;
FIG. 4 is a schematic diagram of an image-based displacement measurement system according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
FIG. 1 is a schematic flow chart of the image-based displacement measurement method according to the present invention. As shown in fig. 1, the method comprises the following steps:
step 100: and constructing a detection network based on the central anchor point. Because the double-stage target detection network has more advantages in detection precision than a single-stage detection network, the invention adopts the classic double-stage detection network fast-RCNN as a basic network.
Because the displacement measurement task needs to accurately position the marker, the existing target detection algorithm (including fast-RCNN) adopts a conventional anchor frame as shown in part (a) of fig. 2 to predict the target position, and the positioning is inaccurate due to slight deviation of the predicted value [ x, y, w, h ]. The method directly predicts the specific position of the target in the image through the central point coordinates (x, y) as shown in the part (b) of figure 2 based on the detection mode of the central anchor point, and is more accurate and simpler compared with the conventional anchor frame prediction. Therefore, the construction of the detection network needs to be improved from three aspects: the anchor frame generation strategy is based on the central anchor point; the area intersection ratio calculation mode is based on a central anchor point; the regression loss function is based on the central anchor point. The central anchor point is the centroid coordinate of the target, the anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame. The specific process is as follows:
(1) and generating a strategy based on the anchor frame detected by the central anchor point. In the original fast-RCNN, the proportion of the anchor frame adopted in the anchor frame generation stage of the RPN network is {1:1, 1:2, 2:1}, and the dimension of the anchor frame is {128 × 128, 256 × 256 and 512 × 512 }; in the detection network of the present invention, since the shape of the labeling frame is square, the anchor frame generation is performed at a ratio of 1:1, and in order to predict the position frame of the marker more accurately, the anchor frames having different areas are added, and the candidate region prediction is performed using the anchor frames having the sizes of 16 × 16, 32 × 32, 64 × 64, 128 × 128, 256 × 256, and 512 × 512.
(2) An Intersection-over-Union (IoU) calculation method based on the area Intersection ratio detected by the central anchor point. In the fast-RCNN, a part of prediction frames with too high overlapping rate, namely IoU too large, need to be removed through an NMS algorithm, but the IoU calculation method adopted by the invention is only suitable for a conventional anchor frame detection mode, so the invention adopts a IoU calculation method based on central anchor point detection. As shown in FIG. 3, the prediction box is located in the upper left corner with center anchor coordinates of（x，y) One half side length ofcThe real bounding box is located at the lower right corner, and the central anchor point coordinate is (G x ，G y ) One half side length ofGcThen IoU calculation formula for the two position boxes is shown in equation 1.
And adopting the IoU calculation formula to screen the prediction box of the NMS algorithm in the Faster-RCNN network.
(3) The invention provides a novel Regression Loss function aiming at the detection of a central anchor Point, namely the Regression Loss function CPR Loss of the central anchor Point.
Firstly, in order to use the network for central anchor point prediction, the dimension of a predicted value output by the last full-connection layer in the regression branch of the fast-RCNN network is modified into 3 dimensions, corresponding to [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities (the dimension of the predicted value output by the last full-connected layer is 4 dimension, which corresponds to [ 2 ]d x ,d y ,d w ,d h ]Four coordinate transformation variables, suitable for conventional anchor frame prediction).
Next, a coordinate regression loss function is designedL x . Let the location information of the prediction box be [ x, y, c ]]Wherein x is the abscissa of the central anchor point of the prediction frame, y is the ordinate of the central anchor point of the prediction frame, and c is the half side length of the prediction frame; setting the position information of the real bounding box as [ 2 ]G x ,G y ,G c ]WhereinG x Is the central anchor point abscissa of the real bounding box,G y is the vertical coordinate of the central anchor point of the real bounding box,G c is one-half of the side length of the real bounding box. Then the coordinate regression loss function Lx corresponds to the prediction error of the x value in the prediction boxThe formula for the loss is shown in equation (2).
In the formula (I), the compound is shown in the specification,d x representing the x value of the prediction box and the true bounding boxG x Coordinate transformation predicted values among the values are output by a full connection layer of the network regression branch;d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values are transformed into real values, and the calculation mode is shown in formula (3).
Subsequently, a coordinate regression loss function is designedL y . Coordinate regression loss functionL y The calculation formula is shown in formula (4) for the prediction error loss of the y value in the prediction box.
In the formula (I), the compound is shown in the specification,d y representing the y-value of the prediction box and the true bounding boxG y Coordinate transformation predicted values among the values are output by a full connection layer of the network regression branch;d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values are transformed into real values, and the calculation mode is shown in formula (5).
Secondly, designing a side length regression loss functionL c . Side length regression loss functionL c Prediction error corresponding to c value in prediction boxThe formula for the loss is shown in equation (6).
In the formula (I), the compound is shown in the specification,d c representing the c-value of the prediction box and the true bounding boxG c Coordinate transformation predicted values among the values are output by a full connection layer of the network regression branch;d' c representing c-values of prediction boxes and true bounding boxesG c The coordinates between the values are transformed into real values, and the calculation method is shown in formula (7).
Finally, based onL x 、L y AndL c design CPR Loss.
CPR Loss=α(L x +L y )+βL c （8）
As shown in equation (8), CPR Loss is regressed by two coordinate regression Loss functionsL x 、L y And a side length regression loss functionL c The three parts are formed. Wherein alpha and beta are regulating factors used for adjusting the proportion of the coordinate regression loss function and the side length regression loss function in the total regression loss function.
And replacing the original anchor frame generation strategy, original IoU calculation method and original CPR Loss function in the fast-RCNN detection algorithm with the anchor frame generation strategy, original IoU calculation method and original CPR Loss function to obtain the detection network of the marker.
Step 200: and training the detection network based on the regression loss function to obtain a marker detection model. Firstly, shooting a marker image for displacement monitoring by using an industrial camera to obtain an image set, and then marking the markers of the image set one by using a central anchor point marking method to obtain a marking file. And finally, updating and optimizing network parameters by adopting a label file based on CPR Loss, and training the detection network to obtain a trained marker detection model.
Step 300: and inputting the marker images acquired at two moments into a marker detection model to obtain a displacement measurement result of the marker. The marker image A acquired at the time A and the marker image B acquired at the time B (the shooting visual fields of the two images are ensured to be consistent) are simultaneously input into the marker detection model, and the position information [ x ] of the marker image A is respectively outputA,yA,cA]And position information [ x ] of marker image BB,yB,cB]. And calculating a displacement measurement result D according to the output position information, namely the displacement of the marker from the moment A to the moment B, wherein the calculation formula is shown as a formula 9.
In specific application, in order to acquire the displacement of the marker in a certain time period, firstly fixing the position of a camera, and acquiring an image to acquire an initial position image of the marker; secondly, when displacement measurement is needed, acquiring an image again to obtain a current position image of the marker, wherein a shooting view field of the camera is ensured to be constant in the acquisition process; the two marker images acquired at different moments are input into a displacement measurement system, and a displacement measurement result D, namely the displacement of the marker generated between the two acquisition moments, can be output.
Based on the method, the invention also provides an image-based displacement measurement system, and fig. 4 is a schematic structural diagram of the image-based displacement measurement system. As shown in fig. 4, the image-based displacement measurement system includes:
a detection network construction module 401, configured to construct a detection network based on a central anchor point; the basic network of the detection network is fast-RCNN, the detection network adopts an anchor frame generation strategy based on a central anchor point, the detection network adopts an area intersection ratio calculation mode based on the central anchor point, and the detection network adopts a regression loss function based on the central anchor point; the central anchor point is the centroid coordinate of the target, the anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame.
A training module 402, configured to train the detection network based on the regression loss function to obtain a marker detection model.
And a displacement measurement module 403, configured to input the marker images acquired at two moments into the marker detection model, so as to obtain a displacement measurement result of the marker.
As a specific embodiment, in the image-based displacement measurement system of the present invention, the detection network constructing module 401 specifically includes:
the anchor frame generation strategy construction unit is used for constructing an anchor frame generation strategy based on the central anchor point, and the process is as follows: the proportion of the anchor frame adopted in the anchor frame generation stage of the RPN network in the Faster-RCNN is adjusted to 1:1, and the sizes of the anchor frame are adjusted to 16 × 16, 32 × 32, 64 × 64, 128 × 128, 256 × 256 and 512 × 512.
The area intersection ratio calculation mode construction unit is used for constructing an area intersection ratio calculation mode based on the central anchor point, and the area intersection ratio calculation formula is as follows:
(ii) a Wherein the content of the first and second substances,IoUin terms of area cross ratio (x，y) For predicting the coordinates of the central anchor point of the frame, the length of one half of the side of the frame is usedcRepresents; (G x ，G y ) The coordinates of the central anchor point of the real bounding box,G c is one-half of the side length of the real bounding box.
The regression loss function construction unit is used for constructing a regression loss function based on the central anchor point, and the process is as follows:
the dimension of the predicted value output by the last full-connection layer in the regression branch in the fast-RCNN network is modified into 3 dimensions, corresponding to [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities;
determining a regression loss function, wherein the formula is as follows: CPR Loss = α (cL x +L y )+βL c (ii) a Wherein α and β are regulatory factors;L x is a function of the regression loss for the x coordinate,
，d x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values are transformed to predict values,d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values transform the real values,
；L y is a function of the regression loss in the y-coordinate,
，d y representing the y-value of the prediction box and the true bounding boxG y The coordinate between the values transforms the predicted value,d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values transform the true values,
；L c is a function of the side length regression loss,
，d c representing c-values of prediction boxes and true bounding boxesG c The coordinate between the values transforms the predicted value,d' c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values transform the true values,
。
as a specific embodiment, in the image-based displacement measurement system of the present invention, the training module 402 specifically includes:
and the image set acquisition unit is used for shooting a marker image for displacement monitoring by adopting an industrial camera to obtain an image set.
And the marking unit is used for marking the markers of the images in the image set one by adopting a central anchor point marking method to obtain a marked file.
And the training unit is used for training the detection network based on the regression loss function by adopting the labeled file to obtain a trained marker detection model.
As a specific embodiment, in the image-based displacement measurement system of the present invention, the displacement measurement module 403 specifically includes:
and the position information output unit is used for inputting the marker images acquired at two moments into the marker detection model and outputting the position information of the markers in the marker images acquired at two moments.
A displacement determination unit for utilizing the formula
Determining the displacement of the marker; wherein D is the displacement of the marker, and the position information output by the marker image acquired at the first moment is [ xA,yA,cA]And the position information output by the marker image acquired at the second moment is [ x ]B,yB,cB]。
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.
Claims (6)
1. An image-based displacement measurement method, comprising:
constructing a detection network based on a central anchor point; the basic network of the detection network is fast-RCNN, the detection network adopts an anchor frame generation strategy based on a central anchor point, the detection network adopts an area intersection ratio calculation mode based on the central anchor point, and the detection network adopts a regression loss function based on the central anchor point; the central anchor point is the centroid coordinate of a target, an anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame;
training the detection network based on the regression loss function to obtain a marker detection model;
inputting the marker images acquired at two moments into the marker detection model to obtain a displacement measurement result of the marker;
the method for constructing the detection network based on the central anchor point specifically comprises the following steps:
an anchor frame generation strategy based on a central anchor point is constructed, and the process is as follows: adjusting the proportion of the anchor frames adopted in the generation stage of the anchor frames of the RPN in the Faster-RCNN to be 1:1, and adjusting the sizes of the anchor frames to be 16 multiplied by 16, 32 multiplied by 32, 64 multiplied by 64, 128 multiplied by 128, 256 multiplied by 256 and 512 multiplied by 512;
constructing an area intersection ratio calculation mode based on a central anchor point, wherein the area intersection ratio calculation formula is as follows:
(ii) a Wherein the content of the first and second substances,IoUin terms of area cross ratio (x，y) For predicting the coordinates of the central anchor point of the frame, the length of one half of the side of the frame is usedcRepresents; (G x ，G y ) The coordinates of the central anchor point of the real bounding box,G c is half of the side length of the real bounding box;
constructing a regression loss function based on the central anchor point, wherein the process is as follows:
modifying the dimension of a predicted value output by the last full-connection layer in the regression branch in the Faster-RCNN network into 3 dimensions, corresponding to [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities;
determining the regression loss function, wherein the formula is as follows: CPR Loss = α (cL x +L y )+βL c (ii) a Wherein α and β are regulatory factors;L x is a function of the regression loss for the x coordinate,
，d x representing the x value of the prediction box and the true bounding boxG x The coordinate between the values transforms the predicted value,d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values transform the true values,
；L y is a function of the regression loss in the y-coordinate,
，d y representing the y-value of the prediction box and the true bounding boxG y The coordinate between the values transforms the predicted value,d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values transform the true values,
；L c is a function of the side length regression loss,
，d c representing the c-value of the prediction box and the true bounding boxG c The coordinate between the values transforms the predicted value,d' c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values transform the true values,
。
2. the image-based displacement measurement method according to claim 1, wherein the training of the detection network based on the regression loss function to obtain a marker detection model specifically comprises:
shooting a marker image for displacement monitoring by using an industrial camera to obtain an image set;
marking the image in the image set one by adopting a central anchor point marking method to obtain a marked file;
and training the detection network based on the regression loss function by adopting the label file to obtain a trained marker detection model.
3. The image-based displacement measurement method according to claim 1, wherein the step of inputting the marker images acquired at two times into the marker detection model to obtain the displacement measurement result of the marker specifically comprises:
inputting the marker images acquired at two moments into the marker detection model, and outputting the position information of the markers in the marker images acquired at two moments;
using formulas
Determining the displacement of the marker; wherein D is the displacement of the marker, the position information of the marker image output acquired at the first momentIs [ x ]A,yA,cA]And the position information output by the marker image acquired at the second moment is [ x ]B,yB,cB]。
4. An image-based displacement measurement system, comprising:
the detection network construction module is used for constructing a detection network based on the central anchor point; the basic network of the detection network is fast-RCNN, the detection network adopts an anchor frame generation strategy based on a central anchor point, the detection network adopts an area intersection ratio calculation mode based on the central anchor point, and the detection network adopts a regression loss function based on the central anchor point; the central anchor point is the centroid coordinate of a target, an anchor frame based on the central anchor point is a square position frame taking the central anchor point as the center, and the marking information of the anchor frame comprises the coordinate of the central anchor point and the half side length of the square position frame;
the training module is used for training the detection network based on the regression loss function to obtain a marker detection model;
the displacement measurement module is used for inputting the marker images acquired at two moments into the marker detection model to obtain the displacement measurement result of the marker;
the detection network construction module specifically includes:
the anchor frame generation strategy construction unit is used for constructing an anchor frame generation strategy based on the central anchor point, and the process is as follows: adjusting the proportion of the anchor frames adopted in the generation stage of the anchor frames of the RPN in the Faster-RCNN to be 1:1, and adjusting the sizes of the anchor frames to be 16 multiplied by 16, 32 multiplied by 32, 64 multiplied by 64, 128 multiplied by 128, 256 multiplied by 256 and 512 multiplied by 512;
the area intersection ratio calculation mode construction unit is used for constructing an area intersection ratio calculation mode based on the central anchor point, and the area intersection ratio calculation formula is as follows:
(ii) a Wherein the content of the first and second substances,IoUin terms of area cross ratio (x，y) For central anchor points of prediction framesCoordinates, for predicting half-length of framecRepresents; (G x ，G y ) The coordinates of the central anchor point of the real bounding box,G c is half of the side length of the real bounding box;
the regression loss function construction unit is used for constructing a regression loss function based on the central anchor point, and the process is as follows:
modifying the dimension of the predicted value output by the last full-connection layer in the regression branch in the Faster-RCNN network into 3 dimension corresponding to the [ 2 ]d x ,d y ,d c ]Three coordinate transformation quantities;
determining the regression loss function, wherein the formula is as follows: CPR Loss = α (cL x +L y )+βL c (ii) a Wherein α and β are regulatory factors;L x is a function of the regression loss in the x-coordinate,
，d x representing the x value of the prediction box and the true bounding boxG x The coordinate between the values transforms the predicted value,d' x representing the x value of the prediction box and the true bounding boxG x The coordinates between the values transform the true values,
；L y is a function of the regression loss in the y-coordinate,
，d y representing the y-value of the prediction box and the true bounding boxG y The coordinate between the values transforms the predicted value,d' y representing the y-value of the prediction box and the true bounding boxG y The coordinates between the values transform the true values,
；L c is a function of the side length regression loss,
，d c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values are transformed to predict values,d' c representing the c-value of the prediction box and the true bounding boxG c The coordinates between the values transform the true values,
。
5. the image-based displacement measurement system of claim 4, wherein the training module specifically comprises:
the image set acquisition unit is used for shooting a marker image for displacement monitoring by adopting an industrial camera to obtain an image set;
the marking unit is used for marking the markers of the images in the image set one by adopting a central anchor point marking method to obtain a marking file;
and the training unit is used for training the detection network based on the regression loss function by adopting the labeled file to obtain a trained marker detection model.
6. The image-based displacement measurement system of claim 4, wherein the displacement measurement module specifically comprises:
the position information output unit is used for inputting the marker images acquired at two moments into the marker detection model and outputting the position information of the markers in the marker images acquired at two moments;
a displacement determination unit for utilizing the formula
Determining the position of the markerMoving; wherein D is the displacement of the marker, and the position information output by the marker image acquired at the first moment is [ xA,yA,cA]And the position information output by the marker image acquired at the second moment is [ x ]B,yB,cB]。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210213628.4A CN114299283B (en) | 2022-03-07 | 2022-03-07 | Image-based displacement measurement method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210213628.4A CN114299283B (en) | 2022-03-07 | 2022-03-07 | Image-based displacement measurement method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114299283A CN114299283A (en) | 2022-04-08 |
| CN114299283B true CN114299283B (en) | 2022-05-20 |
Family
ID=80978531
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210213628.4A Active CN114299283B (en) | 2022-03-07 | 2022-03-07 | Image-based displacement measurement method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114299283B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2019101142A4 (en) * | 2019-09-30 | 2019-10-31 | Dong, Qirui MR | A pedestrian detection method with lightweight backbone based on yolov3 network |
| CN111666839A (en) * | 2020-05-25 | 2020-09-15 | 东华大学 | Road pedestrian detection system based on improved Faster RCNN |
| CN112949482A (en) * | 2021-03-01 | 2021-06-11 | 浙江大学 | Non-contact type rail sleeper relative displacement real-time measurement method based on deep learning and visual positioning |
| CN112949483A (en) * | 2021-03-01 | 2021-06-11 | 浙江大学 | Non-contact rail stretching displacement real-time measurement method based on fast R-CNN |
| CN113011283A (en) * | 2021-03-01 | 2021-06-22 | 浙江大学 | Non-contact type rail sleeper relative displacement real-time measurement method based on video |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190095762A1 (en) * | 2017-09-25 | 2019-03-28 | Gyrfalcon Technology Inc. | Communications Between Internet of Things Devices Using A Two-dimensional Symbol Containing Multiple Ideograms |
| CN110490170B (en) * | 2019-08-27 | 2023-01-06 | 浙江中正智能科技有限公司 | Face candidate frame extraction method |
-
2022
- 2022-03-07 CN CN202210213628.4A patent/CN114299283B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2019101142A4 (en) * | 2019-09-30 | 2019-10-31 | Dong, Qirui MR | A pedestrian detection method with lightweight backbone based on yolov3 network |
| CN111666839A (en) * | 2020-05-25 | 2020-09-15 | 东华大学 | Road pedestrian detection system based on improved Faster RCNN |
| CN112949482A (en) * | 2021-03-01 | 2021-06-11 | 浙江大学 | Non-contact type rail sleeper relative displacement real-time measurement method based on deep learning and visual positioning |
| CN112949483A (en) * | 2021-03-01 | 2021-06-11 | 浙江大学 | Non-contact rail stretching displacement real-time measurement method based on fast R-CNN |
| CN113011283A (en) * | 2021-03-01 | 2021-06-22 | 浙江大学 | Non-contact type rail sleeper relative displacement real-time measurement method based on video |
Non-Patent Citations (1)
| Title |
|---|
| Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks;Shaoqing Ren et al;《arXiv:1506.01497v3 [cs.CV]》;20160106;正文第3章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114299283A (en) | 2022-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105930858A (en) | Fast high-precision geometric template matching method enabling rotation and scaling functions | |
| Hou et al. | Detecting structural components of building engineering based on deep-learning method | |
| CN109598794A (en) | The construction method of three-dimension GIS dynamic model | |
| CN113916130B (en) | Building position measuring method based on least square method | |
| CN103440667A (en) | Automatic device for stably tracing moving targets under shielding states | |
| CN109870106A (en) | A kind of building volume measurement method based on unmanned plane picture | |
| Ying et al. | A mask R-CNN based approach to automatically construct as-is IFC BIM objects from digital images | |
| Wang et al. | A synthetic dataset for Visual SLAM evaluation | |
| CN101685532B (en) | Method for correcting simple linear wide-angle lens | |
| CN112270278A (en) | Key point-based blue top house detection method | |
| Cheng et al. | Vision-based trajectory monitoring for assembly alignment of precast concrete bridge components | |
| CN114066985B (en) | Method for calculating hidden danger distance of power transmission line and terminal | |
| CN113947714B (en) | Multi-mode collaborative optimization method and system for video monitoring and remote sensing | |
| Xu et al. | UAV-based bridge geometric shape measurement using automatic bridge component detection and distributed multi-view reconstruction | |
| CN114299283B (en) | Image-based displacement measurement method and system | |
| CN112215308B (en) | Single-order detection method and device for hoisted object, electronic equipment and storage medium | |
| CN107255458B (en) | Resolving method of vertical projection grating measurement simulation system | |
| CN112884135A (en) | Data annotation correction method based on frame regression | |
| CN105894505A (en) | Quick pedestrian positioning method based on multi-camera geometrical constraint | |
| CN108592789A (en) | A kind of steel construction factory pre-assembly method based on BIM and machine vision technique | |
| CN110826501B (en) | Face key point detection method and system based on sparse key point calibration | |
| CN106874837A (en) | A kind of vehicle checking method based on Computer Vision | |
| Liu et al. | Visual odometry algorithm based on deep learning | |
| CN105447875A (en) | Automatic geometric correction method for electronic topographical map | |
| CN110599587A (en) | 3D scene reconstruction technology based on single image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |