CN115930813A - Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision - Google Patents
Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision Download PDFInfo
- Publication number
- CN115930813A CN115930813A CN202211520949.5A CN202211520949A CN115930813A CN 115930813 A CN115930813 A CN 115930813A CN 202211520949 A CN202211520949 A CN 202211520949A CN 115930813 A CN115930813 A CN 115930813A
- Authority
- CN
- China
- Prior art keywords
- target
- deformation
- result
- fitting
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000006073 displacement reaction Methods 0.000 title claims abstract description 38
- 238000001514 detection method Methods 0.000 claims abstract description 61
- 238000004590 computer program Methods 0.000 claims description 23
- 238000006243 chemical reaction Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 230000002776 aggregation Effects 0.000 claims description 8
- 238000004220 aggregation Methods 0.000 claims description 8
- 238000012806 monitoring device Methods 0.000 claims description 6
- 238000006116 polymerization reaction Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method and a device for monitoring micro-jitter deformation and displacement resistance based on machine vision. The method comprises the following steps: acquiring an image shot by a camera; carrying out target detection on the image to obtain a target initial position and multi-scale features; determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features; judging whether the target of the image is lost or not according to the fitting result and the regression result; and if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target. By implementing the method provided by the embodiment of the invention, the target can be quickly found back when the camera shakes, the deformation of the target is continuously detected, and the accuracy and the stability of deformation monitoring are improved.
Description
Technical Field
The invention relates to a machine vision algorithm, in particular to a method and a device for monitoring micro-jitter deformation and displacement resistance based on machine vision.
Background
With the development of machine vision algorithm technology, more and more researchers apply the machine vision algorithm technology to safety monitoring, in particular to a non-contact deformation monitoring direction based on machine vision. The currently common monitoring technology is a characteristic point matching or template matching algorithm based on a template, namely a natural target or an artificial target, and the technical process is as follows: manually calibrating a target; calculating a current scale factor; carrying out feature point matching/template matching to calculate the pixel deformation; conversion into actual deformation
Existing monitoring techniques often need to be based on the following assumptions: the focal length of the camera does not change during operation; the camera is stationary. However, in an actual outdoor monitoring scene, the above assumptions are often not realized, especially if the camera is not moved. Therefore, the deformation monitoring technology based on manual calibration has the advantages that the monitoring visual field is small, and only the deformation in the calibrated ROI area can be monitored; if the camera moves in a shaking way, sinking and the like, the target is likely to be lost; the automatic calibration function is not available, and the intelligent degree is not enough.
Therefore, it is necessary to design a new method for rapidly retrieving the target and continuously detecting the deformation of the target when the camera shakes, so as to improve the accuracy and stability of deformation monitoring.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a device for monitoring micro-jitter deformation and displacement resistance based on machine vision.
In order to achieve the purpose, the invention adopts the following technical scheme: the micro-jitter deformation and displacement resisting monitoring method based on machine vision comprises the following steps:
acquiring an image shot by a camera;
carrying out target detection on the image to obtain a target initial position and multi-scale features;
determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features;
judging whether the target of the image is lost or not according to the fitting result and the regression result;
and if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target.
The further technical scheme is as follows: the target detection of the image to obtain the initial position of the target and the multi-scale features comprises:
and carrying out target detection on the image by adopting a YOLOv7 model to obtain the initial position of the target and multi-scale features.
The further technical scheme is as follows: the determining a fitting result according to the target initial position and determining a regression result according to the multi-scale features comprises:
performing position regression on the multi-scale features to obtain a regression result;
and performing geometric center fitting on the initial position of the target, and calculating a scale factor to obtain a fitting result.
The further technical scheme is as follows: performing position regression on the multi-scale features to obtain a regression result, wherein the position regression includes:
carrying out non-maximum threshold processing on the detection frames with the confidence degrees larger than a set value in the regressed target frames in the multi-scale features to obtain three optimal detection frames;
and calculating the average value of the three optimal detection boxes to obtain a regression result.
The further technical scheme is as follows: the sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitoring target comprises the following steps:
performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value;
and performing scale conversion by using the final pixel deformation value to obtain a real physical deformation of the monitoring target.
The further technical scheme is as follows: performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value, including:
calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result;
and calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result to obtain a final pixel deformation value.
The further technical scheme is as follows: the performing scale conversion by using the final pixel deformation value to obtain a real physical deformation amount of the monitoring target includes:
and multiplying the final pixel deformation value by the scale factor to obtain the real physical deformation of the monitoring target.
The invention also provides a micro-jitter deformation and displacement resisting monitoring device based on machine vision, which comprises:
an image acquisition unit for acquiring an image taken by a camera;
the target detection unit is used for carrying out target detection on the image to obtain a target initial position and multi-scale features;
the result determining unit is used for determining a fitting result according to the initial position of the target and determining a regression result according to the multi-scale features;
the judging unit is used for judging whether the target of the image is lost or not according to the fitting result and the regression result;
and the sub-pixel fitting unit is used for performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target if the target of the image is not lost.
The invention also provides computer equipment which comprises a memory and a processor, wherein the memory is stored with a computer program, and the processor realizes the method when executing the computer program.
The invention also provides a storage medium storing a computer program which, when executed by a processor, implements the method described above.
Compared with the prior art, the invention has the beneficial effects that: according to the method, the target detection is carried out on the image, the geometric center fitting is carried out on the initial position of the target, the position regression is carried out on the multi-scale features, the sub-pixel fitting effect is achieved in the mode of averaging the matching results and the geometric center fitting under different scales of the feature pyramid, the purpose that when a camera shakes is achieved, the target can be found back quickly, the deformation of the target is detected continuously, and the precision and the stability of deformation monitoring are improved.
The invention is further described below with reference to the accompanying drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a micro-jitter deformation and displacement resisting monitoring method based on machine vision according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a micro-jitter deformation and displacement resisting monitoring method based on machine vision according to an embodiment of the present invention;
FIG. 3 is a schematic view of a sub-flow of a method for monitoring micro-jitter deformation and displacement based on machine vision according to an embodiment of the present invention;
FIG. 4 is a schematic view of a sub-flow of a method for monitoring micro-jitter deformation and displacement based on machine vision according to an embodiment of the present invention;
FIG. 5 is a schematic view of a sub-flow of a method for monitoring micro-jitter deformation and displacement based on machine vision according to an embodiment of the present invention;
FIG. 6 is a schematic view of a sub-flow of a method for monitoring deformation and displacement of anti-micro-jitter based on machine vision according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of a micro-jitter deformation and displacement resisting monitoring device based on machine vision according to an embodiment of the present invention;
FIG. 8 is a schematic block diagram of a computer apparatus provided by an embodiment of the present invention;
fig. 9 is a schematic diagram of an example of monitoring micro-jitter deformation and displacement resistance based on machine vision according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a micro-jitter deformation and displacement resisting monitoring method based on machine vision according to an embodiment of the present invention. Fig. 2 is a schematic flowchart of a method for monitoring micro-jitter deformation and displacement resistance based on machine vision according to an embodiment of the present invention. The micro-jitter deformation and displacement resisting monitoring method based on machine vision is applied to a server. The server performs data interaction with the camera, can quickly retrieve the target when the camera shakes, continues to detect the deformation of the target, and improves the accuracy and stability of deformation monitoring. The end-to-end training can be carried out, the efficiency is high, and the fastest speed can reach 150FPS; when the monitoring device generates certain jitter and deviation to cause the loss of a monitoring target, the monitoring device can be quickly retrieved and continuously monitored; the deep learning algorithm technology and the traditional image processing algorithm technology are combined, so that the whole monitoring system is more stable and is easy to migrate.
Fig. 2 is a schematic flow chart of a method for monitoring micro-jitter deformation and displacement resistance based on machine vision according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S150.
And S110, acquiring an image shot by the camera.
In the present embodiment, the image refers to a visual image captured with a camera.
And S120, carrying out target detection on the image to obtain a target initial position and multi-scale features.
In this embodiment, the target initial position refers to position information of the target after the target is identified for the image; the multi-scale features refer to feature maps of three different scales output by a target detection model.
Specifically, a YOLOv7 model is adopted to perform target detection on the image so as to obtain a target initial position and multi-scale features.
In this embodiment, labeling a target in an image through labelImg labeling software to obtain a sample picture; building a YOLOv7 algorithm model based on a Pythrch deep learning framework; and carrying out target detection on the target image by using a YOLOv7 algorithm model.
Specifically, a target image with a size of 640 × 640 is input into the YOLOv7 model, and feature maps of 1/8,1/16, and 1/32 of the original image and the initial position of the target are sequentially acquired.
And S130, determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features.
In this embodiment, the fitting result refers to a result formed by performing geometric center fitting on the initial position of the target; the regression result refers to a result formed after position regression is carried out on the multi-scale features.
In an embodiment, referring to fig. 3, the step S130 may include steps S131 to S132.
S131, carrying out position regression on the multi-scale features to obtain a regression result.
In an embodiment, referring to fig. 4, the step S131 may include steps S1311 to S1312.
S1311, performing non-maximum threshold processing on the detection frames with the confidence degrees larger than a set value in the regression target frames in the multi-scale features to obtain three optimal detection frames;
and S1312, calculating the average value of the three optimal detection frames to obtain a regression result.
Specifically, the upper left coordinate and the lower right coordinate of the optimal detection frame obtained by the characteristics of three scales (1/8, 1/16 and 1/32) are recorded as a vector V i ；V i ＝(x 1 ,y 1 ,x 2 ,y 1 )i＝{1/8,1/16,1/32}；
Specifically, a non-maximum threshold (NMS) is carried out on detection frames with reliability greater than 0.6 in target frames regressed by 1/8,1/16 and 1/32 feature maps with different scales, optimal detection frames are respectively obtained, the optimal detection frames are averaged, and the sub-pixel effect is achieved.
S132, performing geometric center fitting on the initial position of the target, and calculating a scale factor to obtain a fitting result.
In this embodiment, the fitting of the geometric center is performed on the circular figure in the final target frame output by the detection model, and the scale factor is calculated.
And S140, judging whether the target of the image is lost or not according to the fitting result and the regression result.
Whether the target is lost or not is judged mainly by two ways, firstly, the target detection branch has two outputs, one is the sub-pixel position information of the monitored target under three different scale characteristics, and the initial position (ROI area) of the target is determined based on the sub-pixel position information. After the initial position is obtained, the system stores an ROI image (init _ ROI _ img) of a target locally, when a new picture is obtained, the hash similarity hash _ value of the target image corresponding to the current picture and the init _ ROI _ img is calculated, and the system performs ellipse fitting in an ROI area determined by target detection to obtain the circle center coordinate, the long axis, the short axis length, a and b of an ellipse. Fitting coefficient E = a/b; s = (hash _ value + E)/2, where S is equal to or greater than 0.9 indicates that the target is not lost, otherwise indicates that the target is lost.
And S150, if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target.
In the present embodiment, the actual physical deformation amount of the monitoring target refers to the actual physical deformation amount of the target.
In an embodiment, referring to fig. 5, the step S150 may include steps S151 to S152.
And S151, performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value.
In this embodiment, the final pixel deformation value is a result of performing sub-pixel aggregation on the fitting result and the regression result.
In an embodiment, referring to fig. 6, the step S151 may include steps S1511 to S1512.
S1511, calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result;
the pixel deformation amount is calculated in two parts, one part is a regression result based on target detection, and the other part is a circle center coordinate result based on ellipse fitting of the ROI area. Firstly, when the system is started, initialization is carried out to obtain the following initial quantities of the monitoring target: the initial coordinate of the circle center (init _ center), and the coordinates of the upper left and lower right of the minimum external moment of the target. When monitoring is carried out, the multi-scale target contour coordinate output by the target detection algorithm module and the circle center coordinate obtained by ellipse fitting are respectively subtracted from the initial circle center coordinate of the initial target external rectangular coordinate to obtain the pixel deformation quantity corresponding to the initial target external rectangular coordinate and the circle center coordinate.
S1512, calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result to obtain a final pixel deformation value.
And S152, performing scale conversion by using the final pixel deformation value to obtain a real physical deformation of the monitored target.
Specifically, the final pixel deformation value is multiplied by the scale factor to obtain the real physical deformation of the monitoring target.
In this embodiment, the pixel deformation values of the fitting result and the regression result are calculated respectively and averaged to be the final pixel deformation value; and finally multiplying the pixel deformation value by a scale factor to obtain the real physical deformation of the monitored target.
If the target of the image is lost, the step S120 is executed.
The method of the embodiment mainly achieves micro-deformation monitoring and target tracking through target detection, geometric figure fitting and image similarity calculation, wherein the target tracking means that when a target jumps out of a search area (ROI area) due to camera shake, a target detection module can make judgment by combining fitting coefficients and hash similarity evaluation so as to update the ROI area, and the effect of not losing the target is achieved.
For example: as shown in fig. 9, when the camera does not shake much (initial state), the algorithm performs full-image target detection on the current frame image according to the calibrated ROI area, and performs ellipse fitting on the current frame image and the ROI area to obtain the coordinates of the center of circle. When the camera shakes to cause the monitored target to jump out of the ROI area and cause the target to be lost, the system triggers the target detection module to obtain the latest position of the target, and frame expansion is carried out according to the detection frame (the expansion scale is 1/3 of the length w of the detection frame). And updating the original ROI area frame by the obtained expansion frame. As shown in "shake shift state" in fig. 9, the rectangular frame is an updated ROI region, the circle deviating from the camera lens is the position of the original target circle, the circle corresponding to the camera lens is the circle fitted to the current frame, and the small rectangular frame is the target detection frame of the current target frame.
According to the micro-jitter deformation and displacement resistance monitoring method based on machine vision, the target detection is carried out on the image, the geometric center fitting is carried out on the initial position of the target, the position regression is carried out on the multi-scale features, the effect of sub-pixel fitting is achieved in the mode of averaging the matching results and the geometric center fitting under different scales of the feature pyramid, the purpose that when a camera shakes is achieved, the target can be quickly found back, the deformation of the target continues to be detected, and the precision and the stability of deformation monitoring are improved.
Fig. 7 is a schematic block diagram of a device 300 for monitoring deformation and displacement of a micro-jitter resistance based on machine vision according to an embodiment of the present invention. As shown in fig. 7, the invention further provides a device 300 for monitoring micro-shaking resistant deformation and displacement based on machine vision, which corresponds to the above method for monitoring micro-shaking resistant deformation and displacement based on machine vision. The device 300 for monitoring the deformation and displacement of the machine vision-based object includes a unit for executing the method for monitoring the deformation and displacement of the machine vision-based object, and the device can be configured in a server. Specifically, referring to fig. 7, the device 300 for monitoring micro-jitter deformation and displacement based on machine vision includes an image obtaining unit 301, a target detecting unit 302, a result determining unit 303, a determining unit 304, and a sub-pixel fitting unit 305.
An image acquisition unit 301 for acquiring an image taken by a camera; a target detection unit 302, configured to perform target detection on the image to obtain a target initial position and a multi-scale feature; a result determining unit 303, configured to determine a fitting result according to the target initial position, and determine a regression result according to the multi-scale feature; a determining unit 304, configured to determine whether the target of the image is lost according to the fitting result and the regression result; a sub-pixel fitting unit 305, configured to, if the target of the image is not lost, perform sub-pixel fitting according to the fitting result and the regression result to obtain a true physical deformation amount of the monitored target.
In an embodiment, the target detection unit 302 is configured to perform target detection on the image by using a YOLOv7 model to obtain an initial position of the target and a multi-scale feature.
In one embodiment, the result determination unit 303 includes a position regression subunit and a geometric fitting subunit.
The position regression subunit is used for carrying out position regression on the multi-scale features to obtain a regression result; and the geometric fitting subunit is used for performing geometric center fitting on the initial position of the target and calculating a scale factor to obtain a fitting result.
In one embodiment, the position regression subunit includes a processing module and an average calculation module.
The processing module is used for carrying out non-maximum threshold processing on the detection frames with the confidence degrees being larger than the set value in the regression target frames in the multi-scale features so as to obtain three optimal detection frames; and the average value calculating module is used for calculating the average value of the three optimal detection frames to obtain a regression result.
In one embodiment, the sub-pixel fitting unit 305 includes an aggregation sub-unit and a scaling sub-unit.
The aggregation subunit is used for performing sub-pixel aggregation according to the fitting result and the regression result to obtain a final pixel deformation value; and the scale conversion subunit is used for performing scale conversion by using the final pixel deformation value to obtain the real physical deformation of the monitored target.
In an embodiment, the aggregation subunit includes a deformation amount calculation module and a final deformation value calculation module.
The deformation quantity calculation module is used for calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result; and the final deformation value calculation module is used for calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result so as to obtain a final pixel deformation value.
In an embodiment, the scaling subunit is configured to multiply the scaling factor by the final pixel deformation value to obtain a true physical deformation amount of the monitoring target.
It should be noted that, as can be clearly understood by those skilled in the art, the detailed implementation process of the micro-jitter deformation resistance and displacement monitoring device 300 based on machine vision and each unit may refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, no further description is provided herein.
The above-mentioned machine vision-based micro-jitter deformation resistant, displacement monitoring apparatus 300 may be implemented in the form of a computer program, which may be run on a computer device as shown in fig. 8.
Referring to fig. 8, fig. 8 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 8, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer programs 5032 include program instructions that, when executed, cause the processor 502 to perform a machine vision-based anti-micro-jitter deformation, displacement monitoring method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the computer program 5032 in the non-volatile storage medium 503 to run, and when the computer program 5032 is executed by the processor 502, the processor 502 can be enabled to execute a micro-jitter deformation and displacement resisting monitoring method based on machine vision.
The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the configuration shown in fig. 8 is a block diagram of only a portion of the configuration relevant to the present teachings and does not constitute a limitation on the computer device 500 to which the present teachings may be applied, and that a particular computer device 500 may include more or less components than those shown, or combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:
acquiring an image shot by a camera; carrying out target detection on the image to obtain a target initial position and multi-scale features; determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features; judging whether the target of the image is lost or not according to the fitting result and the regression result; and if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target.
In an embodiment, when implementing the step of performing target detection on the image to obtain the initial position of the target and the multi-scale features, the processor 502 specifically implements the following steps:
and carrying out target detection on the image by adopting a YOLOv7 model to obtain the initial position of the target and the multi-scale features.
In an embodiment, when the step of determining the fitting result according to the initial position of the target and determining the regression result according to the multi-scale feature is implemented by the processor 502, the following steps are specifically implemented:
performing position regression on the multi-scale features to obtain a regression result; and performing geometric center fitting on the initial position of the target, and calculating a scale factor to obtain a fitting result.
In an embodiment, when implementing the step of performing position regression on the multi-scale features to obtain a regression result, the processor 502 specifically implements the following steps:
carrying out non-maximum threshold processing on the detection frames with the confidence degrees larger than a set value in the regressed target frames in the multi-scale features to obtain three optimal detection frames; and calculating the average value of the three optimal detection boxes to obtain a regression result.
In an embodiment, when implementing the step of performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation amount of the monitoring target, the processor 502 specifically implements the following steps:
performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value; and carrying out scale conversion by using the final pixel deformation value to obtain the real physical deformation of the monitored target.
In an embodiment, when implementing the step of performing sub-pixel aggregation according to the fitting result and the regression result to obtain the final pixel deformation value, the processor 502 specifically implements the following steps:
calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result; and calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result to obtain a final pixel deformation value.
In an embodiment, when the processor 502 implements the step of performing the scale conversion by using the final pixel deformation value to obtain the real physical deformation amount of the monitoring target, the following steps are specifically implemented:
and multiplying the final pixel deformation value by the scale factor to obtain the real physical deformation of the monitoring target.
It should be understood that in the embodiment of the present Application, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will be understood by those skilled in the art that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program instructing associated hardware. The computer program includes program instructions, and the computer program may be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the steps of:
acquiring an image shot by a camera; carrying out target detection on the image to obtain a target initial position and multi-scale features; determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features; judging whether the target of the image is lost or not according to the fitting result and the regression result; and if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target.
In an embodiment, when the processor executes the computer program to implement the step of performing the target detection on the image to obtain the initial position of the target and the multi-scale features, the processor specifically implements the following steps:
and carrying out target detection on the image by adopting a YOLOv7 model to obtain the initial position of the target and the multi-scale features.
In an embodiment, when the step of determining the fitting result according to the initial position of the target and determining the regression result according to the multi-scale features is implemented by the processor executing the computer program, the following steps are specifically implemented:
performing position regression on the multi-scale features to obtain a regression result; and performing geometric center fitting on the initial position of the target, and calculating a scale factor to obtain a fitting result.
In an embodiment, when the processor executes the computer program to implement the step of performing position regression on the multi-scale features to obtain a regression result, the following steps are specifically implemented:
carrying out non-maximum threshold processing on the detection frames with the confidence degrees larger than a set value in the regressed target frames in the multi-scale features to obtain three optimal detection frames; and calculating the average value of the three optimal detection boxes to obtain a regression result.
In an embodiment, when the processor executes the computer program to implement the step of performing sub-pixel fitting according to the fitting result and the regression result to obtain the true physical deformation amount of the monitoring target, the following steps are specifically implemented:
performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value; and carrying out scale conversion by using the final pixel deformation value to obtain the real physical deformation of the monitored target.
In an embodiment, when the processor executes the computer program to implement the step of performing sub-pixel aggregation according to the fitting result and the regression result to obtain the final pixel deformation value, the following steps are specifically implemented:
calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result; and calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result to obtain a final pixel deformation value.
In an embodiment, when the processor executes the computer program to implement the step of performing the scaling with the final pixel deformation value to obtain the real physical deformation amount of the monitoring target, the following steps are specifically implemented:
and multiplying the final pixel deformation value by the scale factor to obtain the real physical deformation of the monitoring target.
The storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, which can store various computer readable storage media.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A micro-jitter deformation and displacement resisting monitoring method based on machine vision is characterized by comprising the following steps:
acquiring an image shot by a camera;
carrying out target detection on the image to obtain a target initial position and multi-scale features;
determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale features;
judging whether the target of the image is lost or not according to the fitting result and the regression result;
and if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target.
2. The machine vision-based micro-jitter deformation and displacement resisting monitoring method according to claim 1, wherein the performing target detection on the image to obtain a target initial position and multi-scale features comprises:
and carrying out target detection on the image by adopting a YOLOv7 model to obtain the initial position of the target and multi-scale features.
3. A machine vision-based method for monitoring anti-micro-jitter deformation and displacement according to claim 1, wherein the determining a fitting result according to the target initial position and determining a regression result according to the multi-scale features comprises:
performing position regression on the multi-scale features to obtain a regression result;
and performing geometric center fitting on the initial position of the target, and calculating a scale factor to obtain a fitting result.
4. The machine vision-based micro-jitter deformation and displacement resistant monitoring method according to claim 3, wherein the performing position regression on the multi-scale features to obtain a regression result comprises:
carrying out non-maximum threshold processing on the detection frames with the confidence degrees larger than a set value in the regressed target frames in the multi-scale features to obtain three optimal detection frames;
and calculating the average value of the three optimal detection boxes to obtain a regression result.
5. The machine vision-based micro-jitter deformation and displacement resistant monitoring method according to claim 3, wherein the performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation quantity of the monitored target comprises:
performing sub-pixel polymerization according to the fitting result and the regression result to obtain a final pixel deformation value;
and performing scale conversion by using the final pixel deformation value to obtain a real physical deformation of the monitoring target.
6. The machine vision-based micro-jitter deformation and displacement monitoring method according to claim 5, wherein the performing sub-pixel aggregation according to the fitting result and the regression result to obtain a final pixel deformation value comprises:
calculating the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result;
and calculating the average value of the pixel deformation quantity of the fitting result and the pixel deformation quantity of the regression result to obtain a final pixel deformation value.
7. The machine vision-based micro-jitter deformation and displacement resistant monitoring method according to claim 5, wherein the performing the scale conversion by using the final pixel deformation value to obtain the real physical deformation quantity of the monitored target comprises:
and multiplying the final pixel deformation value by the scale factor to obtain the real physical deformation of the monitoring target.
8. Machine vision-based micro-jitter-resistant deformation and displacement monitoring device, which is characterized by comprising:
an image acquisition unit for acquiring an image taken by a camera;
the target detection unit is used for carrying out target detection on the image to obtain a target initial position and multi-scale features;
the result determining unit is used for determining a fitting result according to the initial position of the target and determining a regression result according to the multi-scale features;
the judging unit is used for judging whether the target of the image is lost or not according to the fitting result and the regression result;
and the sub-pixel fitting unit is used for performing sub-pixel fitting according to the fitting result and the regression result to obtain the real physical deformation of the monitored target if the target of the image is not lost.
9. A computer device, characterized in that the computer device comprises a memory, on which a computer program is stored, and a processor, which when executing the computer program implements the method according to any of claims 1 to 7.
10. A storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211520949.5A CN115930813A (en) | 2022-11-30 | 2022-11-30 | Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211520949.5A CN115930813A (en) | 2022-11-30 | 2022-11-30 | Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115930813A true CN115930813A (en) | 2023-04-07 |
Family
ID=86648366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211520949.5A Pending CN115930813A (en) | 2022-11-30 | 2022-11-30 | Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115930813A (en) |
-
2022
- 2022-11-30 CN CN202211520949.5A patent/CN115930813A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021174939A1 (en) | Facial image acquisition method and system | |
| KR102320649B1 (en) | Methods and devices for determining facial image quality, electronic devices and computer storage media | |
| US11830216B2 (en) | Information processing apparatus, information processing method, and storage medium | |
| US20210227132A1 (en) | Method for tracking target in panoramic video, and panoramic camera | |
| US20150103184A1 (en) | Method and system for visual tracking of a subject for automatic metering using a mobile device | |
| CN110660090B (en) | Subject detection method and apparatus, electronic device, and computer-readable storage medium | |
| CN108253596B (en) | Air supply method of air conditioner, air conditioner and computer readable storage medium | |
| WO2021027325A1 (en) | Video similarity acquisition method and apparatus, computer device and storage medium | |
| JP7479324B2 (en) | Information processing device, information processing method, and program | |
| CN107578424B (en) | Dynamic background difference detection method, system and device based on space-time classification | |
| WO2020014864A1 (en) | Pose determination method and device, and computer readable storage medium | |
| WO2021084972A1 (en) | Object tracking device and object tracking method | |
| CN114445482A (en) | Method and system for detecting target in image based on Libra-RCNN and elliptical shape characteristics | |
| CN110111341B (en) | Image foreground obtaining method, device and equipment | |
| CN113781511B (en) | Conveyor belt edge wear detection method, conveyor belt edge wear detection device, computer equipment and storage medium | |
| WO2020019239A1 (en) | Positioning method and device, terminal and readable storage medium | |
| CN115930813A (en) | Micro-jitter deformation and displacement resisting monitoring method and device based on machine vision | |
| JP7243372B2 (en) | Object tracking device and object tracking method | |
| CN111901499A (en) | Method and equipment for calculating actual pixel distance in video image | |
| CN111612831A (en) | Depth estimation method and device, electronic equipment and storage medium | |
| CN116430069A (en) | Machine vision fluid flow velocity measuring method, device, computer equipment and storage medium | |
| WO2022174603A1 (en) | Pose prediction method, pose prediction apparatus, and robot | |
| CN111738085B (en) | System construction method and device for realizing automatic driving simultaneous positioning and mapping | |
| CN114638921A (en) | Motion capture method, terminal device, and storage medium | |
| CN110796680B (en) | Target tracking method and device based on similar template updating |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Yang Ping Inventor after: Hu Hui Inventor after: Song Jie Inventor after: Zhang Liang Inventor after: Li Xianhong Inventor after: He Yifan Inventor before: Li Xianhong Inventor before: He Yifan Inventor before: Yang Ping Inventor before: Hu Hui Inventor before: Song Jie |