CN112991376A - Equipment contour labeling method and system in infrared image - Google Patents
Equipment contour labeling method and system in infrared image Download PDFInfo
- Publication number
- CN112991376A CN112991376A CN202110369188.7A CN202110369188A CN112991376A CN 112991376 A CN112991376 A CN 112991376A CN 202110369188 A CN202110369188 A CN 202110369188A CN 112991376 A CN112991376 A CN 112991376A
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- offset
- infrared
- rotation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 47
- 239000011159 matrix material Substances 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 25
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000009529 body temperature measurement Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 239000004568 cement Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method and a system for labeling an equipment outline in an infrared image, wherein the method comprises the following steps: collecting a visible light image and an infrared image of the equipment in the current state; matching the visible light image of the equipment in the current state with the visible light template image to obtain a visible light image characteristic matching matrix; calculating a first offset and a second offset of a visible light image of the equipment in the current state; carrying out equipment contour labeling on a prestored infrared template image to obtain coordinates of a plurality of labeling points; and determining the coordinates of a plurality of points on the equipment outline in the infrared image of the equipment in the current state according to the coordinates of the plurality of marking points, the first offset, the second offset, the offset generated by the visible light image along with the rotation of the holder and the offset generated by the infrared image along with the rotation of the holder. According to the invention, the visible light image is used for assisting the infrared image to mark the outline of the equipment to be tested, so that the accuracy of the outline marking of the industrial field temperature measuring equipment is ensured.
Description
Technical Field
The invention relates to the technical field of image processing and machine vision, in particular to a method and a system for labeling equipment outlines in infrared images.
Background
In an industrial field, such as a power plant or a cement plant, there are a lot of equipments which need to measure temperature at any time, such as a motor or a cable joint, and the purpose of the equipment is to ensure that the equipment does not have various abnormalities due to over-high temperature, such as failure of normal operation, danger, fire hazard, even explosion, etc. At present, the two temperature measurement modes are mainly used, one mode is that a worker carries out on-site temperature measurement on equipment needing temperature measurement, whether the equipment normally runs or not is judged, because the industrial field generally has high temperature, high pressure, high noise and high radiation, the worker is in the severe environment for a long time, the body is greatly damaged, in order to avoid the exposure of the worker to a harmful environment, the other temperature measurement mode for carrying out temperature measurement through an infrared image temperature measurement device is adopted at present, in order to distinguish the equipment to be measured, the infrared image temperature measurement device is required to collect an infrared image for carrying out contour detection and drawing on the equipment to be measured.
The inventor finds that when the infrared image temperature measuring equipment detects the outline of the equipment to be detected, the outline of the equipment to be detected is not obvious in the image due to low infrared image resolution and the influence of the surrounding environment, and particularly when the temperature of the equipment to be detected is close to the temperature of the surrounding environment, the equipment to be detected displayed by the infrared image is fuzzy, so that the infrared image temperature measuring equipment cannot directly detect the equipment outline in the infrared image, the outline of the equipment to be detected cannot be drawn, and further the equipment to be detected cannot be correctly distinguished.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
Disclosure of Invention
The invention aims to provide a method and a system for marking an equipment outline in an infrared image, which are used for marking the outline of equipment to be tested by using a visible light image to assist a low-resolution infrared image, so that the accuracy of the outline marking of industrial field temperature measuring equipment is ensured.
In order to achieve the above object, the present invention provides a method for labeling an equipment outline in an infrared image, which comprises: the method comprises the following steps that a collecting device collects visible light images and infrared images of equipment in a current state, wherein a holder is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the holder; matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix; calculating a first shift amount of the visible light image of the device in the current state with respect to the visible light template image in a horizontal axis direction of an image coordinate system based on the visible light image feature matching matrix, and calculating a second shift amount of the visible light image of the device in the current state with respect to the visible light template image in a vertical axis direction of the image coordinate system based on the visible light image feature matching matrix; carrying out equipment contour labeling on a prestored infrared template image to obtain coordinates of a plurality of labeling points; and determining the coordinates of a plurality of points on the equipment outline in the infrared image of the equipment in the current state according to the relationship among the coordinates of a plurality of labeling points on the infrared template image, the first offset, the second offset, the offset generated by the rotation of a preset visible light image along with a holder and the offset generated by the rotation of an infrared image along with the holder, wherein the visible light image and the visible light template image are acquired by the acquisition device at the same position and the shooting multiples of the visible light camera are the same multiple, and the infrared image and the infrared template image are acquired by the acquisition device at the same position.
In an embodiment of the present invention, the apparatus contour labeling method further includes: the method comprises the following steps of predetermining the relationship between the offset of a visible light image generated along with the rotation of a holder and the offset of an infrared image generated along with the rotation of the holder, wherein the predetermining the relationship between the offset of the visible light image generated along with the rotation of the holder and the offset of the infrared image generated along with the rotation of the holder comprises the following steps: determining the offset of the visible light image in the horizontal axis direction and the offset in the longitudinal axis direction of the image coordinate system when the holder rotates once, and determining the offset of the infrared image in the horizontal axis direction and the offset in the longitudinal axis direction when the holder rotates once; determining a first ratio between the offset of the infrared image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system; and determining a second ratio between the offset of the infrared image of each degree of rotation of the holder in the longitudinal axis direction of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the longitudinal axis direction of the image coordinate system.
In an embodiment of the present invention, matching the visible light image of the device in the current state with the pre-stored visible light template image to obtain a visible light image feature matching matrix includes: and matching the visible light image of the equipment in the current state with the pre-stored visible light template image based on an accelerated robust feature algorithm to obtain a visible light image feature matching matrix.
In one embodiment of the present invention, calculating the first offset amount and calculating the second offset amount includes: acquiring the central coordinate of the visible light template image in an image coordinate system, and performing coordinate transformation on the central coordinate of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate; subtracting the coordinate value of the horizontal axis of the central coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and subtracting the coordinate value of the longitudinal axis of the central coordinate on the visible light template image from the coordinate value of the longitudinal axis of the transformed coordinate to obtain the second offset.
In one embodiment of the invention, the infrared template is used as a referenceDetermining the coordinates of a plurality of points on the device profile in the infrared image of the device in the current state according to the relationship among the coordinates of a plurality of labeling points on the image, the first offset, the second offset, the offset generated by the rotation of the cradle head of the preset visible light image and the offset generated by the rotation of the infrared image along with the cradle head, wherein the relationship comprises: and determining the coordinates of points on the device outline on the infrared image according to a first equation, wherein the first equation is as follows: d (X, Y) ═ M + Xsw×Off_Rel_X，N+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y is the abscissa and ordinate of the D point in the image coordinate system, respectively, M, N is the abscissa and ordinate of a certain annotation point in the image coordinate system, respectively, and X isswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
Based on the same inventive concept, the invention also provides an equipment contour labeling system in the infrared image, which comprises: the device comprises a collecting device, a matching module, an offset calculating module, a labeling module and a contour coordinate determining module. The cloud platform is arranged on the acquisition device, the visible light camera and the infrared camera are arranged on the cloud platform, and the acquisition device is used for acquiring visible light images and infrared images of the equipment in the current state. And the matching module is coupled with the acquisition device and is used for matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix. The offset calculating module is coupled to the matching module and configured to calculate a first offset of the visible light image of the device in the current state relative to the visible light template image in a horizontal axis direction of an image coordinate system based on the visible light image feature matching matrix, and the offset calculating module is further configured to calculate a second offset of the visible light image of the device in the current state relative to the visible light template image in a vertical axis direction of the image coordinate system based on the visible light image feature matching matrix. And the marking module is used for marking the contour of the equipment on the pre-stored infrared template image to obtain the coordinates of a plurality of marking points. The contour coordinate determination module is coupled with the labeling module and the offset calculation module, and is used for determining the coordinates of a plurality of points on the equipment contour in the infrared image of the equipment in the current state according to the relationship among the coordinates of a plurality of labeling points on the infrared template image, the first offset, the second offset, the offset generated by the rotation of a preset visible light image along with the holder and the offset generated by the rotation of the infrared image along with the holder. Wherein the visible light image and the visible light template image are both acquired by the acquisition device at the same position and the shooting multiples of the visible light camera are all the same, and the infrared image and the infrared template image are both acquired by the acquisition device at the same position.
In an embodiment of the present invention, the device contour labeling system further includes: an offset relation determining module, coupled to the contour coordinate determining module, for determining a relation between an offset of the visible light image generated along with the rotation of the pan/tilt and an offset of the infrared image generated along with the rotation of the pan/tilt, wherein the offset relation determining module is configured to determine an offset of the visible light image in a horizontal axis direction and an offset in a vertical axis direction of the image coordinate system for each degree of rotation of the pan/tilt, and determine an offset of the infrared image in the horizontal axis direction and an offset in the vertical axis direction of the image coordinate system for each degree of rotation of the pan/tilt; the offset relation determining module is used for determining a first ratio between the offset of the infrared image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system; the offset relation determining module is further configured to determine a second ratio between an offset of the infrared image of each degree of rotation of the pan/tilt in a longitudinal axis direction of the image coordinate system and an offset of the visible light image of each degree of rotation of the pan/tilt in the longitudinal axis direction of the image coordinate system.
In an embodiment of the present invention, the offset calculation module is configured to obtain a center coordinate of the visible light template image in an image coordinate system, and perform coordinate transformation on the center coordinate of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate; the offset calculation module is used for subtracting a horizontal axis coordinate value of a central coordinate on the visible light template image from a horizontal axis coordinate value of the transformed coordinate to obtain the first offset; the offset calculation module is further configured to subtract a longitudinal coordinate value of the central coordinate on the visible light template image from a longitudinal coordinate value of the transformed coordinate to obtain the second offset.
In an embodiment of the present invention, the contour coordinate determination module is configured to determine coordinates of a point on the contour of the device on the infrared image according to a first equation, where the first equation is: d (X, Y) ═ M + Xsw×Off_Rel_X，N+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y is the abscissa and ordinate of the D point in the image coordinate system, respectively, M, N is the abscissa and ordinate of a certain annotation point in the image coordinate system, respectively, and X isswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
Based on the same inventive concept, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the device contour labeling method in the infrared image according to any one of the above embodiments.
Compared with the prior art, according to the method and the system for labeling the equipment contour in the infrared image, in the process of labeling the equipment contour of the infrared image, the visible light image, the infrared image and the respective template images are collected at the same position and under the same visible light camera multiple by utilizing the collecting device carrying the visible light camera, the infrared camera and the holder, the equipment in the infrared image with low resolution is subjected to auxiliary labeling by utilizing the characteristics of relatively high resolution of the visible light image, the relation between the offset of the visible light image and the infrared image, the offset of the visible light image relative to the visible light template image in the current state, the relation between the collected infrared template image and the infrared image and other factors, the contour of temperature measuring equipment in the infrared image can be accurately labeled, the problem that when the resolution of the infrared image is low or the temperature of target equipment to be measured is close to the ambient temperature is effectively solved, the method and the system are suitable for various types of equipment needing temperature measurement in industrial fields, and have high universality and good robustness.
Drawings
FIG. 1 is a method for device contouring in infrared images in accordance with an embodiment of the present invention;
FIG. 2 is a system for device contouring in infrared images in accordance with an embodiment of the present invention;
FIG. 3 is a system for device contouring in infrared images in accordance with an embodiment of the present invention.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
In order to solve the problem that the existing industrial field infrared image temperature measuring equipment cannot accurately detect and draw the outline of the equipment to be detected due to low infrared image resolution and the influence of the surrounding environment, an embodiment provides an equipment outline labeling method in an infrared image, and as shown in fig. 1, the equipment outline labeling method comprises a step S1-a step S5. It should be noted that, in this specification, the visible light image and the visible light template image are both acquired by the acquisition device at the same position and the shooting multiples of the visible light camera are all the same multiples, and the infrared image and the infrared template image are both acquired by the acquisition device at the same position.
In step S1, a collecting device collects a visible light image and an infrared image of the device in a current state, wherein a cradle head is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the cradle head. Preferably, collection system can be the robot, can patrol and examine equipment at any time according to the demand.
In step S2, the visible light image of the device in the current state is matched with a pre-stored visible light template image of the device, so as to obtain a visible light image feature matching matrix.
Optionally, the visible light image of the device in the current state may be matched with the pre-stored visible light template image by using an accelerated robust feature algorithm, so as to obtain a 2 × 3 visible light image feature matching matrix
In step S3, a first shift amount of the visible light image of the device in the current state with respect to the visible light template image in the direction of the horizontal axis of the image coordinate system is calculated based on the visible light image feature matching matrix, and a second shift amount of the visible light image of the device in the current state with respect to the visible light template image in the direction of the vertical axis of the image coordinate system is calculated based on the visible light image feature matching matrix.
Alternatively, the method of calculating the first offset amount and the second offset amount in step S3 is as follows. Firstly, the central coordinates of the visible light template image in an image coordinate system are obtained
Wherein Width is the Width of the template image, and Height is the Height of the template image. Then, coordinate transformation is carried out on the center coordinate of the visible light template image under an image coordinate system based on the visible light image feature matching matrix to obtain the transformed center coordinateCoordinates of the object
Finally, subtracting the coordinate value of the horizontal axis of the central coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset
And subtracting the coordinate value of the longitudinal axis of the central coordinate on the visible light template image from the coordinate value of the longitudinal axis of the transformed coordinate to obtain the second offset
In step S4, device contour labeling is performed on the pre-stored ir template image to obtain coordinates of a plurality of labeling points. The shape of the outline label and the number of the label points can be determined according to the complexity of the outline. For example, quadrilaterals or other polygons may be labeled. In the present embodiment, a quadrangle is labeled according to the shape of the device, and four vertex coordinates, D respectively, are obtainedUpper left of(a，b)，DUpper right part(c，d)，DLeft lower part(e，f)，DLower right(g，h)。
In step S5, the coordinates of the plurality of points on the device outline in the infrared image of the device in the current state are determined according to the coordinates of the plurality of labeled points on the infrared template image, the first offset, the second offset, the offset of the preset visible light image generated by the rotation of the pan-tilt, and the offset of the infrared image generated by the rotation of the pan-tilt.
Specifically, the relationship between the amount of displacement of the visible light image due to the rotation of the pan/tilt head and the amount of displacement of the infrared image due to the rotation of the pan/tilt head is predetermined. In an alternative embodiment, determining the relationship between the offsets comprises: firstly, determining the offset of a visible light image in the horizontal axis direction and the offset of a visible light image in the longitudinal axis direction of an image coordinate system when the holder rotates once, and determining the offset of an infrared image in the horizontal axis direction and the offset of the infrared image in the longitudinal axis direction when the holder rotates once; then determining a first ratio between the offset of the infrared image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system; and determining a second ratio between the offset of the infrared image in the longitudinal axis direction of the image coordinate system every time the holder rotates once and the offset of the visible light image in the longitudinal axis direction of the image coordinate system every time the holder rotates once.
Preferably, when determining the offset of the holder rotating a visible light image or an infrared image every time, the holder may be rotated many times, and a plurality of groups of images are collected to calculate the offset. If the holder can be adjusted for 5 times, the visible light image and the infrared image are collected once after the holder rotates for 1 degree every time, the offset is calculated once, and the offset x of the 5 visible light images in the horizontal axis direction of the image coordinate system is obtained in totalvis1、xvis2、xvis3、xvis4、xvis5And the offset y of the 5 visible light images in the direction of the longitudinal axis of the image coordinate systemvis1、yvis2、yvis3、yvis4、yvis5And obtaining the offset x of the 5 infrared images in the horizontal axis direction of the image coordinate systeminf1、xinf2、xinf3、xinf4、xinf5And the offset y of the 5 infrared images in the direction of the longitudinal axis of the image coordinate systeminf1、yinf2、yinf3、yinf4、yinf5. Then calculating the offset of the visible light image in the horizontal axis direction of the image coordinate system when the holder rotates once
And calculating the head perThe shift amount of the one-degree rotation visible light image in the direction of the longitudinal axis of the image coordinate system is
Calculating the offset of the infrared image in the horizontal axis direction of the image coordinate system every rotation once
And calculating the offset of the infrared image in the longitudinal axis direction of the image coordinate system at every rotation of one degree as
Calculating the first ratio as Off _ Rel _ x ═ xinf_ave/xvis_aveAnd calculating the second ratio as Off _ Rel _ Y ═ Yinf_ave/yvis_ave。
Specifically, in step 5, the coordinates of a point on the device profile on the infrared image are determined according to a first equation: d (X, Y) ═ M + Xsw×Off_Rel_X，N+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y is the abscissa and ordinate of the D point in the image coordinate system, respectively, M, N is the abscissa and ordinate of a certain annotation point in the image coordinate system, respectively, and X isswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
Since four vertex coordinates, each D, are acquired in step S4Upper left of(a，b)，DUpper right part(c，d)，DLeft lower part(e，f)，DLower right(g, h). Coordinates of four points on the device contour on the infrared image can be obtained according to the first formula, the coordinates of the four points are coordinates of four vertexes on the device contour on the infrared image, and the coordinates are respectively DUpper left new＝(a+Xsw*Off_Rel_X，b+Ysh*Off_Rel_Y)；DUpper right new＝(c+Xsw*Off_Rel_X，d+Ysh*Off_Rel_Y)；DLower left new＝(e+Xsw*Off_Rel_X，f+Ysh*Off_Rel_Y)；DLower right new＝(g+Xsw*Off_Rel_X，h+YshOff _ Rel _ Y). And drawing the contour of the equipment in the infrared image according to the calculated coordinates of the four vertexes on the contour of the equipment in the infrared image.
Therefore, in the process of labeling the device profile of the infrared image in the embodiment, the visible light image, the infrared image and the template images thereof are collected at the same position and the same visible light camera multiple by using the collecting device carrying the visible light camera, the infrared camera and the holder, and the device in the infrared image with low resolution is labeled in an auxiliary way by using the characteristics of relatively high resolution of the visible light image, the relationship between the offset of the visible light image and the infrared image, the offset of the visible light image relative to the visible light template image in the current state, the relationship between the collected infrared template image and the infrared image and other factors, so that the profile of the temperature measuring device in the infrared image can be labeled accurately, the problem that when the resolution of the infrared image is low or the temperature of the target device to be measured is close to the ambient temperature is solved effectively, the method is suitable for various types of equipment needing temperature measurement in industrial fields, and has high universality and good robustness.
Based on the same inventive concept, an embodiment further provides a device contour labeling system in an infrared image, as shown in fig. 2, the device contour labeling system in the infrared image includes: the device comprises an acquisition device 10, a matching module 11, an offset calculation module 12, a labeling module 13 and a contour coordinate determination module 14.
The cloud deck is arranged on the acquisition device 10, the cloud deck is provided with a visible light camera and an infrared camera, and the acquisition device 10 is used for acquiring visible light images and infrared images of equipment in the current state. Alternatively, the harvesting device 10 may be a robot.
The matching module 11 is coupled to the acquisition device 10, and is configured to match the visible light image of the device in the current state with a pre-stored visible light template image of the device, so as to obtain a visible light image feature matching matrix. Optionally, an accelerated robust feature algorithm may be employed to match the visible light image of the device in the current state with the pre-stored visible light template image,
the offset calculating module 12 is coupled to the matching module 11, and configured to calculate a first offset of the visible light image of the device in the current state relative to the visible light template image in a horizontal axis direction of an image coordinate system based on the visible light image feature matching matrix, and the offset calculating module 12 is further configured to calculate a second offset of the visible light image of the device in the current state relative to the visible light template image in a vertical axis direction of the image coordinate system based on the visible light image feature matching matrix.
Optionally, the offset calculation module 12 calculates the first offset and the second offset as follows: acquiring the central coordinate of the visible light template image in an image coordinate system, and performing coordinate transformation on the central coordinate of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate; subtracting the coordinate value of the horizontal axis of the central coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and subtracting the coordinate value of the longitudinal axis of the central coordinate on the visible light template image from the coordinate value of the longitudinal axis of the transformed coordinate to obtain the second offset.
The labeling module 13 is configured to perform device contour labeling on a pre-stored infrared template image to obtain coordinates of a plurality of labeling points.
The contour coordinate determination module 14 is coupled to the labeling module 13 and the offset calculation module 12, and configured to determine coordinates of a plurality of points on the device contour in the infrared image of the device in the current state according to a relationship between coordinates of a plurality of labeled points on the infrared template image, the first offset, the second offset, an offset generated by a preset visible light image rotating with the pan/tilt head, and an offset generated by an infrared image rotating with the pan/tilt head.
Optionally, in an embodiment, the apparatus contour labeling system further includes: and an offset relation determination module 15, as shown in fig. 3. The offset relation determining module 15 is coupled to the contour coordinate determining module 14, and is configured to determine a relation between an offset generated by the visible light image when the pan/tilt head rotates and an offset generated by the infrared image when the pan/tilt head rotates, where the offset relation determining module 15 is specifically configured to determine an offset of the visible light image in a horizontal axis direction and an offset of the infrared image in a vertical axis direction of the image coordinate system when the pan/tilt head rotates once, and determine an offset of the infrared image in the horizontal axis direction and an offset of the infrared image in the vertical axis direction when the pan/tilt head rotates once; the offset relation determining module 15 is configured to determine a first ratio between an offset of the infrared image of each degree of rotation of the pan/tilt in a horizontal axis direction of the image coordinate system and an offset of the visible light image of each degree of rotation of the pan/tilt in the horizontal axis direction of the image coordinate system; the offset relation determining module 15 is further configured to determine a second ratio between an offset of the infrared image of each degree of rotation of the pan/tilt in the longitudinal axis direction of the image coordinate system and an offset of the visible light image of each degree of rotation of the pan/tilt in the longitudinal axis direction of the image coordinate system.
Specifically, the contour coordinate determination module 14 is further configured to determine coordinates of a point on the contour of the device on the infrared image according to a first equation: d (X, Y) ═ a + Xsw×Off_Rel_X，b+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y is the abscissa and ordinate of the D point in the image coordinate system, a, b are the abscissa and ordinate of an annotated point in the image coordinate system, XswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
Based on the same inventive concept, an embodiment further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the device contouring method according to any of the above embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.
Claims (10)
1. A method for labeling an equipment contour in an infrared image is characterized by comprising the following steps:
the method comprises the following steps that a collecting device collects visible light images and infrared images of equipment in a current state, wherein a holder is arranged on the collecting device, and a visible light camera and an infrared camera are arranged on the holder;
matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix;
calculating a first shift amount of the visible light image of the device in the current state with respect to the visible light template image in a horizontal axis direction of an image coordinate system based on the visible light image feature matching matrix, and calculating a second shift amount of the visible light image of the device in the current state with respect to the visible light template image in a vertical axis direction of the image coordinate system based on the visible light image feature matching matrix;
carrying out equipment contour labeling on a prestored infrared template image to obtain coordinates of a plurality of labeling points; and
determining the coordinates of a plurality of points on the equipment outline in the infrared image of the equipment in the current state according to the coordinates of a plurality of marking points on the infrared template image, the first offset, the second offset, the offset generated by the rotation of a preset visible light image along with the pan-tilt and the offset generated by the rotation of an infrared image along with the pan-tilt,
wherein the visible light image and the visible light template image are both acquired by the acquisition device at the same position and the shooting multiples of the visible light camera are all the same, and the infrared image and the infrared template image are both acquired by the acquisition device at the same position.
2. The device contour labeling method in an infrared image according to claim 1, characterized in that the device contour labeling method further comprises: the method comprises the following steps of predetermining the relationship between the offset of a visible light image generated along with the rotation of a holder and the offset of an infrared image generated along with the rotation of the holder, wherein the predetermining the relationship between the offset of the visible light image generated along with the rotation of the holder and the offset of the infrared image generated along with the rotation of the holder comprises the following steps:
determining the offset of the visible light image in the horizontal axis direction and the offset in the longitudinal axis direction of the image coordinate system when the holder rotates once, and determining the offset of the infrared image in the horizontal axis direction and the offset in the longitudinal axis direction when the holder rotates once;
determining a first ratio between the offset of the infrared image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system; and
and determining a second ratio between the offset of the infrared image of each degree of rotation of the holder in the longitudinal axis direction of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the longitudinal axis direction of the image coordinate system.
3. The method for labeling the device profile in the infrared image according to claim 1, wherein the matching the visible light image of the device in the current state with the pre-stored visible light template image to obtain the visible light image feature matching matrix comprises:
and matching the visible light image of the equipment in the current state with the pre-stored visible light template image based on an accelerated robust feature algorithm to obtain a visible light image feature matching matrix.
4. The method of claim 1, wherein calculating the first offset and calculating the second offset comprises:
acquiring the central coordinate of the visible light template image in an image coordinate system, and performing coordinate transformation on the central coordinate of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain a transformed coordinate;
subtracting the coordinate value of the horizontal axis of the central coordinate on the visible light template image from the coordinate value of the horizontal axis of the transformed coordinate to obtain the first offset; and
and subtracting the coordinate value of the longitudinal axis of the central coordinate on the visible light template image from the coordinate value of the longitudinal axis of the transformed coordinate to obtain the second offset.
5. The method for labeling the device contour in the infrared image according to claim 2, wherein determining the coordinates of the plurality of points on the device contour in the infrared image of the device in the current state according to the relationship among the coordinates of the plurality of labeling points on the infrared template image, the first offset, the second offset, the offset of the preset visible light image generated by the rotation of the pan-tilt and the offset of the infrared image generated by the rotation of the pan-tilt comprises:
and determining the coordinates of points on the device outline on the infrared image according to a first equation, wherein the first equation is as follows: d (X, Y) ═ M + Xsw×Off_Rel_X，N+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y are the abscissa and ordinate of the D point in the image coordinate system, M, M,N is respectively the abscissa and ordinate of a certain marking point in the image coordinate system, XswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
6. An apparatus contour labeling system in an infrared image, characterized in that the apparatus contour labeling system comprises:
the device comprises a collecting device, a control device and a display device, wherein a holder is arranged on the collecting device, a visible light camera and an infrared camera are arranged on the holder, and the collecting device is used for collecting a visible light image and an infrared image of the equipment in the current state;
the matching module is coupled with the acquisition device and used for matching the visible light image of the equipment in the current state with a pre-stored visible light template image of the equipment to obtain a visible light image characteristic matching matrix;
a shift amount calculation module, coupled to the matching module, for calculating a first shift amount of the visible light image of the device in the current state relative to the visible light template image in a horizontal axis direction of an image coordinate system based on the visible light image feature matching matrix, and calculating a second shift amount of the visible light image of the device in the current state relative to the visible light template image in a vertical axis direction of the image coordinate system based on the visible light image feature matching matrix;
the marking module is used for marking the outline of the equipment on the prestored infrared template image to obtain the coordinates of a plurality of marking points; and
a contour coordinate determination module, coupled to the labeling module and the offset calculation module, for determining coordinates of a plurality of points on the device contour in the infrared image of the device in the current state according to a relationship between the coordinates of the plurality of labeled points on the image of the infrared template, the first offset, the second offset, an offset generated by a preset visible light image along with the rotation of the pan-tilt, and an offset generated by an infrared image along with the rotation of the pan-tilt,
wherein the visible light image and the visible light template image are both acquired by the acquisition device at the same position and the shooting multiples of the visible light camera are all the same, and the infrared image and the infrared template image are both acquired by the acquisition device at the same position.
7. The device silhouette annotation system according to claim 6, wherein the device silhouette annotation system further comprises:
an offset relation determining module, coupled to the contour coordinate determining module, configured to determine a relation between an offset of the visible light image generated along with rotation of the pan/tilt and an offset of the infrared image generated along with rotation of the pan/tilt, where the offset relation determining module is configured to determine an offset of the visible light image in a horizontal axis direction and an offset in a vertical axis direction of the image coordinate system for each rotation of the pan/tilt, and determine an offset of the infrared image in the horizontal axis direction and an offset in the vertical axis direction of the image coordinate system for each rotation of the pan/tilt; the offset relation determining module is used for determining a first ratio between the offset of the infrared image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system and the offset of the visible light image of each degree of rotation of the holder in the direction of the transverse axis of the image coordinate system; the offset relation determining module is further configured to determine a second ratio between an offset of the infrared image of each degree of rotation of the pan/tilt in a longitudinal axis direction of the image coordinate system and an offset of the visible light image of each degree of rotation of the pan/tilt in the longitudinal axis direction of the image coordinate system.
8. The system for device contour labeling in infrared images of claim 6, wherein the offset calculation module is configured to obtain the center coordinates of the visible light template image in an image coordinate system, and perform coordinate transformation on the center coordinates of the visible light template image in the image coordinate system based on the visible light image feature matching matrix to obtain transformed coordinates; the offset calculation module is used for subtracting a horizontal axis coordinate value of a central coordinate on the visible light template image from a horizontal axis coordinate value of the transformed coordinate to obtain the first offset; the offset calculation module is further configured to subtract a longitudinal coordinate value of the central coordinate on the visible light template image from a longitudinal coordinate value of the transformed coordinate to obtain the second offset.
9. The system for labeling a device contour in an infrared image as claimed in claim 7, wherein said contour coordinates determination module is configured to determine coordinates of points on the device contour on the infrared image according to a first equation, wherein the first equation is: d (X, Y) ═ M + Xsw×Off_Rel_X，N+YshX Off _ Rel _ Y), where D is a point on the device outline on the infrared image, X, Y is the abscissa and ordinate of the D point in the image coordinate system, respectively, M, N is the abscissa and ordinate of a certain annotation point in the image coordinate system, respectively, and X isswIs the first offset, YshFor the second offset, Off _ Rel _ X is the first ratio and Off _ Rel _ Y is the second ratio.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for device contouring in infrared images of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369188.7A CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110369188.7A CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112991376A true CN112991376A (en) | 2021-06-18 |
| CN112991376B CN112991376B (en) | 2024-06-14 |
Family
ID=76339222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110369188.7A Active CN112991376B (en) | 2021-04-06 | 2021-04-06 | Equipment contour labeling method and system in infrared image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112991376B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115914792A (en) * | 2022-12-22 | 2023-04-04 | 长春理工大学 | Real-time multidimensional imaging self-adaptive adjustment system and method based on deep learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120070320A (en) * | 2010-12-21 | 2012-06-29 | 엘지디스플레이 주식회사 | Display system including stereo camera and position detecting method using the same |
| CN103902962A (en) * | 2012-12-28 | 2014-07-02 | 汉王科技股份有限公司 | Shielding or light source self-adaption human face recognition method and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
| CN109801336A (en) * | 2019-01-09 | 2019-05-24 | 南京理工大学 | Airborne target locating system and method based on visible light and infrared light vision |
| CN110570454A (en) * | 2019-07-19 | 2019-12-13 | 华瑞新智科技（北京）有限公司 | Method and device for detecting foreign matter invasion |
| CN111274940A (en) * | 2020-01-19 | 2020-06-12 | 厦门中控智慧信息技术有限公司 | Face recognition method, device, equipment and storage medium |
| CN111429518A (en) * | 2020-03-24 | 2020-07-17 | 浙江大华技术股份有限公司 | Labeling method, labeling device, computing equipment and storage medium |
| CN112200840A (en) * | 2020-10-27 | 2021-01-08 | 北京深睿博联科技有限责任公司 | Moving object detection system in visible light and infrared image combination |
-
2021
- 2021-04-06 CN CN202110369188.7A patent/CN112991376B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120070320A (en) * | 2010-12-21 | 2012-06-29 | 엘지디스플레이 주식회사 | Display system including stereo camera and position detecting method using the same |
| CN103902962A (en) * | 2012-12-28 | 2014-07-02 | 汉王科技股份有限公司 | Shielding or light source self-adaption human face recognition method and device |
| CN108010085A (en) * | 2017-11-30 | 2018-05-08 | 西南科技大学 | Target identification method based on binocular Visible Light Camera Yu thermal infrared camera |
| CN109801336A (en) * | 2019-01-09 | 2019-05-24 | 南京理工大学 | Airborne target locating system and method based on visible light and infrared light vision |
| CN110570454A (en) * | 2019-07-19 | 2019-12-13 | 华瑞新智科技（北京）有限公司 | Method and device for detecting foreign matter invasion |
| CN111274940A (en) * | 2020-01-19 | 2020-06-12 | 厦门中控智慧信息技术有限公司 | Face recognition method, device, equipment and storage medium |
| CN111429518A (en) * | 2020-03-24 | 2020-07-17 | 浙江大华技术股份有限公司 | Labeling method, labeling device, computing equipment and storage medium |
| CN112200840A (en) * | 2020-10-27 | 2021-01-08 | 北京深睿博联科技有限责任公司 | Moving object detection system in visible light and infrared image combination |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115914792A (en) * | 2022-12-22 | 2023-04-04 | 长春理工大学 | Real-time multidimensional imaging self-adaptive adjustment system and method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112991376B (en) | 2024-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9488589B2 (en) | Mapping damaged regions on objects | |
| US6690841B2 (en) | Method and apparatus for image registration | |
| JP5671281B2 (en) | Position / orientation measuring apparatus, control method and program for position / orientation measuring apparatus | |
| CN108662987B (en) | Calibration method of 2D camera type laser measuring head | |
| KR102016636B1 (en) | Calibration apparatus and method of camera and rader | |
| EP3514525B1 (en) | Interactive semi-automated borescope video analysis and damage assessment system and method of use | |
| Liu et al. | An improved online dimensional measurement method of large hot cylindrical forging | |
| JP2004127239A (en) | Method and system for calibrating multiple cameras using calibration object | |
| JP2020113268A (en) | Method for calculating tow hitch position | |
| Fawzy | Study the accuracy of digital close range photogrammetry technique software as a measuring tool | |
| Xia et al. | Global calibration of non-overlapping cameras: State of the art | |
| CN111627070B (en) | Method, device and storage medium for calibrating rotation shaft | |
| CN107367245B (en) | Invalid point detection and elimination method in optical three-dimensional profile measurement | |
| Kiddee et al. | A practical and intuitive calibration technique for cross-line structured light | |
| JP5976089B2 (en) | Position / orientation measuring apparatus, position / orientation measuring method, and program | |
| CN112991376A (en) | Equipment contour labeling method and system in infrared image | |
| JP2002132341A (en) | Field inspection device | |
| CN106123808B (en) | A method of it is measured for the deflection of automobile rearview mirror specular angle degree | |
| CN113983951B (en) | Three-dimensional target measuring method, device, imager and storage medium | |
| Usamentiaga | Easy rectification for infrared images | |
| CN106643566B (en) | A kind of method of automatic measurement wheel hub spoke initial angle | |
| CN112734842B (en) | Auxiliary positioning method and system for centering installation of large ship equipment | |
| JPH10318732A (en) | Shape measuring device and image formation apparatus of shape measurement | |
| Uyanik et al. | A method for determining 3D surface points of objects by a single camera and rotary stage | |
| CN116309606B (en) | Machine vision-based steel bar bending angle calibration method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |