CN112017218B - Image registration method and device, electronic equipment and storage medium - Google Patents

Image registration method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112017218B
CN112017218B CN202010941346.7A CN202010941346A CN112017218B CN 112017218 B CN112017218 B CN 112017218B CN 202010941346 A CN202010941346 A CN 202010941346A CN 112017218 B CN112017218 B CN 112017218B
Authority
CN
China
Prior art keywords
registered
value
image
pixel point
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010941346.7A
Other languages
Chinese (zh)
Other versions
CN112017218A (en
Inventor
方吉庆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN202010941346.7A priority Critical patent/CN112017218B/en
Publication of CN112017218A publication Critical patent/CN112017218A/en
Application granted granted Critical
Publication of CN112017218B publication Critical patent/CN112017218B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10048Infrared image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The image registration method, the device, the electronic equipment and the storage medium provided by the embodiment of the invention can acquire the image set to be registered; performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images; detecting connected domains of each binarized image to obtain one or more connected domains; detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain; and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image. Therefore, the images to be registered in the image set to be registered can be registered and fused according to the coordinates of at least four points of the targets to be registered of the plurality of images to be registered, so that the fused images with higher definition can be obtained, and the details in the images can be conveniently identified.

Description

Image registration method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of information technologies, and in particular, to an image registration method, an image registration device, an electronic device, and a storage medium.
Background
At present, image acquisition and snapshot by a camera are widely applied. For example, the traffic camera is used for capturing the vehicle in the running process, so that the monitoring and recording of the illegal behaviors are facilitated.
However, when the image is collected and captured by the traffic camera, the conditions of unclear capture, blurred pictures and the like often occur due to the captured object, weather and the like, so that difficulty is caused in identifying the captured pictures.
Disclosure of Invention
The embodiment of the invention aims to provide an image registration method, an image registration device, electronic equipment and a storage medium, so as to generate a picture image with higher definition through a plurality of pictures. The specific technical scheme is as follows:
in a first aspect of an embodiment of the present application, there is provided an image registration method, including:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images;
detecting connected domains of each binarized image to obtain one or more connected domains;
Detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image.
Optionally, binarizing each image to be registered in the image set to be registered to obtain a plurality of binarized images, including:
Carrying out graying treatment on each image to be registered in the image set to be registered to obtain a plurality of gray images;
respectively calculating the gray threshold value of each gray level image by using the maximum inter-class difference method OTSU;
for each gray image, setting pixels in the gray image with gray values greater than the gray image gray threshold as a first gray value and pixels in the gray image with gray values not greater than the gray image gray threshold as a second gray value, thereby obtaining a plurality of binarized images.
Optionally, performing connected domain detection on each binarized image to obtain one or more connected domains, including:
detecting connected domains of the binarized images, and marking a numerical value for each pixel point in the binarized images, wherein the numerical values of the pixel points of the same connected domain are equal;
Re-marking the numerical value of each pixel point in the binarized image through a preset rule to obtain the re-marked numerical value of each pixel point;
and dividing the pixel points with the same value into the same connected domain according to the re-marked value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
For any pixel point, the value of any pixel point is a first value, and the values of a plurality of pixel points adjacent to any pixel point are second values;
When the first value is equal to the second value, the value of any pixel point is not re-marked;
and when the first value is not equal to the second value, the value of any pixel point is re-marked as the second value.
Optionally, the preset rule includes:
for any pixel point, the value of any pixel point is a first value, and when the values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
Selecting the value with the smallest value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target value;
the value of any pixel is marked as a target value.
Optionally, detecting the target to be registered for each connected domain to obtain a position coordinate of the target to be registered for each connected domain, including:
identifying the target to be registered in each connected domain to obtain a binary image of the target to be registered in each binary image;
And projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered of each connected domain.
Optionally, based on coordinates of at least four points of the target to be registered, registering and fusing each image to be registered in the image set to be registered to obtain a fused image, including:
Calculating to obtain homography matrixes among the images to be registered based on coordinates of at least four points of the target to be registered;
Obtaining a mapping relation between the images to be registered according to the homography matrix between the images to be registered;
And registering and fusing the images to be registered in the image set to be registered according to the mapping relation between the images to be registered, so as to obtain fused images.
In a second aspect of an embodiment of the present application, there is provided an image registration apparatus including:
the image acquisition module is used for acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
The binarization processing module is used for carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images;
the connected domain detection module is used for carrying out connected domain detection on each binarized image to obtain one or more connected domains;
The target detection module is used for detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
the image fusion module is used for registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, and obtaining a fused image.
Optionally, the binarization processing module includes:
Carrying out graying treatment on each image to be registered in the image set to be registered to obtain a plurality of gray images;
respectively calculating the gray threshold value of each gray level image by using the maximum inter-class difference method OTSU;
for each gray image, setting pixels in the gray image with gray values greater than the gray image gray threshold as a first gray value and pixels in the gray image with gray values not greater than the gray image gray threshold as a second gray value, thereby obtaining a plurality of binarized images.
Optionally, the connected domain detection module includes:
the numerical value marking sub-module is used for carrying out connected domain detection on each binarized image, marking a numerical value for each pixel point in the binarized image, wherein the numerical values of the pixel points of the same connected domain are equal;
the re-marking sub-module is used for re-marking the numerical value of each pixel point in the binarized image through a preset rule to obtain the re-marked numerical value of each pixel point;
And the connected domain dividing submodule is used for dividing the pixel points with the same value into the same connected domain according to the re-marked value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
For any pixel point, the value of any pixel point is a first value, and the values of a plurality of pixel points adjacent to any pixel point are second values;
When the first value is equal to the second value, the value of any pixel point is not re-marked;
and when the first value is not equal to the second value, the value of any pixel point is re-marked as the second value.
Optionally, the preset rule includes:
for any pixel point, the value of any pixel point is a first value, and when the values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
Selecting the value with the smallest value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target value;
the value of any pixel is marked as a target value.
Optionally, the target detection module includes:
The target recognition sub-module is used for recognizing the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image;
The coordinate acquisition sub-module is used for projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered of each connected domain.
Optionally, the image fusion module includes:
The matrix acquisition sub-module is used for calculating a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
The relationship acquisition sub-module is used for acquiring the mapping relationship between the images to be registered according to the homography matrix between the images to be registered;
And the image fusion sub-module is used for registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered, so as to obtain fused images.
The embodiment of the application also provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface, and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
A processor for implementing the image registration method of any one of claims 1-7 when executing a program stored on a memory.
Embodiments of the present application also provide a computer readable storage medium having a computer program stored therein, which when executed by a processor, implements the image registration method of any of claims 1-7.
Embodiments of the present invention also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform any of the above described image registration methods.
The embodiment of the invention has the beneficial effects that:
The image registration method, the device, the electronic equipment and the storage medium provided by the embodiment of the invention can acquire the image set to be registered; performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images; detecting connected domains of each binarized image to obtain one or more connected domains; detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain; and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image. Therefore, the images to be registered in the image set to be registered can be registered and fused according to the coordinates of at least four points of the targets to be registered of the plurality of images to be registered, so that the fused images with higher definition can be obtained, and the details in the images can be conveniently identified.
Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an image registration method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating binarization processing of each image to be registered according to an embodiment of the present application;
FIG. 3 is a flowchart of connected domain detection according to an embodiment of the present application;
FIG. 4 is a flow chart of detection of an object to be registered according to an embodiment of the present application;
FIG. 5 is a flowchart of registering and fusing images to be registered according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an image registration apparatus according to an embodiment of the present application;
fig. 7 is an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
First, technical terms related to the present application will be explained:
and (3) communicating domain: that is, the connected Region (Connected Component) generally refers to an image Region composed of foreground pixels having the same pixel value and adjacent positions in the image, and the corresponding english is Region or Blob.
Image registration: and matching or superposing two or more images acquired at different times, with different equipment or under different conditions (pose, climate, angle and the like).
Projection histogram: the projection histogram is a graph obtained by plotting the relative frequency of different levels of a variable by rectangular blocks, and in the patent of the invention, the gradient image is mainly used for making the projection histogram in the horizontal direction and the vertical direction.
In a first aspect of an embodiment of the present application, there is provided an image registration method, including:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images;
detecting connected domains of each binarized image to obtain one or more connected domains;
Detecting the target to be registered of each connected domain to obtain the position coordinates of the target to be registered of each connected domain, wherein the position coordinates of the target to be registered comprise the coordinates of at least four points in the target to be registered;
and registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image.
Therefore, by the image registration method, the image with higher definition after fusion can be obtained by acquiring the coordinates of at least four points of the target to be registered of the plurality of images to be registered and registering and fusing each image to be registered in the image set to be registered according to the coordinates of the at least four points, so that details in the image can be conveniently identified.
The following detailed description first describes an application scenario of an embodiment of the present application. When a plurality of pictures are acquired for a certain target, problems such as unclear details in the image may occur. In this case, the obtained multiple pictures can be registered and fused to obtain a fused image, and then the details in the image are identified according to the fused image. For example:
When the traffic camera shoots a plurality of visible light or infrared light pictures of a certain vehicle and the details in the images of a driver or a license plate and the like are required to be identified. Two visible light images or one visible light image, one infrared light image and the like can be selected, the selected images are registered and fused, an image with higher definition is obtained, and details in the images of a driver or a license plate and the like are identified according to the fused image.
Referring to fig. 1, fig. 1 is a flowchart of an image registration method according to an embodiment of the present application, including:
Step S11, acquiring an image set to be registered.
The image set to be registered comprises at least two images to be registered, each image to be registered comprises the same target to be registered, and the images to be registered can be visible light images, infrared light images and the like. For example, when a traffic camera is used for capturing images of a certain automobile driver, at least two images of the front automobile window are continuously captured, and in the images of the front automobile window, the target to be registered can be the front automobile window. For another example, when the front license plate and the rear license plate of a certain automobile are captured through the traffic camera, at least two pictures aiming at the front license plate and the rear license plate of the automobile are continuously captured, and the target to be registered is the front license plate and the rear license plate of the automobile.
The embodiment of the application aims at the images to be registered in the intelligent terminal, so that the images to be registered can be executed by the intelligent terminal, and particularly, the intelligent terminal can be an intelligent video camera, a hard disk video recorder, a computer or a server and the like.
Step S12, binarizing each image to be registered in the image set to be registered to obtain a plurality of binarized images.
The binarizing process may be performed on the image to be registered, where the image to be registered is first gray-scaled, each pixel point in the image to be registered is set to a certain value between 0 and 255, then the gray threshold of the image to be registered is calculated by assuming that the image is divided into a background and a foreground, and the value of each pixel point in the image to be registered is re-marked by the gray threshold. For example, pixels below the gray threshold may be marked 0 and pixels above the gray threshold may be marked 255. The specific binarization method can be referred to as a binarization method in the related art, for example, OTSU (maximum difference between classes) binarization.
And step S13, detecting the connected domain of each binarized image to obtain one or more connected domains.
And detecting the connected domain of each binarized image. For example, a pixel point larger than a gray threshold in the binary image may be marked as 1, and a pixel point smaller than the gray threshold may be marked as 0 by a two-pass scanning method, so that a value is marked for each pixel point in the binary image, and a connected domain is detected according to a relationship between the marked value and the pixel point, thereby obtaining one or more connected domains in the binary image.
And S14, detecting the target to be registered for each connected domain to obtain the position coordinates of the target to be registered of each connected domain.
The position coordinates of the target to be registered comprise coordinates of at least four points in the target to be registered. The detection of the target to be registered is performed on each connected domain, and the detection of the target to be registered may be performed on each connected domain according to the shape of the target to be registered, so as to obtain images of the target to be registered in one or more connected domains, and further obtain position coordinates of at least four points in the images of the target to be registered.
For example, when a plurality of pictures of a front window of a certain automobile are registered by a traffic camera, the target to be registered is the front window of the automobile, and the coordinates of the target to be registered may be the coordinates of four vertexes of the front window of the automobile. For another example, when the pictures of the front license plate and the rear license plate of the automobile shot by the traffic camera are aligned, the target to be aligned is the front license plate and the rear license plate of the automobile, and the coordinates of the target to be aligned can be the coordinates of four vertexes of the front license plate and the rear license plate of the automobile.
And step S15, registering and fusing each image to be registered in the image set to be registered based on the coordinates of at least four points of the target to be registered, so as to obtain a fused image.
Based on the coordinates of at least four points of the target to be registered, registering and fusing each image to be registered in the image set to be registered, wherein each image to be registered in the image set to be registered can be registered according to the corresponding relation of the coordinates of at least four points of each image to be registered in the plurality of images to be registered. For example, the coordinates of four points of the target to be registered include coordinates of four points of the target to be registered, and then the four points of the other image to be registered are registered with the four points of the reference image to be registered respectively by taking the four points of the target to be registered as the reference, and the fused images are obtained by fusing the registered images.
For example, according to the coordinates of four vertexes of the front window of the automobile, registering and fusing the images of the front windows of the automobile to obtain the fused image of the front window of the automobile with higher definition. For another example, the front license plate and the rear license plate of the automobile are fused according to the coordinates of the four vertexes of the front license plate and the rear license plate of the automobile, and the fused pictures of the front license plate and the rear license plate of the automobile with higher definition are obtained.
Therefore, by the image registration method, the image with higher definition after fusion can be obtained by acquiring the coordinates of at least four points of the target to be registered of the plurality of images to be registered and registering and fusing each image to be registered in the image set to be registered according to the coordinates of the at least four points, so that details in the image can be conveniently identified.
Optionally, referring to fig. 2, step S12 performs binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images, including:
Step S121, performing graying processing on each image to be registered in the image set to be registered, to obtain a plurality of gray images.
The graying processing of the images to be registered can be achieved in various modes, for example, gamma correction can be performed on each image to be registered in the image set to be registered, and gray images corresponding to each image to be registered are obtained, so that a plurality of gray images are obtained. The graying process may set each pixel point in the image to be registered to a value between 0 and 255.
In step S122, the gray threshold of each gray image is calculated by OTSU algorithm.
The assumption of the OTSU algorithm is that there is a threshold Th that divides all pixels of the image into two classes C1 (less than Th) and C2 (greater than Th), and the respective average of these two classes of pixels is m1, m2, and the global average of the image is mgs. While the probability of a pixel being classified into C1 and C2 is p1, p2, respectively. Thus there is:
p1*m1+p2*m2=mG
p1+p2=1
The expression of class variance is:
σ2=p1(m1-mG)2+p2(m2-mG)2
Thereby obtaining the following steps:
σ2=p1*p2(m1-m2)2
when the variance of the above formula is maximized, the obtained mean mG is the threshold value TH, and the formula is as follows:
Wherein L is the total gray level number, and i is the number of pixel points.
Step S123, for each gray-scale image, sets a pixel in the gray-scale image having a gray-scale value greater than the gray-scale threshold of the gray-scale image as a first gray-scale value, and sets a pixel in the gray-scale image having a gray-scale value not greater than the gray-scale threshold of the gray-scale image as a second gray-scale value, thereby obtaining a plurality of binarized images.
Pixels in the grayscale image having a grayscale value greater than the grayscale image grayscale threshold are set to a first grayscale value, e.g., 255, and pixels in the grayscale image having a grayscale value not greater than the grayscale image grayscale threshold are set to a second grayscale value, e.g., 0. Thereby facilitating the examination of the connected domain for the binarized image.
Optionally, referring to fig. 3, step S13 performs connected domain detection on each binarized image to obtain one or more connected domains, including:
step S131, connected domain detection is performed on each binarized image, and a numerical value is marked for each pixel point in the binarized image.
For example, the binarized image is scanned, a label value is set for each effective pixel, for example, a pixel greater than the gray threshold in the binarized image is marked as 1, and a pixel less than the gray threshold is marked as 0.
Or by the following rules:
1) For a certain pixel point, when lable of the left adjacent pixel point and the upper adjacent pixel point of the pixel are invalid values, namely lable is not set, setting the pixel point as a new label value, label++;
2) For a certain pixel point, when one of the left adjacent pixel point or the upper adjacent pixel point of the pixel point is an effective value, the label value of the pixel is set to be the same as the label of the pixel point of the effective value;
3) For a certain pixel, when the left adjacent pixel and the upper adjacent pixel of the pixel are both valid values, the label value of the pixel is set to be the same as the smaller label value.
In step S132, the numerical value of each pixel in the binarized image is re-marked by a preset rule, so as to obtain the re-marked numerical value of each pixel.
In the actual use process, the numerical value of each pixel point in the binarized image is re-marked by a preset rule, which can be that the label belonging to the same communication area but with different values is marked as the same leble, namely the equal relation between the label and the label is recorded.
In step S133, according to the re-marked values of the pixel points, the pixel points with the same values are divided into the same connected domains, so as to obtain one or more connected domains.
In the actual use process, according to the re-marked numerical value of each pixel point, the pixel points with the same numerical value are divided into the same connected areas, for example, the recorded pixel points with the equal relation can be classified into one connected area and the same label can be given.
Therefore, by the connected domain detection method provided by the embodiment of the application, not only can the connected domain detection be performed on each binarized image, but also the connected domain can be re-divided by the pixel points belonging to the same connected domain, so that one or more connected domains can be obtained.
Optionally, the preset rule includes: for any pixel point, the value of any pixel point is a first value, and the values of a plurality of pixel points adjacent to any pixel point are second values; when the first value is equal to the second value, the value of any pixel point is not re-marked; and when the first value is not equal to the second value, the value of any pixel point is re-marked as the second value.
Optionally, the preset rule includes: for any pixel point, the value of any pixel point is a first value, and when the values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point; selecting the value with the smallest value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target value; the value of any pixel is marked as a target value.
For example, an image is scanned by the method described above:
(1) First scanning:
Accessing the current pixel B (x, y), if B (x, y) = 1:
If the pixel values in the B (x, y) field are all 0, a new label is given to B (x, y):
label+=1,B(x,y)=label;
If there are pixels Neighbors in the field of B (x, y) with pixel values > 1:
1) The minimum in Neighbors is given to B (x, y):
B(x,y)=min{Neighbors}
2) Recording the equality relation between all values (label) in Neighbors, namely that the values (label) belong to the same communication area;
labelSet [ i ] = { label_m, ], label_n }, labelSet [ i ] all belong to the same connected region (note: there can be many implementations as long as the relationship between these labels with equal relationship can be recorded
(2) Second scan:
accessing the current pixel B (x, y), if B (x, y) >1:
Finding a minimum label value in the equivalent relationship with label=b (x, y) and assigning the minimum label value to B (x, y);
after the scanning is completed, pixels with the same label value in the image form the same connected region.
Therefore, through the preset rule of the embodiment of the application, the connected domain to which the pixel point belongs can be judged, and the pixel points belonging to the same connected domain are subjected to the repartition of the connected domain, so that one or more connected domains are obtained.
Optionally, referring to fig. 4, step S14 detects the target to be registered for each connected domain, to obtain the position coordinates of the target to be registered for each connected domain, including:
Step S141, identifying the target to be registered for each connected domain, and obtaining a binary image of the target to be registered in each binary image.
The recognition of the target to be registered is performed on each connected domain, and the recognition can be performed according to the shape and the like of the target to be registered, so as to obtain a binary image of the target to be registered in each binary image. For example, when the object to be registered is a front window of an automobile, the connected domain with the trapezoid shape in each connected domain can be identified to obtain a binary image corresponding to the front window of the automobile, and for example, when the object to be registered is a front license plate and a rear license plate of the automobile, the connected domain with the rectangle shape in each connected domain can be identified to obtain a binary image corresponding to the license plate of the automobile.
Step S142, projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered of each connected domain.
And projecting the binary image of the target to be registered in each image to be registered, carrying out horizontal histogram projection on the binary image of the target to be registered to obtain a coordinate in the vertical direction, and carrying out statistics on the binary image to obtain a horizontal coordinate.
For example, the position coordinates of the target to be registered include coordinates of four points, and the target to be registered is an automobile window. And after obtaining the horizontal projection histogram, counting the transformation value of w/2 columns of pixels, wherein the positions where the two jumps occur are the upper and lower coordinates y1 and y2 of the window, counting the change rule of pixel values at (y1+y2)/2 rows in the connected domain binary image, and obtaining the x coordinates, x1 and x2 of the left and right contours of the window according to the change of the pixel values from white to black. Four vertex coordinates P1 (x 1, y 1), P2 (x 1, y 2), P3 (x 2, y 1) and P4 (x 4, y 4) of the plane in which the window lies are obtained from these four coordinates.
Therefore, by the method for detecting the target to be registered for each connected domain, the target to be registered for each connected domain can be identified and projected, and the position coordinates of the target to be registered for each connected domain can be obtained, so that the registration and fusion of the images to be registered can be conveniently carried out according to the position coordinates of the target to be registered, and the fused images can be obtained.
Optionally, referring to fig. 5, step S15 registers and fuses each image to be registered in the image set to be registered based on coordinates of at least four points of the target to be registered, to obtain a fused image, including:
step S151, calculating to obtain a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered.
For example, the target to be registered is an automobile window, the image set to be registered comprises two photos of the same automobile, 4 vertex coordinates P1-P4 of an automobile window area are obtained, and then a homography matrix H which is mapped to the next frame in the previous frame is obtained through calculation according to the fact that a certain point coordinate in four points of the previous frame is (u f,vf) and a certain point coordinate in four points of the next frame is (u s,vs);
step S152, according to the homography matrix between the images to be registered, the mapping relation between the images to be registered is obtained.
According to the above example, the object to be registered is an automobile window, and the two images are mapped according to the homography matrix H between the images to be registeredHas the following components
And step 153, registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered, so as to obtain fused images.
Therefore, the homography matrix among the images to be registered is obtained through calculation based on the coordinates of at least four points of the target to be registered, so that the mapping relation among the images to be registered is obtained, and according to the mapping relation among the images to be registered, the images to be registered in the image set to be registered are registered and fused, so that the images to be registered can be registered and fused, an image with higher definition than the images to be registered can be obtained conveniently, and details in the images to be registered can be identified conveniently.
Referring to fig. 6, in a second aspect of the embodiment of the present application, there is provided an image registration apparatus including:
An image acquisition module 601, configured to acquire an image set to be registered, where the image set to be registered includes at least two images to be registered, and each image to be registered includes a same target to be registered;
the binarization processing module 602 is configured to perform binarization processing on each image to be registered in the image set to be registered, so as to obtain a plurality of binarized images;
a connected domain detection module 603, configured to perform connected domain detection on each binarized image, so as to obtain one or more connected domains;
The target detection module 604 is configured to detect a target to be registered for each connected domain, so as to obtain a position coordinate of the target to be registered for each connected domain, where the position coordinate of the target to be registered includes coordinates of at least four points in the target to be registered;
The image fusion module 605 is configured to register and fuse each image to be registered in the image set to be registered based on coordinates of at least four points of the target to be registered, so as to obtain a fused image.
Optionally, the binarization processing module 602 includes:
Carrying out graying treatment on each image to be registered in the image set to be registered to obtain a plurality of gray images;
respectively calculating the gray threshold value of each gray level image by using the maximum inter-class difference method OTSU;
for each gray image, setting pixels in the gray image with gray values greater than the gray image gray threshold as a first gray value and pixels in the gray image with gray values not greater than the gray image gray threshold as a second gray value, thereby obtaining a plurality of binarized images.
Optionally, the connected domain detecting module 603 includes:
the numerical value marking sub-module is used for carrying out connected domain detection on each binarized image, marking a numerical value for each pixel point in the binarized image, wherein the numerical values of the pixel points of the same connected domain are equal;
the re-marking sub-module is used for re-marking the numerical value of each pixel point in the binarized image through a preset rule to obtain the re-marked numerical value of each pixel point;
And the connected domain dividing submodule is used for dividing the pixel points with the same value into the same connected domain according to the re-marked value of each pixel point to obtain one or more connected domains.
Optionally, the preset rule includes:
For any pixel point, the value of any pixel point is a first value, and the values of a plurality of pixel points adjacent to any pixel point are second values;
When the first value is equal to the second value, the value of any pixel point is not re-marked;
and when the first value is not equal to the second value, the value of any pixel point is re-marked as the second value.
Optionally, the preset rule includes:
for any pixel point, the value of any pixel point is a first value, and when the values of all the pixel points are not completely the same in a plurality of pixel points adjacent to any pixel point;
Selecting the value with the smallest value of each pixel point from a plurality of adjacent pixel points of any pixel point as a target value;
the value of any pixel is marked as a target value.
Optionally, the target detection module 604 includes:
The target recognition sub-module is used for recognizing the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image;
The coordinate acquisition sub-module is used for projecting the binary image of the target to be registered in each image to be registered to obtain the position coordinates of the target to be registered of each connected domain.
Optionally, the image fusion module 605 includes:
The matrix acquisition sub-module is used for calculating a homography matrix between the images to be registered based on the coordinates of at least four points of the target to be registered;
The relationship acquisition sub-module is used for acquiring the mapping relationship between the images to be registered according to the homography matrix between the images to be registered;
And the image fusion sub-module is used for registering and fusing the images to be registered in the image set to be registered according to the mapping relation among the images to be registered, so as to obtain fused images.
Therefore, by the image registration device provided by the embodiment of the application, the images to be registered in the image set to be registered can be registered and fused by acquiring the coordinates of at least four points of the targets to be registered of the plurality of images to be registered and according to the coordinates of the at least four points, so that the fused images with higher definition can be obtained, and the details in the images can be conveniently identified.
The embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702, and the memory 703 perform communication with each other through the communication bus 704,
A memory 703 for storing a computer program;
the processor 701 is configured to implement any of the above-described image registration methods when executing the program stored in the memory 703.
The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (DIGITAL SIGNAL Processing, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
In yet another embodiment of the present invention, a computer readable storage medium is also provided, in which a computer program is stored, which when executed by a processor, implements the steps of any of the image registration methods described above.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the image registration method of any of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device, since it is substantially similar to the method embodiment, the description is relatively simple, and reference is made to the part of the description of the method embodiment for relevant points.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (5)

1. A method of image registration, comprising:
acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
performing binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images;
detecting connected domains of the binarized images, and marking a numerical value for each pixel point in the binarized images, wherein the numerical values of the pixel points of the same connected domain are equal; re-marking the numerical value of each pixel point in the binarized image through a preset rule to obtain the re-marked numerical value of each pixel point; dividing the pixel points with the same value into the same connected domain according to the re-marked value of each pixel point to obtain one or more connected domains;
Identifying the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image; performing horizontal histogram projection on the binary image of the target to be registered in each image to be registered to obtain a horizontal projection histogram of the target to be registered of each connected domain; counting the transformation value of w/2 columns of pixels in the horizontal projection histogram, and determining the position where the jump occurs twice as the upper and lower coordinates y1 and y2 of the target to be registered; counting the transformation rule of pixel values of (y1+y2)/2 rows of pixels in the horizontal projection histogram, and obtaining left and right coordinates x1 and x2 of the target to be registered according to the change from white to black of the pixel values; obtaining coordinates of four points of the target to be registered according to the upper and lower coordinates y1 and y2 and the left and right coordinates x1 and x2;
Calculating to obtain homography matrixes among the images to be registered based on coordinates of four points of the target to be registered; obtaining a mapping relation between the images to be registered according to the homography matrix between the images to be registered; registering and fusing the images to be registered in the image set to be registered according to the mapping relation between the images to be registered to obtain fused images;
The detecting the connected domain of each binarized image, marking a numerical value for each pixel point in the binarized image, includes:
for a certain pixel point, when the values of the left adjacent pixel point and the upper adjacent pixel point of the pixel point are invalid values, setting the value of the pixel point as a new value;
For a certain pixel point, when the value of the left adjacent pixel point of the pixel point is an effective value or the value of the upper adjacent pixel point is an effective value, setting the value of the pixel point as a value corresponding to the effective value;
for a certain pixel point, when the values of the left adjacent pixel point and the upper adjacent pixel point of the pixel point are effective values, setting the value of the pixel point as a value with smaller value in the effective values;
Wherein, the preset rule includes:
for any pixel point, the value of the any pixel point is a first value, and the values of a plurality of pixel points adjacent to the any pixel point are second values;
when the first value is equal to the second value, the value of any pixel is not re-marked;
When the first value is not equal to the second value, the value of any pixel point is re-marked as the second value;
For any pixel point, the value of the any pixel point is a first value, and when the values of all the pixel points are not identical in a plurality of pixel points adjacent to the any pixel point;
selecting the value with the smallest value of each pixel point from the plurality of adjacent pixel points of any pixel point as a target value;
And marking the numerical value of any pixel point as the target numerical value.
2. The method according to claim 1, wherein binarizing each image to be registered in the set of images to be registered to obtain a plurality of binarized images, comprises:
Carrying out graying treatment on each image to be registered in the image set to be registered to obtain a plurality of gray images;
respectively calculating the gray threshold value of each gray level image by using a maximum inter-class difference method OTSU;
for each gray image, setting pixels in the gray image with gray values greater than the gray image gray threshold as a first gray value and pixels in the gray image with gray values not greater than the gray image gray threshold as a second gray value, thereby obtaining a plurality of binarized images.
3. An image registration apparatus, comprising:
The image acquisition module is used for acquiring an image set to be registered, wherein the image set to be registered comprises at least two images to be registered, and each image to be registered comprises the same target to be registered;
The binarization processing module is used for carrying out binarization processing on each image to be registered in the image set to be registered to obtain a plurality of binarized images;
The connected domain detection module is used for carrying out connected domain detection on each binarized image, and marking a numerical value for each pixel point in the binarized image, wherein the numerical values of the pixel points of the same connected domain are equal; re-marking the numerical value of each pixel point in the binarized image through a preset rule to obtain the re-marked numerical value of each pixel point; dividing the pixel points with the same value into the same connected domain according to the re-marked value of each pixel point to obtain one or more connected domains;
The target detection module is used for identifying the target to be registered for each connected domain to obtain a binary image of the target to be registered in each binary image; performing horizontal histogram projection on the binary image of the target to be registered in each binary image to obtain a horizontal projection histogram of the target to be registered of each connected domain; counting the transformation value of w/2 columns of pixels in the horizontal projection histogram, and determining the position where the jump occurs twice as the upper and lower coordinates y1 and y2 of the target to be registered; counting the transformation rule of pixel values of (y1+y2)/2 rows of pixels in the horizontal projection histogram, and obtaining left and right coordinates x1 and x2 of the target to be registered according to the change from white to black of the pixel values; obtaining coordinates of four points of the target to be registered according to the upper and lower coordinates y1 and y2 and the left and right coordinates x1 and x2;
The image fusion module is used for calculating a homography matrix between the images to be registered based on the coordinates of the four points of the target to be registered; obtaining a mapping relation between the images to be registered according to the homography matrix between the images to be registered; registering and fusing the images to be registered in the image set to be registered according to the mapping relation between the images to be registered to obtain fused images;
The connected domain detection module is specifically configured to set, for a certain pixel, a value of a pixel to be a new value when values of a left adjacent pixel and an upper adjacent pixel of the pixel are both invalid values; for a certain pixel point, when the value of the left adjacent pixel point of the pixel point is an effective value or the value of the upper adjacent pixel point is an effective value, setting the value of the pixel point as a value corresponding to the effective value; for a certain pixel point, when the values of the left adjacent pixel point and the upper adjacent pixel point of the pixel point are effective values, setting the value of the pixel point as a value with smaller value in the effective values;
Wherein, the preset rule includes:
for any pixel point, the value of the any pixel point is a first value, and the values of a plurality of pixel points adjacent to the any pixel point are second values;
when the first value is equal to the second value, the value of any pixel is not re-marked;
When the first value is not equal to the second value, the value of any pixel point is re-marked as the second value;
For any pixel point, the value of the any pixel point is a first value, and when the values of all the pixel points are not identical in a plurality of pixel points adjacent to the any pixel point;
selecting the value with the smallest value of each pixel point from the plurality of adjacent pixel points of any pixel point as a target value;
And marking the numerical value of any pixel point as the target numerical value.
4. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for implementing the method steps of any of claims 1-2 when executing a program stored on a memory.
5. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-2.
CN202010941346.7A 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium Active CN112017218B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010941346.7A CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010941346.7A CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112017218A CN112017218A (en) 2020-12-01
CN112017218B true CN112017218B (en) 2024-08-02

Family

ID=73522131

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010941346.7A Active CN112017218B (en) 2020-09-09 2020-09-09 Image registration method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112017218B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112668676A (en) * 2020-12-12 2021-04-16 浙江大华技术股份有限公司 Package security inspection image processing method, electronic device and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108345888A (en) * 2018-02-11 2018-07-31 浙江华睿科技有限公司 A kind of connected domain extracting method and device
CN111028189A (en) * 2019-12-09 2020-04-17 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106097344A (en) * 2016-06-15 2016-11-09 武汉理工大学 A kind of image processing method detecting geometric form impurity in rubber for tire and system
CN107833238B (en) * 2017-11-14 2020-05-01 京东方科技集团股份有限公司 Maximum connected domain marking method, target tracking method and augmented reality/virtual reality device
CN108510455B (en) * 2018-03-27 2020-07-17 长春理工大学 Laser irradiator image fusion method and system
CN109166077B (en) * 2018-08-17 2023-04-18 广州视源电子科技股份有限公司 Image alignment method and device, readable storage medium and computer equipment
CN109191415B (en) * 2018-08-22 2020-12-15 成都纵横自动化技术股份有限公司 Image fusion method and device and electronic equipment
CN109509200B (en) * 2018-12-26 2023-09-29 深圳市繁维医疗科技有限公司 Checkerboard corner detection method based on contour extraction and computer readable storage medium
WO2020198997A1 (en) * 2019-03-29 2020-10-08 深圳市大疆创新科技有限公司 Connected domain processing method, data processing device and computer readable storage medium
CN110189368B (en) * 2019-05-31 2023-09-19 努比亚技术有限公司 Image registration method, mobile terminal and computer readable storage medium
CN110971841B (en) * 2019-12-09 2022-07-15 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108345888A (en) * 2018-02-11 2018-07-31 浙江华睿科技有限公司 A kind of connected domain extracting method and device
CN111028189A (en) * 2019-12-09 2020-04-17 Oppo广东移动通信有限公司 Image processing method, image processing device, storage medium and electronic equipment

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于图像配准的车标识别算法研究;高梦珠;《中国优秀硕士学位论文全文数据库 信息科技辑》;I138-411 *

Also Published As

Publication number Publication date
CN112017218A (en) 2020-12-01

Similar Documents

Publication Publication Date Title
Pun et al. Multi-scale noise estimation for image splicing forgery detection
TWI409718B (en) Method of locating license plate of moving vehicle
CN107301405A (en) Method for traffic sign detection under natural scene
CN109389122B (en) License plate positioning method and device
CN111027535A (en) License plate recognition method and related equipment
CN110599516A (en) Moving target detection method and device, storage medium and terminal equipment
CN112052907A (en) Target detection method and device based on image edge information and storage medium
CN113505643A (en) Violation target detection method and related device
CN114820621A (en) Bolt loss defect detection method, system and device
CN115083008A (en) Moving object detection method, device, equipment and storage medium
CN114155285B (en) Image registration method based on gray histogram
Barcellos et al. Shadow detection in camera-based vehicle detection: survey and analysis
CN112017218B (en) Image registration method and device, electronic equipment and storage medium
CN114648734A (en) Camera working abnormity detection method based on image processing
CN112052859A (en) License plate accurate positioning method and device in free scene
CN110097569B (en) Oil tank target detection method based on color Markov chain significance model
CN110765875A (en) Method, equipment and device for detecting boundary of traffic target
CN111695374A (en) Method, system, medium, and apparatus for segmenting zebra crossing region in monitoring view
CN114724119A (en) Lane line extraction method, lane line detection apparatus, and storage medium
CN111027560B (en) Text detection method and related device
Deb et al. Optical Recognition of Vehicle license plates
CN114419531A (en) Object detection method, object detection system, and computer-readable storage medium
Monfared et al. Optimized road lane detection through a combined Canny edge detection, Hough transform, and scaleable region masking toward autonomous driving
CN112686845B (en) Image processing method and device and computer readable medium
Bartolomé-Hornillos et al. A self-adaptive automatic incident detection system for road surveillance based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant