CN115984588A - Image background similarity analysis method and device, electronic equipment and storage medium - Google Patents
Image background similarity analysis method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115984588A CN115984588A CN202211652830.3A CN202211652830A CN115984588A CN 115984588 A CN115984588 A CN 115984588A CN 202211652830 A CN202211652830 A CN 202211652830A CN 115984588 A CN115984588 A CN 115984588A
- Authority
- CN
- China
- Prior art keywords
- image
- background
- feature
- fused
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the field of image processing, and discloses an image background similarity analysis method, which comprises the following steps: inputting the first image into a portrait segmentation model for segmentation to obtain a binary image, and fusing the binary image and the first image to obtain a fused image; extracting color features, local features and global features of a background region from the fusion image to generate a background feature set; extracting at least one second image from the database, executing the step of generating a background feature set by the first image on the second image, and generating a background feature set of the second image; and respectively calculating pairwise similarity of the color features, the local features and the global features corresponding to the background feature sets of the first image and the second image, and performing weight calculation on each similarity score to obtain the similarity score of the background region of the first image and the second image. The invention also provides an image background similarity analysis device, electronic equipment and a storage medium. The invention realizes more accurate analysis of the relevance between the images.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image background similarity analysis method and apparatus, an electronic device, and a storage medium.
Background
In order to analyze the relevance among different customers in financial business, the customers are required to upload self-shot images, the images mainly comprise a portrait and a background, and the portrait area is located in the central area of the images, is obvious and occupies most area of the images.
The image uploaded by the client and the image uploaded by different clients in the database are usually subjected to characteristic comparison of human image regions, the correlation between the client and other clients is obtained through the result of the characteristic comparison, the similarity analysis of the background region of the image is often ignored, the information of the background region of the image also has very important reference value, and if the similarity analysis is ignored, the problem that the correlation analysis of the image is inaccurate easily occurs.
Disclosure of Invention
In view of the above, it is necessary to provide an image background similarity analysis method, which aims to solve the problem of inaccurate correlation analysis of images in the prior art.
The image background similarity analysis method provided by the invention comprises the following steps:
s1, inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
s2, extracting color features, local features and global features of a background region from the fusion image to generate a background feature set of the first image;
s3, extracting at least one second image from a preset database, and executing the steps S1-S2 on the second image to generate a background feature set of the second image;
and S4, respectively calculating the color feature and the local feature of the background feature set of the first image and the pairwise similarity between the global feature and the corresponding color feature and the local feature in the background feature set of the second image, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
Optionally, the inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image includes:
marking the pixel value of a portrait area of the first image as 0 and the pixel value of a background area as 1;
and segmenting the marked portrait region and the marked background region to obtain the binary image.
Optionally, the fusing the binary image and the first image to obtain a fused image of the first image includes:
performing expansion or/and corrosion treatment on the binary image according to a preset morphological algorithm to obtain a treated binary image;
and fusing the portrait area with the pixel value of 0 in the processed binary image with the first image to obtain a fused image of the first image.
Optionally, the extracting color features, local features, and global features of the background region from the fused image includes:
extracting key point features from the fusion image according to a preset feature extraction algorithm to obtain local features of a background region of the fusion image;
converting the RGB color space of the fused image into an LAB color space to obtain the color characteristics of the background area of the fused image;
and extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image.
Optionally, the extracting, according to a preset feature extraction algorithm, the feature of a key point from the fusion image to obtain a local feature of a background region of the fusion image includes:
extracting candidate key feature points of the fused image and adjacent pixels of the candidate key feature points to perform pixel difference value comparison, selecting a preset number of comparison result values to convert the comparison result values into binary strings, and obtaining all key feature points of the fused image;
removing key feature points of the portrait area in the fusion image to obtain key feature points of a background area in the fusion image;
and calculating the feature center of each key feature point of the background area, and counting the distribution of each key feature point in the corresponding feature center to obtain the local feature of the background area.
Optionally, the converting the RGB color space of the fused image into an LAB color space to obtain the color feature of the background region of the fused image includes:
converting the RGB color space of the fused image into an XYZ space to obtain the XYZ color space of the fused image;
and converting the XYZ color space of the fused image into an LAB space to obtain the color characteristics of the background area of the fused image.
Optionally, the extracting, according to a preset global feature extraction model, a global feature from the background region of the fused image to obtain a global feature of the background region of the fused image includes:
zooming the fused image to a preset size, and inputting the fused image into the global feature extraction model;
and carrying out pixel filling processing on the human image area with the pixel value of 0 in the fused image after the size scaling to obtain the global characteristic of the background area of the fused image.
In order to solve the above problem, the present invention further provides an image background similarity analysis device, including:
the fusion module is used for inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
the generation module is used for extracting color features, local features and global features of a background region from the fusion image and generating a background feature set of the first image;
the extraction module is used for extracting at least one second image from a preset database, executing the steps S1-S2 on the second image and generating a background feature set of the second image;
and the similarity calculation module is used for calculating the similarity of every two of the color features and the local features of the background feature set of the first image and the color features and the local features of the global features corresponding to the background feature set of the second image respectively, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores an image background similarity analysis program executable by the at least one processor, the image background similarity analysis program being executable by the at least one processor to enable the at least one processor to perform the image background similarity analysis method described above.
In order to solve the above problem, the present invention also provides a computer-readable storage medium having an image background similarity analysis program stored thereon, the image background similarity analysis program being executable by one or more processors to implement the above image background similarity analysis method.
Compared with the prior art, the method and the device have the advantages that the first image is input into a preset portrait segmentation model to be segmented to obtain the binary image corresponding to the first image, the binary image and the first image are fused to obtain the fused image of the first image, the color feature, the local feature and the global feature of the background area are extracted from the fused image, and the background feature set of the first image is generated. The method comprises the following steps of filtering remarkable human pixels in a first image, and accurately extracting a background feature set of the first image from multiple dimensions, wherein the background feature set comprises: the local features, the global features and the color features are mutually compensated by utilizing the features, and the extracted background feature set is more comprehensive;
respectively calculating the color feature, the local feature and the global feature of the background feature set of the first image and the color feature, the local feature and the global feature corresponding to the background feature set of the second image, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image; when the similarity of the background feature sets of the first image and the second image is compared, the similarity of local features, the similarity of global features and the similarity of color features of a background region are comprehensively considered, so that the background similarity score is more accurate, and the relevance analysis of the images is more accurate.
Drawings
Fig. 1 is a schematic flowchart of an image background similarity analysis method according to an embodiment of the present invention;
fig. 2 is a schematic block diagram of an image background similarity analysis apparatus according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device for implementing an image background similarity analysis method according to an embodiment of the present invention;
the objects, features and advantages of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The invention provides an image background similarity analysis method. Fig. 1 is a schematic flow chart of an image background similarity analysis method according to an embodiment of the present invention. The method is performed by an electronic device.
S1, inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
in this embodiment, the preset portrait segmentation model includes, but is not limited to, a trained portrait segmentation model; when a self-timer image (a first image) uploaded by a user is obtained, the first image is input into a trained humanseg portrait segmentation model, the image is segmented by adopting threshold segmentation, a binary image with the same pixels and the same size as the first image is obtained, the pixel value of the binary image is 0 or not, the pixel value of the binary image is 0, the pixel value of the binary image belongs to a portrait area, and the pixel value of the binary image belongs to a background area is 1.
And removing discrete points in the binary image by using a morphological algorithm, and filling a small hollow area in the binary image to obtain a processed binary image. And the binary image is processed by a morphological algorithm, so that the extracted background region is prevented from containing portrait information.
And fusing the processed binary image and the first image to obtain a fused image of the first image. The pixel value belonging to the human image area in the fused image is 0, and the pixel value belonging to 1 in the fused image is equal to the pixel value of the background area of the first image.
In one embodiment, the inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image includes:
marking the pixel value of a portrait area of the first image as 0 and the pixel value of a background area as 1;
and segmenting the marked portrait region and the marked background region to obtain the binary image.
Obtaining a trained portrait segmentation model (humanseg portrait segmentation model) through the following steps:
a10, collecting a preset number (for example, 50000) of images to generate a training data set, labeling a portrait area on the training data set, labeling a pixel value of the portrait area as 0, and labeling a pixel value of a background area as 1;
a20, utilizing an open-source pre-training humanseg model, and setting training parameters to finely adjust the humanseg model;
and A30, calculating training loss on the training data set by using a loss function and cross entropy until the network of the pre-training humanseg model is converged, and obtaining a trained humanseg portrait segmentation model.
Inputting the first image into a trained human segmentation model, labeling a human region of the first image (for example, labeling a pixel value of the human region as 0 and labeling a pixel value of a background region as 1) by the human segmentation model, and segmenting two regions with the pixel value of 1 and the pixel value of 0 to obtain a binary image of the first image. The human image region and the background region of the first image can be accurately segmented through the humanseg human image segmentation model.
In one embodiment, the fusing the binary image with the first image to obtain a fused image of the first image includes:
performing expansion corrosion treatment on the binary image according to a preset morphological algorithm to obtain a treated binary image;
and fusing the portrait area with the pixel value of 0 in the processed binary image with the first image to obtain a fused image of the first image.
The preset morphological algorithm includes, but is not limited to, a dilation and erosion morphological algorithm that changes the shape of a portion (black portion) having a pixel value of 0 or a portion (white portion) having a pixel value of 1 in the binary image, for example, by thinning the white portion in the binary image through erosion or by fatening the white portion in the binary image through dilation. In other embodiments, other morphological algorithms may be employed, and are not limited herein.
According to a preset morphological algorithm, performing convolution on a partial region (for example, a part with a pixel value of 0 or a part with a pixel value of 1) of the binary image and a kernel of the morphological algorithm through expansion or/and corrosion treatment to obtain a processed binary image; the kernel of the morphological algorithm can be any shape and size, having a separately defined reference point, an operation of local minimum (i.e., erosion process) for a partial region of the binary image by the reference point, or an operation of local maximum (i.e., dilation process) for a partial region of the binary image by the reference point.
And (3) placing the processed binary image above the first image for alignment and overlapping (the two images have the same size), and fusing a region with a pixel value of 0 in the processed binary image into the first image to obtain a fused image of the first image. Through the fusion processing, the accuracy of obtaining the background area of the fusion image is improved.
S2, extracting color features, local features and global features of a background region from the fusion image to generate a background feature set of the first image;
the step S2 specifically includes:
extracting key point features from the fusion image according to a preset feature extraction algorithm to obtain local features of a background region of the fusion image;
converting the RGB color space of the fusion image into an LAB color space to obtain the color characteristics of the background area of the fusion image;
and extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image.
In this embodiment, the preset feature extraction algorithm includes, but is not limited to, an ORB (organized FAST and Rotated BRIEF) feature extraction algorithm in an OpenCV library; openCV is a cross-platform computer vision and machine learning software library issued based on apache2.0 licensing (open source), which can run on Linux, windows, android, and Mac OS operating systems.
When the ORB feature extraction algorithm is used for extracting the feature of the key point, multi-scale feature extraction is carried out in a pyramid mode by using a single fusion image to obtain the key feature point of a background area in the fusion image under multiple scales, and normalization processing is carried out on the key feature point of the background area in the fusion image under multiple scales to obtain the local feature of the background of the fusion image.
In one embodiment, the extracting, according to a preset feature extraction algorithm, a feature of a key point from the fused image to obtain a local feature of a background region of the fused image includes:
extracting candidate key feature points of the fusion image and adjacent pixels of the candidate key feature points to carry out pixel difference value comparison, selecting a preset number of comparison result values to convert the comparison result values into binary strings, and obtaining all key feature points of the fusion image;
removing key feature points of the portrait area in the fused image to obtain key feature points of the background area in the fused image;
and calculating the feature center of each key feature point of the background area, and counting the distribution of each key feature point in the corresponding feature center to obtain the local feature of the background area.
In one embodiment, the extracting of the candidate key feature points of the fused image and the comparing of the pixel difference values of the adjacent pixels of the candidate key feature points, and selecting a preset number of comparison result values to convert into a binary string to obtain the key feature points of the fused image includes:
extracting pixels of which the pixel difference values are larger than a preset threshold value in the fusion image as candidate key feature points according to a preset feature extraction algorithm;
comparing the candidate key feature points with the adjacent pixels of the candidate key feature points in terms of pixel difference values, and sequencing the obtained comparison result values from high to low;
and selecting comparison result values with the preset number ranked at the top from the sequence, and converting the comparison result values into binary strings to obtain key feature points of the fused image.
The method for extracting the key point features of the fused image by the ORB feature extraction algorithm comprises the following two steps: detecting and describing feature points;
step 1 (feature point detection): calculating and extracting a pixel difference value between each pixel value and an adjacent pixel value in the fused image, and selecting a pixel with the pixel difference value larger than a preset threshold (for example, the threshold is 0.7) as a candidate key feature point;
step 2 (feature point description): comparing the candidate key feature points with the adjacent pixels of the candidate key feature points by pixel difference values, sorting the obtained comparison result values from high to low, selecting the comparison result values with the first preset number (for example, the preset number is 100) from the sorting, and converting the comparison result values with the first preset number and the first preset number into binary strings to obtain the key feature points of the fused image.
In one embodiment, the calculating a feature center of each key feature point of the background region, and counting a distribution of each key feature point in a corresponding feature center to obtain a local feature of the background region of the fused image includes:
b10, calculating the distance between any key feature point of the background area and each preset feature center in the first image according to a preset nearest neighbor algorithm to obtain a plurality of distance values;
b20, selecting the feature center with the minimum distance value from the plurality of distance values as the feature center of the key feature point;
and B30, repeatedly executing the steps B10-B20 until the feature centers of all key feature points of the background area are obtained, counting the distribution of each key feature point in the corresponding feature center, and performing normalization processing to obtain the local features of the background area.
The predetermined Nearest Neighbor algorithm includes, but is not limited to, KNN (K-Nearest Neighbor) Nearest Neighbor algorithm.
For example, the distance between the key point feature a of the background region and each feature center in the first image is calculated by a nearest neighbor algorithm to obtain a plurality of distance values (for example, the distance values are 0.6, 0.8, 1.21, and 2.3, respectively), and the feature center with the minimum distance value of 0.6 is selected as the feature center of the key feature point a; obtaining the characteristic center by other key point characteristics of the background area according to the key point characteristic A to obtain the respective characteristic center;
then, the distribution of each key feature point in the corresponding feature center is counted, normalization processing is carried out, a 1*K vector is output, the 1*K vector is used as the background local feature of the obtained fusion image, and K is the number of the feature centers.
The feature center is constructed in advance, and the construction process of the feature center comprises the following steps: a large number of fusion images are collected, background local features of each fusion image are extracted by adopting an ORB feature extraction algorithm to obtain a background local ORB feature library, the ORB feature library is clustered by adopting a K-means clustering algorithm to obtain K feature centers, and a K value is obtained through experiments.
The local background features of the fused image are obtained by fusing the key feature points of the background region in the image, so that more detailed features of the first image can be obtained, and the accuracy of the local background features can be improved.
In one embodiment, the converting the RGB color space of the fused image into the LAB color space to obtain the color feature of the background region of the fused image includes:
converting the RGB color space of the fused image into an XYZ space to obtain the XYZ color space of the fused image;
and converting the XYZ color space of the fused image into an LAB space to obtain the color characteristics of the background area of the fused image.
The RGB color space cannot be directly converted into an LAB color space, and the XYZ color space is required to be used for transition; for example, an image uploaded by an acquired user usually belongs to an sRGB color space, the sRGB color space of the fused image is converted into an RGB color space through gamma conversion, the RGB color space is converted into an XYZ color space through linear mapping, then the XYZ color space is converted into an LAB color space through a preset normalization manner and a nonlinear conversion manner, LAB channel mean values of respective pixels belonging to a background region in the fused image are counted and a vector of 1*3 is output, and the vector of 1*3 is used as a color feature of the background region of the fused image.
In the curve in the Lab color space mode, the brightness is separately stored in the L channel, and the brightness may be adjusted without changing the color information, or the color may be adjusted without changing the brightness, which is not possible with the curve in the RGB color space mode. Therefore, by converting the RGB color space of the fusion image into the LAB color space, accurate adjustment of the brightness and color of the first image can be achieved.
In an embodiment, the extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image includes:
zooming the fused image to a preset size, and inputting the fused image into the global feature extraction model;
and carrying out pixel filling processing on the human image area with the pixel value of 0 in the fused image after the size scaling to obtain the global characteristics of the background area of the fused image.
The preset global feature extraction model comprises but is not limited to a trained VGG model;
obtaining a well-trained VGG model (global feature extraction model) by the following steps:
c10, collecting a preset number (for example, 50000) of fused images to generate a training data set, and manually labeling each fused image of the training data set, wherein similar fused images are labeled as 0, and non-acquainted fused images are labeled as 1;
c20, setting the output parameters of the classification layer of the pre-trained VGG model to be 512 dimensions by using the open-source pre-trained VGG model, and finely adjusting the pre-trained VGG model;
and C30, calculating the training loss of the training data set by utilizing the cosine similarity until the pre-trained VGG model is converged to obtain the trained VGG model.
The fusion image is zoomed to a preset size (for example, the preset size is 224 × 224 pixels), the fusion image zoomed to the preset size is input into a global feature extraction model after being normalized, pixel filling processing is carried out on a human pixel area with a pixel value of 0 in the fusion image through the global feature extraction model processing, and a vector with a pixel value of 1 × 512 is output to obtain the background global feature of the fusion image.
The pixel values of the portrait areas in the fusion image are all 0, the pixel value of the background area is the pixel value of the corresponding position of the first image, and the portrait area with the pixel value of 0 is treated as a filling pixel when being treated by the trained VGG model and is ignored by the global feature extraction model, so that the background global feature is extracted by the trained VGG model finally, and the background global feature of the fusion image is accurately obtained.
Through the S2 step, the background feature set of the first image is extracted from multiple dimensions, the background feature set comprises local features, global features and color features, all the features are mutually compensated, and the extracted background feature set is more comprehensive.
S3, extracting at least one second image from a preset database, and executing the steps S1-S2 on the second image to generate a background feature set of the second image;
in this embodiment, the preset database includes, but is not limited to, a database which is established by an enterprise autonomously and used for storing images; at least one (for example, 50) images (second images) of other users taken by self are extracted from the database, the steps S1-S2 are executed on the second images to obtain the background local features of the second images, the color features of the background area and the background global features, and the background local features of the second images, the color features of the background area and the background global features are used as the background feature set of the second images.
And S4, respectively calculating the color feature and the local feature of the background feature set of the first image and the pairwise similarity between the global feature and the corresponding color feature and the local feature in the background feature set of the second image, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
The step S4 specifically includes:
according to a preset first similarity algorithm, carrying out similarity calculation on the background local features of the first image and the background local features of the second image to obtain a first similarity score;
according to a preset second similarity algorithm, similarity calculation is carried out on the color features of the background region of the first image and the color features of the background region of the second image, and a second similarity score is obtained;
according to a preset third similarity algorithm, similarity calculation is carried out on the background global features of the first image and the background global features of the second image, and a third similarity score is obtained;
and performing weight calculation on the first similarity score, the second similarity score and the third similarity score according to a preset weight formula to obtain a similarity score of the background region of the first image and the second image.
In this embodiment, the preset first similarity algorithm includes, but is not limited to, a js divergence similarity algorithm, the js divergence similarity algorithm can describe distances of feature distributions, and the distribution of key feature points of each image in corresponding feature centers is compared and calculated by the js divergence similarity algorithm, so as to obtain a first similarity score.
The preset second similarity algorithm includes, but is not limited to, a euclidean distance similarity algorithm, and the euclidean distance similarity algorithm is used to calculate the distance between the vectors of the color features of the background region of each image to judge the similarity degree of the vectors, wherein the closer the distance is, the more similar the vectors are, and the describing the LAB space color distance by the euclidean distance similarity algorithm can reduce the vector differentiation in calculating the LAB space color distance.
The preset third similarity algorithm includes, but is not limited to, a cosine distance similarity algorithm, and the background global feature distance of each image can be compared more accurately through the cosine distance similarity algorithm.
The preset weight formula is a formula in which an enterprise allocates different proportional weights to each weight factor according to the weight factors which influence the image by the background local characteristics, the color characteristics of the background area and the background global characteristics of the image; the weight formula includes:
score＝a*score+β*score+γ*score
where score is the total weight of the second image, a is the weight factor of the background local feature of the second image, β is the weight factor of the color feature of the background region of the second image, and γ second is the weight factor of the background global feature of the image.
For example, the first similarity score, the second similarity score, and the third similarity score are obtained as 0.7, 0.5, and 0.8, respectively, according to the setting of the weight formula: a is 50, β is 30, γ is 20 (e.g., a + β + γ = 100), and then 0.7 × 50+0.5 × 30+0.8 × 20 is used to obtain the background similarity score of the first image, and the background similarity score of the first image and each second image is calculated by the above method.
According to the background similarity scores of the first image and each second image, the higher the background similarity score is, the stronger the relevance between the first image and the corresponding second image is.
Through the steps of S1-S4, the significant human pixels in the first image are filtered, and the background feature set of the first image is accurately extracted from multiple dimensions, and the method comprises the following steps: the local features, the global features and the color features are mutually compensated by utilizing the features, and the extracted background feature set is more comprehensive.
When the similarity of the background feature sets of the first image and the second image is compared, the similarity of the local features, the similarity of the global features and the similarity of the color features of the background region are comprehensively considered, so that the background similarity score is more reasonable and accurate.
The image background similarity analysis method can be applied to the business of comparing the image background similarity of the identity card. The human image segmentation model can be replaced by a vehicle segmentation model, and the method can be applied to the business of comparing the similar backgrounds of the vehicle images. By analogy, the method can be popularized to other scenes with similar comparison backgrounds.
Fig. 2 is a schematic block diagram of an image background similarity analysis apparatus according to an embodiment of the present invention.
The image background similarity analysis apparatus 100 according to the present invention may be installed in an electronic device. According to the implemented functions, the image background similarity analysis apparatus 100 may include a fusion module 110, a generation module 120, an extraction module 130, and a similarity calculation module 140. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
In the present embodiment, the functions of the respective modules/units are as follows:
a fusion module 110, configured to input a first image into a preset portrait segmentation model for segmentation, to obtain a binary image corresponding to the first image, and fuse the binary image and the first image to obtain a fused image of the first image;
a generating module 120, configured to extract color features, local features, and global features of a background region from the fused image, and generate a background feature set of the first image;
an extracting module 130, configured to extract at least one second image from a preset database, perform steps S1-S2 on the second image, and generate a background feature set of the second image;
the similarity calculation module 140 is configured to calculate pairwise similarities between the color feature, the local feature, and the global feature of the background feature set of the first image and the color feature, the local feature, and the global feature corresponding to the background feature set of the second image, and perform weight calculation on each similarity score according to a preset weight formula to obtain a similarity score of the background region of the first image and the second image.
In one embodiment, the inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image includes:
labeling the pixel value of a human image area of the first image as 0 and labeling the pixel value of a background area of the first image as 1;
and segmenting the marked portrait region and the marked background region to obtain the binary image.
In one embodiment, the fusing the binary image with the first image to obtain a fused image of the first image includes:
performing expansion or/and corrosion treatment on the binary image according to a preset morphological algorithm to obtain a treated binary image;
and fusing the portrait area with the pixel value of 0 in the processed binary image with the first image to obtain a fused image of the first image.
In one embodiment, the extracting color features, local features, and global features of a background region from the fused image includes:
extracting key point features from the fusion image according to a preset feature extraction algorithm to obtain local features of a background region of the fusion image;
converting the RGB color space of the fused image into an LAB color space to obtain the color characteristics of the background area of the fused image;
and extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image.
In one embodiment, the extracting, according to a preset feature extraction algorithm, a feature of a key point from the fused image to obtain a local feature of a background region of the fused image includes:
extracting candidate key feature points of the fused image and adjacent pixels of the candidate key feature points to perform pixel difference value comparison, selecting a preset number of comparison result values to convert the comparison result values into binary strings, and obtaining all key feature points of the fused image;
removing key feature points of the portrait area in the fusion image to obtain key feature points of a background area in the fusion image;
and calculating the feature center of each key feature point of the background area, and counting the distribution of each key feature point in the corresponding feature center to obtain the local feature of the background area.
In an embodiment, the converting the RGB color space of the fused image into the LAB color space to obtain the color feature of the background region of the fused image includes:
converting the RGB color space of the fused image into an XYZ space to obtain the XYZ color space of the fused image;
and converting the XYZ color space of the fused image into an LAB space to obtain the color characteristics of the background area of the fused image.
In an embodiment, the extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image includes:
zooming the fused image to a preset size, and inputting the fused image into the global feature extraction model;
and carrying out pixel filling processing on the human image area with the pixel value of 0 in the fused image after the size scaling to obtain the global characteristic of the background area of the fused image.
Fig. 3 is a schematic structural diagram of an electronic device for implementing an image background similarity analysis method according to an embodiment of the present invention.
In the present embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13, which are communicatively connected to each other through a system bus, wherein the memory 11 stores an image background similarity analysis program 10, and the image background similarity analysis program 10 is executable by the processor 12. While fig. 3 shows only the electronic device 1 with the components 11-13 and the image background similarity analysis program 10, it will be understood by those skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may include fewer or more components than shown, or some components may be combined, or a different arrangement of components.
The storage 11 includes a memory and at least one type of readable storage medium. The memory provides cache for the operation of the electronic equipment 1; the readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1; in other embodiments, the non-volatile storage medium may also be an external storage device of the electronic device 1, such as a plug-in hard disk provided on the electronic device 1, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. In this embodiment, the readable storage medium of the memory 11 is generally used for storing an operating system and various types of application software installed in the electronic device 1, for example, codes of the image background similarity analysis program 10 in an embodiment of the present invention, and the like. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is generally configured to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run the image background similarity analysis program 10.
The network interface 13 may comprise a wireless network interface or a wired network interface, and the network interface 13 is used for establishing a communication connection between the electronic device 1 and a terminal (not shown).
Optionally, the electronic device 1 may further include a user interface, the user interface may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface may further include a standard wired interface and a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the embodiments described are illustrative only and are not to be construed as limiting the scope of the claims.
The image background similarity analysis program 10 stored in the memory 11 of the electronic device 1 is a combination of a plurality of instructions, and when running in the processor 12, can realize:
s1, inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
s2, extracting color features, local features and global features of a background region from the fusion image to generate a background feature set of the first image;
s3, extracting at least one second image from a preset database, and executing the steps S1-S2 on the second image to generate a background feature set of the second image;
and S4, respectively calculating the color feature and the local feature of the background feature set of the first image and the pairwise similarity between the global feature and the corresponding color feature and the local feature in the background feature set of the second image, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
Specifically, the processor 12 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the image background similarity analysis program 10, which is not described herein again.
Further, the integrated modules/units of the electronic device 1, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. The computer readable medium may be non-volatile or non-volatile. The computer-readable medium may include: any entity or device capable of carrying said computer program code, a recording medium, a usb-disk, a removable hard disk, a magnetic diskette, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The computer-readable storage medium stores an image background similarity analysis program 10, where the image background similarity analysis program 10 may be executed by one or more processors, and the specific implementation of the computer-readable storage medium of the present invention is substantially the same as that of each embodiment of the image background similarity analysis method, and is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it will be obvious that the term "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the same, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. An image background similarity analysis method applied to an electronic device is characterized by comprising the following steps:
s1, inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
s2, extracting color features, local features and global features of a background region from the fusion image to generate a background feature set of the first image;
s3, extracting at least one second image from a preset database, and executing the steps S1-S2 on the second image to generate a background feature set of the second image;
and S4, respectively calculating the color feature and the local feature of the background feature set of the first image and the pairwise similarity between the global feature and the corresponding color feature and the local feature in the background feature set of the second image, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
2. The method for analyzing the similarity of the background of the image according to claim 1, wherein the step of inputting the first image into a preset human image segmentation model for segmentation to obtain a binary image corresponding to the first image comprises:
marking the pixel value of a portrait area of the first image as 0 and the pixel value of a background area as 1;
and segmenting the marked portrait region and the marked background region to obtain the binary image.
3. The method for analyzing the similarity between the background and the image according to claim 1, wherein the fusing the binary image and the first image to obtain a fused image of the first image comprises:
performing expansion or/and corrosion treatment on the binary image according to a preset morphological algorithm to obtain a treated binary image;
and fusing the portrait area with the pixel value of 0 in the processed binary image with the first image to obtain a fused image of the first image.
4. The method for analyzing the similarity between the background and the image according to claim 1, wherein the extracting the color feature, the local feature and the global feature of the background region from the fused image comprises:
extracting key point features from the fusion image according to a preset feature extraction algorithm to obtain local features of a background region of the fusion image;
converting the RGB color space of the fused image into an LAB color space to obtain the color characteristics of the background area of the fused image;
and extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image.
5. The image background similarity analysis method according to claim 4, wherein the extracting key point features from the fused image according to a preset feature extraction algorithm to obtain local features of the background region of the fused image comprises:
extracting candidate key feature points of the fused image and adjacent pixels of the candidate key feature points to perform pixel difference value comparison, selecting a preset number of comparison result values to convert the comparison result values into binary strings, and obtaining all key feature points of the fused image;
removing key feature points of the portrait area in the fusion image to obtain key feature points of a background area in the fusion image;
and calculating the feature center of each key feature point of the background area, and counting the distribution of each key feature point in the corresponding feature center to obtain the local feature of the background area.
6. The image background similarity analysis method according to claim 4, wherein the converting the RGB color space of the fused image into the LAB color space to obtain the color feature of the background region of the fused image comprises:
converting the RGB color space of the fused image into an XYZ space to obtain the XYZ color space of the fused image;
and converting the XYZ color space of the fused image into an LAB space to obtain the color characteristics of the background area of the fused image.
7. The image background similarity analysis method according to claim 4, wherein the extracting global features from the background region of the fused image according to a preset global feature extraction model to obtain the global features of the background region of the fused image comprises:
zooming the fused image to a preset size, and inputting the fused image into the global feature extraction model;
and carrying out pixel filling processing on the human image area with the pixel value of 0 in the fused image after the size scaling to obtain the global characteristics of the background area of the fused image.
8. An image background similarity analysis apparatus, characterized in that the apparatus comprises:
the fusion module is used for inputting a first image into a preset portrait segmentation model for segmentation to obtain a binary image corresponding to the first image, and fusing the binary image and the first image to obtain a fused image of the first image;
the generating module is used for extracting color features, local features and global features of a background region from the fusion image and generating a background feature set of the first image;
the extraction module is used for extracting at least one second image from a preset database, executing the steps S1-S2 on the second image and generating a background feature set of the second image;
and the similarity calculation module is used for calculating the similarity of every two of the color features and the local features of the background feature set of the first image and the color features and the local features of the global features corresponding to the background feature set of the second image respectively, and performing weight calculation on each similarity score according to a preset weight formula to obtain the similarity score of the background region of the first image and the second image.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores an image background similarity analysis program executable by the at least one processor to enable the at least one processor to perform the image background similarity analysis method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon an image background similarity analysis program executable by one or more processors to implement the image background similarity analysis method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211652830.3A CN115984588A (en) | 2022-12-20 | 2022-12-20 | Image background similarity analysis method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211652830.3A CN115984588A (en) | 2022-12-20 | 2022-12-20 | Image background similarity analysis method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115984588A true CN115984588A (en) | 2023-04-18 |
Family
ID=85964211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211652830.3A Pending CN115984588A (en) | 2022-12-20 | 2022-12-20 | Image background similarity analysis method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115984588A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435156A (en) * | 2023-12-20 | 2024-01-23 | 汉朔科技股份有限公司 | Display information generation method, device, equipment and medium of electronic price tag |
| CN117435156B (en) * | 2023-12-20 | 2024-05-28 | 汉朔科技股份有限公司 | Display information generation method, device, equipment and medium of electronic price tag |
-
2022
- 2022-12-20 CN CN202211652830.3A patent/CN115984588A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435156A (en) * | 2023-12-20 | 2024-01-23 | 汉朔科技股份有限公司 | Display information generation method, device, equipment and medium of electronic price tag |
| CN117435156B (en) * | 2023-12-20 | 2024-05-28 | 汉朔科技股份有限公司 | Display information generation method, device, equipment and medium of electronic price tag |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11657602B2 (en) | Font identification from imagery | |
| CN111476284B (en) | Image recognition model training and image recognition method and device and electronic equipment | |
| WO2021164534A1 (en) | Image processing method and apparatus, device, and storage medium | |
| EP2808827A1 (en) | System and method for OCR output verification | |
| Bhunia et al. | Text recognition in scene image and video frame using color channel selection | |
| RU2760471C1 (en) | Methods and systems for identifying fields in a document | |
| WO2020253508A1 (en) | Abnormal cell detection method and apparatus, and computer readable storage medium | |
| US20170076448A1 (en) | Identification of inflammation in tissue images | |
| Vanetti et al. | Gas meter reading from real world images using a multi-net system | |
| Zhu et al. | Top-down saliency detection via contextual pooling | |
| CN108229289B (en) | Target retrieval method and device and electronic equipment | |
| CN114092938B (en) | Image recognition processing method and device, electronic equipment and storage medium | |
| CN110796145B (en) | Multi-certificate segmentation association method and related equipment based on intelligent decision | |
| CN113780116A (en) | Invoice classification method and device, computer equipment and storage medium | |
| Al-Jubouri | Content-based image retrieval: Survey | |
| CN108664968B (en) | Unsupervised text positioning method based on text selection model | |
| Chakraborty et al. | Application of daisy descriptor for language identification in the wild | |
| US20230138491A1 (en) | Continuous learning for document processing and analysis | |
| Zhang et al. | Scene categorization by deeply learning gaze behavior in a semisupervised context | |
| Kataria et al. | CNN-bidirectional LSTM based optical character recognition of Sanskrit manuscripts: A comprehensive systematic literature review | |
| CN115984588A (en) | Image background similarity analysis method and device, electronic equipment and storage medium | |
| CN115880702A (en) | Data processing method, device, equipment, program product and storage medium | |
| Li et al. | Person re-identification using salient region matching game | |
| Poodikkalam et al. | Optical character recognition based on local invariant features | |
| Fan et al. | Robust visual tracking via bag of superpixels |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |