CN116883703A - Image semantic matching method - Google Patents
Image semantic matching method Download PDFInfo
- Publication number
- CN116883703A CN116883703A CN202310873455.3A CN202310873455A CN116883703A CN 116883703 A CN116883703 A CN 116883703A CN 202310873455 A CN202310873455 A CN 202310873455A CN 116883703 A CN116883703 A CN 116883703A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- feature
- feature images
- symbol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000000605 extraction Methods 0.000 claims abstract description 51
- 239000000284 extract Substances 0.000 claims abstract description 44
- 238000012216 screening Methods 0.000 claims abstract description 22
- 238000010586 diagram Methods 0.000 claims abstract description 21
- 238000001514 detection method Methods 0.000 claims description 36
- 238000012545 processing Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 description 16
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of image matching, in particular to a method for acquiring a first image by a receiving module in an image semantic matching method; the first extraction module extracts a plurality of first image feature images and second feature images; the second extraction module obtains a plurality of first character feature diagrams and first symbol feature diagrams; the third extraction module obtains a plurality of second image feature images, second text feature images and second symbol feature images; the first matching module respectively matches a plurality of feature images with the storage unit; the first classification module outputs a plurality of fourth character feature diagrams and fourth symbol feature diagrams; the second classification module outputs a plurality of fourth image feature images, a fifth character feature image and a fifth symbol feature image; the fourth extraction module outputs a fifth image feature map, a sixth text feature map and a sixth symbol feature map; the second matching module screens out a plurality of third target images; the determination module determines a final target image. The invention can accurately match the result by screening the images, the characters and the symbols respectively.
Description
Technical Field
The invention relates to the field of image matching, in particular to an image semantic matching method.
Background
Image matching was originally proposed in the application studies of the 70 th united states for aircraft assisted navigation systems, terminal guidance for weapon projection systems, and the like. After the 80 s, its application has gradually expanded from the original pure military application to other fields. With the development of scientific technology, image matching technology has become an extremely important technology in the field of modern information processing, "there are wide and practical applications in many fields, such as: pattern recognition, automatic navigation, medical diagnosis, computer vision, three-dimensional image reconstruction, remote sensing image processing and other fields. Image matching is a bottleneck problem in these application fields, and many important computer vision studies are currently being conducted under the assumption that the matching problem has been solved. Therefore, further intensive research on image matching is of great importance.
A zero sample image target detection method and device based on deep learning is disclosed in patent document with publication number of CN113255829B, wherein an image to be detected and a target description text are given in the method; inputting the image to be detected and the target description text into a text semantic guidance detection model to obtain a target detection result output by the text semantic guidance detection model; the text semantic guidance detection model is derived based on a block semantic guidance detection model, and the block semantic guidance detection model is trained based on an image clipping recall method; the text semantic features coded by the text semantic guidance detection model are matched with the tile semantic features coded by the tile semantic guidance detection model, and the descriptive text corresponding to the text semantic features and the tile corresponding to the tile semantic features represent the same target.
The problem of inaccurate image matching results can be caused by the fact that text images are matched simultaneously by the image semantic matching method in the prior art.
Disclosure of Invention
Therefore, the invention provides an image semantic matching method, which solves the problem of inaccurate image matching caused by the fact that images, characters and symbols in images are matched with images in a target image library simultaneously in the actual image semantic matching process.
In order to achieve the above object, the present invention provides an image semantic matching method, which includes a receiving module receiving an input original image and denoising the original image to obtain a first image;
the first extraction module extracts the first image according to the area of the region to obtain a plurality of first image feature images and a plurality of second feature images;
the second extraction module extracts a plurality of second feature images according to the effective pixel values to obtain a plurality of first character feature images and a plurality of first symbol feature images;
the third extraction module re-extracts the first image feature images to obtain a plurality of second image feature images, a plurality of second character feature images and a plurality of second symbol feature images;
the first matching module respectively carries out corresponding matching on the second image feature images, the first character feature images, the first symbol feature images, the second character feature images and the second symbol feature images with the first image library, the first character library and the first symbol library, if the matching is successful, a plurality of third image feature images, a plurality of third character feature images and a plurality of third symbol feature images are output, and if the matching is unsuccessful, a first output feature image is output;
the first classification module classifies the first output feature images again to output a plurality of fourth character feature images and a plurality of fourth symbol feature images;
the second classification module sorts the third image feature images, the fourth character feature images and the fourth symbol feature images according to the features and outputs the sorted third image feature images, the fourth character feature images and the fourth symbol feature images as fourth image feature images, fifth character feature images and fifth symbol feature images;
the fourth extraction module extracts a fifth image feature map, a sixth character feature map and a sixth symbol feature map which are respectively positioned at the first position in the fourth image feature map, the fifth character feature map and the fifth symbol feature map;
the second matching module is used for matching and screening a plurality of first target images from the fifth image feature image and the target image library, matching and screening a plurality of second target images from the sixth text feature image and a plurality of first target images, and matching and screening a plurality of third target images from the sixth symbol feature image and a plurality of second target images;
the determining module compares the similarity between the first image and a plurality of third target images to determine a final target image.
Further, a first detection unit in the first extraction module detects edge contours of continuous areas in the first images and extracts areas contained in different edge contours to obtain a plurality of first detection images;
the first calculating unit calculates the areas of a plurality of first detection images to obtain a plurality of first area values;
the first classification unit sets a first area standard value, takes a plurality of corresponding continuous areas with the first area values larger than the first area standard value as a plurality of first image feature images, and takes a plurality of corresponding continuous areas with the first area values smaller than the first area standard value as a plurality of second feature images;
the first extraction unit extracts a plurality of first image feature images and a plurality of second image feature images.
Further, a second detection unit in the second extraction module detects the second feature map to obtain a plurality of first effective pixel values of a plurality of second feature maps;
the second classification unit sets a first effective pixel standard value, takes continuous areas corresponding to a plurality of first effective pixel values larger than the first effective pixel standard value as a plurality of first character feature images, and takes continuous areas corresponding to a plurality of first effective pixel values smaller than the first effective pixel standard value as a plurality of first symbol feature images;
and the second extraction unit extracts a plurality of the first character feature diagrams and a plurality of the first symbol feature diagrams.
Further, a third detection unit in the third extraction module performs effective pixel detection on the plurality of first image feature images to obtain a plurality of second effective pixel values;
the third classification unit sets a second effective pixel standard value, takes continuous areas corresponding to a plurality of second effective pixel values larger than the second effective pixel standard value as a plurality of second character feature images, and takes continuous areas corresponding to a plurality of second effective pixel values smaller than the second effective pixel standard value as a plurality of second symbol feature images;
and the third extraction unit extracts a plurality of second character feature images and a plurality of second symbol feature images, and the extracted plurality of first image feature images are a plurality of second image feature images.
Further, a first storage unit in the first matching module stores a first image library, a first text library and a first symbol library;
the first matching unit correspondingly matches the second image feature images, the first character feature images, the first symbol feature images, the second character feature images and the second symbol feature images with the storage unit, if the storage unit has a matching result, the storage unit outputs the second image feature images, the first character feature images and the third symbol feature images as a plurality of third image feature images, and if the storage unit does not have the matching result, the storage unit outputs the third image feature images, the third character feature images and the third symbol feature images as a plurality of first output feature images for reclassifying.
Further, a fourth detection unit in the first classification module performs region contour detection on a plurality of the first output feature maps;
the second calculation unit calculates the areas of the first output feature images to output a plurality of first output area values;
the fourth classification unit sets a second area standard value, takes a plurality of continuous areas with the first output area value larger than the second area standard value as a plurality of fourth character feature images, and takes a plurality of continuous areas with the first output area value smaller than the second area standard value as a plurality of fourth symbol feature images;
and the fourth extraction unit extracts a plurality of the fourth character feature images and a plurality of the fourth symbol feature images.
Further, a first receiving unit in the second classification module receives a plurality of third image feature diagrams, a plurality of fourth text feature diagrams and a plurality of fourth symbol feature diagrams;
the third calculating unit calculates the area output of the plurality of third image features as a plurality of second output area values, calculates the area effective pixel values of the plurality of fourth character feature images as a plurality of third effective pixel values, and calculates the area effective pixel values of the plurality of fourth symbol feature images as a plurality of fourth effective pixel values;
the fifth classification unit sorts the second output area values, the third effective pixel values and the fourth effective pixel values from large to small respectively;
the fifth extraction unit outputs the feature images corresponding to the sequences into a plurality of fourth image feature images, a plurality of fifth character feature images and a plurality of fifth symbol feature images.
Further, the fourth extraction module extracts feature images, which are first in the plurality of fourth image feature images, to output as a fifth image feature image, extracts feature images, which are first in the plurality of fifth character feature images, to output as a sixth character feature image, and extracts feature images, which are first in the plurality of fifth symbol feature images, to output as a sixth symbol feature image.
Further, a second storage unit in the second matching module stores a target image library;
the second matching unit performs contour matching on the fifth image feature map and the target image library to screen out a plurality of first target images;
the third matching unit matches the sixth character feature image with a plurality of first target images according to the effective pixel values to screen out a plurality of second target images;
the fourth matching unit is used for matching and screening a plurality of third target images from the sixth symbol feature image according to the effective pixel values and a plurality of second target images;
and the sixth extraction unit extracts a plurality of third target images.
Further, a fourth calculation unit in the determination module calculates the similarity between the first image and a plurality of third target images;
the sixth classification unit sorts and outputs the similarity and the corresponding target image into a fourth target image;
and the seventh extraction unit extracts an image at the first position in the fourth target image as a final target image.
Compared with the prior art, the method has the beneficial effects that the original image is clearer by denoising the original image through the receiving module, so that the feature extraction in the first image is facilitated; the first image is extracted through the first extraction module, the second extraction module and the third extraction module to obtain a plurality of preliminary feature images of images, characters and symbols, so that the first image is segmented, and the image matching process is respectively determined by the images, the characters and the symbols; the first matching module is used for checking the preliminary feature images of the images, the characters and the symbols to judge the correctness of the plurality of preliminary feature images, the first classifying module is used for reclassifying the preliminary feature images to obtain a plurality of final feature images, the feature images of the images, the characters and the symbols are checked to judge the accuracy of the classifying result, and the part with inaccurate classifying result is reclassifed to enable the classifying result to be more accurate, so that the subsequent matching process is facilitated; extracting the final feature images through the second classifying module and the fourth extracting module to obtain feature images with first sequences in a plurality of final feature images, and obtaining representative feature images in the image, text and symbol feature images through sequencing to enable the representative feature images to be matched with the target image library to determine a plurality of third target images; the target image library is screened in sequence by the second matching module according to the processes of image matching, text matching and symbol matching, so that a matching result is accurate; and comparing the similarity by the determining module to determine that the final target image achieves the effect of accurate matching.
In particular, the first detection unit obtains a plurality of first detection images through the edge contour of the continuous area, so that the processing of the first images is simplified, the efficiency of the processing of the first images is improved, the first calculation unit calculates the areas of a plurality of first detection images through the contour of a plurality of first detection images, the calculation result is accurate, the accuracy is high, and the first classification unit distinguishes the images from the character symbol by setting the standard value of the first area, so that the effect of distinguishing a plurality of first image feature images from a plurality of second feature images is achieved.
In particular, the second classifying unit distinguishes the second feature map by setting the first effective pixel standard value to obtain a plurality of first character feature maps and a plurality of first symbol feature maps, so that the effect of distinguishing characters and symbols is achieved, the operation of distinguishing and extracting characters and symbols is facilitated by distinguishing the characters and symbols by the first effective pixel standard value, and the efficiency of image processing is improved.
In particular, the third detection unit detects effective pixel values of a plurality of first image feature images to distinguish characters and symbol features contained in the first image feature images, and distinguishes characters and symbols through second effective pixel standard values to achieve the effect of keeping the definition of the characters and symbols, so that the efficiency of image processing is improved.
In particular, the first matching unit performs corresponding matching on the plurality of second image feature images, the plurality of first character feature images, the plurality of first symbol feature images, the plurality of second character feature images and the plurality of second symbol feature images and the storage unit to check the feature images of the images, the characters and the symbols so as to achieve the effect of more accurately processing the image, the characters and the symbol feature images, and facilitate the subsequent processing of the image.
In particular, the fourth classifying unit distinguishes a plurality of first output feature images through the second area standard value, reclassifies the character and symbol feature images which cannot be matched in the storage unit, achieves the effect of accurate classification, improves the efficiency of subsequent image recognition processing, and achieves the effect of accurately distinguishing the characters and symbols again under the condition of effective pixel value distinguishing failure through the second area standard value.
In particular, the third calculating unit calculates the area of the third image features to achieve the effect of distinguishing the specific gravity of the image features, and calculates the effective pixel values of the fourth character feature images and the fourth symbol feature images to achieve the distinction of different character and symbol features, so that the subsequent image processing can be realized.
In particular, the fourth extraction module extracts the first feature images in the optimal solution among the extracted images, the characters and the symbol features through respectively extracting a plurality of fourth image feature images, a plurality of fifth character feature images and a plurality of fifth symbol feature images, so that subsequent image matching is facilitated, and the image matching efficiency is improved.
In particular, the first images are screened out through contour matching between the fifth image feature map and the target image library to achieve a primary screening process of the target image library, the second images are screened out through matching between the sixth character feature map and the first images according to effective pixel values to achieve a secondary screening process of the target image library, the third images are screened out through matching between the sixth symbol feature map and the second images according to effective pixel values to achieve a tertiary screening process of the target image library, and the matching screening of the target image library through images, characters and symbols achieves an accurate effect of the matching process.
And particularly, the final screening is carried out on the plurality of third target images through the similarity, various features of the first image and the plurality of third target images are carefully compared and analyzed, so that the matching result of the final target images and the first image is accurate, and the matching accuracy is improved.
Drawings
FIG. 1 is a schematic flow chart of an image semantic matching method according to an embodiment of the present invention;
FIG. 2 is a first functional block diagram of an image semantic matching method according to an embodiment of the present invention;
fig. 3 is a second functional block diagram of an image semantic matching method according to an embodiment of the present invention.
Detailed Description
In order that the objects and advantages of the invention will become more apparent, the invention will be further described with reference to the following examples; it should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.
It should be noted that, in the description of the present invention, terms such as "upper," "lower," "left," "right," "inner," "outer," and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, which are merely for convenience of description, and do not indicate or imply that the apparatus or elements must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Furthermore, it should be noted that, in the description of the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those skilled in the art according to the specific circumstances.
Referring to fig. 1 and fig. 2, an image semantic matching method according to an embodiment of the present invention includes: step S110, the receiving module 10 receives an input original image, and performs denoising processing on the original image to obtain a first image;
step S120, the first extraction module 20 extracts the first image according to the area of the region to obtain a plurality of first image feature images and a plurality of second feature images;
step S130, the second extraction module 30 extracts a plurality of the second feature maps according to the effective pixel values to obtain a plurality of first text feature maps and a plurality of first symbol feature maps;
step S140, the third extraction module 40 re-extracts the first image feature map to obtain a plurality of second image feature maps, a plurality of second text feature maps, and a plurality of second symbol feature maps;
step S150, the first matching module 50 respectively performs corresponding matching on the plurality of second image feature maps, the plurality of first text feature maps, the plurality of first symbol feature maps, the plurality of second text feature maps, and the plurality of second symbol feature maps with the first image library, the first text library, and the first symbol library, and if the matching is successful, outputs a plurality of third image feature maps, a plurality of third text feature maps, and a plurality of third symbol feature maps, and if the matching is unsuccessful, outputs a first output feature map;
step S160, the first classification module 60 classifies the first output feature images again to output a plurality of fourth text feature images and a plurality of fourth symbol feature images;
step S170, the second classification module 70 sorts the third image feature images, the fourth text feature images and the fourth symbol feature images according to the features to output the fourth image feature images, the fifth text feature images and the fifth symbol feature images;
step S180, the fourth extracting module 80 extracts a fifth image feature map, a sixth text feature map, and a sixth symbol feature map respectively located at the first position in the plurality of fourth image feature maps, the plurality of fifth text feature maps, and the plurality of fifth symbol feature maps;
step S190, the second matching module 90 performs matching screening on the fifth image feature map and the target image library to obtain a plurality of first target images, performs matching screening on the sixth text feature map and the plurality of first target images to obtain a plurality of second target images, and performs matching screening on the sixth symbol feature map and the plurality of second target images to obtain a plurality of third target images;
in step S200, the determining module 100 compares the similarities between the first image and the plurality of third target images to determine a final target image.
Specifically, the receiving module 10 performs denoising processing on the original image to make the original image clearer, so that image extraction is facilitated, the first image is extracted by the first extracting module 20, the second extracting module 30 and the third extracting module 40 to obtain a plurality of preliminary feature images of images, characters and symbols, the first matching module 50 is used for checking the preliminary feature images of the images, characters and symbols to determine the correctness of the plurality of preliminary feature images, the first classifying module 60 is used for classifying the preliminary feature images again to obtain a plurality of final feature images, the second classifying module 70 and the fourth extracting module 80 are used for extracting the final feature images to obtain a first feature image of the plurality of final feature images, and the second matching module 90 and the determining module 100 are used for determining the final target image to achieve the effect of accurate matching.
Referring to fig. 2, specifically, the first detecting unit 21 in the first extracting module 20 detects edge contours of continuous areas in the first image and extracts areas included in the different edge contours to obtain a plurality of first detected images; the first calculating unit 22 calculates the areas of the first detection images to obtain first area values; the first classifying unit 23 sets a first area standard value, uses a plurality of corresponding continuous areas with the first area value larger than the first area standard value as a plurality of first image feature maps, and uses a plurality of corresponding continuous areas with the first area value smaller than the first area standard value as a plurality of second feature maps; the first extraction unit 24 extracts a plurality of the first image feature maps and a plurality of the second feature maps.
Specifically, the first detecting unit 21 obtains a plurality of first detected images through the edge contour of the continuous area, so that the processing of the first images is simplified, the efficiency of the processing of the first images is improved, the first calculating unit 22 calculates the areas of the plurality of first detected images through the contour of the plurality of first detected images, so that the calculation result is accurate and the accuracy is high, and the first classifying unit 23 distinguishes the images from the character symbol by setting the first area standard value, so that the effect of distinguishing the plurality of first image feature images from the plurality of second feature images is achieved.
Specifically, the second detection unit 31 in the second extraction module 30 detects the second feature map to obtain a number of first valid pixel values of a number of the second feature maps; the second classifying unit 32 sets a first effective pixel standard value, uses a continuous area corresponding to a plurality of first effective pixel values larger than the first effective pixel standard value as a plurality of first character feature maps, and uses a continuous area corresponding to a plurality of first effective pixel values smaller than the first effective pixel standard value as a plurality of first symbol feature maps; the second extracting unit 33 extracts a plurality of the first text feature diagrams and a plurality of the first symbol feature diagrams.
Specifically, the second classifying unit 32 distinguishes the second feature map by setting the first effective pixel standard value to obtain a plurality of first character feature maps and a plurality of first symbol feature maps, so as to achieve the effect of distinguishing characters from symbols, and the first effective pixel standard value distinguishes the characters from symbols to facilitate the operation of recognizing and extracting the characters and symbols, thereby improving the efficiency of image processing.
Specifically, the third detection unit 41 in the third extraction module 40 performs effective pixel detection on the plurality of first image feature maps to obtain a plurality of second effective pixel values; the third classifying unit 42 sets a second effective pixel standard value, uses continuous areas corresponding to a plurality of second effective pixel values larger than the second effective pixel standard value as a plurality of second character feature maps, and uses continuous areas corresponding to a plurality of second effective pixel values smaller than the second effective pixel standard value as a plurality of second symbol feature maps; the third extracting unit 43 extracts a plurality of the second text feature diagrams and a plurality of the second symbol feature diagrams, where the extracted plurality of the first image feature diagrams are a plurality of the second image feature diagrams.
Specifically, the third detecting unit 41 distinguishes the characters and symbols contained in the plurality of first image feature maps by detecting the effective pixel values of the plurality of first image feature maps, and distinguishes the characters and symbols by the second effective pixel standard values, thereby achieving the effect of maintaining the definition of the characters and symbols, and improving the efficiency of image processing.
Specifically, the first storage unit 51 in the first matching module 50 stores a first image library, a first text library, and a first symbol library; the first matching unit 52 performs corresponding matching on the plurality of second image feature maps, the plurality of first text feature maps, the plurality of first symbol feature maps, the plurality of second text feature maps, and the plurality of second symbol feature maps with the first storage unit 51, and if there is a matching result in the first storage unit 51, outputs the matching result as a plurality of third image feature maps, a plurality of third text feature maps, and a plurality of third symbol feature maps, and if the matching result in the first storage unit 51 does not exist, outputs the matching result as a plurality of first output feature maps, and reclassifies the matching result.
Specifically, it will be understood by those skilled in the art that the first image library, the first text library, and the first symbol library are not specifically required, and any one of the image libraries may be selected as the first image library, the first text library, and the first symbol library.
Specifically, the fourth detection unit 61 in the first classification module 60 performs region contour detection on a plurality of the first output feature maps; the second calculating unit 62 performs area calculation on the plurality of first output feature maps to output a plurality of first output area values; the fourth classification unit 63 sets a second area standard value, uses a plurality of continuous areas with the first output area value larger than the second area standard value as a plurality of fourth character feature maps, and uses a plurality of continuous areas with the first output area value smaller than the second area standard value as a plurality of fourth symbol feature maps; the fourth extraction unit 64 extracts a plurality of the fourth text feature diagrams and a plurality of the fourth symbol feature diagrams.
Specifically, the fourth classifying unit 63 distinguishes a plurality of first output feature images through the second area standard value, reclassifies the character and symbol feature images that cannot be matched in the first storage unit 51, achieves the effect of accurate classification, improves the efficiency of subsequent image recognition processing, and achieves the effect of accurately distinguishing the characters and symbols again under the condition of failure in distinguishing the effective pixel values through the second area standard value.
Specifically, the first receiving unit 71 in the second classification module 70 receives a plurality of the third image feature maps, a plurality of fourth text feature maps, and a plurality of fourth symbol feature maps; the third calculating unit 72 calculates a plurality of area outputs of the third image feature as a plurality of second output area values, calculates a plurality of area effective pixel values of the fourth text feature map as a plurality of third effective pixel values, and calculates a plurality of area effective pixel values of the fourth symbol feature map as a plurality of fourth effective pixel values; the fifth sorting unit 73 sorts the plurality of second output area values, the plurality of third effective pixel values, and the plurality of fourth effective pixel values from large to small, respectively; the fifth extraction unit 74 outputs the feature images corresponding to the ranks as a number of fourth image feature maps, a number of fifth text feature maps, and a number of fifth symbol feature maps.
Specifically, the fourth extraction module 80 extracts a first feature image of the plurality of fourth image feature images to output as a fifth image feature image, extracts a first feature image of the plurality of fifth text feature images to output as a sixth text feature image, and extracts a first feature image of the plurality of fifth symbol feature images to output as a sixth symbol feature image.
Specifically, the second storage unit 91 in the second matching module 90 stores a target image library; the second matching unit 92 performs contour matching on the fifth image feature map and the target image library to screen out a plurality of first target images; the third matching unit 93 matches the sixth text feature map with the first target images according to the effective pixel values to screen out second target images; the fourth matching unit 94 matches the sixth symbol feature map with the second target images according to the effective pixel values to screen out third target images; the sixth extraction unit 95 extracts a plurality of the third target images.
Specifically, the first images are screened out through contour matching between the fifth image feature map and the target image library to achieve a primary screening process of the target image library, the second images are screened out through matching between the sixth character feature map and the first images according to effective pixel values to achieve a secondary screening process of the target image library, the third images are screened out through matching between the sixth symbol feature map and the second images according to effective pixel values to achieve a tertiary screening process of the target image library, and the matching screening of the target image library through images, characters and symbols achieves an accurate effect of the matching process.
Specifically, the fourth calculation unit 101 in the determination module 100 calculates the similarity between the first image and the plurality of third target images; the sixth classification unit 102 sorts and outputs the similarity and the target image corresponding to the similarity as a fourth target image; the seventh extraction unit 103 extracts an image at the top of the fourth target image as a final target image.
Specifically, the final screening is performed on the plurality of third target images through the similarity, various features of the first image and the plurality of third target images are carefully compared and analyzed, so that the matching result of the final target images and the first image is accurate, and the matching accuracy is improved.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will be within the scope of the present invention.
The foregoing description is only of the preferred embodiments of the invention and is not intended to limit the invention; various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An image semantic matching method is characterized in that
The receiving module receives an input original image and performs denoising processing on the original image to obtain a first image;
the first extraction module extracts the first image according to the area of the region to obtain a plurality of first image feature images and a plurality of second feature images;
the second extraction module extracts a plurality of second feature images according to the effective pixel values to obtain a plurality of first character feature images and a plurality of first symbol feature images;
the third extraction module re-extracts the first image feature images to obtain a plurality of second image feature images, a plurality of second character feature images and a plurality of second symbol feature images;
the first matching module respectively carries out corresponding matching on the second image feature images, the first character feature images, the first symbol feature images, the second character feature images and the second symbol feature images with the first image library, the first character library and the first symbol library, if the matching is successful, a plurality of third image feature images, a plurality of third character feature images and a plurality of third symbol feature images are output, and if the matching is unsuccessful, a first output feature image is output;
the first classification module classifies the first output feature images again to output a plurality of fourth character feature images and a plurality of fourth symbol feature images;
the second classification module sorts the third image feature images, the fourth character feature images and the fourth symbol feature images according to the features and outputs the sorted third image feature images, the fourth character feature images and the fourth symbol feature images as fourth image feature images, fifth character feature images and fifth symbol feature images;
the fourth extraction module extracts a fifth image feature map, a sixth character feature map and a sixth symbol feature map which are respectively positioned at the first position in the fourth image feature map, the fifth character feature map and the fifth symbol feature map;
the second matching module is used for matching and screening a plurality of first target images from the fifth image feature image and the target image library, matching and screening a plurality of second target images from the sixth text feature image and a plurality of first target images, and matching and screening a plurality of third target images from the sixth symbol feature image and a plurality of second target images;
the determining module compares the similarity between the first image and a plurality of third target images to determine a final target image.
2. The image semantic matching method according to claim 1, wherein a first detection unit in the first extraction module detects edge contours of continuous areas in the first image and extracts areas contained in the different edge contours to obtain a plurality of first detection images;
the first calculating unit calculates the areas of a plurality of first detection images to obtain a plurality of first area values;
the first classification unit sets a first area standard value, takes a plurality of corresponding continuous areas with the first area values larger than the first area standard value as a plurality of first image feature images, and takes a plurality of corresponding continuous areas with the first area values smaller than the first area standard value as a plurality of second feature images;
the first extraction unit extracts a plurality of first image feature images and a plurality of second image feature images.
3. The image semantic matching method according to claim 2, wherein a second detection unit in the second extraction module detects the second feature map to obtain a plurality of first effective pixel values of a plurality of the second feature maps;
the second classification unit sets a first effective pixel standard value, takes continuous areas corresponding to a plurality of first effective pixel values larger than the first effective pixel standard value as a plurality of first character feature images, and takes continuous areas corresponding to a plurality of first effective pixel values smaller than the first effective pixel standard value as a plurality of first symbol feature images;
and the second extraction unit extracts a plurality of the first character feature diagrams and a plurality of the first symbol feature diagrams.
4. The image semantic matching method according to claim 3, wherein a third detection unit in the third extraction module performs effective pixel detection on the plurality of first image feature maps to obtain a plurality of second effective pixel values;
the third classification unit sets a second effective pixel standard value, takes continuous areas corresponding to a plurality of second effective pixel values larger than the second effective pixel standard value as a plurality of second character feature images, and takes continuous areas corresponding to a plurality of second effective pixel values smaller than the second effective pixel standard value as a plurality of second symbol feature images;
and the third extraction unit extracts a plurality of second character feature images and a plurality of second symbol feature images, and the extracted plurality of first image feature images are a plurality of second image feature images.
5. The method of claim 4, wherein a first storage unit in the first matching module stores a first image library, a first text library, and a first symbol library;
the first matching unit correspondingly matches the second image feature images, the first character feature images, the first symbol feature images, the second character feature images and the second symbol feature images with the storage unit, if the storage unit has a matching result, the storage unit outputs the second image feature images, the first character feature images and the third symbol feature images as a plurality of third image feature images, and if the storage unit does not have the matching result, the storage unit outputs the third image feature images, the third character feature images and the third symbol feature images as a plurality of first output feature images for reclassifying.
6. The image semantic matching method according to claim 5, wherein a fourth detection unit in the first classification module performs region contour detection on a plurality of the first output feature maps;
the second calculation unit calculates the areas of the first output feature images to output a plurality of first output area values;
the fourth classification unit sets a second area standard value, takes a plurality of continuous areas with the first output area value larger than the second area standard value as a plurality of fourth character feature images, and takes a plurality of continuous areas with the first output area value smaller than the second area standard value as a plurality of fourth symbol feature images;
and the fourth extraction unit extracts a plurality of the fourth character feature images and a plurality of the fourth symbol feature images.
7. The image semantic matching method according to claim 6, wherein a first receiving unit in the second classification module receives a plurality of the third image feature maps, a plurality of fourth text feature maps, and a plurality of fourth symbol feature maps;
the third calculating unit calculates the area output of the plurality of third image features as a plurality of second output area values, calculates the area effective pixel values of the plurality of fourth character feature images as a plurality of third effective pixel values, and calculates the area effective pixel values of the plurality of fourth symbol feature images as a plurality of fourth effective pixel values;
the fifth classification unit sorts the second output area values, the third effective pixel values and the fourth effective pixel values from large to small respectively;
the fifth extraction unit outputs the feature images corresponding to the sequences into a plurality of fourth image feature images, a plurality of fifth character feature images and a plurality of fifth symbol feature images.
8. The method for matching image semantics according to claim 7, wherein the fourth extraction module extracts a first feature image output of the plurality of fourth image feature images as a fifth image feature image, extracts a first feature image output of the plurality of fifth text feature images as a sixth text feature image, and extracts a first feature image output of the plurality of fifth symbol feature images as a sixth symbol feature image.
9. The image semantic matching method according to claim 8, wherein a second storage unit in the second matching module stores a target image library;
the second matching unit performs contour matching on the fifth image feature map and the target image library to screen out a plurality of first target images;
the third matching unit matches the sixth character feature image with a plurality of first target images according to the effective pixel values to screen out a plurality of second target images;
the fourth matching unit is used for matching and screening a plurality of third target images from the sixth symbol feature image according to the effective pixel values and a plurality of second target images;
and the sixth extraction unit extracts a plurality of third target images.
10. The image semantic matching method according to claim 9, wherein a fourth calculation unit in the determination module calculates a similarity of the first image and a plurality of the third target images;
the sixth classification unit sorts and outputs the similarity and the corresponding target image into a fourth target image;
and the seventh extraction unit extracts an image at the first position in the fourth target image as a final target image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310873455.3A CN116883703A (en) | 2023-07-14 | 2023-07-14 | Image semantic matching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310873455.3A CN116883703A (en) | 2023-07-14 | 2023-07-14 | Image semantic matching method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116883703A true CN116883703A (en) | 2023-10-13 |
Family
ID=88265812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310873455.3A Pending CN116883703A (en) | 2023-07-14 | 2023-07-14 | Image semantic matching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883703A (en) |
-
2023
- 2023-07-14 CN CN202310873455.3A patent/CN116883703A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6339651B1 (en) | Robust identification code recognition system | |
| US6246794B1 (en) | Method of reading characters and method of reading postal addresses | |
| US6291785B1 (en) | Address reader, sorting machine such as a mail thing and character string recognition method | |
| US6687401B2 (en) | Pattern recognizing apparatus and method | |
| Rothfeder et al. | Using corner feature correspondences to rank word images by similarity | |
| KR100249055B1 (en) | Character recognition apparatus | |
| EP0436819B1 (en) | Handwriting recognition employing pairwise discriminant measures | |
| US8200043B2 (en) | Page orientation detection based on selective character recognition | |
| Maldonado-Bascon et al. | Traffic sign recognition system for inventory purposes | |
| Wu et al. | Morphology-based text line extraction | |
| CN112215216A (en) | Character string fuzzy matching system and method for image recognition result | |
| Haboubi et al. | Discrimination between Arabic and Latin from bilingual documents | |
| JP3313272B2 (en) | Address reading method and identification function weight vector generation method | |
| CN107103289B (en) | Method and system for handwriting identification by using handwriting outline characteristics | |
| CN116883703A (en) | Image semantic matching method | |
| Tran et al. | A novel approach for text detection in images using structural features | |
| CN114120309A (en) | Instrument reading identification method and device and computer equipment | |
| JP3675511B2 (en) | Handwritten character recognition method and apparatus | |
| JP2005250786A (en) | Image recognition method | |
| CN112200217A (en) | Identification algorithm evaluation method and system based on infrared image big data | |
| Ballerini et al. | Rectangular traffic sign recognition | |
| Patel et al. | Gujarati Text Localization, Extraction and Binarization from Images | |
| CN115690472A (en) | Image recognition method and device based on image recognition model | |
| CN117546209A (en) | Image processing apparatus, image processing method, and image processing system | |
| JPH06215182A (en) | Character segmenting device and character recognizing device using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |