CN110162657A - A kind of image search method and system based on high-level semantics features and color characteristic - Google Patents
A kind of image search method and system based on high-level semantics features and color characteristic Download PDFInfo
- Publication number
- CN110162657A CN110162657A CN201910453427.XA CN201910453427A CN110162657A CN 110162657 A CN110162657 A CN 110162657A CN 201910453427 A CN201910453427 A CN 201910453427A CN 110162657 A CN110162657 A CN 110162657A
- Authority
- CN
- China
- Prior art keywords
- image
- level semantics
- characteristic
- semantics features
- color characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The present disclosure discloses a kind of image search method and system based on high-level semantics features and color characteristic carries out image preprocessing this method comprises: receiving query image and the image set that is retrieved；The color characteristic of image is extracted using color histogram；The high-level semantics features matrix based on capable form of characteristics of image figure is extracted using the convolutional neural networks constructed based on deep learning；The color characteristic of be retrieved image and query image and high-level semantics features matrix are subjected to similarity mode respectively, obtain color characteristic similarity and high-level semantics features similarity, weighted sum obtains final similarity, the image that descending arrangement output retrieves.
Description
Technical field
The disclosure belongs to the technical field of image retrieval, is related to a kind of image based on high-level semantics features and color characteristic
Search method and system.
Background technique
Only there is provided background technical informations relevant to the disclosure for the statement of this part, it is not necessary to so constitute first skill
Art.
With the fast development of internet, digital picture is in explosive increase on internet, how from the figures of substantial amounts
As quickly and accurately retrieving image interested to people in database, further as the research heat of field of image search
Point.Under normal conditions, image search method can be divided into two kinds: text based image retrieval (Text-based Image
Retrieval, TBIR) and content-based image retrieval (content-based image retrieval, CBIR).
TBIR method is exactly to be described by being manually labeled as image addition text label with the main contents to image,
Image retrieval problem is become into text retrieval problem, this image search method retrieval rate is higher, however, inventor is grinding
Hair during find, but because of human factor, there is great defects for TBIR method: manually mark it is at high cost, be not suitable for greatly
The retrieval of scale image data；Artificial subjectivity is too strong, and different people may be different to the mark of same piece image, thus
Lead to the otherness of image searching result；The content that image is included cannot be fully described in text label, cause search result quasi-
True rate reduces.
CBIR method be exactly allow user input a width query image, by query image and it is all be retrieved image into
Row feature extraction, characteristic similarity matching, to retrieve image similar with query image content in image data base.
Although CBIR method solves the defect being manually labeled to image, however, inventor has found in R&D process, the side CBIR
Method can be by " semantic letter existing between the characteristics of the underlying image of computer identification and the high-level semantic concept of human perception
Ditch " causes retrieval effectiveness still unsatisfactory.How to reduce and is even led across this " wide gap " as present image retrieval research
The big hot spot of the one of domain.
In recent years, deep learning achieved original achievement in terms of image procossing, and the feature based on deep learning exists
Also there is achievement in every Computer Vision Task.The more abstract, Shandong by the extracted image high-level semantics features of deep learning
Stick is more preferable, can preferably characterize picture material.Therefore, how preferably using deep learning to the high-level semantics features of image
Extract a focus for also becoming field of image search.
Summary of the invention
For the deficiencies in the prior art, one or more other embodiments of the present disclosure provide a kind of based on high-rise language
The image search method and system of adopted feature and color characteristic, effectively improve the performance of image retrieval.
According to the one aspect of one or more other embodiments of the present disclosure, provide a kind of based on high-level semantics features and color
The image search method of feature.
A kind of image search method based on high-level semantics features and color characteristic, this method comprises:
It receives query image and the image set that is retrieved, carries out image preprocessing；
The color characteristic of image is extracted using color histogram；
The high level based on capable form of characteristics of image figure is extracted using the convolutional neural networks constructed based on deep learning
Semantic feature matrix；
The color characteristic of be retrieved image and query image and high-level semantics features matrix are subjected to similarity mode respectively,
Color characteristic similarity and high-level semantics features similarity are obtained, weighted sum obtains final similarity, descending arrangement output inspection
The image that rope arrives.
Further, in the method, the specific steps for carrying out image preprocessing include:
Convert query image to 1024 × 1024 resolution ratio；
Noise reduction process is carried out using 3 × 3 mean filter to the image after resolution ratio conversion.
Further, in the method, the specific step of the color characteristic that query image is extracted using color histogram
Suddenly include:
RGB mode is converted the image into using color histogram；
The image of RGB mode is mapped to HSV space；The H of the HSV space is divided into 16 grades, and S is divided into 4 grades, and V is drawn
It is divided into 4 grades；
The number of the pixel of each color value in statistical picture, the color for obtaining 256 dimensions corresponding to each image are special
Levy vector.
Further, in the method, the convolutional neural networks based on deep learning building include sequentially connected 3
A convolutional layer；First convolutional layer includes the convolution kernel and 1 linear amending unit of 16 sizes 3 × 3, is connected with maximum pond layer
It connects；Second convolutional layer includes the convolution kernel and 1 linear amending unit that 4 sizes are 3 × 3；Third convolutional layer includes 4 sizes
For 3 × 3 convolution kernel and 1 linear amending unit, it is connect with mean value pond layer.
Further, in the method, training set of images is received, convolutional neural networks are trained, calculates convolution mind
Error function through network output valve updates network parameter using back-propagation algorithm and small lot gradient descent algorithm.
Further, in the method, described to extract image spy using the convolutional neural networks constructed based on deep learning
The specific steps of the high-level semantics features vector based on capable form of sign figure include:
It to query image and is retrieved at image set respectively using the convolutional neural networks constructed based on deep learning
Reason, corresponding 256 characteristic patterns of each image, corresponding 1 256 × 256 matrix of each characteristic pattern；
It is grouped every data line of each characteristic pattern as one, seeks the mean value of data, variance, mode in the grouping
And maximum, and the branching characteristic vector as the row；
The rule in middle position is distributed according to the main contents of image, by preceding c1 row corresponding to every width characteristic pattern with after
The branching characteristic vector of c1 row is multiplied by the first weight, and the branching characteristic vector of center row is multiplied by the second weight, and the second weight is big
In the first weight, the sum of two weights are 1；
Each branching characteristic vector of 256 characteristic patterns corresponding to every piece image mutually gone together connect without change
It connects, the row vector of the high-level semantics features matrix as image obtains high-level semantics features matrix.
Further, in the method, described respectively by the color characteristic of be retrieved image and query image and high-rise language
The specific steps that adopted feature vector carries out similarity mode include:
Using characteristic similarity matching process calculate query image color characteristic and it is all be retrieved color of image feature it
Between similarity；
Each width is retrieved high level corresponding to image using characteristic similarity matching process identical with color characteristic
Each row feature vector of semantic feature matrix all row features with the high-level semantics features matrix of query image respectively
Vector successively carries out similarity mode, is maximized respectively, sums to obtain every width for maximum value and is retrieved image and query image
Similarity on high-level semantics features.
Further, this method further include: receive user feedback as a result, carrying out according to user feedback result to retrieving
Adjustment, is retrieved again.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of image search method based on high-level semantics features and color characteristic.
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed described one kind and is based on for storing a plurality of instruction, described instruction
The image search method of high-level semantics features and color characteristic.
According to the one aspect of one or more other embodiments of the present disclosure, provide a kind of based on high-level semantics features and color
The image retrieving apparatus of feature.
A kind of image retrieving apparatus based on high-level semantics features and color characteristic is based on high-rise language based on described one kind
The image search method of adopted feature and color characteristic, comprising:
Preprocessing module is configured as receiving query image and the image set that is retrieved, carries out image preprocessing；
Color feature extracted module is configured as extracting the color characteristic of image using color histogram；
High level feature extraction module is configured as utilizing the convolutional neural networks extraction figure constructed based on deep learning
As the high-level semantics features vector based on capable form of characteristic pattern；
Similarity mode module, the color characteristic and high-level semantic of be configured to will to be retrieved image and query image
Feature vector carries out similarity mode, obtains color characteristic similarity and high-level semantics features similarity, weighted sum obtains most
Whole similarity, the image that descending arrangement output retrieves.
The disclosure the utility model has the advantages that
(1) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides,
During image characteristics extraction, the primary colour feature and high-level semantics features of image have been comprehensively considered, so that characteristics of image
Representativeness better than individually considers the representativeness of characteristics of the underlying image.
(2) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides, it is comprehensive
It closes and considers similarity that query image and every width are retrieved between image different piece content to measure query image and every width quilt
The similarity between image is retrieved, the contrast between image can be further enhanced.
(3) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides, will
The weight during weight, final similarity mode during High level feature extraction is set as dynamic parameter, can be into one
Step improves image search method to the adaptability of different query images.
(4) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides leads to
It crosses user satisfaction and carries out information feedback, can be further improved the performance of image retrieval.
Detailed description of the invention
The accompanying drawings constituting a part of this application is used to provide further understanding of the present application, and the application's shows
Meaning property embodiment and its explanation are not constituted an undue limitation on the present application for explaining the application.
Fig. 1 is a kind of image retrieval side based on high-level semantics features and color characteristic according to one or more embodiments
Method flow chart.
Specific embodiment:
Below in conjunction with the attached drawing in one or more other embodiments of the present disclosure, to one or more other embodiments of the present disclosure
In technical solution be clearly and completely described, it is clear that described embodiment is only disclosure a part of the embodiment,
Instead of all the embodiments.Based on one or more other embodiments of the present disclosure, those of ordinary skill in the art are not being made
Every other embodiment obtained under the premise of creative work belongs to the range of disclosure protection.
It is noted that following detailed description is all illustrative, it is intended to provide further instruction to the application.Unless another
It indicates, all technical and scientific terms that the present embodiment uses have and the application person of an ordinary skill in the technical field
Normally understood identical meanings.
It should be noted that term used herein above is merely to describe specific embodiment, and be not intended to restricted root
According to the illustrative embodiments of the application.As used herein, unless the context clearly indicates otherwise, otherwise singular
Also it is intended to include plural form, additionally, it should be understood that, when in the present specification using term "comprising" and/or " packet
Include " when, indicate existing characteristics, step, operation, device, component and/or their combination.
It should be noted that flowcharts and block diagrams in the drawings show according to various embodiments of the present disclosure method and
The architecture, function and operation in the cards of system.It should be noted that each box in flowchart or block diagram can represent
A part of one module, program segment or code, a part of the module, program segment or code may include one or more
A executable instruction for realizing the logic function of defined in each embodiment.It should also be noted that some alternately
Realization in, function marked in the box can also occur according to the sequence that is marked in attached drawing is different from.For example, two connect
The box even indicated can actually be basically executed in parallel or they can also be executed in a reverse order sometimes,
This depends on related function.It should also be noted that each box and flow chart in flowchart and or block diagram
And/or the combination of the box in block diagram, the dedicated hardware based system that functions or operations as defined in executing can be used are come
It realizes, or the combination of specialized hardware and computer instruction can be used to realize.
In the absence of conflict, the feature in the embodiment and embodiment in the disclosure can be combined with each other, and tie below
It closes attached drawing and embodiment is described further the disclosure.
Embodiment one
According to the one aspect of one or more other embodiments of the present disclosure, provide a kind of based on high-level semantics features and color
The image search method of feature.
In order to further increase the representativeness of image feature vector, this method is special using high-level semantics features and primary colour
The mode that combines is levied to characterize the feature of image, so that the feature of image is more comprehensive, to make comparison between image more
Comprehensively, deeply.Meanwhile comprehensively considering query image and similarity that every width is retrieved between image different piece content is measured
Query image and every width are retrieved the similarity between image, can further enhance the contrast between image.
As shown in Figure 1, a kind of image search method based on high-level semantics features and color characteristic, this method comprises:
A kind of image search method based on high-level semantics features and color characteristic, this method comprises:
S1 image preprocessing: pre-processing image, reduces the noise jamming in image；
The color characteristic of S2 extraction image: the color characteristic of image is extracted using color histogram；
The high-level semantics features of S3 extraction image: convolutional neural networks of the building based on deep learning utilize convolutional Neural
The high-level semantics features vector based on capable form of network extraction characteristics of image figure；
S4 characteristics of image similarity mode: firstly, by the color characteristic progress phase of all be retrieved image and query images
It is matched like degree；Secondly, by every width be retrieved all rows of characteristics of image figure eigenmatrix respectively with the every row of query image characteristic pattern
Eigenmatrix carry out similarity mode, and each matching result of the matched maximum value as this time is taken, by each secondary phase of gained
It sums like degree matching result, to obtain the similarity on high-level semantics features；Finally, by color characteristic similarity and height
Layer semantic feature similarity is weighted summation, is retrieved the final similarity of image and query image to obtain every width, so
The image being retrieved is exported to user according to the sequence of final similarity from big to small afterwards.
In the step S1 of the present embodiment, image preprocessing process are as follows:
Convert image to 1024 × 1024 resolution ratio；Image is carried out at noise reduction using 3 × 3 mean filter
Reason, to reduce the interference of noise on image feature extraction generation.
In the step S2 of the present embodiment, the extraction process of color feature vector are as follows:
Using color histogram, RGB mode is first converted the image into, the image of RGB mode is then mapped to HSV sky
Between, wherein H is divided into 16 grades, and S is divided into 4 grades, and V is divided into 4 grades, to obtain 256 kinds of different color values；It counts respectively
The number of the pixel of each color value in image, so that the color feature vector of 256 dimensions corresponding to each image is obtained,
The color feature vector of query image is set as VCq, the image that is retrieved color feature vector be VCri(i=1,2 ..., n, wherein
N is the amount of images that is retrieved)).
In the step S3 of the present embodiment, the process of High level feature extraction network is constructed are as follows:
Convolutional neural networks are constructed based on deep learning, which includes 3 convolutional layers, 3 linear amendments
Unit, 2 pond layers.The characteristic pattern that each convolutional layer is handled is used as the input data of next convolutional layer.1st volume
Lamination includes the convolution kernel (step-length 1) and 1 linear amending unit that 16 sizes are 3 × 3, be followed by 1 maximum pond layer,
The size of core is 3 × 3 (step-lengths 2)；2nd convolutional layer includes the convolution kernel (step-length 1) and 1 line that 4 sizes are 3 × 3
Property amending unit；3rd convolutional layer includes the convolution kernel (step-length 1) and 1 linear amending unit that 4 sizes are 3 × 3, it
It is followed by 1 mean value pond layer, the size of core is 3 × 3 (step-lengths 2).After 3 layers of convolution, each image can all correspond to 256
A characteristic pattern.Wherein the first width characteristic pattern of convolutional neural networks input refers to the image to be processed of HSV mode.Utilize image
Training set is trained convolutional neural networks, calculates the error function of convolutional neural networks output valve, is calculated using backpropagation
Method and small lot gradient descent algorithm update network parameter.
In the step S3 of the present embodiment, high-level semantic of the characteristics of image figure based on capable form is extracted using deep learning
The process of feature vector are as follows:
Firstly, query image and all images that are retrieved are handled respectively using the convolutional neural networks after training,
Then 256 characteristic patterns that the last one pond layer corresponding to all images exports are handled, each characteristic pattern is right
Answer 1 256 × 256 matrix；Secondly, being grouped every data line of each characteristic pattern as one, number in the grouping is sought
According to mean μ, variances sigma, mode v, maximum m, use this 4 data characteristicses as the branching characteristic vector (μ of the rowi,j,k,
σi,j,k,vi,j,k,mi,j,k) (i=1,2 ..., 256, j=1,2 ..., 256, k=1,2,3,4, wherein i indicates the i-th width feature
Figure, j indicate that jth row, k indicate k-th of data characteristics)；Finally, being distributed in the rule in middle position according to the main contents of image
Rule, by the branching characteristic vector of preceding c1 row corresponding to every width characteristic pattern and rear c1 row multiplied by weight w1, the branch spy of center row
Vector is levied multiplied by weight w2, wherein w2>w1, and w1+w2=1.By the mutually colleague of 256 characteristic patterns corresponding to every piece image
Each branching characteristic vector (μi,j,k,σi,j,k,vi,j,k,mi,j,k) (i=1,2 ..., 256, j=1,2 ..., 256, k=1,2,
3,4) it carries out connecting without change, the row vector of the high-level semantics features matrix as image(wherein, i=1,2 ..., 256),
Each row vector is made of 256 × 4 data, to obtain 1 256 × 1024 high-level semantics features matrix.
In the step S4 of the present embodiment, color of image characteristic similarity matching process are as follows:
Appropriate characteristic similarity matching process is found, query image color characteristic and all color of image that are retrieved are calculated
Similarity between feature.If Sci(i=1,2 ..., n) is query image and the i-th width is retrieved the spy of color corresponding to image
Similarity between sign.
Image high-level semantics features similarity mode process are as follows:
Using Similarity Match Method identical with previous step by each width be retrieved high-level semantic corresponding to image spy
Levy matrix each row feature vector respectively all row feature vectors with the high-level semantics features matrix of query image according to
Secondary carry out similarity mode, is maximized respectively, this 256 maximum values are summed, obtain every width be retrieved image with look into
Ask similarity of the image on high-level semantics features.If Sgi(i=1,2 ..., n) is query image and the i-th width is retrieved image
Similarity between corresponding high-level semantics features.
The final similarity mode process of image are as follows:
Every width is retrieved the similarity S of image and query image on color characteristicciWith on high-level semantics features
Similarity SgiIt is weighted summation, i.e. Si=a1Sci+a2Sgi, wherein a1+a2=1, acquired results SiIt is retrieved image as every width
With the final similarity of query image, finally the image retrieved is exported to use according to the sequence of final similarity from big to small
Family.
Further, retrieving is adjusted according to user feedback result.If user is satisfied to search result,
Stop retrieving, otherwise, adjustment feature extracting method and characteristic similarity matching process appropriate and then re-starts inspection
Rope.
Embodiment two
According to the one aspect of one or more other embodiments of the present disclosure, a kind of computer readable storage medium is provided.
A kind of computer readable storage medium, wherein being stored with a plurality of instruction, described instruction is suitable for by terminal device
Reason device loads and executes a kind of image search method based on high-level semantics features and color characteristic.
Embodiment three
According to the one aspect of one or more other embodiments of the present disclosure, a kind of terminal device is provided.
A kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；Meter
Calculation machine readable storage medium storing program for executing is suitable for being loaded by processor and being executed described one kind and is based on for storing a plurality of instruction, described instruction
The image search method of high-level semantics features and color characteristic.
These computer executable instructions execute the equipment according to each reality in the disclosure
Apply method or process described in example.
In the present embodiment, computer program product may include computer readable storage medium, containing for holding
The computer-readable program instructions of row various aspects of the disclosure.Computer readable storage medium, which can be, can keep and store
By the tangible device for the instruction that instruction execution equipment uses.Computer readable storage medium for example can be-- but it is unlimited
In-- storage device electric, magnetic storage apparatus, light storage device, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned
Any appropriate combination.The more specific example (non exhaustive list) of computer readable storage medium includes: portable computing
Machine disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or
Flash memory), static random access memory (SRAM), Portable compressed disk read-only memory (CD-ROM), digital versatile disc
(DVD), memory stick, floppy disk, mechanical coding equipment, the punch card for being for example stored thereon with instruction or groove internal projection structure, with
And above-mentioned any appropriate combination.Computer readable storage medium used herein above is not interpreted instantaneous signal itself,
The electromagnetic wave of such as radio wave or other Free propagations, the electromagnetic wave propagated by waveguide or other transmission mediums (for example,
Pass through the light pulse of fiber optic cables) or pass through electric wire transmit electric signal.
Computer-readable program instructions described herein can be downloaded to from computer readable storage medium it is each calculate/
Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network
Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway
Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted
Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment
In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing present disclosure operation can be assembly instruction, instruction set architecture (ISA)
Instruction, machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programmings
The source code or object code that any combination of language is write, the programming language include the programming language-of object-oriented such as
C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer-readable program refers to
Order can be executed fully on the user computer, partly be executed on the user computer, as an independent software package
Execute, part on the user computer part on the remote computer execute or completely on a remote computer or server
It executes.In situations involving remote computers, remote computer can include local area network by the network-of any kind
(LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize internet
Service provider is connected by internet).In some embodiments, by being believed using the state of computer-readable program instructions
Breath comes personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or programmable logic
Array (PLA), the electronic circuit can execute computer-readable program instructions, to realize the various aspects of present disclosure.
Example IV
According to the one aspect of one or more other embodiments of the present disclosure, provide a kind of based on high-level semantics features and color
The image retrieving apparatus of feature.
A kind of image retrieving apparatus based on high-level semantics features and color characteristic is based on high-rise language based on described one kind
The image search method of adopted feature and color characteristic, comprising:
Preprocessing module is configured as receiving query image and the image set that is retrieved, carries out image preprocessing；
Color feature extracted module is configured as extracting the color characteristic of image using color histogram；
High level feature extraction module is configured as utilizing the convolutional neural networks extraction figure constructed based on deep learning
As the high-level semantics features vector based on capable form of characteristic pattern；
Similarity mode module, the color characteristic and high-level semantic of be configured to will to be retrieved image and query image
Feature vector carries out similarity mode, obtains color characteristic similarity and high-level semantics features similarity, weighted sum obtains most
Whole similarity, the image that descending arrangement output retrieves.
It should be noted that although being referred to several modules or submodule of equipment in the detailed description above, it is this
Division is only exemplary rather than enforceable.In fact, in accordance with an embodiment of the present disclosure, two or more above-described moulds
The feature and function of block can embody in a module.Conversely, the feature and function of an above-described module can be with
Further division is to be embodied by multiple modules.
The disclosure the utility model has the advantages that
(1) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides,
During image characteristics extraction, the primary colour feature and high-level semantics features of image have been comprehensively considered, so that characteristics of image
Representativeness better than individually considers the representativeness of characteristics of the underlying image.
(2) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides, it is comprehensive
It closes and considers similarity that query image and every width are retrieved between image different piece content to measure query image and every width quilt
The similarity between image is retrieved, the contrast between image can be further enhanced.
(3) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides, will
The weight during weight, final similarity mode during High level feature extraction is set as dynamic parameter, can be into one
Step improves image search method to the adaptability of different query images.
(4) a kind of image search method and system based on high-level semantics features and color characteristic that the disclosure provides leads to
It crosses user satisfaction and carries out information feedback, can be further improved the performance of image retrieval.
The foregoing is merely preferred embodiment of the present application, are not intended to limit this application, for the skill of this field
For art personnel, various changes and changes are possible in this application.Within the spirit and principles of this application, made any to repair
Change, equivalent replacement, improvement etc., should be included within the scope of protection of this application.Therefore, the disclosure is not intended to be limited to this
These embodiments shown in text, and it is to fit to the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. a kind of image search method based on high-level semantics features and color characteristic, which is characterized in that this method comprises:
It receives query image and the image set that is retrieved, carries out image preprocessing；
The color characteristic of image is extracted using color histogram；
The high-level semantic based on capable form of characteristics of image figure is extracted using the convolutional neural networks constructed based on deep learning
Eigenmatrix；
The color characteristic of be retrieved image and query image and high-level semantics features matrix are subjected to similarity mode respectively, obtained
Color characteristic similarity and high-level semantics features similarity, weighted sum obtain final similarity, and descending arrangement output retrieves
Image.
2. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In in the method, the specific steps for carrying out image preprocessing include:
Convert query image to 1024 × 1024 resolution ratio；
Noise reduction process is carried out using 3 × 3 mean filter to the image after resolution ratio conversion.
3. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In in the method, the specific steps of the color characteristic that query image is extracted using color histogram include:
RGB mode is converted the image into using color histogram；
The image of RGB mode is mapped to HSV space；The H of the HSV space is divided into 16 grades, and S is divided into 4 grades, and V is divided into 4
Grade；
The number of the pixel of each color value in statistical picture, obtain corresponding to each image the color characteristics of 256 dimensions to
Amount.
4. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In in the method, the convolutional neural networks based on deep learning building include sequentially connected 3 convolutional layers；First
Convolutional layer includes the convolution kernel and 1 linear amending unit of 16 sizes 3 × 3, is connect with maximum pond layer；Second convolutional layer packet
Include convolution kernel and 1 linear amending unit that 4 sizes are 3 × 3；Third convolutional layer includes the convolution kernel that 4 sizes are 3 × 3
With 1 linear amending unit, it is connect with mean value pond layer；
In the method, training set of images is received, convolutional neural networks are trained, calculates convolutional neural networks output valve
Error function updates network parameter using back-propagation algorithm and small lot gradient descent algorithm.
5. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In, in the method, it is described using convolutional neural networks construct based on deep learning extraction characteristics of image figure based on capable
The specific steps of the high-level semantics features vector of form include:
Query image and the image set that is retrieved are handled respectively using the convolutional neural networks constructed based on deep learning, often
Width image corresponds to 256 characteristic patterns, corresponding 1 256 × 256 matrix of each characteristic pattern；
It is grouped every data line of each characteristic pattern as one, seeks the mean value of data, variance, mode and pole in the grouping
Big value, and the branching characteristic vector as the row；
The rule in middle position is distributed according to the main contents of image, by preceding c1 row corresponding to every width characteristic pattern and rear c1 row
Branching characteristic vector multiplied by the first weight, the branching characteristic vector of center row is multiplied by the second weight, and the second weight is greater than the
One weight, the sum of two weights are 1；
Each branching characteristic vector of 256 characteristic patterns corresponding to every piece image mutually gone together connect without change, is made
For the row vector of the high-level semantics features matrix of image, high-level semantics features matrix is obtained.
6. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In in the method, described respectively to carry out the color characteristic of be retrieved image and query image and high-level semantics features vector
The specific steps of similarity mode include:
Query image color characteristic is calculated using characteristic similarity matching process and all is retrieved between color of image feature
Similarity；
Each width is retrieved high-level semantic corresponding to image using characteristic similarity matching process identical with color characteristic
Each row feature vector of eigenmatrix all row feature vectors with the high-level semantics features matrix of query image respectively
Similarity mode is successively carried out, is maximized respectively, maximum value is summed to obtain every width and is retrieved image and query image in height
Similarity in layer semantic feature.
7. a kind of image search method based on high-level semantics features and color characteristic as described in claim 1, feature exist
In this method further include: receive user feedback as a result, being adjusted according to user feedback result to retrieving, carry out again
Retrieval.
8. a kind of computer readable storage medium, wherein being stored with a plurality of instruction, which is characterized in that described instruction is suitable for by terminal
The processor of equipment is loaded and is executed as claim 1-7 is described in any item a kind of based on high-level semantics features and color characteristic
Image search method.
9. a kind of terminal device comprising processor and computer readable storage medium, processor is for realizing each instruction；It calculates
Machine readable storage medium storing program for executing is for storing a plurality of instruction, which is characterized in that described instruction is suitable for being loaded by processor and being executed such as power
Benefit requires a kind of described in any item image search methods based on high-level semantics features and color characteristic of 1-7.
10. a kind of image retrieving apparatus based on high-level semantics features and color characteristic, which is characterized in that based on such as claim
A kind of described in any item image search methods based on high-level semantics features and color characteristic of 1-7, comprising:
Preprocessing module is configured as receiving query image and the image set that is retrieved, carries out image preprocessing；
Color feature extracted module is configured as extracting the color characteristic of image using color histogram；
High level feature extraction module is configured as extracting image spy using the convolutional neural networks constructed based on deep learning
Levy the high-level semantics features vector based on capable form of figure；
Similarity mode module, the color characteristic and high-level semantics features of be configured to will to be retrieved image and query image
Vector carries out similarity mode, obtains color characteristic similarity and high-level semantics features similarity, weighted sum obtains most last phase
Like degree, descending arrangement exports the image retrieved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910453427.XA CN110162657B (en) | 2019-05-28 | 2019-05-28 | Image retrieval method and system based on high-level semantic features and color features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910453427.XA CN110162657B (en) | 2019-05-28 | 2019-05-28 | Image retrieval method and system based on high-level semantic features and color features |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110162657A true CN110162657A (en) | 2019-08-23 |
| CN110162657B CN110162657B (en) | 2021-04-02 |
Family
ID=67629639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910453427.XA Expired - Fee Related CN110162657B (en) | 2019-05-28 | 2019-05-28 | Image retrieval method and system based on high-level semantic features and color features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110162657B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110516689A (en) * | 2019-08-30 | 2019-11-29 | 北京达佳互联信息技术有限公司 | Image processing method, device and electronic equipment, storage medium |
| CN112584146A (en) * | 2019-09-30 | 2021-03-30 | 复旦大学 | Method and system for evaluating interframe similarity |
| CN113344012A (en) * | 2021-07-14 | 2021-09-03 | 马上消费金融股份有限公司 | Article identification method, device and equipment |
| CN113343015A (en) * | 2021-05-31 | 2021-09-03 | 北京达佳互联信息技术有限公司 | Image query method and device, electronic equipment and computer readable storage medium |
| CN113392257A (en) * | 2021-06-23 | 2021-09-14 | 泰康保险集团股份有限公司 | Image retrieval method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298775A (en) * | 2014-10-31 | 2015-01-21 | 北京工商大学 | Multi-feature content-based image retrieval method and system |
| CN108009191A (en) * | 2017-09-24 | 2018-05-08 | 贵州师范学院 | A kind of image search method |
| CN108897791A (en) * | 2018-06-11 | 2018-11-27 | 云南师范大学 | A kind of image search method based on depth convolution feature and semantic similarity amount |
| CN109635141A (en) * | 2019-01-29 | 2019-04-16 | 京东方科技集团股份有限公司 | For retrieving method, electronic equipment and the computer readable storage medium of image |
| US10268952B2 (en) * | 2015-06-04 | 2019-04-23 | Oath Inc. | Image searching |
-
2019
- 2019-05-28 CN CN201910453427.XA patent/CN110162657B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298775A (en) * | 2014-10-31 | 2015-01-21 | 北京工商大学 | Multi-feature content-based image retrieval method and system |
| US10268952B2 (en) * | 2015-06-04 | 2019-04-23 | Oath Inc. | Image searching |
| CN108009191A (en) * | 2017-09-24 | 2018-05-08 | 贵州师范学院 | A kind of image search method |
| CN108897791A (en) * | 2018-06-11 | 2018-11-27 | 云南师范大学 | A kind of image search method based on depth convolution feature and semantic similarity amount |
| CN109635141A (en) * | 2019-01-29 | 2019-04-16 | 京东方科技集团股份有限公司 | For retrieving method, electronic equipment and the computer readable storage medium of image |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110516689A (en) * | 2019-08-30 | 2019-11-29 | 北京达佳互联信息技术有限公司 | Image processing method, device and electronic equipment, storage medium |
| CN110516689B (en) * | 2019-08-30 | 2020-10-27 | 北京达佳互联信息技术有限公司 | Image processing method, image processing device, electronic device and storage medium |
| CN112584146A (en) * | 2019-09-30 | 2021-03-30 | 复旦大学 | Method and system for evaluating interframe similarity |
| CN112584146B (en) * | 2019-09-30 | 2021-09-28 | 复旦大学 | Method and system for evaluating interframe similarity |
| CN113343015A (en) * | 2021-05-31 | 2021-09-03 | 北京达佳互联信息技术有限公司 | Image query method and device, electronic equipment and computer readable storage medium |
| CN113392257A (en) * | 2021-06-23 | 2021-09-14 | 泰康保险集团股份有限公司 | Image retrieval method and device |
| CN113392257B (en) * | 2021-06-23 | 2023-06-16 | 泰康保险集团股份有限公司 | Image retrieval method and device |
| CN113344012A (en) * | 2021-07-14 | 2021-09-03 | 马上消费金融股份有限公司 | Article identification method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110162657B (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110162657A (en) | A kind of image search method and system based on high-level semantics features and color characteristic | |
| WO2021042828A1 (en) | Neural network model compression method and apparatus, and storage medium and chip | |
| Xu et al. | Light-YOLOv3: fast method for detecting green mangoes in complex scenes using picking robots | |
| CN110458107A (en) | Method and apparatus for image recognition | |
| CN107016405A (en) | A kind of insect image classification method based on classification prediction convolutional neural networks | |
| CN110188209A (en) | Cross-module state Hash model building method, searching method and device based on level label | |
| CN110209859A (en) | The method and apparatus and electronic equipment of place identification and its model training | |
| CN111310518B (en) | Picture feature extraction method, target re-identification method, device and electronic equipment | |
| CN108509892A (en) | Method and apparatus for generating near-infrared image | |
| CN109740681A (en) | A kind of fruit method for sorting, device, system, terminal and storage medium | |
| CN112347888A (en) | Remote sensing image scene classification method based on bidirectional feature iterative fusion | |
| CN108280451A (en) | Semantic segmentation and network training method and device, equipment, medium, program | |
| US11941865B2 (en) | Hyperspectral image classification method based on context-rich networks | |
| CN104063686A (en) | System and method for performing interactive diagnosis on crop leaf segment disease images | |
| Ma et al. | Unsupervised domain adaptation augmented by mutually boosted attention for semantic segmentation of vhr remote sensing images | |
| CN110647941A (en) | Vegetable identification method and equipment based on convolutional neural network | |
| CN114445664A (en) | Image classification and identification method and device based on adaptive dynamic convolution network and computer equipment | |
| CN107766828A (en) | UAV Landing Geomorphological Classification method based on wavelet convolution neutral net | |
| CN108171208A (en) | Information acquisition method and device | |
| CN108363962B (en) | Face detection method and system based on multi-level feature deep learning | |
| CN113435254A (en) | Sentinel second image-based farmland deep learning extraction method | |
| CN112668638A (en) | Image aesthetic quality evaluation and semantic recognition combined classification method and system | |
| CN110019849A (en) | A kind of video concern moment search method and device based on attention mechanism | |
| CN115393635A (en) | Infrared small target detection method based on super-pixel segmentation and data enhancement | |
| CN110688966A (en) | Semantic-guided pedestrian re-identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210402 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |