CN111581410B - Image retrieval method, device, medium and system thereof - Google Patents
Image retrieval method, device, medium and system thereof Download PDFInfo
- Publication number
- CN111581410B CN111581410B CN202010475291.5A CN202010475291A CN111581410B CN 111581410 B CN111581410 B CN 111581410B CN 202010475291 A CN202010475291 A CN 202010475291A CN 111581410 B CN111581410 B CN 111581410B
- Authority
- CN
- China
- Prior art keywords
- database
- search
- retrieval
- image
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000000605 extraction Methods 0.000 claims abstract description 12
- 230000015654 memory Effects 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000000875 corresponding effect Effects 0.000 description 95
- 238000010586 diagram Methods 0.000 description 6
- 239000011521 glass Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000037308 hair color Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2452—Query translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2468—Fuzzy queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Automation & Control Theory (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to an image retrieval method and device, medium and system thereof. The method comprises the following steps: at least one image to be processed is obtained, and structural feature extraction is carried out on the image to be processed to obtain a plurality of corresponding structural information; converting the plurality of structured information into keywords, and concatenating the keywords into a string; inserting the character strings into a first database, and establishing indexes for the character strings in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one, and the indexes are related to the identification information; determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string; inputting the search character string into a first database to search out a target index, and determining information about the target identification based on the target index; and based on the target identification information determined in the first database, retrieving key information corresponding to the retrieval condition in the second database.
Description
Technical Field
The present application relates to the field of computer data information retrieval, and in particular, to an image retrieval method, and an apparatus, medium and system thereof.
Background
With the development of the internet and multimedia technology, multimedia information, particularly image information, is being generated and propagated at a speed of explosion. The image retrieval technology can enable a user to quickly and accurately find and access a required image in various image information, and the requirements of the user on the speed of image retrieval and processing are higher and higher.
In the related art, storage and retrieval of image structural information is generally performed using a conventional relational database. Because the traditional relational database does not support field expansion, the structured fields cannot be directly added, deleted and modified, and old data needs to be migrated when fields are newly added. And in the process of structured information retrieval, a fuzzy query mode is generally adopted, so that the efficiency is low.
Disclosure of Invention
The embodiment of the application provides an image retrieval method, an image retrieval device, an image retrieval medium and an image retrieval system. The technical scheme of the application is that the structural information of the image is classified and converted into keywords, then the keywords corresponding to the structural information are connected into character strings by adopting special characters (such as spaces), and the character strings are inserted into a first database (such as a full text search database) and are indexed. After the search condition is determined, the search condition is converted into a search keyword corresponding to the structured information, and the search keyword is connected into a search string using special characters (e.g., spaces). Searching the corresponding target index in the first database by the search keyword, determining the identification information corresponding to the target index, and searching the key information (such as the storage position, the appearance time and the like of the image to be searched) of the target image corresponding to the search condition in the second database (such as a KV database) based on the identification information. Compared with the traditional relational database (such as SQL database) for image retrieval, the method adopts the fuzzy query mode to search the fields in the database one by one. The scheme can quickly and effectively retrieve the related information of the target image.
In a first aspect, an embodiment of the present application provides an image retrieval method, including: at least one image to be processed is obtained, and structural feature extraction is carried out on the image to be processed, so that a plurality of pieces of structural information corresponding to the image to be processed one by one are obtained; converting each of the plurality of structured information into a keyword according to a preset classification, and connecting the keywords corresponding to each of the plurality of structured information into a character string; inserting the character strings into a first database, and establishing indexes for the character strings in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one, and the indexes and the identification information corresponding to the character strings are related; determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string; inputting the search character string into a first database for searching, obtaining a target index corresponding to the search character string, and determining target identification information related to the target index based on the obtained target index; and based on the target identification information determined in the first database, retrieving key information of the target image corresponding to the retrieval condition in the second database.
In a possible implementation of the first aspect, the method further includes: the first database includes a full text search database, inserting the character string into the first database, and indexing the character string in the first database includes:
the character string is inserted into the full text retrieval database, and an index is established for the character string in the full text retrieval database.
In a possible implementation of the first aspect, the method further includes: the second database comprises a KV database, and the key information of the target image corresponding to the retrieval condition is retrieved from the second database based on the target identification information determined in the first database, wherein the key information comprises the following components:
and searching Key information of the target image corresponding to the search condition in the KV database based on the target identification information determined in the full-text search database, wherein the Key of the KV database is taken as the target identification information, and the Key information of the target image corresponding to the search condition is taken as the Value of the KV database. During retrieval, after the Key (namely the target identification information) is determined in the full-text retrieval database, the Value corresponding to the Key can be directly obtained, namely the Key information corresponding to the target image under the retrieval condition is retrieved.
In a possible implementation of the first aspect, the method further includes: the key information corresponding to the search condition includes: at least one of storage location information of a search target corresponding to the search condition and time information at which the search target corresponding to the search condition appears.
In a possible implementation of the first aspect, the method further includes: connecting keywords corresponding to each of the plurality of structured information into a string of characters includes: keywords corresponding to each of the plurality of structured information are concatenated into a string by a special connector.
In a possible implementation of the first aspect, the method further includes: the step of connecting the search keywords into a search string includes: the search keywords are connected into a search string through special connectors.
In a possible implementation of the first aspect, the method further includes: the special character includes a space.
In a second aspect, an embodiment of the present application provides an image retrieval apparatus, including:
the feature extraction module is used for obtaining at least one image to be processed, carrying out structural feature extraction on the image to be processed, and obtaining a plurality of pieces of structural information corresponding to the image to be processed one by one;
The keyword conversion module is used for converting each of the plurality of structured information into keywords according to preset classification and connecting the keywords corresponding to each of the plurality of structured information into character strings;
the index creating module is used for inserting the character strings into the first database and creating indexes for the character strings in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one and is related to the indexes and the identification information corresponding to the character strings;
a search condition processing module for determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string;
the first retrieval module is used for inputting the retrieval character string into the first database for retrieval, obtaining a target index corresponding to the retrieval character string, and determining target identification information related to the target index based on the obtained target index;
and the second retrieval module is used for retrieving the key information of the target image corresponding to the retrieval condition from the second database based on the target identification information determined in the first database.
In a possible implementation manner of the second aspect, the apparatus further includes: the first database includes a full text search database, and the index creation module inserts character strings into the first database and indexes the character strings in the first database by:
the character string is inserted into the full text retrieval database, and an index is established for the character string in the full text retrieval database.
In a possible implementation manner of the second aspect, the apparatus further includes: the second database comprises a KV database, and the second retrieval module retrieves key information of the target image corresponding to the retrieval condition from the second database based on the target identification information determined in the first database by the following method:
and searching Key information of the target image corresponding to the search condition in the KV database based on the target identification information determined in the full-text search database, wherein the Key of the KV database is taken as the target identification information, and the Key information of the target image corresponding to the search condition is taken as the Value of the KV database.
In a possible implementation manner of the second aspect, the apparatus further includes: the key information corresponding to the search condition includes: at least one of storage location information of a search target corresponding to the search condition and time information at which the search target corresponding to the search condition appears.
In a possible implementation manner of the second aspect, the apparatus further includes: the keyword conversion module connects keywords corresponding to each of the plurality of structured information into a string by:
keywords corresponding to each of the plurality of structured information are concatenated into a string by a special connector.
In a possible implementation manner of the second aspect, the apparatus further includes: the search condition processing module connects the search keywords into a search string by:
the search keywords are connected into a search string through special connectors.
In a possible implementation manner of the second aspect, the apparatus further includes: the special character includes a space.
In a third aspect, embodiments of the present application provide a machine-readable medium having stored thereon instructions which, when executed on a machine, cause the machine to perform the above-described first aspect and possibly the image retrieval methods in implementations of the first aspect.
In a fourth aspect, embodiments of the present application provide a system comprising:
a memory for storing instructions for execution by one or more processors of the system, and
a processor, which is one of the processors of the system, for executing the image retrieval method in the first aspect and possible implementations of the first aspect.
Drawings
FIG. 1 illustrates an image retrieval scene graph according to some embodiments of the application;
FIG. 2 illustrates a flow chart of an image retrieval method, according to some embodiments of the application;
FIG. 3 illustrates a block diagram of an image retrieval device, according to some embodiments of the application;
FIG. 4 illustrates a block diagram of a system, according to some embodiments of the application;
fig. 5 illustrates a block diagram of a system on a chip (SoC) in accordance with some embodiments of the present application.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
Illustrative embodiments of the application include, but are not limited to, an image retrieval method and apparatus, medium and system thereof. The technical scheme of the application is that the structural information of the image is classified and converted into keywords, then the keywords corresponding to the structural information are connected into character strings by adopting special characters (such as spaces), and the character strings are inserted into a first database (such as a full text search database) and are indexed. After the search condition is determined, the search condition is converted into a search keyword corresponding to the structured information, and the search keyword is connected into a search string using special characters (e.g., spaces). Searching the corresponding target index in the first database by the search keyword, determining the identification information corresponding to the target index, and searching the key information (such as the storage position, the appearance time and the like of the image to be searched) of the target image corresponding to the search condition in the second database (such as a KV database) based on the identification information. Compared with the traditional relational database (such as SQL database) for image retrieval, the method adopts the fuzzy query mode to search the fields in the database one by one. The scheme can quickly and effectively retrieve the related information of the target image.
Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Fig. 1 discloses an image retrieval scenario, according to some embodiments of the application. In the scenario shown in fig. 1, comprising a terminal 100 and an image acquisition device 101 (e.g. a camera), the terminal 100 is connected to the image acquisition device 101. Wherein the image capturing apparatus 101 captures an image and transmits the captured image to the terminal 100. For example, the camera transmits the acquired image (e.g., a face image, etc.) to the terminal 100 in real time.
The terminal 100 may extract structural information of the images from the received large number of original images, and key information associated with each image, such as storage location of the images, time of occurrence, etc. In some embodiments, the structured information extraction may be performed on a large number of original images through a three-dimensional convolutional neural network model, resulting in structured information corresponding to each original image.
In some embodiments, the structured information includes various characteristic attribute information of pedestrians, such as human body characteristics, clothing characteristics, ornament characteristics, and carrying characteristics, etc. Human body characteristics include hair, face, limbs, gender, etc. The clothing features include: features of a coat, pants, dress, shoe, etc. The ornament features include: hats, sunglasses, eyeglasses, scarves, belts, waistbands, etc. The carrying object features include: features of single shoulder bags, backpack, handbags, draw-bar boxes, umbrellas, etc. In some embodiments, the structured information includes features of the vehicle. For example, a license plate number, a body color, a vehicle brand, a vehicle type, a sub-brand, a vehicle year, and various vehicle feature information (e.g., annual check marks, sun visors, pendants, ornaments, tissue boxes, seat belts, etc.); the type of non-motor vehicle, the body color, etc.
In some embodiments, the terminal 100 classifies each of the plurality of structured information according to a preset condition, resulting in a plurality of classification data corresponding to each structured information. In some embodiments, each structured message may be classified according to decimal, resulting in a plurality of decimal classification data corresponding to each structured message. For example, whether or not a person wears glasses is divided into 2 kinds, 0 means that the person does not wear glasses, and 1 means that the person wears glasses; dividing whether people have beards into 2 types, wherein 0 indicates that the people have beards, and 1 indicates that the people do not have beards; the types of hairstyles of humans are divided into 7: 0 for head, 1 for bald, 2 for cun head, 3 for short hair, 4 for medium and long hair, 5 for long hair, and 6 for braid.
Then converting a plurality of classification data (e.g., decimal data) corresponding to each structured information into keywords, e.g., in some embodiments, classifying the person's hairstyle into the 7 types described above, and then converting the 7 hairstyles into shapen, respectively; BALD; BUZZCUT; SHORT; MEDIUM; TRESS; and SNOOD.
And at least two of the plurality of keywords corresponding to each of the plurality of structured information are connected into a character string by a special connector (e.g., a space). And inserting the character strings into a first database, and establishing indexes for the character strings in the first database, wherein the first database comprises identification information (such as ID information of the images) corresponding to the acquired target images one by one, and the indexes corresponding to the character strings are related to the identification information. Wherein the first database comprises a full text retrieval database.
When a user needs to search an image, firstly determining the search condition of the image to be searched, converting the determined search condition into a corresponding search keyword according to structural information by a method similar to the above description, and connecting the search keyword into a search character string through a special connector. During retrieval, a retrieval character string corresponding to an image to be retrieved is input into a first database, and identification information related to the index is determined based on the index retrieved in the first database; and based on the identification information determined in the first database, retrieving key information corresponding to the image to be retrieved in the second database, and obtaining a corresponding image through the key information corresponding to the image to be retrieved in the second database. Thus, the image retrieval speed can be greatly improved. In some embodiments, the second database may be a KV database.
It will be appreciated that the terminal 100 shown in fig. 1 includes, but is not limited to: small computing terminal devices (e.g., image processing boxes, etc.), servers, cell phones, tablet computers, laptop computers, desktop computers, personal digital assistants, virtual reality or augmented reality devices, electronic devices such as televisions having one or more processors embedded or coupled therein, and the like.
It will be appreciated that the image retrieval scenario shown in fig. 1 is only one example of a scenario in which embodiments of the present application may be implemented, and that embodiments of the present application are not limited to the scenario shown in fig. 1. In other embodiments, the scenario illustrated in FIG. 1 may include more or fewer devices or components than the illustrated embodiments, or certain components may be combined, certain components may be split, or different component arrangements.
Fig. 2 illustrates a flow chart of an image retrieval method, according to some embodiments of the application. As shown in fig. 2, the method specifically includes;
1) At least one image to be processed is acquired, and structural feature extraction is carried out on the image to be processed, so that a plurality of pieces of structural information (202) corresponding to the image to be processed one by one are obtained. Wherein the image to be processed may be acquired from an image acquisition device. For example, a large number of images acquired over a period of time by a monitoring camera. In some embodiments, the three-dimensional convolutional neural network model is used to extract structural information of a large number of images to be processed, so as to obtain structural information of each image to be processed, such as human body features, clothing features, ornament features, carrying features and the like. It will be appreciated that each image corresponds to a set of structured information, for example image a corresponds to: long hair, hat, skirt, middle-aged women; image B corresponds to: wearing glasses, retaining beards, pulling dogs, and old men.
2) Each of the plurality of structured information is converted into a keyword according to a preset classification, and the keywords corresponding to each of the plurality of structured information are concatenated into a character string (204). Wherein the special character may be a space.
In some embodiments, each of the plurality of structured information may be classified in decimal terms, resulting in a plurality of decimal data corresponding to each structured information. For example, in some embodiments, the person's age may be divided into 7 (0 to 15 years old, 1 to 20 years old, 2 to 30 years old, 3 to 35 years old, 4 to 35 years old, 5 to 45 years old, 6 to 55 years old), sex into two (0 to men, 1 to women, etc.), body state into 3 (0 to fat, 1 to thin, 2 to middle), upper body coat color into 9 (0 to black, 1 to white, 2 to red, 3 to yellow, 4 to blue, 5 to green, 6 to purple, 7 to brown, 8 to polychromic, etc.), hair style characteristics into 7 (0 to optical head, 1 to bald, 2 to size, 3 to short hair, 4 to long hair, 5 to long hair, 6 to zai), hair color into 8 (0 to black, 1 to white, 2 to red, 3 to yellow, 4 to blue, 5 to green, 6 to purple, 7 to brown, etc.), face to the front (0 to the back, 3 to the front, etc.).
The classification result of each structured message is then converted to a keyword, e.g., in some embodiments, the 7 ages of the above-described partitions are represented as KID, kid_ YOUTH, YOUTH, YOUTH _ ADULT, ADULT, ADULT _old, OLD. The 2 sexes of the human are denoted as MALE, FEMALE. The classification value of hair color is denoted as BLACK, WHITE, BROWN, YELLOW, PURPLE, BLUE. The classification value of whether the above-mentioned divided person has a beard or not is expressed as: BEARD_YES, BEARD_NO.
In some embodiments, keywords corresponding to each of the plurality of structured information may be concatenated into a string using a special connector, e.g., "middle aged men with beard" represented by a string as: "ADULT BEARD_YES MALE".
It can be understood that a plurality of pieces of structured information can be extracted from one image, the structured information is classified according to a preset rule, and then the structured information is represented by keywords.
It can be understood that the structural information of the image related to the image retrieval method provided by the embodiment of the present application is not limited to the structural information described in the above embodiment. And, the classification manner of the structured information is not limited to the classification manner described in the above embodiment either.
3) The character string is inserted into a first database, and an index is established for the character string in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one, and the index corresponding to the character string is related to the identification information (206).
In some embodiments, the first database comprises a full text retrieval database. In some embodiments, the identification information is an ID created for each string. In some embodiments, for a full text search database, the string of structured information converted may be passed to a word segmentation component (Tokenizer) that segments the document into individual words based on special characters. The result obtained after the word segmentation is called Token (Token). The obtained word elements are transmitted to a language processing component for language processing, and the result processed by the language processing component is called word (Term) (namely key words). The obtained word (Term) is transmitted to an index component (index), the index component firstly creates the obtained word as a mapping dictionary of character strings and keyword IDs, and finally an inverted list is formed.
4) A search condition is determined, the search condition is converted into a search keyword corresponding to the structured information, and the search keyword is connected into a search string (208).
In some embodiments, when a user wants to find an image, a search condition corresponding to the image is determined, the determined search condition is converted into a corresponding search keyword according to structural information in a similar manner to the above description, and the search keywords are connected into a search string through a special connector.
5) Inputting the search string into the first database for searching, obtaining a target index corresponding to the search string, and determining target identification information related to the target index based on the obtained target index (210)
In some embodiments, a search string is entered in a full-text search engine (e.g., lucene), a target index corresponding to the search string is retrieved, and in some embodiments, the target index is zero, i.e., no target index corresponding to a search condition is retrieved in the full-text search engine. In some embodiments, the target index is multiple, i.e. multiple target indexes corresponding to the search condition are searched in the full text search engine. ID information associated with the index is determined based on the index retrieved in the full-text search engine.
6) Key information of the target image corresponding to the search condition is searched in the second database based on the target identification information determined in the first database (212).
In some embodiments, key information corresponding to the image to be retrieved is retrieved in the second database based on the ID information determined in the first database. The key information comprises information such as storage positions of pictures, appearance time and the like. In particular, in some embodiments, ID information in the full text search database may be correlated with the KV database. Because the KV database only has two attributes of Key-Value, key information corresponding to the image to be searched can be used as Value, and ID information in the full-text search database can be used as Key. It can be understood that after the ID information corresponding to the search condition is determined in the full-text search database, the Value corresponding to the ID information can be searched in the KV database through the ID information, that is, the key information of the target image corresponding to the search condition is searched, and the target image to be searched finally can be directly found through the key information, without comparing the key information of the images one by one in the KV database.
Fig. 3 illustrates a block diagram of an image retrieval device, according to some embodiments of the application. As shown in fig. 3, specifically, the method includes:
the feature extraction module 302 is configured to obtain at least one image to be processed, and perform structural feature extraction on the image to be processed to obtain a plurality of pieces of structural information corresponding to the image to be processed one by one;
A keyword conversion module 304, configured to convert each of the plurality of structured information into a keyword according to a preset classification, and connect the keywords corresponding to each of the plurality of structured information into a character string; wherein the keyword conversion module connects keywords corresponding to each of the plurality of structured information into a string by: keywords corresponding to each of the plurality of structured information are concatenated into a string by a special connector. In some embodiments, the special character includes a space.
The index creating module 306 is configured to insert a character string into a first database, and create an index for the character string in the first database, where the first database includes identification information corresponding to the acquired images to be processed one by one, and the index and the identification information corresponding to the character string are related; wherein the first database comprises a full text search database, the index creation module inserts character strings into the first database, and indexes the character strings in the first database by: the character string is inserted into the full text retrieval database, and an index is established for the character string in the full text retrieval database.
A search condition processing module 308 for determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string; the search condition processing module connects the search keywords into a search string by: the search keywords are connected into a search string through special connectors. In some embodiments, the special character includes a space.
A first search module 310, configured to input a search string into a first database for searching, obtain a target index corresponding to the search string, and determine target identification information related to the target index based on the obtained target index;
the second retrieving module 312 is configured to retrieve key information of the target image corresponding to the retrieval condition from the second database based on the target identification information determined in the first database. The second database comprises a KV database, and the second retrieval module retrieves key information of the target image corresponding to the retrieval condition from the second database based on the target identification information determined in the first database by the following method: and searching Key information of the target image corresponding to the search condition in the KV database based on the target identification information determined in the full-text search database, wherein the Key of the KV database is taken as the target identification information, and the Key information of the target image corresponding to the search condition is taken as the Value of the KV database. Wherein, the key information corresponding to the search condition includes: at least one of storage location information of a search target corresponding to the search condition and time information at which the search target corresponding to the search condition appears.
It can be understood that the image retrieval device 300 shown in fig. 3 corresponds to the image retrieval method provided by the present application, and the technical details in the above detailed description about the image retrieval method provided by the present application still apply to the image retrieval device 300 shown in fig. 3, and the detailed description is referred to above and will not be repeated here.
Fig. 4 is a block diagram of a system 400 according to some embodiments of the application. In some embodiments, system 400 may include one or more processors 404, system control logic 408 coupled to at least one of processors 404, system memory 412 coupled to system control logic 408, non-volatile memory (NVM) 416 coupled to system control logic 408, and network interface 420 coupled to system control logic 408.
In some embodiments, processor 404 may include one or more single-core or multi-core processors. In some embodiments, processor 404 may include any combination of general-purpose and special-purpose processors (e.g., graphics processor, application processor, baseband processor, etc.). The processor 404 may be configured to perform feature extraction on the acquired image to be processed to obtain a plurality of structured information corresponding to the image to be processed one by one; each of the plurality of structured information is converted into a keyword according to a preset classification, and the keywords corresponding to each of the plurality of structured information are connected into a character string. The processor 404 is further configured to insert a character string into the first database, and index the character string in the first database, where the first database includes identification information corresponding to the acquired image to be processed one by one, and the index corresponding to the character string and the identification information are related. When image retrieval is required, the processor 404 converts the retrieval conditions into retrieval keywords corresponding to the structured information according to the determined retrieval conditions, and connects the retrieval keywords into a retrieval character string; inputting the search character string into a first database for searching, obtaining a target index corresponding to the search character string, and determining target identification information related to the target index based on the obtained target index; and based on the target identification information determined in the first database, retrieving key information of the target image corresponding to the retrieval condition in the second database.
In some embodiments, system control logic 408 may include any suitable interface controller to provide any suitable interface to at least one of processors 404 and/or any suitable device or component in communication with system control logic 408.
In some embodiments, system control logic 408 may include one or more memory controllers to provide an interface to system memory 412. The system memory 412 may be used to load and store data and/or instructions. The memory 412 of the system 400 may include any suitable volatile memory in some embodiments, such as a suitable Dynamic Random Access Memory (DRAM).
NVM/memory 416 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions. For example, the acquired picture information is stored. In some embodiments, NVM/memory 416 may include any suitable nonvolatile memory, such as flash memory, and/or any suitable nonvolatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) Drive, a DVD (Digital Versatile Disc ) Drive.
NVM/memory 416 may include a portion of a storage resource on the device of installation system 400 or it may be accessed by, but not necessarily part of, the apparatus. For example, NVM/storage 416 may be accessed over a network via network interface 420.
In particular, system memory 412 and NVM/storage 416 may each include: a temporary copy and a permanent copy of instructions 424. The instructions 424 may include: instructions that, when executed by at least one of the processors 404, cause the system 400 to implement the method shown in fig. 2. In some embodiments, instructions 424, hardware, firmware, and/or software components thereof may additionally/alternatively be disposed in system control logic 408, network interface 420, and/or processor 404.
Network interface 420 may include a transceiver to provide a radio interface for system 400 to communicate with any other suitable device (e.g., front end module, antenna, etc.) over one or more networks. In some embodiments, network interface 420 may be integrated with other components of system 400. For example, network interface 420 may be integrated with at least one of processor 404, system memory 412, nvm/storage 416, and a firmware device (not shown) having instructions that, when executed by at least one of processor 404, implement an image retrieval method as shown in fig. 2.
Network interface 420 may further include any suitable hardware and/or firmware to provide a multiple-input multiple-output radio interface. For example, network interface 420 may be a network adapter, a wireless network adapter, a telephone modem, and/or a wireless modem.
In one embodiment, at least one of the processors 404 may be packaged together with logic for one or more controllers of the system control logic 408 to form a System In Package (SiP). In one embodiment, at least one of the processors 404 may be integrated on the same die with logic for one or more controllers of the system control logic 408 to form a system on a chip (SoC).
The system 400 may further include: input/output (I/O) devices 432. The I/O device 432 may include a user interface to enable a user to interact with the system 400; the design of the peripheral component interface enables peripheral components to also interact with the system 400. In some embodiments, the system 400 further includes a sensor for determining at least one of environmental conditions and location information associated with the system 400.
Fig. 5 shows a block diagram of a SoC (System on Chip) 500, in accordance with an embodiment of the present application. In fig. 5, similar parts have the same reference numerals. In addition, the dashed box is an optional feature of a more advanced SoC. In fig. 5, the SoC500 includes: an interconnect unit 550 coupled to the application processor 510; a system agent unit 570; bus controller unit 580; an integrated memory controller unit 540; a set or one or more coprocessors 520 which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a Static Random Access Memory (SRAM) unit 530; a Direct Memory Access (DMA) unit 560. In one embodiment, coprocessor 520 includes a special-purpose processor, such as, for example, a network or communication processor, compression engine, GPU, high-throughput MIC processor, embedded processor, or the like.
Embodiments of the disclosed mechanisms may be implemented in hardware, software, firmware, or a combination of these implementations. Embodiments of the application may be implemented as a computer program or program code that is executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
Program code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.
The program code may be implemented in a high level procedural or object oriented programming language to communicate with a processing system. Program code may also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described in the present application are not limited in scope by any particular programming language. In either case, the language may be a compiled or interpreted language.
In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy diskettes, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random Access Memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information in the form of an electrical, optical, acoustical or other propagated signal using the internet, such as carrier waves, infrared signal digital signals, etc. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).
In the drawings, some structural or methodological features may be shown in a particular arrangement and/or order. However, it should be understood that such a particular arrangement and/or ordering may not be required. Rather, in some embodiments, these features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, may not be included or may be combined with other features.
It should be noted that, in the embodiments of the present application, each unit/module mentioned in each device is a logic unit/module, and in physical terms, one logic unit/module may be one physical unit/module, or may be a part of one physical unit/module, or may be implemented by a combination of multiple physical units/modules, where the physical implementation manner of the logic unit/module itself is not the most important, and the combination of functions implemented by the logic unit/module is only a key for solving the technical problem posed by the present application. Furthermore, in order to highlight the innovative part of the present application, the above-described device embodiments of the present application do not introduce units/modules that are less closely related to solving the technical problems posed by the present application, which does not indicate that the above-described device embodiments do not have other units/modules.
It should be noted that in the examples and descriptions of this patent, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
While the application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.
Claims (14)
1. An image retrieval method, the method comprising:
at least one image to be processed is obtained, and structural feature extraction is carried out on the image to be processed to obtain a plurality of pieces of structural information corresponding to the image to be processed one by one;
converting each of the plurality of structured information into keywords according to a preset classification, and connecting the keywords corresponding to each of the plurality of structured information into character strings;
inserting the character strings into a first database, and establishing indexes for the character strings in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one, and the indexes corresponding to the character strings are related to the identification information; the first database comprises a full text retrieval database;
determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string;
inputting the search character string into the first database for searching, obtaining a target index corresponding to the search character string, and determining target identification information related to the target index based on the obtained target index;
Retrieving key information of a target image corresponding to the retrieval condition from a second database based on the target identification information determined in the first database;
the second database comprises a KV database, and the searching of the key information of the target image corresponding to the searching condition in the second database based on the target identification information determined in the first database comprises the following steps:
and searching Key information of the target image corresponding to the search condition in a KV database based on the target identification information determined in the full-text search database, wherein the Key of the KV database is taken as the target identification information, and the Key information of the target image corresponding to the search condition is taken as the Value of the KV database in the KV database.
2. The image retrieval method of claim 1, wherein the inserting the character string into a first database and indexing the character string in the first database comprises:
and inserting the character string into a full text retrieval database, and establishing an index for the character string in the full text retrieval database.
3. The image retrieval method according to claim 1, wherein the key information of the target image corresponding to the retrieval condition includes: at least one of storage location information of a search target corresponding to the search condition and time information at which the search target corresponding to the search condition appears.
4. The image retrieval method according to claim 1, wherein the concatenating keywords corresponding to each of the plurality of structured information into a character string includes:
and connecting keywords corresponding to each of the plurality of structured information into a character string through special connectors.
5. The image retrieval method according to claim 1, wherein the connecting the retrieval keywords into a retrieval string comprises:
and connecting the search keywords into a search character string through special connectors.
6. The image retrieval method as recited in claim 4 or 5, wherein the special connector includes a space.
7. An image retrieval apparatus, the apparatus comprising:
the feature extraction module is used for obtaining at least one image to be processed, carrying out structural feature extraction on the image to be processed, and obtaining a plurality of pieces of structural information corresponding to the image to be processed one by one;
A keyword conversion module, configured to convert each of the plurality of structured information into a keyword according to a preset classification, and connect the keywords corresponding to each of the plurality of structured information into a character string;
the index creation module is used for inserting the character strings into a first database and establishing indexes for the character strings in the first database, wherein the first database comprises identification information corresponding to the acquired images to be processed one by one, and the indexes corresponding to the character strings are related to the identification information; the first database comprises a full text retrieval database;
a search condition processing module for determining a search condition, converting the search condition into a search keyword corresponding to the structured information, and connecting the search keyword into a search string;
the first retrieval module is used for inputting the retrieval character string into the first database for retrieval, obtaining a target index corresponding to the retrieval character string, and determining target identification information related to the target index based on the obtained target index;
the second retrieval module is used for retrieving key information of the target image corresponding to the retrieval condition from a second database based on the target identification information determined in the first database;
The second database comprises a KV database, and the searching of the key information of the target image corresponding to the searching condition in the second database based on the target identification information determined in the first database comprises the following steps:
and searching Key information of the target image corresponding to the search condition in a KV database based on the target identification information determined in the full-text search database, wherein the Key of the KV database is taken as the target identification information, and the Key information of the target image corresponding to the search condition is taken as the Value of the KV database in the KV database.
8. The image retrieval device of claim 7, wherein the index creation module inserts the character string into a first database and indexes the character string in the first database by:
and inserting the character string into a full text retrieval database, and establishing an index for the character string in the full text retrieval database.
9. The image retrieval device according to claim 7, wherein the key information of the target image corresponding to the retrieval condition includes: at least one of storage location information of a search target corresponding to the search condition and time information at which the search target corresponding to the search condition appears.
10. The image retrieval apparatus according to claim 7, wherein the keyword conversion module connects keywords corresponding to each of the plurality of structured information into character strings by:
and connecting keywords corresponding to each of the plurality of structured information into a character string through special connectors.
11. The image retrieval device according to claim 7, wherein the retrieval condition processing module connects the retrieval keywords into a retrieval character string by:
and connecting the search keywords into a search character string through special connectors.
12. The image retrieval device of claim 10 or 11, wherein the special connector comprises a space.
13. A machine-readable medium having stored thereon instructions which, when executed on a machine, cause the machine to perform the image retrieval method of any of claims 1 to 6.
14. A system, comprising:
a memory for storing instructions for execution by one or more processors of the system, and
a processor, which is one of the processors of the system, for performing the image retrieval method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010475291.5A CN111581410B (en) | 2020-05-29 | 2020-05-29 | Image retrieval method, device, medium and system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010475291.5A CN111581410B (en) | 2020-05-29 | 2020-05-29 | Image retrieval method, device, medium and system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111581410A CN111581410A (en) | 2020-08-25 |
| CN111581410B true CN111581410B (en) | 2023-11-14 |
Family
ID=72111226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010475291.5A Active CN111581410B (en) | 2020-05-29 | 2020-05-29 | Image retrieval method, device, medium and system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111581410B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112286506B (en) * | 2020-10-30 | 2024-05-07 | 杭州海康威视数字技术股份有限公司 | Data association method, device, server and storage medium |
| CN113138637B (en) * | 2021-04-16 | 2024-04-05 | 北京沃东天骏信息技术有限公司 | Computer configuration information processing method, device and storage medium thereof |
| CN114201258B (en) * | 2021-12-10 | 2024-06-07 | 金蝶软件（中国）有限公司 | Copy and paste method, apparatus, computer device and storage medium |
| CN114265952B (en) * | 2022-03-02 | 2023-03-14 | 浙江宇视科技有限公司 | Target retrieval method and device |
| CN117932120A (en) * | 2024-03-18 | 2024-04-26 | 支付宝(杭州)信息技术有限公司 | Data storage method and device of graph database |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107729518A (en) * | 2017-10-26 | 2018-02-23 | 山东浪潮云服务信息科技有限公司 | The text searching method and device of a kind of relevant database |
| CN108255972A (en) * | 2017-12-27 | 2018-07-06 | 浪潮通用软件有限公司 | A kind of text searching method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5799621B2 (en) * | 2011-07-11 | 2015-10-28 | ソニー株式会社 | Information processing apparatus, information processing method, and program |
-
2020
- 2020-05-29 CN CN202010475291.5A patent/CN111581410B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107729518A (en) * | 2017-10-26 | 2018-02-23 | 山东浪潮云服务信息科技有限公司 | The text searching method and device of a kind of relevant database |
| CN108255972A (en) * | 2017-12-27 | 2018-07-06 | 浪潮通用软件有限公司 | A kind of text searching method and system |
Non-Patent Citations (1)
| Title |
|---|
| 郑庆华,张炜.超文本全文检索技术的研究与实现.西安交通大学学报.2001,(04),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111581410A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111581410B (en) | Image retrieval method, device, medium and system thereof | |
| CN106560809B (en) | Modifying at least one attribute of an image with at least one attribute extracted from another image | |
| US9430719B2 (en) | System and method for providing objectified image renderings using recognition information from images | |
| CN106560810B (en) | Searching using specific attributes found in images | |
| US8649572B2 (en) | System and method for enabling the use of captured images through recognition | |
| US7809192B2 (en) | System and method for recognizing objects from images and identifying relevancy amongst images and information | |
| US7809722B2 (en) | System and method for enabling search and retrieval from image files based on recognized information | |
| EP4113325A2 (en) | System and method of saving digital content classified by person-based clustering | |
| TWI761803B (en) | Image processing method and image processing device, processor and computer-readable storage medium | |
| JP7165752B2 (en) | Image processing method and apparatus, processor, storage medium | |
| KR20210002410A (en) | System, method and program of constructing dataset for training appearance recognition model | |
| WO2006122164A2 (en) | System and method for enabling the use of captured images through recognition | |
| EP2084625B1 (en) | Dominant color descriptors | |
| Rubio et al. | Multi-modal joint embedding for fashion product retrieval | |
| CN111444373B (en) | Image retrieval method, device, medium and system thereof | |
| Nie et al. | Deep feature ranking for person re-identification | |
| CN110413818B (en) | Label paper recommending method, device, computer readable storage medium and computer equipment | |
| KR20200141387A (en) | System, method and program for searching image data by using deep-learning algorithm | |
| CN111178409A (en) | Image matching and recognition system based on big data matrix stability analysis | |
| CN113129410B (en) | Sketch image conversion method and related product | |
| WO2022222779A1 (en) | Image generation method and apparatus, and electronic device and computer-readable medium | |
| Wu et al. | A novel image retrieval approach with Bag-of-Word model and Gabor feature | |
| Boini Madhavi | INTEGRATED APPROACH TO CONTENT BASED IMAGE RETRIVAL USING CLUSTERING | |
| Padala et al. | An approach for effective image retrievals based on semantic tagging and generalized gaussian mixture model | |
| CN118152609A (en) | Image generation method, device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |