CN111460185A - Book searching method, device and system - Google Patents
Book searching method, device and system Download PDFInfo
- Publication number
- CN111460185A CN111460185A CN202010235494.7A CN202010235494A CN111460185A CN 111460185 A CN111460185 A CN 111460185A CN 202010235494 A CN202010235494 A CN 202010235494A CN 111460185 A CN111460185 A CN 111460185A
- Authority
- CN
- China
- Prior art keywords
- book
- feature vector
- cover image
- similarity
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5846—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using extracted text
Abstract
The invention discloses a book searching method, a book searching device and a book searching system, wherein the method comprises the following steps: acquiring a cover image of a book; converting the cover image into a feature vector as an input feature vector; and calculating the similarity of the input feature vector and a plurality of pre-stored feature vectors, and determining the target book to be obtained according to the similarity. The invention can search the book cover by using a machine learning method and then search the needed book, the book cover has strict uniqueness relative to the bar code of the book, the efficiency and the accuracy of obtaining book information and information which is not contained in the book, such as analysis of answer knowledge points related to the book and the like by a user can be improved by searching the book through the book cover, high-quality answers and knowledge points can be better transmitted to the user, and the educational fairness can be better realized.
Description
Technical Field
The invention relates to the field of computer information processing, in particular to a book searching method, device, system and system.
Background
With the development of the modern society, the demand of people on the knowledge quantity is continuously increased, more and more people use the network teaching platform to learn, in the process of searching books, the answers of the whole book are obtained once, the analysis of knowledge points can greatly improve the checking efficiency, the repeated labor of obtaining answers aiming at a single topic page is reduced, a user can locate the books to be searched through scanning the bar codes of the books, but the same bar code can correspond to a plurality of books, so that the bar codes do not have strict uniqueness, and the needed books are difficult to be quickly and accurately searched.
Disclosure of Invention
The invention provides a book searching method, device and system, and aims to solve the technical problem of how to quickly and accurately search related information of a target book.
An aspect of the present invention provides a book searching method for searching to obtain a target book based on a cover image of the book, including:
acquiring a cover image of a book;
converting the cover image into a feature vector as an input feature vector;
and calculating the similarity between the input feature vector and a pre-stored feature vector, and determining the target book to be obtained according to the similarity.
According to a preferred embodiment of the present invention, the step of converting the cover image into an input feature vector comprises:
processing the cover image to obtain a pattern feature vector;
acquiring character information in the cover image, and processing the character information to obtain character characteristic vectors;
and combining the pattern feature vector and the character feature vector to obtain the input feature vector.
According to a preferred embodiment of the present invention, the processing the cover image to obtain a pattern feature vector further includes:
processing the cover image through an image vectorization model to obtain a pattern image characteristic vector subjected to special coding;
optionally, the image vectorization model is a convolutional neural network model, and includes at least a VGG model and a resnet model.
According to a preferred embodiment of the present invention, the acquiring the text information in the cover image, and processing the text information to obtain a text feature vector further includes:
carrying out character detection on the book cover image by using an anchor point regression method;
adopting a recurrent neural network to identify the detected characters;
converting the recognized characters into character characteristic vectors by using a word embedding model;
optionally, the combining the pattern feature vector and the text feature vector to obtain the input feature vector further includes:
performing vector splicing on the processed pattern image characteristic vector and the character characteristic vector;
splicing the pattern feature vector and the character feature vector end to obtain an input feature vector converted from the cover image;
the dimension of the input feature vector is equal to the sum of the dimensions of the pattern image feature vector and the character feature vector.
According to a preferred embodiment of the invention, the method further comprises:
establishing a content database, storing book contents in the content database, and distributing different book IDs for the book contents of different books;
converting the cover images of the books in the content database into feature vectors, and establishing a corresponding relationship between the feature vectors and the IDs of the books;
optionally, the step of establishing a corresponding relationship between the feature vector and each book ID includes:
establishing an index database for storing the corresponding relation;
optionally, the index database and the content database are built on the same server.
According to a preferred embodiment of the invention, the method further comprises:
the server acquires a cover image of a book;
converting the cover image into the input feature vector;
calculating the similarity between the input feature vector and the feature vector in the index data, and determining the book ID of the target book to be obtained according to the similarity;
optionally, the step of determining the book ID of the target book to be obtained according to the similarity includes:
the similarity of the input feature vector and a plurality of pre-stored feature vectors is sorted from high to low;
determining books corresponding to the feature vectors with the similarity greater than a preset value with the input feature vectors as target books, or determining books with the similarity in the order of the input feature vectors and the preset number of books in the order of the similarity in the order of the input feature vectors as the target books;
acquiring a book ID corresponding to the target book;
optionally, the similarity is calculated by a cosine distance between the feature vectors.
According to a preferred embodiment of the present invention, after acquiring the book ID corresponding to the target book, the method further includes:
the server acquires the content of the target book from the content database according to the book ID corresponding to the target book and sends the content of the target book to the client according to the request of the client;
the client displays any one of the following of the target book: book ID of the target book, cover image of the target book, book content of the target book.
A second aspect of the present invention provides a book searching apparatus for searching to obtain a target book based on a cover image of the book, comprising:
the image acquisition module is used for acquiring a cover image of the book;
the image conversion module is used for converting the cover image into a characteristic vector as an input characteristic vector;
and the target searching module is used for calculating the similarity between the input feature vector and a plurality of pre-stored feature vectors and determining the target book to be obtained according to the similarity.
A third aspect of the present invention provides a book search system for searching for a target book based on a cover image of the book, comprising a client and a server, wherein,
the client comprises:
the image acquisition module is used for acquiring a cover image of a book;
an image transfer module for sending the cover image to the server;
the content display module is used for displaying the target book acquired from the server to a user;
the server includes:
the feature extraction module is used for receiving the cover image, converting the cover image into a feature vector and sending the feature vector to the feature matching module as an input feature vector;
the characteristic database is used for storing characteristic vectors corresponding to the book cover images;
the characteristic matching module is used for receiving the input characteristic vector, calculating the similarity between the input characteristic vector and a plurality of characteristic vectors in the characteristic database which is stored in advance, and determining a target book to be obtained according to the similarity;
and the content database is used for storing book contents.
A fourth aspect of the present invention provides a client for searching according to a cover image of a book to obtain a target book, including:
the image acquisition module is used for acquiring a cover image of a book;
an image transfer module for sending the cover image to the server;
and the content display module is used for displaying the target book acquired from the server to the user.
A fifth aspect of the present invention provides a book server for providing a book search and book data service to a client, the book server comprising:
the feature extraction module is used for receiving the cover image, converting the cover image into a feature vector and sending the feature vector to the feature matching module as an input feature vector;
the characteristic database is used for storing characteristic vectors corresponding to the book cover images;
the feature matching module is used for calculating the similarity between the input feature vector sent by the feature extraction module and a plurality of feature vectors in the feature database and determining a target book to be obtained according to the similarity;
and the content database is used for storing book contents.
A sixth aspect of the invention provides a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement any of the methods.
The technical scheme of the invention has the following beneficial effects:
the invention searches book covers by using a machine learning method and then searches required books, the book covers have strict uniqueness relative to bar codes of the books, the efficiency and accuracy of obtaining book information and information which is not contained in the books, such as analysis of answer knowledge points related to the books and the like, by searching the books through the book covers can be improved, high-quality answers and knowledge points can be better transmitted to users, and the educational fairness can be better realized.
Drawings
In order to make the technical problems solved by the present invention, the technical means adopted and the technical effects obtained more clear, the following will describe in detail the embodiments of the present invention with reference to the accompanying drawings. It should be noted, however, that the drawings described below are only drawings of exemplary embodiments of the invention, from which other embodiments can be derived by those skilled in the art without inventive step.
Fig. 1 is a schematic view of an application scenario of a book search system according to the present invention;
FIG. 2 is a flow chart of a book searching method according to the present invention;
FIG. 3 is a block diagram of a book searching apparatus according to the present invention;
FIG. 4 is a block diagram of a book search system according to the present invention;
FIG. 5 is a schematic diagram of a computer-readable storage medium of the present invention.
Detailed Description
Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. The exemplary embodiments, however, may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. The same reference numerals denote the same or similar elements, components, or parts in the drawings, and thus their repetitive description will be omitted.
Features, structures, characteristics or other details described in a particular embodiment do not preclude the fact that the features, structures, characteristics or other details may be combined in a suitable manner in one or more other embodiments in accordance with the technical idea of the invention.
In describing particular embodiments, the present invention has been described with reference to features, structures, characteristics or other details that are within the purview of one skilled in the art to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific features, structures, characteristics, or other details.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these terms should not be construed as limiting. These phrases are used to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the present invention.
The term "and/or" and/or "includes any and all combinations of one or more of the associated listed items.
In order that the objects, technical solutions and advantages of the present invention will become more apparent, the present invention will be further described in detail with reference to the accompanying drawings in conjunction with the following specific embodiments.
Fig. 1 is a schematic view of an application scenario of a book search system according to the present invention. As shown in fig. 1, a user opens an online learning client by using a terminal, and searches and positions a required book by uploading a book cover image through an interface of the online learning client, because information such as year, region, publisher, cover background and the like can be visually listed in a cover, the book searching through the cover image has strict uniqueness relative to the barcode searching through the book, the online learning client sends the book cover image uploaded by the user to a server, and the server compares the cover image after vectorization processing through an image vectorization model with a book cover image vector stored in the server, screens out the book cover required by the user, and displays information such as the cover, content and the like of the whole book to the user, so that the user can quickly and accurately find out the required book content.
The book in the embodiment of the invention can be a book in the aspect of test questions, such as a problem book, a test paper and the like, the content of the book is corresponding answers and knowledge point analysis and the like, a user uses a certain problem book to test, the user wants to check the answers and the analysis after answering, but the needed answers and the analysis cannot be found from the problem book, at the moment, the problem book can be searched through an online learning client, a cover image of the problem book is uploaded to the online learning client, the online learning client sends the cover image to a server, the server converts the cover image into a characteristic vector and compares the characteristic vector with the characteristic vector in a database, finally, the corresponding problem book is found, and the test questions, the answers, the knowledge point analysis and the like corresponding to the problem book are sent to the online learning client for the user to look up.
Fig. 2 is a schematic flow chart of a book searching method according to the present invention. As shown in fig. 2, the method includes:
s201, acquiring a cover image of the book.
Specifically, after the user starts the online client, the user can select to search for books, and the online client pops up a dialog box according to the operation of the user for the user to select: uploading book cover image files or shooting book covers, if a user selects to upload the book cover image files, popping up a dialog box at a client, so that the user uploads the book cover image files stored in the terminal to the client, wherein the file format can be a common picture format such as jpg, jpeg, png and the like; if the user selects to shoot the book covers, the client calls the terminal camera shooting tool, the user aims the book covers at the camera shooting tool to shoot and recognize, and the client stores the book cover images successfully shot and uploads the images to the server through the network.
S202, converting the cover image into a feature vector as an input feature vector.
Specifically, firstly, the server processes the book cover image uploaded by the user through the client through an image vectorization model to obtain a specially coded pattern feature vector, the image vectorization model can be a convolutional neural network model (CNN), the convolutional neural network model can specifically select a VGG model, a resnet model and the like, and the accuracy and speed requirements required by large-scale book cover retrieval can be met. Extracting characters in the book cover image, such as the publication year, region, publication agency, book name, author, etc., first using the anchor point regression method to detect the characters in the book cover image, then using a cyclic neural network to identify characters, finally using a word embedding model (word2vec) to convert the identified characters into character characteristic vectors, carrying out vector splicing on the processed pattern characteristic vectors and the character characteristic vectors, the pattern feature vector and the character feature vector can be spliced end to obtain an input feature vector converted from the cover image, the dimension of the input feature vector is equal to the sum of the dimensions of the pattern feature vector and the character feature vector, for example, if the dimension of the pattern feature vector is 256 dimensions and the dimension of the text feature vector is 128 dimensions, the dimension of the input feature vector is 384 dimensions.
Wherein, the pattern means: the book cover image contains the contents of the background in addition to the text. The characters refer to: chinese characters, English letters, and other language characters or numbers.
S203, calculating the similarity between the input feature vector and a plurality of pre-stored feature vectors, and determining the target book to be obtained according to the similarity.
Specifically, before calculating the similarity, a database is first established in the server for storing the feature vectors corresponding to the book cover images and the book contents. The database can be divided into a content database and an index database, and the content database is used for storing related content information such as cover images, answers and knowledge point analysis of all books and corresponding book IDs. The collected books are collected as many as possible, and different book IDs are assigned to different books. For example, firstly, collected books are classified, the collected books can be classified according to the technical field corresponding to the book content, and the collected books can also be classified according to specific characteristics, so that the searching speed is improved conveniently; then different book IDs are allocated to different books, the different books correspond to the book IDs one by one, for example, book ID of book a is 000001, book ID of book B is 000002, then the book cover image of each book is processed by the image vectorization model in sequence to obtain the corresponding feature vector, one of the books corresponds to one feature vector, each feature vector is different, the feature vector obtained here is the same as the method for obtaining the input feature vector in the above embodiment, and is obtained by splicing the pattern feature vector converted from the cover image and the character feature vector, and finally, the corresponding relationship is established between each book ID and the corresponding feature vector, for example, the cover feature vector of the book A is a, when the feature vector is found to be a, the corresponding book ID000001 is obtained, and the index database is used for storing the corresponding relationship.
After the corresponding relation is established and stored, the server processes book cover images uploaded by a user into input feature vectors, sends the input feature vectors to an index database of the server, and sends a feature matching instruction to the index database, the feature vectors are stored in the index database, similarity comparison is carried out on the received input feature vectors and each feature vector stored in the index database, the similarity can be cosine distance between the two feature vectors, the larger the cosine value is, the higher the similarity of the two feature vectors is, and if the value is 1, the two feature vectors are completely the same. After the similarity between the input feature vector and each feature vector stored in the index database is obtained, the similarity ranked first is highest for all the similarities in descending order, for example, if the similarity between the input feature vector corresponding to the book cover image uploaded by the user and the feature vector corresponding to the book a cover image is 80%, the similarity between the feature vector corresponding to the book B cover image is 30%, the similarity between the feature vector corresponding to the book C cover image is 97%, the similarity between the feature vector corresponding to the book D cover image is 52%, the similarity ranking is 97%, 80%, 52%, 30%, and the corresponding book ranking is book C, book a, book D, and book B.
After the similarity ranking is obtained, in order to obtain a target book, a threshold of the similarity may be set, and a book corresponding to a feature vector with the similarity greater than a preset value is determined as the target book, for example, if the threshold is set to 75%, then book C and book a are determined as the target book, and the similarity between book B and book D does not reach the threshold, so that the similarity ranking is not considered; the number of target books may be set, and a predetermined number of books with the highest similarity rank may be determined as target books, for example, if the number of target books is set to 3, book C, book a, and book D may be determined as target books, and book B may be disregarded because of ranking at the 4 th position.
After the target book is determined, the server sends the book ID of the target book to the client, the client acquires the book ID corresponding to the target book and sends the corresponding book ID to the content database of the server, and requests the content database to provide the book content corresponding to the book ID, for example, the client acquires the book IDs of the book a and the book C and acquires the book contents of the book a and the book C through the content database, the covers of the book a and the book C are displayed to the user through the client interface and viewed by the user, and when the user clicks the cover of the book a, the client displays the book content and the related content of the book a to the user; and when the user clicks the cover of the book C, the client displays the book content of the book C to the user. The user can select the needed books through the book covers and the book contents, and meanwhile, the method can enable the user to know the book contents similar to the target books, help the user to expand knowledge and know more knowledge.
Those skilled in the art will appreciate that all or part of the steps to implement the above-described embodiments are implemented as programs (computer programs) executed by a computer data processing apparatus. When the computer program is executed, the method provided by the invention can be realized. Furthermore, the computer program may be stored in a computer readable storage medium, which may be a readable storage medium such as a magnetic disk, an optical disk, a ROM, a RAM, or a storage array composed of a plurality of storage media, such as a magnetic disk or a magnetic tape storage array. The storage medium is not limited to centralized storage, but may be distributed storage, such as cloud storage based on cloud computing.
Embodiments of the apparatus of the present invention are described below, which may be used to perform method embodiments of the present invention. The details described in the device embodiments of the invention should be regarded as complementary to the above-described method embodiments; reference is made to the above-described method embodiments for details not disclosed in the apparatus embodiments of the invention.
Fig. 3 is a schematic diagram of a module architecture of a book searching apparatus according to the present invention. As shown in fig. 3, the apparatus 300 includes:
an image acquisition module 301, configured to acquire a cover image of a book;
an image conversion module 302, configured to convert the cover image into a feature vector as an input feature vector;
and the target searching module 303 is configured to calculate similarities between the input feature vector and a plurality of feature vectors stored in advance, and determine a target book to be obtained according to the similarities.
Specifically, a user opens an online learning client by using a terminal, a book cover image is uploaded by an image acquisition module 301 of the online learning client to search and position a needed book, because the book cover has strict uniqueness relative to a bar code of the book, information such as year, region, publisher, cover background and the like can be visually listed in the cover, an image conversion module 302 of the device 300 carries out vectorization processing on the book cover image uploaded by the user through an image vectorization model to obtain an input characteristic vector, a target search module 303 compares the input characteristic vector with a book cover image vector stored in a server to screen out the book cover needed by the user, and the information such as the cover, content and the like of the whole book is displayed to the user, so that the user can quickly and accurately find out the needed book content.
Those skilled in the art will appreciate that the modules in the above-described embodiments of the apparatus may be distributed as described in the apparatus, and may be correspondingly modified and distributed in one or more apparatuses other than the above-described embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Fig. 4 is a schematic block diagram of a book search system according to the present invention. As shown in fig. 4, the system 400 includes a client 401 and a server 402, wherein,
the client 401 includes:
the image acquisition module 4011 is configured to obtain a cover image of a book;
an image transfer module 4012 configured to send the cover image to the server;
a content display module 4013, configured to display the target book obtained from the server to a user;
the server 402 includes:
the feature extraction module 4021 is configured to receive the cover image, convert the cover image into a feature vector, and send the feature vector to the feature matching module as an input feature vector;
the feature database 4022 is used for storing feature vectors corresponding to the book cover images;
the feature matching module 4023 is configured to receive the input feature vector, calculate similarity between the input feature vector and a plurality of feature vectors in the feature database stored in advance, and determine a target book to be obtained according to the similarity;
the content database 4023 is configured to store the book cover images and the book content.
Specifically, a user opens the online learning client 401 of the book search system 400 by using a terminal, searches and locates a required book by uploading an image of a book cover through the image acquisition module 4011 of the online learning client 401, because the book cover has strict uniqueness relative to a barcode of the book, information such as year, region, publisher, cover background and the like can be visually listed in the cover, the image transmission module 4012 of the online learning client 401 sends the image of the book cover uploaded by the user to the feature extraction module 4021 of the server 402, the feature extraction module 4021 performs vectorization processing through an image vectorization model to obtain an input feature vector and sends the input feature vector to the feature matching module 4023, the feature matching module 4023 obtains a book cover image vector from the feature database 4022 of the server 402 and compares the input feature vector, the target book ID corresponding to the book cover image vector required by the user is screened out and sent to the content display module 4013, the content display module 4013 sends a request for obtaining the target book corresponding to the target book ID to the content database 4024 of the server 402 according to the target book ID, the target book is downloaded from the content database 4024, and information such as the cover and the content of the downloaded target book is displayed to the user, so that the user can quickly and accurately find the required book content.
Another embodiment of the present invention also provides a client for searching according to a cover image of a book to obtain a target book, including:
the image acquisition module is used for acquiring a cover image of a book;
an image transfer module for sending the cover image to the server;
and the content display module is used for displaying the target book acquired from the server to the user.
Another embodiment of the present invention also provides a book server for providing a book search and book data service to a client, the book server including:
the feature extraction module is used for receiving the cover image, converting the cover image into a feature vector and sending the feature vector to the feature matching module as an input feature vector;
the characteristic database is used for storing characteristic vectors corresponding to the book cover images;
the feature matching module is used for calculating the similarity between the input feature vector sent by the feature extraction module and a plurality of feature vectors in the feature database and determining a target book to be obtained according to the similarity;
and the content database is used for storing book contents.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments of the present invention described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the present invention. The computer program, when executed by a data processing apparatus, enables the computer readable medium to implement the above-described method of the invention, namely: a book searching method for searching to obtain a target book based on a cover image of the book, comprising: acquiring a cover image of a book; converting the cover image into a feature vector as an input feature vector; and calculating the similarity of the input feature vector and a plurality of pre-stored feature vectors, and determining the target book to be obtained according to the similarity.
FIG. 5 is a schematic diagram of a computer-readable storage medium of the present invention. The computer program may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, C + +, or the like, as well as conventional procedural programming languages, such as the "C" language or similar programming languages.
In summary, the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functionality of some or all of the components in embodiments in accordance with the invention may be implemented in practice using a general purpose data processing device such as a microprocessor or a Digital Signal Processor (DSP). The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
While the foregoing embodiments have described the objects, aspects and advantages of the present invention in further detail, it should be understood that the present invention is not inherently related to any particular computer, virtual machine or electronic device, and various general-purpose machines may be used to implement the present invention. The invention is not to be considered as limited to the specific embodiments thereof, but is to be understood as being modified in all respects, all changes and equivalents that come within the spirit and scope of the invention.
Claims (10)
1. A book searching method for searching to obtain a target book based on a cover image of the book, comprising:
acquiring a cover image of a book;
converting the cover image into a feature vector as an input feature vector;
and calculating the similarity between the input feature vector and a pre-stored feature vector, and determining the target book to be obtained according to the similarity.
2. The method of claim 1, wherein the step of converting the cover image into an input feature vector comprises:
processing the cover image to obtain a pattern feature vector;
acquiring character information in the cover image, and processing the character information to obtain character characteristic vectors;
and combining the pattern feature vector and the character feature vector to obtain the input feature vector.
3. The method of any of claims 1 to 2, wherein processing the cover image to obtain a pattern feature vector further comprises:
processing the cover image through an image vectorization model to obtain a pattern image characteristic vector subjected to special coding;
optionally, the image vectorization model is a convolutional neural network model, and includes at least a VGG model and a resnet model.
4. The method of any one of claims 1 to 3, wherein the obtaining text information in the cover image and processing the text information to obtain a text feature vector further comprises:
carrying out character detection on the book cover image by using an anchor point regression method;
adopting a recurrent neural network to identify the detected characters;
converting the recognized characters into character characteristic vectors by using a word embedding model;
optionally, the combining the pattern feature vector and the text feature vector to obtain the input feature vector further includes:
performing vector splicing on the processed pattern image characteristic vector and the character characteristic vector;
splicing the pattern feature vector and the character feature vector end to obtain an input feature vector converted from the cover image;
the dimension of the input feature vector is equal to the sum of the dimensions of the pattern image feature vector and the character feature vector.
5. The method according to any one of claims 1 to 4, further comprising:
establishing a content database, storing book contents in the content database, and distributing different book IDs for the book contents of different books;
converting the cover images of the books in the content database into feature vectors, and establishing a corresponding relationship between the feature vectors and the IDs of the books;
optionally, the step of establishing a corresponding relationship between the feature vector and each book ID includes:
establishing an index database for storing the corresponding relation;
optionally, the index database and the content database are built on the same server.
6. The method according to any one of claims 1 to 5, further comprising:
the server acquires a cover image of a book;
converting the cover image into the input feature vector;
calculating the similarity between the input feature vector and the feature vector in the index data, and determining the book ID of the target book to be obtained according to the similarity;
optionally, the step of determining the book ID of the target book to be obtained according to the similarity includes:
the similarity of the input feature vector and a plurality of pre-stored feature vectors is sorted from high to low;
determining books corresponding to the feature vectors with the similarity greater than a preset value with the input feature vectors as target books, or determining books with the similarity in the order of the input feature vectors and the preset number of books in the order of the similarity in the order of the input feature vectors as the target books;
acquiring a book ID corresponding to the target book;
optionally, the similarity is calculated by a cosine distance between the feature vectors.
7. The method according to any one of claims 1 to 6, wherein after acquiring the book ID corresponding to the target book, the method further comprises:
the server acquires the content of the target book from the content database according to the book ID corresponding to the target book and sends the content of the target book to the client according to the request of the client;
the client displays any one of the following of the target book: book ID of the target book, cover image of the target book, book content of the target book.
8. A book searching apparatus for searching to obtain a target book based on a cover image of the book, comprising:
the image acquisition module is used for acquiring a cover image of the book;
the image conversion module is used for converting the cover image into a characteristic vector as an input characteristic vector;
and the target searching module is used for calculating the similarity between the input feature vector and a plurality of pre-stored feature vectors and determining the target book to be obtained according to the similarity.
9. A book searching system for searching to obtain a target book based on a cover image of the book, comprising a client and a server, wherein,
the client comprises:
the image acquisition module is used for acquiring a cover image of a book;
an image transfer module for sending the cover image to the server;
the content display module is used for displaying the target book acquired from the server to a user;
the server includes:
the feature extraction module is used for receiving the cover image, converting the cover image into a feature vector and sending the feature vector to the feature matching module as an input feature vector;
the characteristic database is used for storing characteristic vectors corresponding to the book cover images;
the characteristic matching module is used for receiving the input characteristic vector, calculating the similarity between the input characteristic vector and a plurality of characteristic vectors in the characteristic database which is stored in advance, and determining a target book to be obtained according to the similarity;
and the content database is used for storing book contents.
10. A book server for providing a book search and book data service to a client, the book server comprising:
the feature extraction module is used for receiving the cover image, converting the cover image into a feature vector and sending the feature vector to the feature matching module as an input feature vector;
the characteristic database is used for storing characteristic vectors corresponding to the book cover images;
the feature matching module is used for calculating the similarity between the input feature vector sent by the feature extraction module and a plurality of feature vectors in the feature database and determining a target book to be obtained according to the similarity;
and the content database is used for storing book contents.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010235494.7A CN111460185A (en) | 2020-03-30 | 2020-03-30 | Book searching method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010235494.7A CN111460185A (en) | 2020-03-30 | 2020-03-30 | Book searching method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111460185A true CN111460185A (en) | 2020-07-28 |
Family
ID=71683344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010235494.7A Pending CN111460185A (en) | 2020-03-30 | 2020-03-30 | Book searching method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111460185A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112560902A (en) * | 2020-12-01 | 2021-03-26 | 中国农业科学院农业信息研究所 | Book identification method and system based on spine visual information |
| CN112564898A (en) * | 2020-11-30 | 2021-03-26 | 南京晓庄学院 | Book safe storage method and device and storage medium |
| CN112632317A (en) * | 2021-01-13 | 2021-04-09 | 深圳市万物志科技有限公司 | Multi-target interaction method and device based on user pictures |
| CN113239234A (en) * | 2021-06-04 | 2021-08-10 | 杭州大拿科技股份有限公司 | Method for providing video book and method for establishing video book |
| CN114661879A (en) * | 2022-03-23 | 2022-06-24 | 国网江苏省电力有限公司连云港供电分公司 | Data searching method, system, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090148059A1 (en) * | 2007-12-10 | 2009-06-11 | Sharp Kabushiki Kaisha | Image processing apparatus, image display apparatus, image forming apparatus, image processing method and storage medium |
| CN107220325A (en) * | 2017-05-22 | 2017-09-29 | 华中科技大学 | A kind of similar icon search methods of APP based on convolutional neural networks and system |
| CN107679070A (en) * | 2017-08-22 | 2018-02-09 | 科大讯飞股份有限公司 | A kind of intelligence, which is read, recommends method and apparatus, electronic equipment |
| CN108829764A (en) * | 2018-05-28 | 2018-11-16 | 腾讯科技（深圳）有限公司 | Recommendation information acquisition methods, device, system, server and storage medium |
| CN109255346A (en) * | 2018-08-31 | 2019-01-22 | 深圳闳宸科技有限公司 | Reading method, device and electronic equipment |
| CN109766465A (en) * | 2018-12-26 | 2019-05-17 | 中国矿业大学 | A kind of picture and text fusion book recommendation method based on machine learning |
-
2020
- 2020-03-30 CN CN202010235494.7A patent/CN111460185A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090148059A1 (en) * | 2007-12-10 | 2009-06-11 | Sharp Kabushiki Kaisha | Image processing apparatus, image display apparatus, image forming apparatus, image processing method and storage medium |
| CN107220325A (en) * | 2017-05-22 | 2017-09-29 | 华中科技大学 | A kind of similar icon search methods of APP based on convolutional neural networks and system |
| CN107679070A (en) * | 2017-08-22 | 2018-02-09 | 科大讯飞股份有限公司 | A kind of intelligence, which is read, recommends method and apparatus, electronic equipment |
| CN108829764A (en) * | 2018-05-28 | 2018-11-16 | 腾讯科技（深圳）有限公司 | Recommendation information acquisition methods, device, system, server and storage medium |
| CN109255346A (en) * | 2018-08-31 | 2019-01-22 | 深圳闳宸科技有限公司 | Reading method, device and electronic equipment |
| CN109766465A (en) * | 2018-12-26 | 2019-05-17 | 中国矿业大学 | A kind of picture and text fusion book recommendation method based on machine learning |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112564898A (en) * | 2020-11-30 | 2021-03-26 | 南京晓庄学院 | Book safe storage method and device and storage medium |
| CN112564898B (en) * | 2020-11-30 | 2022-06-14 | 南京晓庄学院 | Book safe storage method and device and storage medium |
| CN112560902A (en) * | 2020-12-01 | 2021-03-26 | 中国农业科学院农业信息研究所 | Book identification method and system based on spine visual information |
| CN112632317A (en) * | 2021-01-13 | 2021-04-09 | 深圳市万物志科技有限公司 | Multi-target interaction method and device based on user pictures |
| CN113239234A (en) * | 2021-06-04 | 2021-08-10 | 杭州大拿科技股份有限公司 | Method for providing video book and method for establishing video book |
| CN113239234B (en) * | 2021-06-04 | 2023-07-18 | 杭州大拿科技股份有限公司 | Method for providing video book and method for establishing video book |
| CN114661879A (en) * | 2022-03-23 | 2022-06-24 | 国网江苏省电力有限公司连云港供电分公司 | Data searching method, system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10824874B2 (en) | Method and apparatus for processing video | |
| CN107346336B (en) | Information processing method and device based on artificial intelligence | |
| CN111460185A (en) | Book searching method, device and system | |
| CN108595583B (en) | Dynamic graph page data crawling method, device, terminal and storage medium | |
| CN104685501B (en) | Text vocabulary is identified in response to visual query | |
| CN107590255B (en) | Information pushing method and device | |
| CN110740389B (en) | Video positioning method, video positioning device, computer readable medium and electronic equipment | |
| KR20180122926A (en) | Method for providing learning service and apparatus thereof | |
| CN108197300B (en) | Question searching method and system | |
| CN108920450B (en) | Knowledge point reviewing method based on electronic equipment and electronic equipment | |
| CN107679070B (en) | Intelligent reading recommendation method and device and electronic equipment | |
| CN110347866B (en) | Information processing method, information processing device, storage medium and electronic equipment | |
| CN108710653B (en) | On-demand method, device and system for reading book | |
| CN112183048A (en) | Automatic problem solving method and device, computer equipment and storage medium | |
| CN107862058B (en) | Method and apparatus for generating information | |
| CN114429635A (en) | Book management method | |
| CN111651674B (en) | Bidirectional searching method and device and electronic equipment | |
| CN113157867A (en) | Question answering method and device, electronic equipment and storage medium | |
| CN109472028B (en) | Method and device for generating information | |
| CN112883218A (en) | Image-text combined representation searching method, system, server and storage medium | |
| CN115759293A (en) | Model training method, image retrieval device and electronic equipment | |
| CN111488513A (en) | Method and device for generating page | |
| CN106570116B (en) | Search result aggregation method and device based on artificial intelligence | |
| CN113569741A (en) | Answer generation method and device for image test questions, electronic equipment and readable medium | |
| CN111666474B (en) | Whole page question searching method and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230613 Address after: 6001, 6th Floor, No.1 Kaifeng Road, Shangdi Information Industry Base, Haidian District, Beijing, 100085 Applicant after: Beijing Baige Feichi Technology Co.,Ltd. Address before: 100085 4001, 4th floor, No.1 Kaifa Road, Shangdi Information Industry base, Haidian District, Beijing Applicant before: XIAOCHUANCHUHAI EDUCATION TECHNOLOGY (BEIJING) CO.,LTD. |
|
| TA01 | Transfer of patent application right |