CN117149947A

CN117149947A - Book query method and device based on VR equipment, VR equipment and medium

Info

Publication number: CN117149947A
Application number: CN202311039031.3A
Authority: CN
Inventors: 谭文亮
Original assignee: Shenzhen Coocaa Network Technology Co Ltd
Current assignee: Shenzhen Coocaa Network Technology Co Ltd
Priority date: 2023-08-16
Filing date: 2023-08-16
Publication date: 2023-12-01

Abstract

The invention relates to a book query method and device based on VR equipment, VR equipment and a medium. The method comprises the steps of obtaining book query request voice information of a user, extracting keywords of a target book to be queried according to the book query request voice information, collecting a first book shelf image in a set area, carrying out text recognition on the first book shelf image, extracting all spine information texts in the first book shelf image, extracting book keywords and spine positions based on all spine information texts in the first book shelf image, sorting the book keywords and the spine positions into a spine text list of the first book shelf and storing the spine text list in the local, and extracting and outputting the spine positions of the target book when the keywords of the target book are successfully matched with the keywords of all books in the spine text list. The method can accurately find the position of the book on the bookshelf, improves the efficiency and accuracy of book finding, can quickly find books when the books are misplaced on other bookshelf, and reduces the bookshelf arranging frequency of a book manager.

Description

Book query method and device based on VR equipment, VR equipment and medium

Technical Field

The invention is suitable for the technical field of virtual reality and the field of book inquiry, and particularly relates to a book inquiry method and device based on VR equipment, VR equipment and a medium.

Background

The traditional book inquiry method is to manually inquire the target books at the entrance by using a book inquiry assistant, and the inquiry assistant can provide the types of the books and the information of the bookshelf where the books are located and needs readers to search for the books by themselves. The existing technology for navigating to the target bookshelf by utilizing the VR glasses facilitates the convenient finding of the bookshelf position under the condition that the bookshelf where books are located is known and the bookshelf position is not known. However, the method is only suitable for finding corresponding types of bookshelf in a huge library, and the user still needs to find the accurate position of the book by himself before the bookshelf is of a type. If books are placed randomly, and books which a book manager does not arrange in time, a user wants to find are misplaced on other bookshelf, and the navigation provided at the moment is guided to the original bookshelf position, so that positioning deviation occurs, the target books are difficult to find according to the position of the bookshelf, the difficulty of finding books is increased, the time for finding books is prolonged, and the book finding efficiency is lower.

Disclosure of Invention

In view of the above, the embodiments of the present invention provide a book query method, apparatus, VR device and medium, so as to solve the problem that in the prior art, the book position cannot be found accurately.

In a first aspect, an embodiment of the present invention provides a book query method, including:

acquiring book query request voice information of a user, and extracting keywords of a target book to be queried according to the book query request voice information;

collecting a first book block image in a set area, performing character recognition on the first book block image, and extracting all spine information texts in the first book block image;

determining a spine text list of the first bookshelf based on all spine information texts in the first bookshelf image, wherein the spine text list comprises keywords and spine positions of all books;

searching a spine text list, and matching keywords of the target books with keywords of each book in the spine text list;

and when the matching is successful, extracting and outputting the spine position of the target book.

In a second aspect, an embodiment of the present invention provides a book inquiry apparatus, including:

the keyword extraction module is used for acquiring book query request voice information of a user and extracting keywords of a target book to be queried according to the book query request voice information;

the information text extraction module is used for acquiring a first book block image in the set area, carrying out text recognition on the first book block image and extracting all the spine information texts in the first book block image;

The text list determining module is used for determining a spine text list of the first bookshelf based on all spine information texts in the first bookshelf image, wherein the spine text list comprises keywords and spine positions of all books;

the matching keyword module is used for searching the book spine text list and matching keywords of the target books with keywords of all books in the book spine text list;

and the extraction and output target position module is used for extracting and outputting the spine position of the target book when the matching is successful.

In a third aspect, an embodiment of the present invention provides a VR device, where the VR device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and where the processor implements the steps of the book inquiry method as in the first aspect when executing the computer program.

In a fourth aspect, embodiments of the present invention provide a computer readable storage medium storing a computer program which, when executed by a processor, performs steps of the book inquiry method as in the first aspect.

In a fifth aspect, embodiments of the present invention provide a computer program product that, when run on a VR device, causes the VR device to perform the steps of the book inquiry method of the first aspect described above.

Compared with the prior art, the invention has the beneficial effects that: the VR equipment scans and identifies the text information of the spine of the front bookshelf, inquires and matches the target books, can accurately find the positions of the target books on the bookshelf, reduces the difficulty of book finding, reduces the time of book finding, and improves the book finding efficiency; the voice recognition user can find the book requirement conveniently and rapidly; when books are not on the corresponding bookshelf, the invention can still quickly find target books within the range, reduce the frequency of arranging the bookshelf by a librarian and lighten the workload of the librarian.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an application environment of a book query method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a book inquiry method based on VR device in accordance with an embodiment of the present invention;

FIG. 3 is a schematic flow chart of extracting keywords of a target book in a book query method according to an embodiment of the invention;

FIG. 4 is a flow chart of the method for extracting text information of spine from collected images in a book inquiry method according to an embodiment of the invention;

FIG. 5 is a flow chart showing the process of sorting the text list of the spine in the book inquiry method according to an embodiment of the present invention;

FIG. 6 is a flow chart of matching keywords in the book query method according to an embodiment of the invention;

FIG. 7 is a schematic diagram of another process after extracting keywords of a target book in the book query method according to an embodiment of the invention;

FIG. 8 is a schematic diagram of a book query apparatus according to the book query method of an embodiment of the invention;

fig. 9 is a schematic structural diagram of a VR device in a book query method according to an embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

It should be understood that the sequence numbers of the steps in the following embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not be construed as limiting the implementation process of the embodiments of the present invention.

In order to illustrate the technical scheme of the invention, the following description is made by specific examples.

The book query method based on the VR device provided by the embodiment of the invention can be applied to an application environment as shown in figure 1, wherein the VR terminal communicates with the server terminal through a network. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers.

Referring to fig. 2, a flow chart of a book query method based on VR device according to an embodiment of the present invention is provided, where the book query method may be applied to the VR end in fig. 1, and VR devices corresponding to the VR end are connected to a target database through a preset application program interface (Application Programming Interface, API). When the target data is driven to run to execute the corresponding task, a corresponding task log is generated, and the task log can be acquired through an API. As shown in fig. 2, the book inquiry method may include the steps of:

step S101, acquiring book query request voice information of a user, and extracting keywords of a target book to be queried according to the book query request voice information.

The voice information of the user is determined by the existing voice recognition technology, the voice recognition technology takes voice as a research object, and a voice recognition module built in VR equipment automatically recognizes and understands the speech spoken by human beings through voice signal processing and pattern recognition.

The user orally sends out an instruction, a voice recognition module built in the VR equipment compares the feature vector of the input voice with each template in the template library in sequence in terms of similarity, and the highest similarity is output as a recognition result. For example, the user issues an instruction: the method comprises the steps that I want to find out 'dream of the red blood cell', the machine recognizes the voice characteristic vector of each word, compares the voice characteristic vector with different voice characteristic vectors in the existing template library, finds out the word, the word and the phrase with the highest similarity, determines each word 'I' want to find 'red' building 'dream' spoken by a user one by one, and finds out the commonly matched word 'dream of the red blood cell' from the word by one, so that the keyword of the target book to be queried is extracted as 'dream of the red blood cell'.

Step S102, collecting a first book block image in a set area, performing text recognition on the first book block image, and extracting all the spine information texts in the first book block image.

Wherein, the characters can be collected and identified by utilizing an optical character identification algorithm, and the optical character identification algorithm refers to a technology of directly identifying an image containing the text as computer characters (a black-and-white dot matrix of a computer); the first bookshelf is the clearest and complete bookshelf which can be shot by the VR device.

In the embodiment, the required red building dreams are more on the literature bookshelf, so the literature bookshelf has a plurality of books in a region, the user wears the VR equipment to stand in front of the literature bookshelf, the VR equipment scans the images of the books on the bookshelf, photographs are photographed and stored at multiple angles, clear and complete bookshelf photographs are selected, gray processing is carried out on the obtained photographs, the photographs are converted into easily-identified gray images, the influence of colors on character recognition is reduced, the definition of the images is improved, characters in the processed images are extracted, characters such as ' three-language ' and ' Shuihu ' in the images are recognized, the characters such as ' red building dreams ', ' unitary character ' in the images are compared with the names in a word database, recognition results are optimized, the three-language ' in the ' red building dreams ', ' western character ' are corrected, the optimized recognition results are output, and all the extracted spine information text records are stored.

Step S103, based on all the spine information texts in the first bookshelf image, extracting book keywords and spine positions, and arranging the book keywords and the spine positions into a spine text list of the first bookshelf, wherein the spine text list is stored locally.

The spine information text comprises, but is not limited to, a name, an author and a publisher, a spine text list of the first bookshelf is arranged according to all the extracted spine information text records, and the spine information is associated with the corresponding position of the spine information in the bookshelf and recorded in the spine text list.

In the above embodiment, the spine text list is sorted into "three-country meaning, luo Guan, first row, 2 nd book" based on the text recognition result; the water is transmitted, shi Naian, the first row of 5 th books; dream of Red mansions, cao Xueqin, second row 3 rd book; west note, wu Chengen, third row, 7 th book ", where non-exemplified books are not shown but still within the list.

Step S104, searching a spine text list, and matching keywords of the target books with keywords of each book in the spine text list;

the spine text list is provided with various book keywords recorded in the first bookshelf, and the spine text list is used as a text library, so that keywords matched with the target book are searched.

Step S105, when the matching is successful, the spine position of the target book is extracted and output.

When the key words of the target book are matched with a book key word in the book spine text list, outputting the spine position corresponding to the book key word in the list, and displaying the spine position in the VR equipment as the highlighting of the corresponding spine position.

According to the book inquiring method, through the steps of voice recognition of a user request, scanning and recognizing surrounding environments, extracting the spine information on the bookshelf, then searching for a matched target book, and outputting the spine position of the target book according to the matching result, the target book can be conveniently and rapidly inquired, the specific position coordinates of the book on a class of bookshelf are found, and the book searching time is shortened.

In other embodiments, as shown in fig. 3, in S101, the method for querying a book provided by the present invention obtains the voice information of a query request of a user, extracts keywords of a target book to be queried according to the voice information of the query request of the book, and includes the following steps:

step S201, obtaining book query request voice information of a user by utilizing VR equipment, and converting the book query request voice information into book query request text information.

The sound receiving device arranged in the VR equipment obtains the sound of the user, obtains the voice feature vector of the user instruction, compares the voice feature vector with templates in the database, compares words with higher similarity, and outputs the text information of the book inquiry request.

In one embodiment, when the user issues an instruction: the method comprises the steps that a book with the name of' sh ǒ u sh is searched, a radio device arranged in VR equipment obtains each voice feature vector in an instruction, and the voice feature vectors are compared with word vectors in a database, and the judgment instruction is "searching for the book with the name of … …", and words which cannot be directly judged are compared for a plurality of times, so that two words which accord with the voice feature vectors are obtained: the jewelry and the gesture are respectively output as text information of a book inquiry request: "find books named jewelry" and "find books named gestures".

Step S202, extracting book names in the book inquiry request text information to obtain keywords of the target books to be inquired.

And extracting information such as book names, authors, publishers and the like from the book query request text information, namely, the keywords of the target books, and storing the keyword records. In the embodiment, text information is requested based on book inquiry, and target book keywords meeting the context, namely 'jewelry' and 'gestures', are found.

In another embodiment, the book query request text message is "find Cao Xueqin book", and the target book keyword is "Cao Xueqin".

According to the book query method, the radio device acquires the voice of the user, converts the voice into text information, extracts keywords of related books in the text, intelligently processes the requirements of the user under different conditions, can accurately query the range, and can comprehensively find target books at the same time, so that the accuracy of querying the books is improved.

In other embodiments, as shown in fig. 4, in S102, that is, a first book shelf image in a set area is collected, text recognition is performed on the first book shelf image, and all the spine information texts in the first book shelf image are extracted, including the following steps:

step S301, a VR device is utilized to acquire a real environment image in real time, and whether a complete bookshelf image exists in the real environment image is judged.

The VR equipment scans the front scene in real time, recognizes the real environment image, shoots bookshelf conditions for many times at multiple angles, distinguishes whether the real environment image has a complete bookshelf condition or not from the real environment image, and directly outputs the bookshelf image if the bookshelf image is complete; if the bookshelf image scanning is incomplete, the complete bookshelf image is completed through the splicing of a plurality of incomplete images, and the next step is convenient to carry out.

Step S302, when it is determined that a complete bookshelf image exists in the real environment image, outputting a first bookshelf image.

And extracting and outputting the scanned or spliced complete bookshelf image, wherein the image clearly covers the spine information of all books on the bookshelf. In an embodiment, the spine font of the book is small and not easy to identify, and the shooting device arranged on the VR equipment can amplify the partial bookshelf image to shoot so as to ensure the definition of the image, and one or a group of high-quality images are selected to be output and stored.

Step S303, performing character recognition on the first book block image by utilizing an optical character recognition technology, and extracting all the spine information texts in the first book block image.

The optical character recognition technology is used for preprocessing a high-quality image, comprising binarization, noise removal, inclination correction and the like, recognizing text information in the processed image, carrying out post-processing on recognition result correction according to a specific language context, extracting all spine information texts in a first book block image, and outputting a spine information list.

According to the book inquiring method, the optical character recognition technology is utilized to scan and recognize the text in the spine photo on the bookshelf, and the multi-angle multiple photos are compared and corrected to output information, so that the accuracy of character recognition is improved, and the accuracy of book inquiring is ensured.

In other embodiments, as shown in fig. 5, in S103, that is, based on all the spine information texts in the first bookshelf image, the method extracts the book keywords and the spine positions, sorts the book keywords and the spine positions into a spine text list of the first bookshelf, and stores the spine text list locally, including the following steps:

step S401, extracting keywords of each book according to all the spine information texts in the first book block image.

And recognizing common phrases according to all the spine information texts recognized by the first book shelf image scanning, extracting keywords of each book from the common phrases, and forming a set covering the book names of each book.

Step S402, determining the spine position of each book according to the position of each spine information text in the first book shelf image.

In the complete bookshelf image, the spine information and the spine positions are in one-to-one correspondence, for example, the spine of books at the positions of the first row of books 2, the first row of books 5, the second row of books 3 and the third row of books 7 are three kingdoms, a waterside transmission, a red building dream and a western tour; correspondingly, books with the spine of three kingdoms, the transmission of the waterfront, the dream of the red building and the western inscription are respectively positioned at the first row of the 2 nd book, the first row of the 5 th book, the second row of the 3 rd book and the third row of the 7 th book, and if the books with the spine of the red building and the dream of the red building are determined, the books can be determined to be positioned at the second row of the 3 rd book.

Step S403, storing the keywords and the spine positions of each book to form a spine text list.

The key words of the books are words extracted from all the spine information texts identified according to the first book shelf image scanning; the position of the spine is the position of the book on the bookshelf. The information is classified and summarized and is collected into a spine text list.

According to the book query method, based on all the spine information texts on the bookshelf images, the book keywords and the spine positions are extracted and are arranged into a spine text list, and the keywords are associated with the spine positions, so that the accuracy of book information on the bookshelf is ensured, and a user can find a target book accurately.

In other embodiments, as shown in fig. 6, in S104, a spine text list is searched, and keywords of a target book are matched with keywords of each book in the spine text list, including the following steps:

step S501, calculating the similarity between the keywords of the target book and the keywords of each book in the spine text list to obtain N keyword similarities, wherein N is greater than 2.

The method comprises the steps of regarding keywords of a target book as target keywords, regarding keywords of each book in a spine text list as example keywords, respectively calculating similarity between the target keywords and each example keyword, regarding the example keyword with the largest similarity as the similar keywords of the target keywords, and comprising the following steps:

Converting the target keywords into target keyword vectors by using a word vector technology, and converting each example keyword into a corresponding example keyword vector according to the word vector technology;

and calculating the similarity between the target keyword vector and N example keyword vectors, and comparing the N similarity, wherein the example keyword with the largest similarity is used as the similar keyword of the target keyword.

Among them, word vector techniques are used to represent text into a series of vectors that can express text semantics, such as One-hot Encoding (One-hot Encoding) and Word2Vec (a Word vector model). In this embodiment, the target keyword is converted into a target keyword vector according to the Word2Vec technique, the target keyword vector may be used to characterize semantic information in the target keyword, and each example keyword is converted into a corresponding example keyword vector according to the Word2Vec technique, the example keyword vector may be used to characterize semantic information in the corresponding example keyword template.

Specifically, the target keyword vector and each example keyword vector can be substituted into a preset similarity calculation formula to obtain the similarity of the target keyword and each corresponding example keyword;

The preset similarity calculation formula is as follows:

wherein X is _i Similarity between the target keyword and the ith exemplary keyword, m is the target keyword vector, s _i Is the i-th example key vector, where i=1, 2, …, N.

The target keyword vector may be used to represent semantic information in the target keyword, and the example keyword vector may be used to represent semantic information in the corresponding example keyword, in this embodiment, by calculating cosine similarity between the target keyword vector and each example keyword vector, the similarity between the target keyword and each corresponding example keyword may be represented, and the greater the cosine similarity, the greater the similarity between the target keyword and the corresponding example keyword may be represented.

And obtaining N example keywords (N is more than 2) from the original corpus, respectively calculating the similarity between the target keywords and each example keyword, and comparing the similarity values. The method is used for obtaining the example keywords, so that the influence of the keyword entity words in keyword matching is reduced, and the matching accuracy between the keywords is effectively improved.

Step S502, the maximum keyword similarity is selected from the N keyword similarities, and when the maximum keyword similarity is larger than a preset similarity threshold value, the keyword of the target book is judged to be successfully matched with the keyword of the book corresponding to the maximum keyword similarity.

And comparing the maximum keyword similarity from the N keyword similarities, comparing the maximum keyword similarity with a set similarity threshold value, and if the similarity is larger than the similarity threshold value, considering that the matching of the keyword of the target book and the book keyword corresponding to the maximum keyword similarity is successful, otherwise, judging that the matching is failed.

In an embodiment, the target keyword is "dream of red blood cell", the example keyword has words such as "three kingdoms" and "Shuihu" and "western tour", the similarity between the target keyword and the example keyword is calculated respectively, the similarity between the calculated "dream of red blood cell" and "three kingdoms" is 0.3, the similarity between the calculated "dream of red blood cell" and "three kingdoms" is 0.6, the similarity between the calculated "dream of red blood cell" and "three kingdoms" and "western tour" is 0.7, if the similarity threshold value is 0.8, the maximum keyword similarity is 0.7 is smaller than the threshold value 0.8, and the matching fails, and there is no target book on the bookshelf.

According to the book query method, the example keyword with the maximum similarity is used as the similar keyword of the target keyword, and the example keyword which is most similar to the target keyword is screened out and used for generating the subsequent target book, so that the accuracy of a dialogue generating result is improved.

In other embodiments, the book query method provided in S105, that is, when the matching is successful, extracts and outputs the spine position of the target book, includes the following steps:

when the matching is successful, marking the spine position of the book corresponding to the maximum keyword similarity in the first book shelf image as the spine position of the target book.

When the key words of the target book are successfully matched with the book key words corresponding to the maximum key word similarity, marking the spine position of the book corresponding to the maximum key word similarity, and taking the spine position as the spine position of the target book.

According to the book query method, the accurate position of the target book is marked, so that a user can more intuitively find a desired book.

In other embodiments, as shown in fig. 7, after S101, the method for querying a book provided by the present invention obtains the voice information of a query request of a user, extracts keywords of a target book to be queried according to the voice information of the query request of the book, and further includes the following steps:

step S701, collecting a second bookshelf image in the set area, carrying out text recognition on the second bookshelf image, and extracting all spine information texts in the second bookshelf image.

The second bookshelf is a bookshelf with definition or integrity inferior to that of the first bookshelf, which can be shot by the VR equipment, and when the first bookshelf can not inquire the target books, the image of the second bookshelf is selected for character scanning and identification.

Step S702, determining a spine text list of the second bookshelf based on all the spine information texts in the second bookshelf image.

Step S703, firstly, retrieving the spine text list of the first bookshelf, and matching the keywords of the target book with the keywords of each book in the spine text list of the first bookshelf.

The steps S701 to S703 are similar to the steps S102 to S104, and the specific implementation process may refer to the descriptions of the steps S102 to S104, which are not repeated here.

Step S704, when the matching is judged to be unsuccessful, the spine text list of the second bookshelf is searched, and the keywords of the target book are matched with the keywords of each book in the spine text list of the second bookshelf.

In one embodiment, the VR device scans more than one bookshelf in front of the bookshelf surface by the user, the nearer bookshelf serves as the first bookshelf to first match the target keyword, the farther bookshelf serves as the second bookshelf as the candidate, when the target book is not on the first bookshelf, the matching is searched in the spine keywords identified by the second bookshelf, and the keyword similarity is calculated until the position of the target book is output.

According to the book query method, the scene that a user faces a plurality of bookshelves is considered, the VR equipment is utilized to photograph the first bookshelf and the second bookshelf successively, and the method is used for searching target books on the first bookshelf and the second bookshelf respectively, which is equivalent to selecting the second bookshelf as an alternative scheme, so that the problem that the target books cannot be found directly from one bookshelf is avoided, the searching range is enlarged, and the time for querying the books is shortened.

Fig. 8 shows a block diagram of a book query apparatus based on VR device according to an embodiment of the present invention, which is applied to VR devices and connected to a target database through a network. When the target database is driven to run to execute the corresponding task, a corresponding task log is generated, and the task log can be acquired through an API. For convenience of explanation, only portions relevant to the embodiments of the present invention are shown.

Referring to fig. 8, the book inquiry apparatus includes: the keyword extraction module 81, the information text extraction module 82, the text list determination module 83, the keyword matching module 84, and the output target position extraction module 85. The functional modules are described in detail as follows:

The keyword extraction module 81 is configured to obtain book query request voice information of a user, and extract keywords of a target book to be queried according to the book query request voice information;

the information text extracting module 82 is configured to collect a first book block image in the set area, perform text recognition on the first book block image, and extract all the spine information text in the first book block image;

a determining text list module 83 for determining a spine text list of the first bookshelf based on all spine information texts in the first bookshelf image, the spine text list including keywords and spine positions of respective books;

a matching keyword module 84, configured to retrieve a spine text list, and match keywords of the target book with keywords of each book in the spine text list;

the extracting and outputting target position module 85 is configured to extract and output the spine position of the target book when the matching is successful.

Optionally, the keyword extraction module 81 includes:

the text information conversion unit is used for acquiring book inquiry request voice information of a user by utilizing the VR equipment and converting the book inquiry request voice information into book inquiry request text information;

the keyword acquisition unit is used for extracting the book names in the book inquiry request text information to obtain keywords of the target books to be inquired.

Optionally, the above-mentioned extracted information text module 82 includes:

the image acquisition unit is used for acquiring a real environment image in real time by utilizing the VR equipment and judging whether a complete bookshelf image exists in the real environment image;

a first book block image output unit configured to output a first book block image when it is determined that a complete bookshelf image exists in the real environment image;

and the spine information text extraction unit is used for carrying out text recognition on the first book block image by utilizing an optical character recognition technology and extracting all the spine information texts in the first book block image.

Optionally, the determining text list module 83 includes:

the keyword extraction unit is used for extracting keywords of each book according to all the spine information texts in the first book shelf image;

the book spine position determining unit is used for determining the book spine position of each book according to the position of each book spine information text in the first book shelf image;

and the text list storage unit is used for storing the keywords and the spine positions of the books to form the spine text list.

Optionally, the matching keyword module 84 includes:

the similarity calculation unit is used for calculating the similarity between the keywords of the target book and the keywords of each book in the spine text list to obtain N keyword similarity, wherein N is more than 2;

And the matching result judging unit is used for screening out the maximum keyword similarity from the N keyword similarities, and judging that the keyword of the target book is successfully matched with the keyword of the book corresponding to the maximum keyword similarity when the maximum keyword similarity is larger than a preset similarity threshold value.

Optionally, the extracting output target location module 85 includes:

and the spine position marking unit is used for marking the spine position of the book corresponding to the maximum keyword similarity in the first book shelf image as the spine position of the target book when the matching is successful.

Optionally, the book inquiry apparatus further includes:

the second bookshelf image acquisition module is used for acquiring a second bookshelf image in a set area, carrying out character recognition on the second bookshelf image and extracting all the spine information texts in the second bookshelf image;

the spine text list determining module is used for determining a spine text list of the second bookshelf based on all the spine information texts in the second bookshelf image;

the first book shelf keyword matching module is used for searching a book spine text list of the first book shelf, and matching keywords of the target books with keywords of all books in the book spine text list of the first book shelf;

And the second bookshelf keyword matching module is used for searching the spine text list of the second bookshelf when the matching is judged to be unsuccessful, and matching the keywords of the target books with the keywords of all books in the spine text list of the second bookshelf.

It should be noted that, because the content of information interaction and execution process between the modules and the embodiment of the method of the present invention are based on the same concept, specific functions and technical effects thereof may be referred to in the method embodiment section, and details thereof are not repeated herein.

Fig. 9 is a schematic structural diagram of a VR device according to an embodiment of the present invention. As shown in fig. 9, the VR device of this embodiment includes: at least one processor (only one shown in fig. 9), a memory, and a computer program stored in the memory and executable on the at least one processor, the processor executing the computer program to perform the steps of any of the various book inquiry method embodiments described above.

The VR device may include, but is not limited to, a processor, a memory. It will be appreciated by those skilled in the art that fig. 9 is merely an example of a VR device and is not intended to limit the VR device, and that the VR device may include more or fewer components than shown, or may combine certain components, or may include different components, such as a network interface, a display screen, an input device, and the like.

The processor may be a CPU, but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory includes a readable storage medium, an internal memory, etc., where the internal memory may be the memory of the VR device, the internal memory providing an environment for the execution of an operating system and computer readable instructions in the readable storage medium. The readable storage medium may be a hard disk of the VR device, and in other embodiments may be an external storage device of the VR device, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. that are provided on the VR device. Further, the memory may also include both internal storage units and external storage devices of the VR device. The memory is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs such as program codes of computer programs, and the like. The memory may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above device may refer to the corresponding process in the foregoing method embodiment, which is not described herein again. The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiment, and may be implemented by a computer program to instruct related hardware, and the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of the method embodiment described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.

The present invention may also be implemented as a computer program product for implementing all or part of the steps of the method embodiments described above, when the computer program product is run on a VR device, causing the VR device to execute.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided herein, it should be understood that the disclosed apparatus/VR devices and methods may be implemented in other manners. For example, the apparatus/VR device embodiments described above are merely illustrative, e.g., the division of modules or elements is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims

1. The book query method based on the VR equipment is characterized by comprising the following steps of:

acquiring book inquiry request voice information of a user, and extracting keywords of a target book to be inquired according to the book inquiry request voice information;

based on all the spine information texts in the first bookshelf image, extracting book keywords and spine positions, and arranging the book keywords and the spine positions into a spine text list of the first bookshelf, wherein the spine text list is stored locally;

searching the spine text list, and matching the keywords of the target books with the keywords of each book in the spine text list;

2. The book query method of claim 1, wherein, in the obtaining the book query request voice information of the user, extracting keywords of the target book to be queried according to the book query request voice information comprises:

obtaining book inquiry request voice information of a user by utilizing VR equipment, and converting the book inquiry request voice information into book inquiry request text information;

and extracting the book names in the book inquiry request text information to obtain the keywords of the target books to be inquired.

3. The book inquiry method according to claim 1, wherein the collecting the first book block image in the set area, performing text recognition on the first book block image, and extracting all the spine information text in the first book block image comprises:

acquiring a real environment image in real time by utilizing the VR equipment, and judging whether a complete bookshelf image exists in the real environment image;

outputting a first bookshelf image when the fact that the complete bookshelf image exists in the real environment image is judged;

and performing character recognition on the first book block image by utilizing an optical character recognition technology, and extracting all the spine information texts in the first book block image.

4. The book inquiry method according to claim 1, wherein the extracting book keywords and book spine positions based on all the book spine information texts in the first book shelf image, collating the book keywords and book spine positions into a book spine text list of the first book shelf, the book spine text list being stored locally, includes:

extracting keywords of each book according to all the spine information texts in the first book shelf image;

determining the spine position of each book according to the position of each spine information text in the first book shelf image;

And storing the keywords and the spine positions of each book to form the spine text list.

5. The book inquiry method according to claim 1, wherein said retrieving said spine text list, matching keywords of said target book with keywords of respective books in said spine text list, comprises:

calculating the similarity between the keywords of the target book and the keywords of each book in the spine text list to obtain N keyword similarity, wherein N is more than 2;

and screening out the maximum keyword similarity from the N keyword similarities, and judging that the keyword of the target book is successfully matched with the keyword of the book corresponding to the maximum keyword similarity when the maximum keyword similarity is larger than a preset similarity threshold value.

6. The book inquiry method according to claim 1, wherein said extracting and outputting the spine position of the target book when the matching is successful comprises:

and when the matching is successful, marking the spine position of the book corresponding to the maximum keyword similarity in the first book shelf image as the spine position of the target book.

7. The book inquiry method according to claim 1, wherein after the book inquiry request voice information extracts keywords of a target book to be inquired, the book inquiry method further comprises:

collecting a second bookshelf image in a set area, performing text recognition on the second bookshelf image, and extracting all spine information texts in the second bookshelf image;

determining a spine text list of the second bookshelf based on all spine information texts in the second bookshelf image;

firstly, searching a spine text list of the first bookshelf, and matching keywords of the target books with keywords of each book in the spine text list of the first bookshelf;

and when the matching is judged to be unsuccessful, searching the spine text list of the second bookshelf, and matching the keywords of the target book with the keywords of each book in the spine text list of the second bookshelf.

8. A book inquiry apparatus, said apparatus comprising:

The information text extraction module is used for acquiring a first book block image in a set area, carrying out character recognition on the first book block image and extracting all book spine information texts in the first book block image;

the matching keyword module is used for searching the spine text list and matching keywords of the target books with keywords of all books in the spine text list;

9. VR device characterized in that it comprises a processor, a memory and a computer program stored in said memory and executable on said processor, said processor implementing the steps of the book inquiry method according to any one of claims 1 to 7 when said computer program is executed.

10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the book inquiry method according to any one of claims 1 to 7.