TW202011219A - System for document searching using results of text analysis and natural language input - Google Patents

Abstract

A system for document searching using results of text analysis and natural language input is disclosed. The system includes a keyword capture unit, a database module, a network platform communication unit, a statement parsing unit, a file list providing unit, and a sentence learning unit. The invention can perform keyword analysis on stored files to enhance searching speed. It can also perform file search by receiving user input of natural language containing keywords on the cloud platform, further facilitating processes to search files and saving time for obtaining related files.

Description

Document search system using text analysis results and natural language input

The invention relates to a document search system, in particular to a document search system using text analysis results and natural language input.

In traditional information file management operations, common file storage methods are classified according to file attributes (such as text files, graphic files, or audiovisual files), characteristics (such as content topics or creation time), file names, etc. In the corresponding folders, all folders are presented in a tree structure to facilitate users to access files. Under this structure, as long as you know the file name or folder attributes, it is very convenient to find a specific file.

When the data is open access on the Internet, unless a specific URL is given, if you want to find a specific file or a file related to a certain topic, the search time will become quite long. First, it is related to the user’s unfamiliar file storage structure; second, the search keyword needs to be further used to compare all files (name or metadata), which also takes time; finally, there may be many uses on the Internet If the file access service function is used at the same time, the processing of the data also causes a delay. Therefore, if you want to reduce the time delay caused by the aforementioned reasons, a good file search system is needed.

Taking Google keyword query as an example, this is a technology that can find and provide related web page links in all web page data on the Internet through keywords. Since the form of a web page is also a folder or file in a storage device, this query method can also be used as a reference for establishing the aforementioned document search system. The search engine of Google's keyword query is actually mainly doing two things, namely crawling and indexing, and using page relevance to sort/rank these search results pages. The search engine works by searching the index page instead of browsing the entire database content of the website. Search engines use data exploration crawlers (a type of software) to crawl keywords entered by users, and provide user-related webpage links based on the keywords of the page and the content and the relevance of the website. The whole process can take less than half a second. Therefore, in theory, as long as the corresponding keyword data is created for the files stored in a cloud database, the desired data can be quickly found through the received keywords.

On the other hand, when users search for files through keywords on a specific or unspecified platform in the cloud, sometimes the keyword cannot be sure to find all related files. For example, if the user enters "facebook" or "I want facebook", it may be to find files related to facebook; however, the source of the file is determined by the file name, content or metadata, and the language is unknown. In addition, if there is a keyword "Facebook" in the file, should we also provide it to the user?

In response to the above requirements, people need a document search system that can perform keyword analysis on stored files to enhance the search speed, and can also search for documents by receiving user-entered natural grammar containing keywords on the cloud platform to It is convenient for users to search files and save time for obtaining related files.

This paragraph extracts and compiles certain features of the invention. Other features will be revealed in subsequent paragraphs. Its purpose is to cover the spirit and scope of the additional patent application scope, various modifications and similar arrangements.

The object of the present invention is to provide a document search system that uses text analysis results and natural language input to perform keyword analysis on stored files to enhance the search speed, and can also be performed by receiving user input of natural grammar containing keywords on a cloud platform Document search, to facilitate users to search files and save time to get related files. The system includes: a keyword retrieval unit for obtaining the corresponding character code of the text information from the file containing the text information, and retrieving at least one file noun keyword from the character code; a database Module, including: a file database to store multiple files; a noun keyword database to store plural noun keywords; and a file keyword link database to store each file in the file database A retrieval table of at least one file noun keyword retrieved by the keyword retrieval unit, wherein the comparison table further notes the keyword sources of all file noun keywords retrieved by the file; a network platform communication unit, which operates to pass The network is connected to a network platform server, receives a query term input in natural language from a user of the network platform running on the network platform server, and sends a file list or a leading sentence for the query term To the server of the network platform for transmission to the user; a sentence parsing unit for retrieving at least one query noun keyword and corresponding keyword source from the query phrase, and when the keyword source is missing, Issue the guide sentence through the network platform communication unit to obtain the source of the keyword; and a file list providing unit for querying the query noun keyword and the source of the keyword to the comparison table and sorting out all matching files The file name is in the file list, and the file list is issued through the network platform communication unit. At least one file noun keyword retrieved by the keyword retrieval unit and at least one query noun keyword retrieved by the sentence analysis unit come from the noun keyword database.

The document search system using text analysis results and natural language input may further include a sentence learning unit for learning the query phrase through a machine learning algorithm. After removing the query noun keyword and the source of the keyword, the remaining text A plurality of equivalent text combinations, and use these equivalent text combinations to assist in determining noun keywords, keyword sources, and/or semantic meaning of query words.

Preferably, the machine learning algorithm may be a TF-IDF algorithm.

In one embodiment, the noun keyword database may further store the corresponding synonymous noun keywords for each noun keyword. When the sentence parsing unit retrieves a query noun keyword, the file list providing unit will query the comparison table based on the query noun keyword, the synonymous noun keyword of the query noun keyword, and the keyword source 2. Arrange the file names of all matching files in the file list, and send the file list through the network platform communication unit.

According to the present invention, the text information type may be a character code, a text image, or a file with embedded text image objects. Text images and files with embedded text image objects are converted into corresponding character codes using optical character recognition technology (Optical Character Recognition, OCR). The file format can be Portable Document Format (PDF), Power Point file format, Power Point file compatible format, Word file format, Word file compatible format, Excel file format, or Excel file compatible format. The source of the keyword can be file name, file content, or file metadata (Metadata).

The retrieval method for querying noun keywords and keyword sources may include the steps of: (a) segmenting the query terms according to words; (b) giving the segmented words part of speech; (c) belonging to the category of nouns, and Words appearing in the noun keyword database are classified as query noun keywords; and (d) Words that belong to the noun category and are synonymous with plural preset keyword sources are classified as keyword sources.

The noun keyword can be derived from the Chinese sentence structure database of the Central Research Institute of the Republic of China.

The invention uses the keyword retrieval unit to find the file noun keywords from the file, so as to establish the relationship between the file and the file noun keywords. At the same time, the sentence parsing unit and the file list providing unit can analyze the query words and sentences of the user terminal to accurately Quickly find the files corresponding to the query noun keywords in the query words and sentences, and present all the files in a list to the user. In this way, it can meet the needs of enhancing the search speed and receiving the user's input of natural grammar containing keywords on the cloud platform to search for documents, thereby facilitating the user to search for files and saving time for obtaining related files.

The present invention will be described more specifically by referring to the following embodiments.

Please refer to FIG. 1, which illustrates the components and operation modes of a document search system (hereinafter referred to as system) using text analysis results and natural language input according to an embodiment of the present invention. The system can be erected on a server 10, and then connected to at least one network platform server 30 via a network 20. The system disclosed by the present invention includes a keyword retrieval unit 110, a database module 120, a network platform communication unit 130, a sentence analysis unit 140, a file list providing unit 150, and a sentence learning unit 160. The functions and interaction methods of each component are introduced below.

The keyword retrieving unit 110 can obtain the corresponding character code of the text information from the file containing the text information, and retrieve at least one file noun keyword from the character code. One aspect of this system is to control the keywords in all the files stored in the database module 120, so as to be able to quickly find the files (text files, graphic files, or other files with embedded graphic files) related to the data that the user wants. Therefore, the keyword retrieval unit 110 is used to "deeply understand" which keywords are in a file. Here, the type of text information contained in the file may be the character encoding itself. The character encoding may be, but not limited to, ASCII, ISO/IEC 646, ISO/IEC 646, DOS character set, Windows character set, Big5, CNS 11643, ISO/IEC 2022, GB 2312, EUC, Unicode, UTF-8, etc. These character codes can present special regional text, all known text and some text images with special codes. In all electronic document editors in the world, it is possible to use at least one of the foregoing to express the text edited by the user. Therefore, the content of the text information will be a series of words, which may be presented in a character encoding. In addition, the text information may also be a text image. The simplest explanation is to take a photo of a page of a book, and the photo must have the text of that page. For text images, the keyword extraction unit 110 may use optical character recognition technology (Optical Character Recognition, OCR) to convert it to the corresponding character encoding. Of course, OCR technology has been used for many years, and related software or hardware can be integrated with the access keyword retrieval unit 110, or used as a support module to operate outside the keyword retrieval unit 110. The text information is more likely to be a file embedded with a text image object, such as a word file or a pdf file embedded with the aforementioned photo. Of course, the text information in the text image object embedded in this file can also be obtained by OCR technology.

There are many file formats, such as Portable Document Format (PDF), Power Point file format, Power Point file compatible format, Word file format, Word file compatible format, Excel file format, and Excel file compatible format . The aforementioned xx file compatible format means that a file format can be opened by two or more editors, and an editor can convert the file format to another specific file format, and the specific file format is the file compatible format. For example, Writer under OpenOffice can open and edit the Word file format at the same time as Microsoft Office Word, but Writer can save the file as a ".odt" file format with a specific file extension. The odt file format is a kind of Word File compatible format.

Since the foregoing character encoding carries many characters, the keyword extracting unit 110 can extract at least one file noun keyword from the character encoding. Here, the extracted at least one file noun keyword is preset in the database module 120, so by comparing the content of the text information, the file noun keyword can be easily found. The so-called "noun keyword" means that the part of speech of the keyword is a noun. In order to have a better understanding of the retrieval of file noun keywords, please refer to Figure 3, which shows the content of a word file, the text content of which is displayed in the upper half of the figure, and the text enclosed by the dotted frame in the lower half is the file Name data and file metadata (Metadata), the latter two are not visible when opening the word file, and can only be found through the file manager. The text enclosed in solid lines in the text content is the file noun keyword of the word file, such as Facebook, article, news, Facebook, source code, friends, description, operation method and version. Other words such as an article, decode, display, care, simple, etc., because they are not nouns, will not be intercepted. However, the term "Wei Gong Bang" is the name of a blog owner. Although it is a noun, it is not listed as a noun keyword in the database module 120 because of its infrequent use or lack of popularity. Will be captured. Conversely, if the blogger becomes famous in the future, the name is often used as facebook, and the database module 120 may list it as a noun keyword, and the keyword retrieval unit 110 retrieves it as a file noun keyword.

The database module 120 includes several databases: a file database 121, a noun keyword database 122 and a file keyword link database 123. The file database 121 stores many of the aforementioned files, and the noun keyword database 122 stores all the aforementioned noun keywords (not limited to the "file" noun keywords from a single file). There are many sources of noun keywords. In this embodiment, the source of the noun keyword is the Chinese sentence structure database of the Central Research Institute of the Republic of China. The file keyword link database 123 stores a comparison table of each file in the file database 121 and at least one file noun keyword retrieved from the keyword retrieval unit 110. The previous example illustrates this. Assume that the word file name is "10244852.doc" and it is stored in the file database 121. Then, the content of the column of the comparison table is "10244852.doc" à "Facebook; article; news; Facebook; source code; friends; description; mode of operation; version", where "à" indicates the connection relationship, that is to say, with The file noun keywords related to the "10244852.doc" file include Facebook, articles, news, Facebook, source code, friends, description, operation mode and version.

According to the present invention, the aforementioned comparison table can additionally note the keyword sources of all file noun keywords for file comparison. Here, the keyword source refers to where the archive noun keyword is found. In this embodiment, the source of the keyword may be the file name, file content, or file metadata. The relationship between these three has been illustrated in Figure 3, and will not be repeated here.

The network platform communication unit 130 operates to connect to the network platform server 30 through the network 20. In addition, the network platform communication unit 130 can also receive a query phrase input in natural language from a user of the network platform 300 running on the network platform server 30. For the convenience of explanation, the user may connect a desktop computer 410 or a smartphone 420 to the network platform server 30 in hardware, log in or connect to the network platform 300, and then connect to the network platform communication unit 130 of the system connection. Here, the network platform 300 refers to various service platforms that spread or interconnect information through the network. In practice, the network platform 300 may be a blog web service platform, and the network platform communication unit 130 can drive a client, such as a desktop computer 410 browser, with a specific javascript code through the WEB format. The network platform 300 may also be a community or communication software platform, such as LINE, facebook, etc. The connection between the network platform communication unit 130 and these network platforms 300 may be through its open or authorized API. Therefore, the network platform communication unit 130 can allow the user to input query terms in natural language.

Natural language is the language that ordinary people can use. There is no specific format, but it may be difficult to interpret because of the characteristics of personal language use. The query term refers to all text information sent by the user who wants to find the file, and may include a single "noun" and a longer "sentence". Several examples of query terms are shown in Figure 5, and the following description will explain them in depth. The network platform communication unit 130 sends a file list for the query phrase to the network platform server 30 after the query phrase is completed for subsequent processing, so as to be transmitted to the user who transmits the query phrase through the network platform 300. If the semantics of the query term is not clear, the sent data will be changed to a leading sentence. The file list and the guide sentence will be explained below.

The sentence analysis unit 140 can extract at least one query noun keyword and the corresponding keyword source from the aforementioned query words and sentences. Please refer to Figure 2 for the process of querying noun keywords and keyword source extraction methods. The steps are to first split query words by words (S01). For example, if a query phrase is "I want to find files related to butterflies", segmenting by words will become "I/want to find/about/butterfly/related/files". Next, the segmented words are given parts of speech (S02). The segmented query with attached part of speech becomes "I (pronoun) / want to find (verb) / about (preposition) / butterfly (noun) / related (adjective) / file (noun)". The third step: classify the words that belong to the noun category and appear in the noun keyword database 122 into query noun keywords (S03). In this example, "butterfly" and "file" are matched, but not all can be classified as query noun keywords. Finally, the words that belong to the noun category and have the same synonyms as the plural preset keyword sources are classified as keyword sources (S04). As mentioned above, the source of keywords can be the file name, file content, and file metadata, and each has a synonym. For example, the default synonyms of file content can include file, file, and content. Therefore, the "file" found in step S03 will be regarded as the target to be found, that is, the source of the keyword. Therefore, the keyword of the query noun retrieved is "butterfly", and the source of the keyword is "file", that is, to find the word "butterfly" in the file content of all files, and mark the file.

Please refer to Figure 5. Each field in the table in Fig. 5 represents a query type. For the convenience of explanation, the bottom line is added below the query noun keyword, and the source of the keyword is indicated in bold italics. The type in the first column has only a single noun. Obviously, the noun is the query noun keyword. However, at this time, the source of the keyword is lacking, and the amount of the searched file is too large. The sentence parsing unit 140 can issue the guide sentence through the network platform communication unit 130 to obtain the source of the keyword. In the second column, "Is there any relevant information on Facebook?" I pointed out that the query noun keyword is "Facebook", but "relevant information" does not know what kind of keyword source, it is best to issue a guide sentence to confirm the keyword source. The query words in the third column are clearly stated, so it can be seen that "facebook" is the query noun keyword, and "file" (that is, file content) is the source of the keyword. The fourth column uses Brin grammar to join together, you can find the query noun keyword of "Facebook" or "facebook", but the source of the keyword is unknown. The fifth column connects two nouns with spaces. The relationship here may be an intersection or a union, which is defined by the sentence analysis unit 140. Similarly, the source of the keyword is unknown. The sixth column specifies that the query noun keyword "Facebook" is used to find the keyword source "file content", and the query noun keyword "facebook" is used to find the keyword source "file name". Since both content and filename are synonymous words corresponding to the source of the keyword, the sentence parsing unit 140 can also process. The sixth column uses Norn's Brin syntax, which is intended to include "facebook" files in the file content, but not excel files. Since the query noun keywords and keyword sources can be retrieved, the sentence analysis unit 140 can also process such query words and sentences. "File" in the last column refers to either the query noun keyword or the keyword source. The sentence parsing unit 140 preferentially handles query noun keywords, so the word "file" will be regarded as a query noun keyword, and a corresponding leading sentence will be sent to the user who transmits the query term. It should be noted that the at least one file noun keyword retrieved by the keyword retrieving unit 110 is the same as the at least one query noun keyword retrieved by the sentence parsing unit 140, from the noun keyword database 122.

The file list providing unit 150 is used to query the noun keywords and keyword sources against the aforementioned comparison table, organize the file names of all matching files in the file list, and issue the file list through the network platform communication unit 130 . In order to have a clearer understanding of the function of the file list providing unit 150, please refer to FIG. 4, which illustrates the operation mode of the LINE interface. The user of the smartphone 420 connects to the system through LINE. Using the LINE API connection, the user can find the information (file) he wants with the system in a "chat" way. First, the user sent out the query phrase "Do you have Facebook information?" Obviously, if the source of the keyword is missing in the query phrase, the sentence parsing unit 140 will issue a guide sentence of "Is the file content, file name or metadata related to facebook?" After the user determines the content (synonyms of the file content), the file list providing unit 150 can perform a table lookup operation and send all files to the user in a file list. The file list may be very long and will be sold separately. The user can click on the desired file name, and the corresponding file will be sent to him via LINE.

The sentence learning unit 160 is used to learn and obtain the query words and sentences after removing the query noun keyword and the source of the keyword through a machine learning algorithm, and a plurality of equivalent text combinations of the remaining text, and using these equivalent text combinations to assist Query noun keywords, keyword sources or/and judgment of sentence semantics. Please refer to Figure 5, the first column to the sixth column are to find the natural language of facebook query noun keywords. After deducting the query noun keywords and keyword sources (if any), "" (empty collection), "have any relevant information?", "I want", "or", "" ( Space) and "in as", these are equivalent text combinations that are willing to be queried. By removing these equivalent text combinations, it is more likely to present the desired query noun keywords and keyword sources. The semantic meaning of the seventh column of FIG. 5 has the meaning of excluding certain query content. The query word and sentence can remove a query noun keyword and the remaining text after the keyword source, and a set of equivalent text combinations can be listed. In this way, multiple equivalent text combinations are subdivided according to the user's semantics, so that the true intentions of customers can be more effectively understood. In addition, in practice, the aforementioned machine learning algorithm can use the TF-IDF algorithm.

In another embodiment, the noun keyword database 122 may further store the corresponding synonymous noun keywords for each noun keyword. This approach allows customers to find more relevant keywords when they search for a single noun keyword. For example, when the query noun keyword is "Facebook", the corresponding synonymous noun keyword "facebook" will also be used to search. In this case, when the sentence parsing unit 140 retrieves a query noun keyword, the file list providing unit 150 will use the query noun keyword, the query noun keyword synonymous noun keyword and the keyword source The query table is used to query, sort out the file names of all matching files in the file list, and send the file list through the network platform communication unit 130. Of course, you can ask the user if you want to accept this practice before proceeding.

Although the present invention has been disclosed as above in the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to the scope defined in the attached patent application.

10: server 110: keyword retrieval unit 120: database module 121: file database 122: noun keyword database 123: file keyword connection database 130: network platform communication unit 140: sentence analysis unit 150 : File list providing unit 160: sentence learning unit 20: network 30: network platform server 300: network platform 410: desktop computer 420: smartphone

FIG. 1 illustrates the components and operation of a document search system using text analysis results and natural language input according to an embodiment of the present invention; FIG. 2 is a flow chart of a method for extracting noun keywords and keyword sources; FIG. 3 illustrates A file content; FIG. 4 illustrates a social APP interface operation mode; and FIG. 5 is a list for explaining query words and sentences.

10: Server

110: Keyword retrieval unit

120: Database module

121: File database

122: noun keyword database

123: File keyword link database

130: Network platform communication unit

140: statement parsing unit

150: File list providing unit

160: sentence learning unit

20: Internet

30: Web platform server

300: Internet platform

410: Desktop computer

420: Smartphone

Claims (11)

A document search system using text analysis results and natural language input includes: a keyword extraction unit for obtaining the corresponding character encoding of the text information from the file containing the text information, and extracting from the character encoding Retrieve at least one file noun keyword; a database module, including: a file database that stores a plurality of files; a noun keyword database that stores a plurality of noun keywords; and a file keyword link database, Storing a comparison table of each file in the file database and at least one file noun keyword retrieved from the keyword retrieval unit, wherein the comparison table additionally notes keywords of all file noun keywords controlled by the file Source; a network platform communication unit, which operates to connect to a network platform server via the network, receives a query sentence in natural language input from a user of the network platform running on the network platform server, and sends A file list or a guiding sentence for the query term is sent to the server of the network platform for transmission to the user; a sentence parsing unit for extracting at least one query noun keyword and corresponding key from the query term Word source, and when the keyword source is missing, issue the guidance sentence through the network platform communication unit to obtain the keyword source; and a file list providing unit for the query noun keyword and the keyword source Query the comparison table, arrange the file names of all matching files in the file list, and issue the file list through the network platform communication unit; wherein, at least one file noun key retrieved by the keyword retrieval unit The word and at least one query noun keyword retrieved by the sentence analysis unit comes from the noun keyword database. The document search system using text analysis results and natural language input as described in item 1 of the patent scope further includes a sentence learning unit for learning to obtain the query phrase through a machine learning algorithm and remove the query noun keyword After the source of the keyword, a plurality of equivalent text combinations of the remaining text, and using these equivalent text combinations to assist in querying noun keywords, keyword sources, and/or semantic judgment of query words. As described in the second item of the patent application, a document search system using text analysis results and natural language input, wherein the machine learning algorithm is a TF-IDF algorithm. As described in item 1 of the patent application scope, a document search system using text analysis results and natural language input, wherein the noun keyword database further stores corresponding synonymous noun keywords for each noun keyword. As described in item 4 of the patent application scope, a document search system using text analysis results and natural language input, when the sentence analysis unit retrieves a query noun keyword, the file list providing unit will use the query noun key Word, the synonymous noun keyword of the query noun keyword and the source of the keyword query the comparison table, sort the file names of all matching files in the file list, and issue the file list through the network platform communication unit . The document search system using text analysis results and natural language input as described in item 1 of the patent application scope, in which the text information type is character encoding, text images, or files with embedded text image objects. A document search system using text analysis results and natural language input as described in item 1 of the patent application scope, in which text images and files with embedded text image objects are converted into optical character recognition technology (Optical Character Recognition, OCR) into Corresponding character encoding. The document search system using text analysis results and natural language input as described in item 1 of the patent scope, where the file format is Portable Document Format (PDF), Power Point file format, Power Point file compatible Format, Word file format, Word file compatible format, Excel file format, or Excel file compatible format. For example, the document search system using text analysis results and natural language input as described in item 1 of the patent application scope, where the source of the keyword is file name, file content, or file metadata (Metadata). The document search system using text analysis results and natural language input as described in item 1 of the patent application scope, in which the retrieval method for querying noun keywords and keyword sources includes the steps of: (a) Separating query phrases by words ; (B) Give the segmented words part-of-speech; (c) Classify the words that belong to the noun category and appear in the noun keyword database into query noun keywords; and (d) Will belong to the noun category, Words with the same synonyms as the plural preset keyword sources are classified as keyword sources. For example, the document search system using text analysis results and natural language input as described in item 1 of the patent application scope, where the noun keyword comes from the Chinese sentence structure tree database of the Central Research Institute of the Republic of China.

Images