JP7377915B2 - Method, computer device, and computer program for providing personalized data retrieval service - Google Patents

Description

The following description relates to techniques for providing personalized data search services.

Searching for text document data is an extremely basic and important operation, and is widely used in the information retrieval field.

A search engine, in a broad sense, refers to a system that collects and searches information on the Internet.It mainly crawls web pages on the Internet, and when a specific search term (query) is input, Refers to a system that shows web pages related to the search term as result values.

For example, Patent Document 1 (registration date: March 2, 2011) discloses a technique for providing a custom search engine to a client.

Generally, an inverted index document structure that indexes terms is used for searching. In the existing document structure, one primary key specifies multiple fields, but in an inverted index, one value (term) specifies the document number that includes the corresponding value.

On the other hand, in recent years, services have been provided that allow searches within individual data such as personal emails, personal files, and messenger chat rooms.

Although an inverted index is the best material structure for search response speed, in an individual data search service, the search target is only a small part of the entire document, so the inverted index material structure is not suitable in terms of cost and resources.

Korean Registered Patent No. 10-1021022

As an engine specializing in individual data search services, we provide a search engine without inverted indexes.

To provide a search engine that can apply a full scan method for partial match search, which is a basic requirement of individual data search services, and at the same time satisfy the response speed of individual data search services.

A method for retrieving discrete data performed on a computing device, the computing device including at least one processor configured to execute computer readable instructions contained in a memory, the method for retrieving discrete data comprising: compressing a search target document corresponding to individual data related to a user into a block-based volume by the at least one processor and storing it in a volume; and compressing a plurality of volumes corresponding to the search request in parallel by the at least one processor. The present invention provides a method for searching individual data, including a step of performing a full scan search.

According to one aspect, the storing step may include generating a compressed volume by collecting and compressing the search target documents in blocks of a certain size.

According to another aspect, when a new document comes in, the storing step includes appending the new document to an increment volume of the search target documents, and adding the increment volume to a predetermined size. The method may include compressing blocks to generate a compressed volume.

According to another aspect, when the existing document is deleted after the compressed volume is generated, the storing step further includes marking the existing document with deletion information, and the marked document may be excluded from search results.

According to another aspect, the searching step may search for documents that partially match the query using a full scan method for the block-based compressed volume without using an inverted index document structure.

According to another aspect, the searching step may include decoding the plurality of volumes in parallel, and performing a string find on the decoded volumes in parallel.

According to another aspect, the storing step may include storing a replica volume of the individual data in a plurality of hosts for server duplication.

According to another aspect, the searching step includes normalizing the query included in the search request and the documents in the plurality of volumes in Unicode, and collating the query using the normalized character string. ) performing a search.

According to another aspect, the storing step includes Unicode normalizing the search target document, and the searching step includes Unicode normalizing the query included in the search request, and normalizing the search target document. The method may include a step of performing a matching search using the character strings obtained.

According to still another aspect, the storing step may further include generating a conversion table including an offset indicating a converted character position and an original character at the corresponding position.

A computer program for causing a computer device to execute the search method is provided.

A computing device including at least one processor configured to execute computer-readable instructions contained in a memory, the at least one processor configured to retrieve documents corresponding to individual data associated with a user. A computer device is provided that includes a document storage section that compresses and stores a volume in blocks, and a parallel search section that performs a full scan search in parallel on a plurality of volumes corresponding to a search request.

According to the embodiments of the present invention, it is possible to provide a search engine that is specialized for individual data search services and can satisfy the response speed of individual data search services without using a transposed index material structure.

According to an embodiment of the present invention, it is possible to provide an intuitive search service with high search efficiency by generating a search target document as a compressed volume in units of blocks and performing a full scan search on the compressed volume in parallel. can.

1 is a diagram illustrating an example of a network environment in an embodiment of the present invention. FIG. 1 is a block diagram illustrating an example of a computer device in an embodiment of the present invention. FIG. 1 is a diagram illustrating an example of components that a processor of a computing device may include in an embodiment of the present invention; FIG. 1 is a flowchart illustrating an example of a method that may be performed by a computing device in an embodiment of the invention. FIG. 3 is an exemplary diagram illustrating a method of reducing input/output time in an embodiment of the present invention. FIG. 3 is an exemplary diagram for explaining a volume generation process that replaces an index in an embodiment of the present invention. FIG. 3 is an exemplary diagram for explaining a volume generation process that replaces an index in an embodiment of the present invention. FIG. 3 is an exemplary diagram for explaining a method of reducing CPU time in an embodiment of the present invention. FIG. 2 is a diagram showing the structure of an individual data search service using a full scan method according to an embodiment of the present invention. FIG. 2 is an exemplary diagram for explaining duplication of search servers in an embodiment of the present invention. FIG. 3 is an exemplary diagram illustrating a normalization process required for a collation search in an embodiment of the present invention.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

Embodiments of the present invention relate to techniques for providing individual data search services.

Embodiments including matters specifically disclosed in this specification can provide a search engine without an inverted index as an engine specialized for individual data search services, thereby improving search efficiency, Considerable advantages can be achieved in aspects such as service intuitiveness and cost savings.

In this specification, individual data refers to documents to be searched, particularly personal email documents generated by email services, personal files generated by Drive services, personal talk messages generated by messaging services, etc. It may refer to a comprehensive set of user's personal documents such as:

The individual data search device according to the embodiment of the present invention may be realized by at least one computer device, and the individual data search method according to the embodiment of the present invention may be realized by at least one computer device included in the individual data search device. May be executed. At this time, the computer program according to the embodiment of the present invention may be installed and executed in the computer device, and the computer device may perform the individual data search according to the embodiment of the present invention under the control of the executed computer program. You may carry out the method. The above-mentioned computer program may be recorded on a computer-readable recording medium in order to be coupled to a computer device and cause the computer to execute the individual data retrieval method.

FIG. 1 is a diagram showing an example of a network environment in an embodiment of the present invention. The network environment of FIG. 1 shows an example including multiple electronic devices 110, 120, 130, 140, multiple servers 150, 160, and a network 170. Such FIG. 1 is only an example for explaining the invention, and the number of electronic devices and the number of servers are not limited as shown in FIG. 1. Furthermore, the network environment in FIG. 1 is merely an example of an environment applicable to this embodiment, and the environment applicable to this embodiment is not limited to the network environment in FIG. 1. .

The plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer devices. Examples of the plurality of electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, PCs (personal computers), notebook PCs, digital broadcasting terminals, PDAs (personal digital assistants), and PMPs (portable multimedia platforms). ayer ), tablets, etc. As an example, although FIG. 1 shows a smartphone as an example of the electronic device 110, in the embodiment of the present invention, the electronic device 110 may utilize a substantially wireless or wired communication method to communicate with others via the network 170. electronic devices 120, 130, 140 and/or servers 150, 160.

The communication method is not limited, and is not limited to communication methods that utilize communication networks that can be included in the network 170 (for example, mobile communication networks, wired Internet, wireless Internet, and broadcasting networks), as well as communication methods that utilize short distances between devices. Wireless communications may also be included. For example, the network 170 is a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), or a WAN (wide area network). e area network), BBN (broadband network), the Internet, etc. may include any one or more of the networks. Additionally, network 170 may include any one or more of network topologies including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. It will not be done.

Each server 150, 160 is implemented by one or more computing devices that communicate with a plurality of electronic devices 110, 120, 130, 140 via a network 170 to provide instructions, code, files, content, services, etc. good. For example, the server 150 may be a system that provides services (financial services, for example) to a plurality of electronic devices 110, 120, 130, and 140 connected via the network 170.

FIG. 2 is a block diagram illustrating an example of a computer device in an embodiment of the present invention. Each of the plurality of electronic devices 110, 120, 130, and 140 and each of the servers 150 and 160 described above may be realized by the computer device 200 shown in FIG. 2.

Such a computing device 200 may include a memory 210, a processor 220, a communication interface 230, and an input/output interface 240, as shown in FIG. Memory 210 is a computer readable storage medium and may include permanent mass storage devices such as random access memory (RAM), read only memory (ROM), and disk drives. Here, a permanent large capacity storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate permanent storage device separate from the memory 210. Additionally, an operating system and at least one program code may be recorded in the memory 210. Such software components may be loaded into memory 210 from a computer-readable storage medium separate from memory 210. Such other computer-readable recording media may include computer-readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, memory cards, and the like. In other embodiments, software components may be loaded into memory 210 through communication interface 230 that is not a computer-readable storage medium. For example, software components may be loaded into memory 210 of computing device 200 based on a computer program installed by a file received over network 170.

Processor 220 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 220 by memory 210 or communication interface 230. For example, processor 220 may be configured to execute instructions received according to program code recorded on a storage device, such as memory 210.

The communication interface 230 may provide functionality for the computing device 200 to communicate with other devices (eg, the recording device described above) via the network 170. As an example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 according to a program code recorded in a storage device such as the memory 210 may be transmitted to others via the network 170 under the control of the communication interface 230. may be transmitted to the device. Conversely, signals, instructions, data, files, etc. from other devices may be received by the computing device 200 via the network 170 and through the communication interface 230 of the computing device 200 . Signals, instructions, data, etc. received through communication interface 230 may be communicated to processor 220 and memory 210, files, etc. may be transferred to a storage medium (such as a persistent storage device as described above) that computing device 200 may further include. May be recorded.

Input/output interface 240 may be a means for interfacing with input/output device 250. For example, input devices may include devices such as a microphone, keyboard, or mouse, and output devices may include devices such as a display and speakers. As another example, input/output interface 240 may be a means for interfacing with a device that has integrated input and output functionality, such as a touch screen. Input/output device 250 may be configured as one device with computer device 200.

Also, in other embodiments, computing device 200 may include fewer or more components than those of FIG. However, most prior art components need not be clearly illustrated. For example, computing device 200 may be implemented to include at least some of the input/output devices 250 described above, and may further include other components such as transceivers, databases, and the like.

In the following, specific embodiments of a method and a computer device for providing an individual data search service will be described.

FIG. 3 is a block diagram illustrating an example of components that a processor of a computer device may include, in an embodiment of the invention, and FIG. 2 is a flowchart illustrating an example of a method that can be used.

The computer device 200 according to the present embodiment may provide an individual data search service to a client by connecting to a dedicated application installed on the client or a web/mobile site associated with the computer device 200. The computer device 200 may include a computer-implemented individual data search device.

Individual data search services have a large inflow of documents, but relatively few search requests. Furthermore, the individual data search service requires, as a basic requirement, a partial match search to search for documents that partially match the query. In the case of the individual data search service, the documents that are actually searched during a search are only a small portion of the total documents.

Individual data search services receive a large amount of documents, so when using the transposed index material structure commonly used for searches, excessive transposed index costs are incurred, leading to wasted resources on the search server. .

Furthermore, in order to provide partial match search with a transposed index document structure, bigram analysis of documents must be performed, but during bigram analysis, the number of terms and volume become large. Difficulties in service.

In this embodiment, in consideration of the characteristics of the individual data search service as described above, as an engine specialized for the individual data search service, the engine satisfies the response speed of the individual data search service without using the transposed index material structure. Provide a search engine that can

The processor 220 of the computer device 200 may include a document storage section 310 and a parallel search section 320, as shown in FIG. 3, as components for executing the individual data search method shown in FIG. Depending on the embodiment, components of processor 220 may be selectively included or excluded from processor 220. Also, depending on the embodiment, components of processor 220 may be separated or combined to express the functionality of processor 220.

Such processor 220 and components of processor 220 may control computer device 200 to perform steps S410 to S420 included in the individual data retrieval method of FIG. 3. For example, processor 220 and components of processor 220 may be implemented to execute instructions in accordance with operating system code and at least one program code contained in memory 210.

Here, the components of processor 220 may be representations of different functions that are performed by processor 220 according to instructions provided by program code recorded on computer device 200. For example, the document storage unit 310 may be used as a functional representation of the processor 220 that controls the computer device 200 according to the instructions described above so that the computer device 200 stores documents to be searched.

Processor 220 may read the necessary instructions from memory 210 loaded with instructions related to controlling computing device 200 . In this case, the read instructions may include instructions for controlling the processor 220 to perform steps S410 to S420 described below.

The steps S410-S420 described below may be performed in a different order than shown in FIG. 4, and some of the steps S410-S420 may be omitted or additional steps may be included. It's okay.

Referring to FIG. 4, in step S410, the document storage unit 310 increases the search volume by storing search target documents corresponding to individual data such as personal emails, personal files, personal chat messages, etc. in block units. May be generated. According to an embodiment of the present invention, the search volume is recorded in the nonvolatile memory 210 (for example, an auxiliary storage device such as a disk) in the form of a file, and when the parallel search unit 320 searches, the search volume is The volume file may be read and processed from 210 (eg, volatile main storage such as RAM). At this time, if the saved file is compressed, the time required to read the file can be reduced, and the search response time can be reduced.

The document storage unit 310 stores incoming documents to be searched in volumes in units of blocks. At this time, in order to minimize search response time, the document storage unit 310 compresses the volumes in units of blocks and stores them as compressed volumes. In other words, the document storage unit 310 may compress and store the search target document during the volume generation stage in order to reduce the time required to read the volume during the search process. At this time, the document storage unit 310 may collect and compress the search target documents in blocks of a predetermined size. The size of each block may be determined based on empirical or experimental values that are large enough to achieve a compression ratio and can be parallelized. May generate volume. By generating a search volume without using an inverted index structure, the process of generating the volume (ie, packing) becomes lighter, and the cost of generating the volume can be significantly reduced. When a document to be searched is compressed, input/output time (I/O time) required for searching can be reduced, as well as volume size and server resource demand.

In step S420, when a search request is received, the parallel search unit 320 may read a compressed volume corresponding to the search request and perform a full scan search on the read compressed volume in parallel. At this time, the parallel search unit 320 may perform a full scan search using a character string find method to search for documents that partially match the query. When the parallel search unit 320 receives a search request, the parallel search unit 320 may perform character string finding after reading a compressed volume corresponding to the search request. In order to perform string finding, it is necessary to read the entire document in the volume, but while the process of reading a compressed volume cannot be parallelized, the decoding of the compressed volume and the string finding process can be parallelized. It is. In other words, the parallel search unit 320 may perform string finding after decoding all compressed volumes corresponding to the search request in parallel. Since the search target in the individual data search service is only a small part of the entire document, a sufficiently fast response speed can be guaranteed by performing a full scan search instead of the transposed index material structure. The full scan search method can store the original document in the search volume as it is and then scan it, so it can greatly reduce the volume generation cost, and it is easy to implement increments, and it is easy to update new documents. It also has the advantage of being quick. In particular, in this embodiment, during full-scan search, CPU time, including volume decoding time and character string finding execution time, can be reduced by block-by-block parallelization.

This embodiment provides an individual data search service using a full scan method without using a transposed index structure.

In full scan search, a volume corresponding to a search request is read, and then all documents in the corresponding volume are read to search for documents that partially match the query. At this time, the search response time includes the input/output time for reading the volume and the CPU time for performing the full scan search.

In the case of an individual data search service using the full scan method, a method of reducing input/output time and a method of reducing CPU time are applied in order to minimize the search response time.

FIG. 5 is an exemplary diagram illustrating a method for reducing input/output time in an embodiment of the present invention.

The method of reducing input/output time in FIG. 5 corresponds to the document storage step S410 described in FIG. 4.

The processor 220 may compress the search target document into blocks of a certain size. Referring to FIG. 5, an incremental volume 50 of search target documents may be compressed into blocks of a constant size. In other words, the processor 220 may generate the compressed volume 60 in units of blocks by separately compressing the search target document, thereby reducing the input/output time of the search response time. For example, if a compression rate of 400% is applied as shown in FIG. 5, input/output time and storage space can be reduced to 1/4.

The material structure for the individual data search service includes an incremental volume 50 indicating a volume in which documents to be searched are collected before being compressed, and a compressed volume 60 indicating a volume compressed into blocks of a fixed size. Incremental volumes may be utilized for the purpose of minimizing reading and decoding performance of volume files in environments with a continuous influx of small volumes of documents. As explained below, once a certain size is reached, compression is preferably performed to create a compressed volume to maintain a relatively small size.

At this time, when a new document comes in, the processor 220 may convert the new document into a serviceable material structure and then append it to the incremental volume 50 to reflect the new document. As shown in FIG. 6, when the incremental volume 50 consisting of search target documents reaches a certain size, for example 1 MB, the processor 220 may compress it into 1 MB blocks and add them to the compressed volume 60. When the average document size is 4 KB and the SSD read speed is 500 MB/s, it is possible to search up to 500,000 documents in an individual volume within 1 second.

The processor 220 can maintain the volume even if the server is shut down during the volume creation process by performing a full scan search at the same time as the volume creation process. By appending only added documents to a volume, the volume can be incremented without having to be regenerated or replaced.

Referring to FIG. 7, in the case of a new document, the processor 220 appends the new document to the incremental volume 50, and then compresses the incremental volume 50 to a compressed volume 60 when the incremental volume 50 reaches a certain size. It may be reflected.

On the other hand, if an existing document is deleted after the compressed volume 60 is generated, the processor 220 may mark the corresponding document with deletion information, and exclude the marked document from the search results when generating the search results. You can process it by method. At this time, an additional document structure for marking deletion information may be used. Depending on the embodiment, it is also possible to organize garbage (unnecessary data (garbage)) including deleted documents and the like periodically or at necessary times to regenerate the compressed volume 60.

FIG. 8 is an exemplary diagram for explaining a method of reducing CPU time in an embodiment of the present invention.

The method of reducing CPU time in FIG. 8 corresponds to the parallel search step S420 described in FIG. 4.

Referring to FIG. 8, the processor 220 performs a process of decoding a volume compressed in units of blocks, that is, a compressed volume 60, and a character string finding process using a full scan for the search target document 50 that has been decompressed by the decoding. Each process may be parallelized. The processor 220 can reduce the CPU time in the search response time by executing parallel decoding and parallel character string finding in block units. The processor 220 may provide a search result 80 corresponding to the search request as a response result by performing block-by-block parallel decoding and parallel string finding.

FIG. 9 is a diagram illustrating the structure of an individual data search service using a full scan method according to an embodiment of the present invention.

Referring to FIG. 9, the individual data search device according to the present invention may include a search application server (SAS) 910 and a search server 920 for the individual data search service. The SAS 910 is responsible for providing search results corresponding to a search request, and the search server 920 is responsible for storing a compressed volume 60 for searching. The search server 920 may perform a volume generation process (packing) instead of an inverted index, and one search server 920 may store tens of thousands of volumes or more.

Since the individual data search device according to the present invention does not use an inverted index, it does not require an index server, and since it applies the compressed volume 60 in blocks, it is possible to reduce the demand for the search server 920.

By using a full scan method instead of a transposed index, the individual data search device according to the present invention can naturally provide a partial match search required by an individual data search service.

Furthermore, the individual data search device according to the present invention may support server duplication. Referring to FIG. 10, two hosts may be randomly designated for each user in the search server 920, and the individual data of the corresponding user may be stored in the designated hosts. The individual data retrieval device may include a database 1030 that stores server locations mapped to each user. In other words, replica volumes for individual data may be stored in a plurality of search servers 920, and this can be used for standby replication rather than for distributing search requests. .

Furthermore, the individual data search device according to the present invention may support a collation function that searches for characters with the same meaning without distinguishing them from each other. The matching function includes a function to search without distinguishing between uppercase and lowercase letters (case insensitive), an alphabet without a sign and an alphabet with a sign (for example,
(Outside 1)
), a function to search without distinguishing between katakana and hiragana (Kana Type insensitive), etc. may be included.

Since the individual data search device according to the present invention consists of one volume for searching and one volume for document summarization (for example, Snippet), it is possible to search if this is normalized. However, there may be a limit that document summarization is not possible.

In order to provide matching, the individual data search device according to the present invention may apply one of a method of performing Unicode normalization at the time of search and a method of performing Unicode normalization at the time of volume generation.

For example, the processor 220 may perform the search after normalizing the query included in the search request and all documents in the volume during the search, and converting the normalized string into a certain format (e.g., downcase). may provide search results by matching. At this time, the processor 220 may also perform the normalization process in parallel to minimize search execution time.

As another example, the processor 220 may perform Unicode normalization on the document in a pre-processing manner during volume generation and then save the document. Referring to FIG. 11, during the volume generation process, the processor 220 may normalize the document in the volume and at the same time generate a conversion table 1140 for restoring the normalized document to its original state. The conversion table 1140 may include an offset indicating a converted character position within a character string and an original character at the corresponding position. Thereafter, processor 220 may normalize the query during a search and search for documents that have already been normalized for the query. In the search process, only the query is normalized, and the search target documents are already normalized, so the search load can be reduced. Since the original document is required when summarizing the document based on the search results, the conversion table 1140 may be used to restore the document to its original state. Since document summarization is performed only on some documents depending on the search results, few documents are restored to their original state, and the search response time is not greatly affected.

As described above, the embodiments of the present invention provide a search engine that is specialized for individual data search services and can satisfy the response speed of individual data search services without using a transposed index material structure. I can do it. Further, according to an embodiment of the present invention, a document to be searched is generated as a compressed volume in units of blocks, and a full scan search is performed on the compressed volume in parallel, thereby providing a highly efficient and intuitive search service. be able to.

The apparatus described above may be realized by hardware components, software components, and/or a combination of hardware and software components. For example, the devices and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or may be implemented using one or more general purpose or special purpose computers, such as various devices capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that execute on the OS. The processing device may also be responsive to execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, one processing device may be described as being used, but those skilled in the art will understand that a processing device may include multiple processing elements and/or multiple types of processing elements. You will understand that. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include computer programs, code, instructions, or a combination of one or more of these that configure a processing device or instruct a processing device, independently or collectively, to perform operations as desired. You may do so. The software and/or data may be embodied in a machine, component, physical device, computer storage medium or device of any kind for being interpreted by or providing instructions or data to a processing device. good. The software may be distributed on computer systems connected by a network, and may be recorded or executed in a distributed manner. The software and data may be recorded on one or more computer readable storage media.

Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on computer-readable media. Here, the medium may be one that continuously records a computer-executable program, or one that temporarily records it for execution or download. Also, the medium may be a variety of recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may be distributed over a network. It may also exist. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, It may also include ROM, RAM, flash memory, etc., and may be configured to record program instructions. Further, other examples of the medium include an application store that distributes applications, a site that supplies or distributes various other software, and a recording medium or storage medium managed by a server.

As mentioned above, although the embodiments have been described based on limited embodiments and drawings, those skilled in the art will be able to make various modifications and variations based on the above description. For example, the techniques described may be performed in a different order than in the manner described, and/or components of the systems, structures, devices, circuits, etc. described may be performed in a different form than in the manner described. Even when combined or combined, opposed or replaced by other components or equivalents, suitable results can be achieved.

Therefore, even if the embodiments are different, if they are equivalent to the scope of the claims, they fall within the scope of the appended claims.

220: Processor 310: Document storage section 320: Parallel search section

Claims (17)

An individual data retrieval method performed on a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The individual data search method is
compressing and storing a search target document corresponding to individual data related to a user into a volume in units of blocks by the at least one processor; and compressing a plurality of volumes corresponding to the search request in parallel by the at least one processor. including a full scan search step,
The storing step includes, when a new document comes in, adding the new document to the incremental volume consisting of the search target document, and compressing the incremental volume into blocks of a constant size to generate a compressed volume. Including individual data search methods. The storing step includes:
If the existing document is deleted after the compressed volume is generated, the method further comprises: marking deletion information for the existing document;
The individual data search method according to claim 1 , wherein the marked document is excluded from search results. The full scan search step includes:
2. The individual data retrieval method according to claim 1, wherein a document that partially matches the query is retrieved by a full scan method for the compressed volume in units of blocks without using a transposed index material structure. The full scan search step includes:
The individual data retrieval method according to claim 1, comprising: decoding the plurality of volumes in parallel; and performing character string finding on the decoded volumes in parallel. The storing step includes:
2. The individual data search method according to claim 1, further comprising the step of: storing replicated volumes for the individual data in a plurality of hosts for server duplication. The full scan search step includes:
2. The method according to claim 1, comprising: normalizing a query included in the search request and documents in the plurality of volumes in Unicode; and performing a matching search using the normalized character string. Individual data search method. The storing step includes:
normalizing the search target document in Unicode;
The full scan search step includes:
The individual data search method according to claim 1, comprising: normalizing a query included in the search request in Unicode; and performing a collation search using the normalized character string. An individual data retrieval method performed on a computer device, the method comprising:
The computer device includes at least one processor configured to execute computer-readable instructions contained in memory;
The individual data search method is
compressing and storing a search target document corresponding to individual data associated with a user into a volume in blocks by the at least one processor;
performing a full-scan search in parallel on a plurality of volumes corresponding to the search request by the at least one processor;
including;
The storing step includes:
normalizing the search target document in Unicode; and
A step of generating a conversion table including an offset indicating the position of the converted character and the original character at the corresponding position.
including;
The full scan search step includes:
Unicode normalizing the query included in the search request; and
Stage of performing a matching search using normalized strings
Individual data search methods, including . A computer program for causing a computer device to execute the individual data search method according to any one of claims 1 to 8 . A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
The at least one processor includes:
It includes a document storage unit that compresses and stores search target documents corresponding to individual data related to the user into volumes in blocks, and a parallel search unit that performs full scan searches in parallel on multiple volumes corresponding to a search request,
When a new document flows in, the document storage unit adds the new document to the incremental volume consisting of the search target document, and compresses the incremental volume into blocks of a constant size to generate a compressed volume. The document storage unit is
11. The computer device according to claim 10 , wherein a compressed volume is generated by collecting and compressing the search target documents in blocks of a fixed size. The document storage unit is
if the existing document is deleted after the compressed volume is generated, marking deletion information for the existing document;
The computer device according to claim 10 , wherein the marked document is excluded from search results. The parallel search unit includes:
13. The method according to claim 10 , wherein a document that partially matches the query is retrieved by a full scan method for the block-based compressed volume without using a transposed index data structure. computer equipment. The parallel search unit includes:
decoding the plurality of volumes in parallel;
The computer device according to any one of claims 10 to 12 , characterized in that character string finding is executed in parallel for the decoded volumes. The parallel search unit includes:
Unicode normalizing a query included in the search request and documents in the plurality of volumes;
The computer device according to any one of claims 10 to 12 , characterized in that a verification search is performed using a normalized character string. The document storage unit is
Unicode normalize the search target document,
The parallel search unit includes:
Unicode normalize the query included in the search request;
The computer device according to any one of claims 10 to 12 , characterized in that a verification search is performed using a normalized character string. A computer device,
at least one processor configured to execute computer-readable instructions contained in the memory;
including;
The at least one processor includes:
a document storage unit that compresses and stores search target documents corresponding to individual data related to the user into a volume in blocks, and
Parallel search unit that performs full scan searches of multiple volumes in parallel in response to search requests
including;
The document storage unit is
Unicode normalize the search target document,
Generates a conversion table containing the offset indicating the converted character position and the original character at the corresponding position,
The parallel search unit includes:
Unicode normalize the query included in the search request;
Performing a matching search using normalized strings
A computer device characterized by:

Images