WO2018066144A1

WO2018066144A1 - Program for generation of indexed data, method for generation of indexed data, system for generation of indexed data, search program, search method, and search system

Info

Publication number: WO2018066144A1
Application number: PCT/JP2016/080034
Authority: WO
Inventors: ゆかり袴田; 隼毅袴田; 片岡　正弘; 亮佑兵庫
Original assignee: 富士通株式会社
Priority date: 2016-10-07
Filing date: 2016-10-07
Publication date: 2018-04-12
Also published as: JP6575688B2; JPWO2018066144A1; US11182341B2; US20190251062A1

Abstract

A program for generation of indexed data causes a computer to execute a process of generating form output format data, from form data which includes a form which includes a plurality of columns. Said program for generation of indexed data causes said computer to execute a process of generating index information with respect to words, characters, or numerical values, said index information including position information which is capable of identifying respective attributes of the plurality columns and position relations in the form data of the data which is included in the form output format data and corresponds to each of the plurality of columns. Said program for generation of indexed data causes said computer to execute a process of outputting an output file which includes the index information and the form output format data.

Description

Indexed data generation program, indexed data generation method, indexed data generation system, search program, search method, and search system

The present invention relates to an indexed data generation program, an indexed data generation method, an indexed data generation system, a search program, a search method, and a search system.

Conventionally, when printing a form, for example, a PDF (Portable Document Format) format is used in order to ensure that it is output in a target output format. A PDF file created for printing may be used not only for printing and storage purposes but also for searching and the like. However, since a PDF file created for printing is required to have a small file size, it cannot be used for advanced search. In order to perform an advanced search, for example, it is known to embed metadata such as position information such as a character string and attribute information in a PDF file by extending a tag. In order to perform an advanced search, for example, it is known to generate an index that associates a form item with the number of pages with a specific key.

JP 2013-045208 A JP 2001-056836 A

However, for example, when the tag is expanded, the file size becomes large and the index is not included, so that the search cannot be performed quickly.

In addition, for example, even when an index that associates words and numerical values with page information is created in units of words and numerical values using conventional techniques, the index has only page information. Can't search for words or numbers. For this reason, even if the index is referred to, it is necessary to confirm whether or not the search candidates individually correspond to the search target items after narrowing down the search candidates by the search.

For example, when searching for a numerical value in a form, even if a numerical value of “100” is searched, a search candidate whose “unit price” is “100”, “quantity” is “100”, and “amount” is “100”. Since it is displayed, it is necessary to confirm whether it corresponds to the target item. Similarly, when searching for words, even if you search for “Kamakura”, search candidates with “branch name” as “Kamakura”, “address” as “Kamakura”, and “name” as “Kamakura” Is displayed.

Therefore, it is necessary to refer to the data of the pages that are candidates for the search of the PDF file and individually check whether or not the items correspond to the search target items.

In one aspect, the purpose is to quickly determine whether or not the item is a search target item.

In the first proposal, the indexed data generation program causes a computer to execute processing for generating form output format data from form data including a form having a plurality of columns. The indexed data generation program stores index information for words, characters, or numerical values in a computer, and includes data for each attribute corresponding to each attribute of the plurality of columns and each of the plurality of columns included in the form output format data. A process of generating index information including position information that can identify the positional relationship in is executed. The indexed data generation program causes a computer to execute a process of outputting an output file including index information and form output format data.

According to one aspect, it is possible to quickly determine whether or not the item corresponds to a search target item.

FIG. 1 is a diagram illustrating an example of the flow of a form file generation process according to the first embodiment. FIG. 2 is a diagram illustrating an example of the bitmap index according to the first embodiment. FIG. 3 is a diagram illustrating an example of the hashed index generation process according to the first embodiment. FIG. 4 is a diagram illustrating a configuration example of an encoded file. FIG. 5 is a diagram illustrating an example of an image in which a search target is input. FIG. 6 is a diagram illustrating an example of a search process according to the first embodiment. FIG. 7 is a diagram illustrating an example of the character string determination process according to the first embodiment. FIG. 8 is a diagram illustrating an example of search condition determination processing according to the first embodiment. FIG. 9 is a diagram illustrating an example of the hashed bitmap restoration process according to the first embodiment. FIG. 10 is a functional block diagram illustrating the configuration of the form output device according to the first embodiment. FIG. 11 is a diagram illustrating an example of a flowchart of the form file generation process according to the first embodiment. FIG. 12 is a functional block diagram illustrating the configuration of the search device according to the first embodiment. FIG. 13 is a diagram illustrating an example of a flowchart of search processing according to the first embodiment. FIG. 14 is a diagram illustrating an example of form output format data and a bitmap index according to the second embodiment. FIG. 15 is a diagram illustrating an example of an image in which a search target is input. FIG. 16 is a diagram illustrating an example of search processing according to the second embodiment. FIG. 17 is a diagram illustrating an example of form output format data and a bitmap index according to the third embodiment. FIG. 18 is a diagram illustrating an example of an image in which a search target is input. FIG. 19 is a diagram illustrating an example of search processing according to the third embodiment. FIG. 20 is a diagram illustrating a hardware configuration example of a computer. FIG. 21 is a diagram illustrating a configuration example of a program operating on a computer. FIG. 22 is a diagram illustrating a configuration example of an apparatus in the system according to the embodiment.

Hereinafter, embodiments of an indexed data generation program, an indexed data generation method, an indexed data generation system, a search program, a search method, and a search system disclosed in the present application will be described in detail with reference to the drawings. This scope of rights is not limited by this embodiment. Each embodiment can be appropriately combined within a range in which processing contents do not contradict each other.

[Example of form file generation according to Embodiment 1]
FIG. 1 is a diagram illustrating an example of the flow of a form file generation process according to the first embodiment. The form output device generates form output format data from the input form data. The form output device, based on the generated form output format data, attributes (hereinafter may be referred to as “items”) of a plurality of items (columns) of the form, and a plurality of items included in the form output format data. A bitmap index including position information capable of specifying the positional relationship in the form data between the data corresponding to each item is generated. Then, the form output device generates a form file including form output format data and a bitmap index. The form data includes a form having a plurality of items.

An example of a form file generation process is described below. For example, a form output device that executes a form file generation process reads a file 3 including form data. Note that “3” of the file 3 indicates that the file ID is “3”. The file 3 has “date”, “product code”, “product name”, and the like as items of the form. For example, the item “date” includes “2015/7/15”. For example, the item “product code” includes “12345”. For example, “product name” includes “a1a2a3”. Here, “a1a2a3” will be described as “refrigerator” displayed in Japanese kanji. The form items are not limited to these.

The form output device generates form output format data from the form data included in the file 3. The form output format data is CSV (Comma-Separated Values) format data. The form output format data includes a record composed of a plurality of items of data separated by commas. Although FIG. 1 shows an example having one record as an example, the form output format data includes a plurality of records.

The form output device encodes the file 3 with the form output form data including the character string composed of words and characters and the numerical string composed of numerical values, and generates the encoded data and the encoding dictionary. To do. Hereinafter, the character string and the numerical value string will be simply referred to as “character string”. For example, the form output device encodes “2015/7/15” of the item “date” into “A000h”. For example, the form output device encodes “12345” in the item “product code” into “B010h”. For example, the form output device encodes “refrigerator” in the item “product name” into “C021h”. Further, for example, when the character string of the form output format data is alphabet, the form output device encodes for each word. For example, “Liquid Crystal Television” is divided into “Liquid”, “Crystal”, and “Television”, and each is encoded. Each data of the form output format data is encoded for each item. The form output device encodes the form output format data and generates encoded data.

Also, the form output device encodes each character when the character string of the form output format data is Japanese. For example, “refrigerator” is divided into “cold”, “brewery”, and “warehouse”, and each is encoded. For example, the form output device encodes “cold” into “D000h”. For example, the form output device encodes “Kura” into “D001h”. For example, the form output device encodes “warehouse” into “D002h”. When the character string of the form output format data is Japanese, the code used for the encoded data is an encoded code for each character string.

Hereinafter, a code obtained by encoding each data of the form output format data is referred to as “word ID”.

The coding dictionary has a static dictionary and a dynamic dictionary. Static dictionaries are based on general English dictionaries, Japanese dictionaries, textbooks, etc., and identify the frequency of words that appear in a document. ID). In the static dictionary, a static code that is a code corresponding to each word is registered in advance. On the other hand, the dynamic dictionary is a dictionary in which words that are not registered in the static dictionary are associated with dynamically assigned dynamic codes (codes). Examples of words that are not registered in the static dictionary include words with low appearance frequency (low frequency words), characters, unknown words, numerical values, time, and tags. The unknown word here is a word that is not registered in the static dictionary and has a feature that appears repeatedly in the document to be encoded. In the dynamic dictionary, words associated with the dynamic code are registered in the order of appearance of words not registered in the static dictionary. Note that the encoding dictionary for generating encoded data and the encoding dictionary for generating a bitmap index may be different dictionaries.

The form output device generates information about the word ID for the file ID. Further, the form output device generates information on the word ID and the position of the character string indicated by the word ID for the file ID. Also, the form output device generates information on the form item and the position of the form item for the file ID. The form output device sets the presence or absence of the character string indicated by the word ID for the file 3 in the two-dimensional bitmap index BI1 based on the generated information. The form output device also sets the word ID and the position of the character string indicated by the word ID for the file 3 in the first three-dimensional bitmap index BI2 based on the generated information. Further, the form output device sets the item of the form and the position indicated by the form item with respect to the file 3 in the second three-dimensional bitmap index BI3 based on the generated information.

When the character string of the form output format data is Japanese, the word ID encoded for each character is used in the two-dimensional bitmap index BI1, the first three-dimensional bitmap index BI2, and the like. For example, when the character string is “refrigerator” in Japanese, three word IDs corresponding to “cold”, “brewery”, and “warehouse” are used. For example, when the character string is the alphabet “Liquid Crystal Television”, three word IDs corresponding to “Liquid”, “Crystal”, and “Television” are used.

Here, the two-dimensional bitmap index BI1 is an index for full-text search, and is a bit string obtained by concatenating a pointer specifying a character string and a bit indicating the presence or absence of the character string in each file. That is, the two-dimensional bitmap index BI1 refers to a bitmap obtained by indexing the presence / absence of each file with respect to a character string included in the file. During the search process, this bitmap can be used as an index indicating whether or not a character string to be searched is included. For example, a word ID is adopted as a pointer for designating a character string. For example, the character string itself may be used as the pointer for designating the character string. That is, the two-dimensional bitmap index BI1 is a collection of bitmaps for each character string indicated by a pointer that designates a character string. As shown in FIG. 1, the X axis of the two-dimensional bitmap index BI1 represents a file ID, and the Y axis represents a word ID. That is, the two-dimensional bitmap index BI1 indicates the presence / absence of the character string indicated by the word ID for each file indicated by the plurality of file IDs.

Here, the first three-dimensional bitmap index BI2 is a bitmap index in which the position of the character string in each file is added to the two-dimensional bitmap index BI1. In other words, the first three-dimensional bitmap index BI2 refers to a bitmap obtained by indexing positions existing for each file with respect to a character string included in the file. As shown in FIG. 1, the X-axis of the first three-dimensional bitmap index BI2 represents Offset (position), the Y-axis represents a word ID, and the Z-axis represents a file ID. The first three-dimensional bitmap index BI2 represents the position for each file indicated by the plurality of file IDs of the character string indicated by the word ID. That is, the first three-dimensional bitmap index BI2 is index information including position information that can specify the positional relationship in the form data between character strings.

Also, the second three-dimensional bitmap index BI3 is a bitmap in which the position of the form item in each file is indexed. As shown in FIG. 1, the X axis of the second three-dimensional bitmap index BI3 represents Offset (position), the Y axis represents a form item, and the Z axis represents a file ID. That is, the second three-dimensional bitmap index BI3 is index information including position information that can specify the positional relationship of the form items in the form data.

Here, processing when the form output device generates each bitmap type index for the file 3 will be described.

The form output device encodes a character string included in the form item of the file 3 into a word ID. When the character string is in Japanese, the form output device encodes each character into a word ID. For example, since “refrigerator” of “product name” is Japanese, it is encoded into a word ID for each character. Here, a process when each bitmap type index is generated for “cold” of the item “product name” of the form will be described. The word ID of “Cold” in “Product Name” is “D000h”. It is assumed that the position of “cold” in the file 3 is “2”.

The form output device sets the presence or absence of the character indicated by the word ID “D000h” for the file 3 in the two-dimensional bitmap index BI1 based on the generated information. In the two-dimensional bitmap index BI1 shown in FIG. 1, the bit value “1” indicating that “D000h” exists in the file with the file ID “3” is a bit corresponding to the file ID “3”. Set to For example, when “D000h” does not exist in the file, the bit value “0” is set in the bit corresponding to the file ID.

The form output device sets the position “2” of the character indicated by the word ID “D000h” in the first three-dimensional bitmap index BI2 for the file 3 based on the generated information. In the first three-dimensional bitmap index BI2 shown in FIG. 1, the bit value “1”, which means that “D000h” exists at the position “2” of the file with the file ID “3”, The bit corresponding to the position “2” of the ID “3” is set. For example, when “D000h” does not exist at a predetermined position of the file, a bit value “0” is set to a bit corresponding to the predetermined position of the file ID.

Also, the form output device sets the position of the item “product name” of the form in the second three-dimensional bitmap index BI3 with respect to the file 3 based on the generated information. In the second three-dimensional bitmap index BI3 shown in FIG. 1, a bit value “1” indicating that the item “product name” of the form starts from the position “2” of the file 3 has a file ID. It is set to the bit corresponding to position “2” of “3”. In the second three-dimensional bitmap index BI3, the bit value “1” is set only for the bit corresponding to the position where the item of the form starts. For example, in the file 3, the item “product name” of the form starts from the position “2” and ends at the position “4”. Therefore, for the item next to the item “product name” in the form, the bit value “1” is set to the bit corresponding to the position “5” in the second three-dimensional bitmap index BI3.

In this way, the form output device, with respect to the file 3, includes a plurality of items of the form and each bit including position information that can specify the positional relationship in the form data between the word IDs obtained by encoding the form data. Map type indexes BI1 to BI3 are generated. The form output device generates each hashed index from each bitmap type index BI1 to BI3. Then, the form output device outputs an encoded file having the index information of each hashed index and a form file including the form output format data as an output file. The hashed index will be described later.

[Example of bitmap index]
Next, the bitmap type index will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of the bitmap index according to the first embodiment. Here, a bitmap type index for a form item in the file 3 will be described as an example. Further, a bitmap type index for the word ID corresponding to “2015/7/15” of the form item “date” and the item “refrigerator” of the form in the file 3 will be described as an example.

In the bitmap type index for the form items, the position and the bit corresponding to the position are allocated in the order of the items from the top record of the file 3. Then, the bit value of the bit corresponding to the item position is set to “1”. When the position and the bit corresponding to the position are assigned to the item of the last form of a certain record, the next position and the next bit corresponding to the position are assigned to the first item of the next record.

The bitmap of the item “date” of the form is “... 1000001”. This indicates that data relating to the item “date” of the form of the first record is stored in the 0th bit at the position “0” of the file 3. In addition, the sixth bit at the position “6” in the file 3 indicates that data relating to the item “date” of the form of the next record is stored. Here, data relating to the item “date” of the form is stored in each bit corresponding to the positions “12”, “21”, “29”, and “37”.

Also, the bit map of the item “product code” in the form is “... 10000010”. This indicates that data relating to “product code” of the form item of the first record is stored in the first bit at the position “1” of the file 3. In addition, the seventh bit at the position “7” of the file 3 indicates that data relating to the item “product code” of the form of the next record is stored. Here, data relating to the item “product code” of the form is stored in each bit corresponding to the positions “13”, “22”, “30”, and “38”.

In addition, the bit map of the item “product name” of the form is “... 100,000,100”, and the bit map of the item “unit price” of the form is “... 100,000,000”. This indicates that data relating to the item “product name” is stored in the second bit to the fourth bit at the positions “2” to “4” of the file 3. In addition, it indicates that data relating to the item “product name” is stored in the 8th to 10th bits of the position “8” to “10” of the file 3. In addition, here, in the bits corresponding to the positions “14” to “19”, “23” to “27”, “31” to “35”, and “39” to “42”, the item “product” Stores data related to "name".

In the bitmap type index for the word ID, the position and the bit corresponding to the position are assigned from the top record of the file 3 in the order of the word ID. Then, the bit value of the bit corresponding to the position of the word ID is set to “1”. When the position and the bit corresponding to the position are assigned to the last word ID of a certain record, the next position and the next bit corresponding to the position are assigned to the first word ID of the next record.

Also, the bitmap of the word ID “A000h” (numerical value “2015/7/15)” is “... 0001”. This indicates that the numerical value “2015/7/15” corresponding to the word ID “A000h” is stored in the 0th bit and not stored in other bits. The bitmap of the word ID “D000h” (character “cold”) is “... 0100”. This indicates that the character “cold” corresponding to the word ID “D000h” is stored in the second bit and not stored in other bits. The bitmap of the word ID “D001h” (character “Kura”) is “... 1000”. This indicates that the character “Kura” corresponding to the word ID “D001h” is stored in the third bit and not stored in other bits. Also, the bitmap of the word ID “D002h” (the character “box”) is “... 10000”. This indicates that the character “kura” corresponding to the word ID “D002h” is stored in the fourth bit and not stored in other bits.

As described above, the bitmap type index is, for example, a bitmap type transposed index storing form items, position information of word IDs in form data, and the like.

[Example of hashed index generation processing according to Embodiment 1]
Next, an example of processing for hashing the bitmap of the bitmap side index will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of the hashed index generation process according to the first embodiment. FIG. 3 illustrates an example of hashing a bitmap of the two-dimensional bitmap index BI1 that represents a file ID as the X axis and a word ID as the Y axis.

The form output device generates a plurality of hashed bitmaps to which a hash function is applied for each bitmap corresponding to the word ID. Here, the form output device assumes a 43-bit register and, as an example, based on the hash values (bottom) of 29 and 31, hash the bitmap of the two-dimensional bitmap index BI1. Specifically, the form output device sets the value of each bit of the bitmap to the remainder of dividing the position of each bit of the bitmap corresponding to the word ID by the bottom for one hashed bitmap. Set. The position of each bit in the bitmap corresponds to each file ID. As an example, the form output device sets the value of each bit of the bitmap in the remainder position obtained by dividing the position of each bit of the bitmap by the base 29 for the hashed bitmap h11 of the base 29. The bit value “1” at the 35th bit position of the bitmap is set to the sixth bit of the hashed bitmap h11. The bit value “1” at the 42nd bit position of the bitmap is set to the 13th bit of the hashed bitmap h11. The form output device sets the value of each bit of the bitmap in the remainder position obtained by dividing the position of each bit of the bitmap by the base 31 for the hashed bitmap h12 of the base 31. The bit value “1” at the 35th bit position of the bitmap is set to the fourth bit of the hashed bitmap h12. The bit value “1” at the 42nd bit position of the bitmap is set to the 11th bit of the hashed bitmap h12. That is, the form output device sets each bit from the 0th bit of the bitmap in order from the 0th bit of the hashed bitmap, and sets up to the (base-1) th bit. Then, the form output device wraps again and sets a value obtained by OR operation with the value already set in the hashed bitmap from the 0th bit of the bitmap. As a result, the form output device generates hashed bitmaps h11 and h12 for the bitmap. In this way, the form output device generates hashed bitmaps h11 and h12 for the bitmap of the two-dimensional bitmap index BI1 in the word ID, respectively. The form output device generates a two-dimensional hashed index HI1 in which the generated hashed bitmaps h11 and h12 are collected.

The form output device generates a plurality of hashed bitmaps to which a hash function is applied from the bitmap of the first three-dimensional bitmap index BI2 as in the case of the bitmap of the two-dimensional bitmap index BI1. . Then, the form output device generates a first three-dimensional hashed index HI2 in which the generated hashed bitmaps are collected. Similarly, the form output device generates a second three-dimensional hashed index HI3 from the second three-dimensional bitmap index BI3. Hereinafter, description will be made assuming that the hash values (bottom) are 29 and 31, for example.

[Configuration example of encoded file]
FIG. 4 is a diagram illustrating a configuration example of an encoded file. As shown in FIG. 4, the encoded file has a header part, encoded data, and a trailer part. The encoded data stores an encoded word ID group for each of a plurality of files. The trailer unit stores the address of each file, dynamic dictionary (common / individual) information, hashed index (2D / 3D), and the like. The address of each file indicates the storage destination address of each encoded data obtained by encoding a plurality of files. As an example, the address of each file is a relative address from the beginning of the encoded data. Individual dynamic dictionary information corresponds to dynamic dictionary information for each of a plurality of files. The common dynamic dictionary information corresponds to the dynamic dictionary information for the entire plurality of files. The header part stores a pointer to the address of each file stored in the trailer part, a pointer to a dynamic dictionary, and a pointer to a hashed index (2D / 3D). When the form output device encodes a plurality of files, the form output device stores the encoded data indicating the encoded results in the encoded file, and stores the stored addresses at the addresses of the files. In the decryption process, the address of the file to be decrypted is referred to from the address of each file in the trailer section using a pointer to the address of each file in the header section. In the decoding process, a pointer to the dynamic dictionary in the header part is used to refer to the dynamic dictionary. In the search processing, the address of the hashed index to be searched is referred to from the address of each hashed index of the trailer unit using a pointer to the address of each hashed index of the header unit.

[Example of Search Processing According to Embodiment 1]
FIG. 5 is a diagram illustrating an example of an image in which a search target is input. As shown in FIG. 5, in the search device, “form”, “page”, and “record” can be selected as the effective range of the search condition. The effective range of the search condition represents a unit for performing the search. For example, when “form” is selected, the search is performed in units of forms. For example, when “page” is selected, the search is performed in units of pages. For example, when “all” is selected, the search is performed without designating the unit for performing the search. In the search device, for example, it is possible to search using three search conditions as item conditions. Here, a case will be described in which “all” is selected as the effective range of the search condition, “product name” is set as the search condition, and “refrigerator” is input as the search target. The search condition may be directly input. When Japanese is input as a search target, the search target character string is divided for each character and the search is performed.

FIG. 6 is a diagram illustrating an example of search processing according to the first embodiment. As shown in FIG. 6, the search device that executes the search process refers to the two-dimensional hashed index HI1 based on the input “refrigerator” to be searched. The search device acquires a file ID in which each word ID corresponding to “cold”, “stored”, and “stored” exists. Then, the search device specifies the word ID and the file ID in the first three-dimensional hashed index HI2, and specifies the position information of the word ID. In addition, the search device specifies the position information of the item “product name” of the form by designating the “product name” and the file ID of the search condition in the second three-dimensional hashed index HI3.

An example of the search process will be described in detail below. When receiving the input of the search target character string, the search device refers to the two-dimensional hashed index HI1 and extracts a hashed bitmap for the word ID indicated by the received search target. The search device divides the “refrigerator” received as the search target into “cold”, “stored”, and “storage”, and reads the word ID corresponding to each character. The search device reads the word ID “D000h” corresponding to “cold”. The search device reads the word ID “D001h” corresponding to “Kura”. The search device reads the word ID “D002h” corresponding to “box”. Then, the search device extracts the hashed bitmap h4 for the word ID “D000h” from the two-dimensional hashed index HI1. The hashed bitmap h4 includes a hashed bitmap h41 at the base 29 and a hashed bitmap h42 at the base 31. The search device extracts the hashed bitmap h5 for the word ID “D001h” from the two-dimensional hashed index HI1. The hashed bitmap h5 includes a hashed bitmap h51 at the base 29 and a hashed bitmap h52 at the base 31. The search device extracts the hashed bitmap h6 for the word ID “D002h” from the two-dimensional hashed index HI1. The hashed bitmap h6 includes a hashed bitmap h61 on the bottom 29 and a hashed bitmap h62 on the bottom 31.

The search device restores a hashed bitmap for each extracted word ID. The hashed bitmap restoration process will be described later. The restoration result is represented by a bitmap corresponding to the word ID. Here, the search device restores the hashed bitmap h4 for the word ID “D000h” and outputs the bitmap b4 as the restoration result. The search device restores the hashed bitmap h5 for the word ID “D001h” and outputs the bitmap b5 as the restoration result. The search device restores the hashed bitmap h6 for the word ID “D002h” and outputs the bitmap b6 as the restoration result.

Also, the search device performs an AND operation on the bitmap b4 with the word ID “D000h”, the bitmap b5 with the word ID “D001h”, and the bitmap with the word ID “D002h”. The search device outputs a file ID whose bit of the AND result is ON (“1”). That is, the search device narrows down file IDs of files including “cold” indicated by the word ID “D000h”, “kura” indicated by the word ID “D001h”, and “box” indicated by the word ID “D002h”. . Here, “3” is output as the file ID.

In addition, after narrowing down the file ID, the search device refers to the first three-dimensional hashed index HI2 and extracts a hashed bitmap for the word ID and the file ID. Here, the search device extracts the hashed bitmap h7 for the word ID “D000h” and the file ID “3” from the first three-dimensional hashed index HI2. The hashed bitmap h7 includes a hashed bitmap h71 at the base 29 and a hashed bitmap h72 at the base 31. Further, the search device extracts the hashed bitmap h8 for the word ID “D001h” and the file ID “3” from the first three-dimensional hashed index HI2. The hashed bitmap h8 includes a hashed bitmap h81 at the bottom 29 and a hashed bitmap h82 at the bottom 31. Further, the search device extracts the hashed bitmap h9 for the word ID “D002h” and the file ID “3” from the first three-dimensional hashed index HI2. The hashed bitmap h9 includes a hashed bitmap h91 at the bottom 29 and a hashed bitmap h92 at the bottom 31.

In addition, the search device restores the extracted hashed bitmaps h7 to h9. The restoration result is represented by a bit map corresponding to the word ID and the file ID. Here, the search device restores the hashed bitmap h7 for the word ID “D000h” and the file ID “3”, and outputs the bitmap b7 as the restoration result. The search device restores the hashed bitmap h8 for the word ID “D001h” and the file ID “3”, and outputs the bitmap b8 as a restoration result. The search device restores the hashed bitmap h9 for the word ID “D002h” and the file ID “3”, and outputs the bitmap b9 as the restoration result.

Also, the search device specifies an offset (position) where the bits of the bitmaps b7 to b9 output as the restoration result indicate ON (“1”). For example, the search device specifies an offset (position) whose bit value is “1” in the bitmap b7 of the file ID “3” and the word ID “D000h”. Here, Offset (position) “2” is specified. Further, the search device specifies an Offset (position) having a bit value “1” in the bitmap b8 of the file ID “3” and the word ID “D001h”. Here, Offset (position) “3” is specified. Further, the search device specifies an Offset (position) having a bit value “1” in the bitmap b9 of the file ID “3” and the word ID “D002h”. Here, Offset (position) “4” is specified.

The search device determines whether or not the search target character string is included in the file based on the specified Offset (position). FIG. 7 is a diagram illustrating an example of the character string determination process according to the first embodiment.

For example, as shown in FIG. 7, when the offset (position) is specified for the word IDs “D000h”, “D001h”, and “D002h”, the search device determines the word ID according to the character string to be searched. Bitmaps b7 to b9 are arranged. Here, since the character string to be searched is “refrigerator”, the search device arranges bitmap b7, bitmap b8, and bitmap b9 in this order. Then, the search device shifts the bit whose bit value is “1” in the “cold” bitmap b7, which is the first character of the character string to be searched, to the left by one. As a result, in the “cold” bitmap b7, the third bit is “1”. The search device performs an AND operation on the shifted “cold” bitmap b7 and the next character “kura” bitmap b8. When the result of the AND operation is “1”, the search device shifts a bit whose bit value is “1” in the AND result bitmap to the left by one. As a result, in the bit map of the AND result, the fourth bit is “1”. The search device performs an AND operation on the bitmap of the shifted AND result and the bitmap “b9” of “K” as the next character. When the result of the AND operation is “1”, the search device determines that the search target is included in the file. Note that when any result of the AND operation is “0”, the search device determines that the search target is not included in the file.

In this way, the search device determines whether or not the search target is included in the file by combining the AND operation and the bit shift.

Referring back to FIG. 6, after narrowing down the file ID, the search device refers to the second three-dimensional hashed index HI3 and extracts a hashed bitmap for the form item included in the file ID. Here, the search device extracts the hashed bitmap for the form item included in the file ID “3” from the second three-dimensional hashed index HI3. The search device uses the second three-dimensional hashed index for the hashed bitmaps h10 to h13 corresponding to the items “date”, “product code”, “product name”, “unit price”, etc. of the form included in the file 3. Extract from HI3. The hashed bitmap h10 includes a hashed bitmap h101 at the bottom 29 and a hashed bitmap h102 at the bottom 31. The hashed bitmap h11 includes a hashed bitmap h111 at the bottom 29 and a hashed bitmap h112 at the bottom 31. The hashed bitmap h12 includes a base 29 hashed bitmap h121 and a base 31 hashed bitmap h122. The hashed bitmap h13 includes a hashed bitmap h131 at the base 29 and a hashed bitmap h132 at the base 31.

Also, the search device restores the extracted hashed bitmaps h10 to h13. The restoration result is represented by a file ID and a bitmap corresponding to the item of the form. Here, the search apparatus restores the hashed bitmaps h10 to h13 and outputs the bitmaps b10 to b13 as the restoration results.

Also, the search device specifies an offset (position) in which the bits of the bitmaps b10 to b13 output as the restoration result indicate ON (“1”). For example, in the bitmap b10 of the file ID “3” and the item “date” of the form, the search device specifies an offset (position) where the bit value of the bitmap b10 is “1”. Here, for example, Offset (position) “0” is specified. Similarly, the search device specifies an offset (position) where the bit values of the bitmaps b11 to b13 are “1” for each item of the form.

The search device determines whether or not the search target character string is included in the search condition item based on the specified offset (position). FIG. 8 is a diagram illustrating an example of search condition determination processing according to the first embodiment.

Here, the search device performs a search condition determination process based on the offset (position) of the identified form item and the result of the AND operation on the “refrigerator” described with reference to FIG. The search device determines whether the Offset (position) where the bit value is “1” is included in the item “product name” selected as the search condition in the final AND result bitmap in FIG. 7. To do. In the final AND result bitmap, the bit value of the fourth bit is “1”. In the “product name” item bitmap, the second bit is “1”, and in the “unit price” item bitmap, the fifth bit is “1”. Therefore, it can be seen that characters included in the item “product name” are stored in the second to fourth bits in the bitmap. In the final AND result bitmap, the fourth bit is “1”. Thereby, the search device determines that the character string “refrigerator” to be searched exists in the item “product name” in the file 3. For example, when the position where the bit value is “1” in the final AND result bit map is the first bit or the fifth bit, the search device sets the character string “refrigerator” to be searched. However, it is determined that the item “product name” does not exist.

The search device outputs, for example, a file name including the character string “refrigerator” in the item “product name” of the form. The search device may display, for example, a file name including the character string “refrigerator” in the item “product name” of the form, and a character string before and after including “refrigerator”. As described above, the search device uses the hashed index (bitmap type index) to search for the search target character string that satisfies the search condition, and outputs the search result. Thereby, the search device can quickly output a search result for the character string to be searched.

[Example of hashed bitmap restoration processing according to Embodiment 1]
Next, an example of processing for restoring the hashed bitmap will be described with reference to FIG. FIG. 9 is a diagram illustrating an example of the hashed bitmap restoration process according to the first embodiment. Here, a case where the hashed bitmap h4 is restored will be described as an example.

The search device expands the hashed bitmaps h41 and h42 included in the hashed bitmap h4 into bitmaps (restoration 1). Here, the search device adds the position of each bit in the hashed bitmap to the value obtained by multiplying the base by an integer (0 to) for the restoration destination bitmap of one base hashed bitmap. The value of each bit of the hashed bitmap is set at the position. As an example, the search device adds the position of each bit of the hashed bitmap h41 to the value obtained by multiplying the base 29 by “0” for the restoration destination bitmap b41 of the hashed bitmap h41 of the base 29. The value of each bit of the hashed bitmap h41 is set. The retrieval apparatus performs hashing at the position obtained by adding the position of each bit of the hashed bitmap h41 to the value obtained by multiplying the base 29 by “1” for the restoration destination bitmap b41 of the hashed bitmap h41 of the base 29. The value of each bit of the bitmap h41 is set. The search device repeats until the value of the bit at the position of the maximum bit of the bitmap b41 in the bottom 29 of the restoration destination is set. Similarly, the search device adds the position of each bit of the hashed bitmap h42 to the value obtained by multiplying the base 31 by “0” for the restoration destination bitmap b42 of the hashed bitmap h42 of the base 31. The value of each bit of the hashed bitmap h42 is set. The retrieval apparatus performs hashing at the position obtained by adding the position of each bit of the hashed bitmap h42 to the value obtained by multiplying the base 31 by “1” for the restoration destination bitmap b42 of the hashed bitmap h42 of the base 31. The value of each bit of the bitmap h42 is set. The search device repeats until the value of the bit at the position of the maximum bit in the restoration destination bitmap b42 is set.

The search device performs an AND operation on the bit at the corresponding position of each restored bitmap (restoration 2). Here, the search device performs an AND operation on the bitmap b41 restored from the hashed bitmap h41 in the base 29 and the bitmap b42 restored from the hashed bitmap h42 in the base 31. The retrieval device outputs the AND result bitmap b4 as a restoration result.

Note that the search device also restores the hashed bitmap of the first three-dimensional hashed index HI2 in the same manner as the hashed bitmap of the two-dimensional hashed index HI1. In addition, the search device also restores the hashed bitmap of the second three-dimensional hashed index HI3 in the same manner as the hashed bitmap of the two-dimensional hashed index HI1.

[Configuration of form output device according to Embodiment 1]
Next, the configuration of the form output apparatus 100 according to the first embodiment will be described with reference to FIG. FIG. 10 is a functional block diagram illustrating the configuration of the form output device 100 according to the first embodiment. As illustrated in FIG. 10, the form output device 100 includes a control unit 110 and a storage unit 120.

The control unit 110 is a processing unit that executes the index generation processing shown in FIG. The control unit 110 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. And the control part 110 respond | corresponds to the electronic circuit of integrated circuits, such as ASIC and FPGA, for example. Or the control part 110 respond | corresponds to electronic circuits, such as CPU and MPU. Further, the control unit 110 includes a form output format data generation unit 111, an encoding processing unit 112, an embedding unit 113, and an output unit 114.

The storage unit 120 corresponds to a storage device such as a nonvolatile semiconductor memory element such as a flash memory or FRAM (registered trademark). The storage unit 120 includes an encoding dictionary 121, a two-dimensional hashed index 122, a first three-dimensional hashed index 123, a second three-dimensional hashed index 124, and encoded data 125.

The form output format data generation unit 111 reads the encoding target file into the storage area. The form output format data generation unit 111 generates form output format data from the form data included in the read file. The form output format data generation unit 111 outputs the generated form output format data to the encoding processing unit 112. Note that the form output format data generation unit 111 extracts the page information of the form data from the form data. For example, the form output format data generation unit 111 searches for a definition file and extracts page information before generating form output format data. Further, for example, the form output format data generation unit 111 may extract page information in the process of generating form output format data. For example, the form output format data generation unit 111 adds page information to the form output format data. Further, for example, the form output format data generation unit 111 may output it as page information separately from the form output format data.

The encoding processing unit 112 includes an encoding unit 112a and an index generation unit 112b. The encoding unit 112 a encodes a character string included in the form output format data based on the encoding dictionary 121, and generates encoded data 125. The encoding unit 112a determines whether or not a character string included in the form output format data is registered in the static dictionary. As an example, the encoding unit 112a determines whether or not the character string hits the bit filter of the static dictionary. When the character string is registered in the static dictionary, the encoding unit 112a encodes the character string based on the static dictionary. As an example, the encoding unit 112a encodes a character string into a static code (word ID) corresponding to the character string based on a static dictionary. The encoding unit 112a outputs the encoded word ID to the index generation unit 112b.

The encoding unit 112a encodes the character string based on the dynamic dictionary when the character string is not registered in the static dictionary. As an example, the encoding unit 112a determines whether or not a character string is stored in the buffer unit of the dynamic dictionary. When the character string is not stored in the buffer unit of the dynamic dictionary, the encoding unit 112a stores the character string in the buffer unit, and stores the storage position where the character string is stored and the stored data length in the address table. To store. The encoding unit 112a encodes the character string into a dynamic code (word ID) in the address table associated with the character string. In addition, when the character string is stored in the buffer unit of the dynamic dictionary, the encoding unit 112a encodes the character string into a dynamic code (word ID) corresponding to the character string. The encoding unit 112a outputs information including the encoded word ID and the position of the character string indicated by the word ID to the index generation unit 112b.

The index generation unit 112b generates a two-dimensional hashed index 122 based on the file ID of the file and the encoded word ID. For example, the index generation unit 112b receives information output from the encoding unit 112a. The index generation unit 112b generates a bitmap corresponding to the file ID of the file in which the word ID exists based on the received information. The index generation unit 112b generates a hashed bitmap using the base α and the base β for the generated bitmap. That is, the index generation unit 112b sets the presence or absence of the character string indicated by the word ID for the file in the two-dimensional hashed index 122.

As an example, the index generation unit 112b generates a hashed bitmap based on two hash values (bottom) for each bitmap corresponding to the word ID. That is, the index generation unit 112b sequentially sets each bit from the 0th bit of the bitmap from the 0th bit of the hashed bitmap, and sets up to the (base-1) th bit. Then, the index generation unit 112b loops again and sets a value obtained by OR operation with a value already set in the hashed bitmap from the 0th bit of the hashed bitmap. Then, when the index generation unit 112b generates hashed bitmaps for the bitmaps corresponding to all the word IDs, the two-dimensional hashed index 122 in which the generated hashed bitmaps are collected is stored in the storage unit 120. To do.

Also, the index generation unit 112b generates the first three-dimensional hashed index 123 based on the file ID of the file, the encoded word ID, and the position of the character string indicated by the word ID. For example, the index generation unit 112b generates a bitmap corresponding to the file ID of the file in which the word ID exists and the position of the word ID in the file based on the received information. The index generation unit 112b generates a hashed bitmap using the base α and the base β for the generated bitmap. That is, the index generation unit 112b sets the word ID and the position of the character string indicated by the word ID for the file (form data) in the first three-dimensional hashed index 123.

Also, the index generation unit 112b generates the second three-dimensional hashed index 124 based on the file ID of the file, the form item, and the position of the form item. For example, the index generation unit 112b generates a bitmap corresponding to the file ID of the file in which the word ID exists and the position of the item of the form based on the received information. The index generation unit 112b generates a hashed bitmap using the base α and the base β for the generated bitmap. That is, the index generation unit 112b sets the position of the item of the form with respect to the file (form data) and the position of the form item in the form data in the second three-dimensional hashed index 124.

Note that the generation method of the first three-dimensional hashed index 123 and the second three-dimensional hashed index 124 is the same as the description of the two-dimensional hashed index 122, and thus the description thereof is omitted. The index generation unit 112b stores the generated first three-dimensional hashed index 123 and second three-dimensional hashed index 124 in the storage unit 120.

The embedding unit 113 embeds an encoded file having a dynamic dictionary of the encoding dictionary 121, encoded data 125, and index information of each hashed index 122 to 124 into a form file, and generates an output file. . As a result, a form file with an index is generated. For example, when the form file is a PDF file, the embedding unit 113 embeds the encoded file in the extension area of the PDF file. For example, when the form file is a unique compressed file of the form product, the embedding unit 113 compresses the encoded file together with the form definition body and generates an output file.

The output unit 114 transfers the output file to, for example, the search device 200 (see FIG. 12).

[Procedure for Form File Generation Processing According to Embodiment 1]
Next, the process procedure of the form file generation process according to the first embodiment will be described with reference to FIG. FIG. 11 is a diagram illustrating an example of a flowchart of the form file generation process according to the first embodiment. The form file is a form file with an index.

The form output format data generation unit 111 reads the encoding target file into the storage area (S10). The form output format data generation unit 111 generates form output format data from the form data included in the read file (S11).

The encoding unit 112a encodes the character string of the form output format data into the word ID based on the encoding dictionary 121 (S12). If the character string is not stored in the encoding dictionary 121 (static dictionary and dynamic dictionary), the encoding unit 112a newly encodes the character string and stores it in the dynamic dictionary.

The index generation unit 112b generates a two-dimensional hashed index 122 based on the file ID and word ID of the file (S13). The index generation unit 112b generates the first three-dimensional hashed index 123 based on the file ID of the file, the word ID, and the position of the word ID in the file (S14). The index generating unit 112b generates the second three-dimensional hashed index 124 based on the file ID of the file, the item of the form, and the position of the form item in the file (S15).

The embedding unit 113 embeds the encoded file having the dynamic dictionary of the encoding dictionary 121 and the hashed indexes 122 to 124 in the form file, and generates an output file (S16).

The output unit 114 outputs the generated output file to, for example, the search device 200 (see FIG. 12) (S17).

[Configuration of Search Device According to Embodiment 1]
Next, the configuration of the search device 200 that executes the search process according to the first embodiment will be described with reference to FIG. FIG. 12 is a functional block diagram illustrating the configuration of the search device 200 according to the first embodiment. As illustrated in FIG. 12, the search device 200 includes a control unit 210 and a storage unit 220.

The control unit 210 is a processing unit that executes the search processing shown in FIGS. The control unit 210 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. And the control part 210 respond | corresponds to the electronic circuit of integrated circuits, such as ASIC and FPGA, for example. Alternatively, the control unit 210 corresponds to an electronic circuit such as a CPU or MPU. In addition, the control unit 210 includes a search target reception unit 211, a first restoration unit 212, a second restoration unit 213, a search processing unit 214, and a search result output unit 215.

The storage unit 220 corresponds to a storage device such as a nonvolatile semiconductor memory element such as a flash memory or FRAM (registered trademark). The storage unit 220 includes an encoding dictionary 221, a two-dimensional hashed index 222, a first three-dimensional hashed index 223, a second three-dimensional hashed index 224, a first restored bitmap 225, and a second restored bit. A map 226, a third restored bitmap 227, a narrowed bitmap 228, and encoded data 229 are included. The first restoration bitmap 225 is a bitmap in which the two-dimensional hashed index 222 is expanded. The second restored bitmap 226 is a bitmap in which the first three-dimensional hashed index 223 is expanded. The third restored bitmap 227 is a bitmap in which the second three-dimensional hashed index 224 is expanded. The narrow-down bitmap 228 is a bitmap of an AND result obtained by performing an AND operation on the second restoration bitmap 226 when the search target is a character string represented by a plurality of word IDs. The bit map of AND corresponds to, for example, the bit map indicating “refrigerator” of the AND result in FIG.

The search target reception unit 211 receives a search target. For example, the search target receiving unit 211 displays an input image as shown in FIG. 5 and receives search conditions and a search target character string.

The first restoration unit 212 extracts a hashed bitmap from the two-dimensional hashed index 222 corresponding to the word ID indicated by the search target character string. The first restoration unit 212 expands the extracted hashed bitmap, and stores the expanded first restoration bitmap 225 in the storage unit 220. When the search target is composed of character strings represented by a plurality of word IDs, the first restoration unit 212 generates a plurality of hashed bitmaps from a plurality of two-dimensional hashed indexes 222 corresponding to each word ID. Extract. Then, the first restoration unit 212 expands each hashed bitmap, and stores the plurality of expanded bitmaps in the storage unit 220 as the first restoration bitmap 225.

The first restoration unit 212 extracts a file ID including all search target character strings from the first restoration bitmap 225. For example, when there are a plurality of first restoration bitmaps 225, the first restoration unit 212 performs an AND operation on the first restoration bitmap 225 as shown in FIG. The first restoration unit 212 extracts a file ID whose bit value of the AND result is “1”. The first restoration unit 212 outputs the extracted file ID to the second restoration unit 213.

The second restoration unit 213 extracts the hashed bitmap from the first three-dimensional hashed index 223 corresponding to the file ID and the word ID indicated by the character string to be searched. The second restoration unit 213 expands the extracted hashed bitmap, and stores the expanded second restoration bitmap 226 in the storage unit 220. When the search target is composed of character strings represented by a plurality of word IDs, the second restoration unit 213 uses the hashed bits from the plurality of first three-dimensional hashed indexes 223 corresponding to the respective word IDs. Extract multiple maps. Then, the second restoration unit 213 expands each hashed bitmap, and stores the plurality of expanded bitmaps as the second restoration bitmap 226 in the storage unit 220.

Also, the second restoration unit 213 extracts the hashed bitmap from the second three-dimensional hashed index 224 corresponding to the file ID and the form item. The second restoration unit 213 expands the extracted hashed bitmap, and stores the expanded third restoration bitmap 227 in the storage unit 220.

The search processing unit 214 searches for a file including a character string to be searched and a position of the character string to be searched. For example, the search processing unit 214 extracts an Offset (position) whose bit value indicates “1” based on the second restoration bitmap 226. When the search target is composed of a character string represented by a plurality of word IDs, the search processing unit 214 sets an offset (position) whose bit value indicates “1” for each second restored bitmap 226. ,Extract. Then, the search processing unit 214 compares the sequence of word IDs, that is, the sequence of character strings with the sequence of Offset (position) where the bit value indicates “1”. That is, the search processing unit 214 determines whether or not the positional relationship of the character string in the search target matches the positional relationship of the word ID with the bit value “1”. For example, in the second restoration bitmap 226, the search processing unit 214 combines the AND operation and the bit shift as shown in FIG. Whether or not matches. When the positional relationship of the character string to be searched matches the positional relationship of the word ID, the search processing unit 214 stores the final AND result bitmap as the narrowed bitmap 228 in the storage unit 220.

Also, the search processing unit 214 searches for a search target that satisfies the search condition. For example, when the search condition is selected in the item condition of FIG. 5, the search processing unit 214 selects a search target that satisfies the search condition based on the third restoration bitmap 227 and the refinement bitmap 228. Search for. For example, the search processing unit 214 determines that the Offset (position) where the bit value is “1” in the narrowed bitmap 228 is included in the Offset (position) indicating the search condition item in the third restoration bitmap 227. It is determined that the search condition is satisfied. In this way, the search processing unit 214 searches for a search target that satisfies the search condition.

The search result output unit 215 outputs the search result of the search processing unit 214. The search result output unit 215 outputs, for example, a file name for a file that satisfies the search condition and includes a search target character string.

[Processing procedure of search processing according to embodiment 1]
Next, a processing procedure of search processing according to the first embodiment will be described with reference to FIG. FIG. 13 is a diagram illustrating an example of a flowchart of search processing according to the first embodiment.

The search target reception unit 211 receives a search target (S20).

The first restoration unit 212 reads the two-dimensional hashed index 222 corresponding to the word ID representing the character string to be searched (S21). The first restoration unit 212 extracts a hashed bitmap from the read two-dimensional hashed index 222 and expands it into the first restored bitmap 225 (S22). Based on the first restoration bitmap 225, the first restoration unit 212 extracts a file ID including all the word IDs representing the character string to be searched (S23).

The second restoration unit 213 reads the first three-dimensional hashed index 223 based on the file ID and the word ID (S24). The second restoration unit 213 extracts the hashed bitmap from the first three-dimensional hashed index 223 and expands it into the second restored bitmap 226 (S25). The second restoration unit 213 reads the second three-dimensional hashed index 224 based on the file ID and the item of the form selected as the search condition (S26). The second restoration unit 213 extracts a hashed bitmap from the second three-dimensional hashed index 224 and expands it into the third restored bitmap 227 (S27). Note that when the search condition is not selected, the second restoration unit 213 proceeds with the process without extracting the hashed bitmap from the second three-dimensional hashed index 224, for example.

The search processing unit 214 determines whether or not all the character strings to be searched are included in the second restoration bitmap 226 (S28). When the second restoration bitmap 226 includes all the search target character strings (Yes in S28), the search processing unit 214 determines whether or not the search condition is satisfied (S29). If no search condition is selected, the search processing unit 214 proceeds with the process without determining whether the search condition is satisfied, for example.

If the search condition is satisfied (Yes in S29), the search result output unit 215 outputs, for example, a file name for the file that satisfies the search condition and includes the search target character string (S30). On the other hand, when any of the search target character strings is not included in the second restoration bitmap 226 (No in S28), the search result output unit 215 indicates that there is no file including the search target character strings. (S31). When the search condition is not satisfied (No at S29), the search result output unit 215 outputs that the search condition does not exist as a search result (S32).

[Effect of Example]
The form output device 100 generates form output format data from the form data. The form output device 100 is index information for a character string, and includes position information that can specify a form item and a positional relationship in the form data between data corresponding to each form item included in the form output format data. Generate index information including. The form output device 100 outputs an output file including index information and form output format data. Thereby, for example, the search device 200 can quickly determine whether there is a search target by referring to the index information.

The index information includes position information that can identify the item of the form and the positional relationship between the data included in the form output format data. Thereby, for example, the search device 200 can quickly determine whether or not the item corresponds to, for example, a search target item by referring to the index information.

The output file includes encoded data obtained by encoding the form output format data and index information in the extended area. For example, when the output file is a PDF file, encoded data and index information are included in the PDF extension area. Thereby, for example, the search device 200 can perform a search in a state where the form output format data is encoded, and can quickly determine whether or not the item corresponds to a search target item.

Note that, for example, it may be possible to embed attribute information or position information such as a character string as metadata in a PDF file. In this case, however, the file size increases. In addition, the time for generating the PDF file becomes longer. On the other hand, the form output device 100 embeds, for example, encoded data obtained by encoding form output format data and index information in the extension area of the PDF file. Thereby, the form output device 100 can reduce the file size of the output file. Further, the form output device 100 can shorten the time for generating a PDF file, for example.

In order to reduce the file size, for example, it is possible to compress the form data in the Zip format. However, if compression is performed in the Zip format, it is necessary to decompress all the compressed files when using compressed form data. Therefore, the search cannot be performed until all decompression is completed, and the search time becomes long. On the other hand, the form output device 100 has index information separately from the encoded data. Thereby, for example, the search device 200 can quickly determine whether or not the item corresponds to the search target item by referring to the index information. Therefore, the search time can be shortened.

Encoded data is data in which the form output format data is encoded for each item of the form. Thereby, the form output apparatus 100 can create an output file compressed at a high compression rate, and can reduce the size of the output file. Further, for example, the search device 200 can expand the encoded data for each item of the form, and can refer to or extract only a necessary part.

The first three-dimensional

hashed indexes

123 and 223 that are index information are bitmap-type transposed index information with the file ID, the word ID, and the position information of the word ID as axes. As a result, the form output device 100 can reduce the size of the output file as compared to, for example, embedding attribute information or position information such as a character string as metadata in a PDF file. Further, for example, the search device 200 can quickly perform a search with high accuracy. *

Next, Example 2 will be described. In the second embodiment, a case where the same character string is included in different items will be described.

[Example of form output format data and bitmap type index according to Embodiment 2]
FIG. 14 is a diagram illustrating the form output format data and the bitmap index according to the second embodiment. The form output format data in FIG. 14 has “branch name”, “address”, “gender”, and the like as items of the form. For example, the item “branch name” includes “Kamakura branch”. Further, for example, the item “address” includes “Kamakura city”. For example, the item “sex” includes “male”. Note that the encoding procedure, hashing procedure, and the like in the second embodiment are the same as those in the first embodiment, and the description in the second embodiment is omitted. Further, hereinafter, each character string will be described using each character string before being encoded into a word ID.

The bitmap type index generated by the form output device includes different form items, for example, “branch name” and “address”, and the character strings “kama” and “kura”. Therefore, in the bit map of “kick”, the bit values of the 0th bit and the 4th bit are set to “1”. In the bitmap in “kura”, the bit values of the first bit and the fifth bit are set to “1”.

[Example of search processing in Embodiment 2]
FIG. 15 is a diagram illustrating an example of an image in which a search target is input. Here, “address” is selected as the search condition, and “Kamakura” is input as the search target.

When the search device receives the input of the search target, the search device sets a mask bit in the bitmap of the item “address” as the search condition, as shown in FIG. FIG. 16 is a diagram illustrating an example of search processing according to the second embodiment. When the mask bit is set in the bit map of the item “address” of the form, a bit map in which the bit values of the fourth to sixth bits are set to “1” is generated.

The search device performs an AND operation on the bitmap of the mask bit and the bitmap of the search result of “Kamakura” (the narrowed-down bitmap 228). The search process for “Kamakura” is the same as the search process described with reference to FIG. Since “Kamakura” exists in the item “branch name” of the form and the item “address” of the form, the bit values of the first bit and the fifth bit in the bitmap of the search result of “Kamakura” are “ 1 ”is set. However, in the bit map of the mask bit, since the first bit does not correspond to the item “address” of the form, the bit value is set to “0”. Therefore, when an AND operation is performed, the bit value of the first bit is “0” and only the bit value of the fifth bit is “1” in the bit map of the AND result. Thus, the search device narrows down search targets that satisfy the search condition by performing an AND operation. That is, even when the same character string is included in a plurality of items of the form, the search device can narrow down the position of the search target according to the search condition.

Next, Example 3 will be described. In the third embodiment, a case where the same character string is included in different items and a search is performed in units of records will be described.

[Example of form output format data and bitmap index according to Embodiment 3]
FIG. 17 is a diagram illustrating the form output format data and the bitmap index according to the third embodiment. The form output format data in FIG. 17 has “branch name”, “address”, and “gender” as the items of the form. The form output format data may have other items. In addition, the form output format data of FIG. 17 includes a plurality of records. For example, the

records

1 and 2 include data different only in the item of “sex”. Note that the encoding procedure, hashing procedure, and the like in the third embodiment are the same as those in the first embodiment, and the description in the third embodiment is omitted. Further, hereinafter, each character string will be described using each character string before being encoded into a word ID.

The bitmap type index generated by the form output device includes character strings “Kama” and “kura” in different form items and different records. For this reason, in the bit map of “kick”, the bit values of the 0th bit, the 4th bit, the 8th bit, and the 12th bit are set to “1”. In the bitmap in “Kura”, the bit values of the first bit, the fifth bit, the ninth bit, and the thirteenth bit are set to “1”. In the bitmap indicating the record delimiters, “1” is set in the 0th, 8th, and 16th bits. In the bitmap indicating the record delimiter, the bit value of the bit corresponding to the first character string of the record is set to “1”. The bitmap indicating the record delimiter is generated as a three-dimensional bitmap index, for example, similarly to the bitmap for the form item. A description of the procedure is omitted.

[Example of search processing in Embodiment 3]
FIG. 18 is a diagram illustrating an example of an image in which a search target is input. Here, search is selected in units of records. In addition, “address” is set as the search condition 1, and “Kamakura” is input as the search target. Furthermore, “all” is set as the search condition 2 and “female” is input as the search target.

When the search device accepts the search target, the search device sets a mask bit in the bitmap of the item “address” as the search condition, as shown in FIG. FIG. 19 is a diagram illustrating an example of search processing according to the third embodiment. When the mask bit is set in the bit map of the item “address” of the form, a bit map in which the bit values of the 4th to 6th bits and the 12th to 14th bits are set to “1” is generated.

The search device performs an AND operation on the bitmap of the mask bit and the bitmap of the search result of “Kamakura” (the narrowed-down bitmap 228). When the AND operation is performed, the bit values of the 5th and 13th bits are “1” in the AND result bitmap. This indicates that “Kamakura” is included in the item “address” of the record 1 record and the item “address” of the record 2 record.

When the search condition is “all”, for example, the search device outputs the bit map of the search result “female” as the AND result bit map as it is without setting the mask bit. In the bit map of the AND result of “woman” to be searched, the bit value of the 15th bit is set to “1”. Note that when the search condition is “all”, the search device may perform an AND operation by setting mask bits in all the bitmaps of the form items.

The search device determines a record satisfying the search condition based on the AND result bitmap, the AND result bitmap of the “female” to be searched, and the bitmap indicating the record delimiter position. In the bit map indicating the record delimiter position, the search device sets the bit value in the bit map of the AND result and the bit map of the AND result of the “female” to be searched to the 0th to 7th bits indicating the record 1. It is determined whether or not “1” is set. In addition, in the bit map indicating the record delimitation position, the search device uses the bit of the AND result bit map and the bit map of the “female” AND result bit map indicating the record 2 in the 8th to 15th bits. It is determined whether or not the value is set to “1”. Here, in the 0th bit to the 7th bit, the bit value is not set to “1” in the bit map of the AND result of “woman” to be searched. Therefore, the search device determines that record 1 does not satisfy the search condition. In the 8th to 15th bits, the bit value of the AND result bitmap of the search target “woman” is set to “1”, and the bit value of the AND result bitmap is set to “1”. Is set. Therefore, the search device determines that record 2 satisfies the search condition. The search device narrows down and outputs records 2 that satisfy the search condition and include “Kamakura” and “Woman” to be searched. That is, even when the same character string is included in a plurality of records and a plurality of items of the form, the search device can narrow down the search target in units of records according to the search conditions.

Now, although the embodiments related to the disclosed apparatus have been described so far, the disclosed technology may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

When there are a plurality of files including search objects, for example, the search device 200 according to the embodiment counts the number of search objects for each file, and outputs a file preferentially as the number of search objects is increased. May be. For example, the search device 200 outputs files in the order that includes many search targets. As a result, it is possible to quickly find a file containing a large number of search targets.

In addition, the form output apparatus 100 according to the embodiment has been described on the assumption that each bit map is hashed based on the hash values (bottom) of 29 and 31 assuming a 32-bit register. In the embodiment, one bit map is described as 44 bits. However, the hash values (bottom) of 29 and 31 are examples, and are not limited to this. The number of bits in the bitmap is also an example and is not limited to this. The two hash values (bases) may be determined according to the number of character strings in the plurality of files. For example, if the number of character strings is 10,000, the two bases are selected so that the two-dimensional matrix represented by the remainder divided by one base and the remainder divided by the other base is about 10,000. The The two bases may be adjacent prime numbers. Two prime numbers to be selected are 97 and 101 as an example when the number of matrices is 10,000. In other words, in a two-dimensional matrix space in which the least common multiple is about 10,000, a surplus set obtained by one hash and the other hash for a character string does not collide with a surplus set obtained for another character string. It is based on the assumption that it will be (non-overlapping).

Further, in the embodiment, the form output device 100 uses each hashed bitmap based on two hash values (bottom) for each of the bitmap corresponding to the word ID and the bitmap corresponding to the word ID and the file ID. It explained that it generates. The form output device 100 may detect a hash collision (hash noise) when generating a hashed bitmap. For example, since an extremely high-frequency character string exists in a plurality of files, bit values at a plurality of positions in a bitmap corresponding to the extremely high-frequency character string are set to “1”. Then, when the bitmap is hashed, “1” may be set redundantly at the same position in the hashed bitmap. Examples of the extremely high frequency character string include “the” and “on”. Therefore, the form output apparatus 100 may perform hash collision reduction with respect to hash noise, and reduce hash noise by measuring a 0/1 ratio or dividing a bitmap. For example, the form output apparatus 100 uses the presence / absence information of the bitmap corresponding to the hashed bitmap in which the collision has occurred when any one of the hashed bitmaps continuously causes a collision (1 / 0) ratios. If the ratio of “1” is greater than the threshold, the form output device 100 divides the bitmap corresponding to the hashed bitmap in which the collision has occurred. As an example, the form output device 100 extracts bits at even-numbered positions in a bitmap corresponding to a hashed bitmap in which a collision has occurred, and newly generates a bitmap. In addition, the form output apparatus 100 extracts bits at odd-numbered positions in the bitmap corresponding to the hashed bitmap in which the collision has occurred, and newly generates a bitmap. Then, the form output device 100 stores the divided new bitmap as, for example, a low-frequency word area as a division destination. The form output device 100 sets a division destination for one of the two hashed bitmaps. Then, the form output device 100 generates a hashed bitmap based on the two hash values (bottom) for each divided bitmap after dividing the bitmap. Thereby, even if the data of the hashed bitmaps collide, the form output device 100 divides the even-numbered data and the odd-numbered data of the bitmap before hashing and performs hashing respectively. Thus, data collision can be avoided.

Further, the form output device 100 according to the embodiment, for example, based on a plurality of adjacent hash values (bottom), for example, two-dimensional (word ID axis and file ID axis) and three-dimensional (word ID axis) It has been explained that a hashed index is generated by applying hashing to the file ID axis and the character string position axis, the form item axis, and the file ID and form item position axis. However, the form output device 100 may use a block axis instead of the file axis. That is, the presence / absence information of the word ID may be in units of blocks.

In addition, when the search target character string is Japanese, the form output device 100 according to the example sets a word ID for each character and generates a bitmap index. However, the form output device 100 may set a word ID for each word and generate a bitmap index.

In addition, the processing procedure, control procedure, specific name, information including various data and parameters shown in the embodiment can be arbitrarily changed unless otherwise specified.

[Hardware configuration]
Hereinafter, hardware and software used in the above-described embodiment will be described. FIG. 20 is a diagram illustrating a hardware configuration example of the computer 1. The computer 1 includes, for example, a processor 301, a RAM (Random Access Memory) 302, a ROM (Read Only Memory) 303, a drive device 304, a storage medium 305, an input interface (I / F) 306, an input device 307, an output interface (I / F) 308, output device 309, communication interface (I / F) 310, SAN (Storage Area Network) interface (I / F) 311, and bus 312. Each piece of hardware is connected via a bus 312.

The RAM 302 is a readable / writable memory device, and for example, a semiconductor memory such as SRAM (Static RAM) or DRAM (Dynamic RAM), or a flash memory even if not a RAM is used. The ROM 303 includes a PROM (Programmable ROM). The drive device 304 is a device that performs at least one of reading and writing of information recorded in the storage medium 305. The storage medium 305 stores information written by the drive device 304. The storage medium 305 is a storage medium such as a hard disk, a flash memory such as an SSD (Solid State Drive), a CD (Compact Disc), a DVD (Digital Versatile Disc), or a Blu-ray disc. Further, for example, the computer 1 is provided with a drive device 304 and a storage medium 305 for each of a plurality of types of storage media.

The input interface 306 is connected to the input device 307 and is a circuit that transmits an input signal received from the input device 307 to the processor 301. The output interface 308 is a circuit that is connected to the output device 309 and causes the output device 309 to execute an output in accordance with an instruction from the processor 301. The communication interface 310 is a circuit that controls communication via the network 3. The communication interface 310 is, for example, a network interface card (NIC). The SAN interface 311 is a circuit that controls communication with a storage device connected to the computer 1 via a storage area network. The SAN interface 311 is, for example, a host bus adapter (HBA).

The input device 307 is a device that transmits an input signal according to an operation. The input signal is, for example, a key device such as a keyboard or a button attached to the main body of the computer 1, or a pointing device such as a mouse or a touch panel. The output device 309 is a device that outputs information according to the control of the computer 1. The output device 309 is, for example, an image output device (display device) such as a display, or an audio output device such as a speaker. For example, an input / output device such as a touch screen is used as the input device 307 and the output device 309. The input device 307 and the output device 309 may be integrated with the computer 1 or may be an apparatus that is not included in the computer 1 and is connected to the computer 1 from the outside, for example.

For example, the processor 301 reads a program stored in the ROM 303 or the storage medium 305 to the RAM 302 and performs the processing of the control units 110 and 210 according to the read program procedure. At that time, the RAM 302 is used as a work area of the processor 301. The functions of the storage units 120 and 220 are such that the ROM 303 and the storage medium 305 are program files (application program AP24, middleware MW23 and OS22 described later) and data files (for example, encoding

dictionaries

121 and 221, two-dimensional

hashed index

122, 222, the first three-dimensional

hashed indexes

123, 223, etc.), and the RAM 302 is used as a work area of the processor 301. A program read by the processor 301 will be described with reference to FIG.

FIG. 21 is a diagram illustrating a configuration example of a program operating on a computer. In the computer 1, an OS (operating system) 22 for controlling the hardware group HW21 (301 to 312) shown in FIG. The processor 301 operates according to the procedure according to the OS 22 to control and manage the hardware group HW21, whereby the processing according to the application program AP24 and the middleware MW23 is executed in the hardware group HW21. Further, in the computer 1, the middleware MW 23 or the application program AP 24 is read into the RAM 302 and executed by the processor 301.

When the form generation function is called, the processor 301 performs processing based on at least a part of the middleware MW23 or the application program AP24 (by controlling the hardware HW group 21 based on the OS 22) ) The function of the control unit 110 is realized. When the search function is called, the processor 301 performs processing based on at least a part of the middleware MW23 or the application program AP24 (by controlling the hardware HW group 21 based on the OS 22). The function of the control unit 210 is realized. The encoding function and the search function may be included in the application program AP24 itself, or may be a part of the middleware MW23 that is executed by being called according to the application program AP24.

FIG. 22 is a diagram illustrating a configuration example of an apparatus in the system of the embodiment. The system in FIG. 22 includes a computer 1 a, a computer 1 b, a base station 2, and a network 3. The computer 1a is connected to the network 3 connected to the computer 1b by at least one of wireless and wired.

The form output device 100 and the search device 200 may be included in either the computer 1a or the computer 1b shown in FIG. The computer 1b may include the function of the form output device 100, the computer 1a may include the function of the search device 200, the computer 1a may include the function of the form output device 100, and the computer 1b may include the function of the search device 200. . Further, both the computer 1a and the computer 1b may have the function of the form output device 100 and the function of the search device 200.

100 Form output device 110 Control unit 111 Form output format data generation unit 112 Encoding processing unit 112a Encoding unit 112b Index generation unit 113 Embedding unit 114 Output unit 120 Storage unit 121 Encoding dictionary 122 Two-dimensional hashed index 123 First The second three-dimensional hashed index 124 The second three-dimensional hashed index 200 The search device 211 The search target receiving unit 212 The first restoration unit 213 The second restoration unit 214 The search processing unit 215 The search result output unit 220 The storage unit 221 Encoding Dictionary 222 Two-dimensional hashed index 223 First three-dimensional hashed index 224 Second three-dimensional hashed index 225 First restored bitmap 226 Second restored bitmap 227 Third restored bitmap 228 Refined bitmap

Claims

On the computer,
Generate form output format data from form data including forms with multiple columns,
Index information for words, characters, or numerical values, which identifies the attributes of each of the plurality of columns and the positional relationship in the form data among the data corresponding to each of the plurality of columns included in the form output format data Generating the index information including possible position information;
An indexed data generation program for executing a process of outputting an output file including the index information and the form output format data.
The indexed data generation program according to claim 1, wherein the output file includes encoded data obtained by encoding the form output format data and the index information in an extension area of the output file.
The indexed data generation program according to claim 2, wherein the encoded data is data in which the form output format data is encoded for each attribute of the form.
The index information is bitmap-type transposed index information with the word, the character, or the numerical value, the file information of the form, and the position information as axes. The indexed data generation program according to any one of the above.
Computer
Generate form output format data from form data including forms with multiple columns,
Index information for words, characters, or numerical values, which identifies the attributes of each of the plurality of columns and the positional relationship in the form data among the data corresponding to each of the plurality of columns included in the form output format data Generating the index information including possible position information;
An indexed data generation method comprising: executing an output file including the index information and the form output format data.
A first generation unit that generates form output format data from form data including a form having a plurality of columns;
Index information for words, characters, or numerical values, which identifies the attributes of each of the plurality of columns and the positional relationship in the form data among the data corresponding to each of the plurality of columns included in the form output format data A second generator for generating the index information including possible position information;
An indexed data generation system comprising: an output unit that outputs an output file including the index information and the form output format data.
On the computer,
Accepts a search target string that is a combination of words, letters, and numbers,
Index information for words, characters, or numerical values, corresponding to each of the plurality of columns included in the form output form data generated from the form data including the attributes of the plurality of columns and the form having the plurality of columns. The word, the character, and the numerical value included in the character string to be searched are referred to as the form output format data with reference to the index information that includes position information that can specify the positional relationship in the form data between the data. And a process for determining whether the positional relationship matches the character string to be searched.
Computer
Accepts a search target string that is a combination of words, letters, and numbers,
Index information for words, characters, or numerical values, corresponding to each attribute of each of a plurality of columns and each of a plurality of columns included in the form output format data generated from the form data including the form having the plurality of columns. With reference to the index information including position information that can identify the positional relationship in the form data between the data, the word, the character, and the numerical value included in the character string to be searched are included in the form output format data. A search method comprising: executing a process of determining whether or not the positional relationship matches the character string to be searched.
A reception unit that accepts a search target character string combining words, characters, and numerical values,
Index information for words, characters, or numerical values, corresponding to each attribute of each of a plurality of columns and each of a plurality of columns included in the form output format data generated from the form data including the form having the plurality of columns. With reference to the index information including position information that can identify the positional relationship in the form data between the data, the word, the character, and the numerical value included in the character string to be searched are included in the form output format data. And a determination unit that determines whether the positional relationship matches the character string to be searched.