JP2019003501A - Information processor, information processing method, and program - Google Patents

Abstract

To provide a storage technology enabling high-speed search of a document including whose data type is not determined and which includes a field name and a value corresponding to the field name.SOLUTION: An information processor includes: an acquisition part acquiring a document whose data type is not determined and which includes a field name and a value corresponding to the field name; a holding part holding field name specification information for specifying the field name described in the document in description order of the field name in the document; a generation part generating description order and type relationship information associating the description order with type information of the document specified in accordance with the description order; and a storage control part storing, into a storage region, type-value relationship information associating a value corresponding to the field name corresponding to the field name specification information held in the description order.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  A database (DB) exists as a mechanism for efficiently using data handled on a computer or an information communication network, and a database management system (DBMS) exists as a system for managing the DB.

  The DB includes a relational DB (RDB) and a management system (RDBMS) that can store data in a two-dimensional table format and combine the tables as necessary.

  However, in recent years, DBMSs other than RDBMSs, so-called NoSQL (Not Only SQL), are often used depending on the usage environment and usage conditions due to an increase in the amount of data handled, response speed, data diversification, and the like. It is coming. There is a database called a document DB as one of NoSQL databases.

  The document DB stores data described in a description format such as JSON (Java Script Object Notation) or XML (Extensible Markup Language). As an example of the document DB, there is a document DB such as MongoDB or MarkLogic.

  As a technique for storing a document (structured document) that can identify individual logical structural elements constituting a document of a predetermined format such as an XML format and for searching the stored document, for example, there are the following techniques.

  As a first technique, there is a structured document retrieval system that retrieves a structured document in which one document has a plurality of logical structures (for example, Patent Document 1). In the first technique, document registration is performed by assigning a predetermined index group identifier to character string data that is highly likely to be collectively referred to during search. Then, a document search is performed using index data having the same index group identifier. As a result, it is possible to realize a structure designation search that designates only a target logical structure as a target at high speed.

  As a second technique, there is a technique for indexing an XML document (for example, Patent Document 2).

  As a third technique, there is a search processing technique for a structured document in binary format. In the third technique, the search expression conversion unit converts each of the nodes constituting the search expression for the structured document into a corresponding index using the vocabulary list, thereby converting the search expression. The document analysis unit specifies an index corresponding to each node constituting the structured document using the vocabulary list. The search expression evaluation unit corresponds to a part of the structured document corresponding to the converted search expression to each index described in the converted search expression and each node specified by the document analysis unit. Search using the index. As a result, a higher-speed search process for a binary structured document can be realized.

JP 2000-3366 A Special table 2007-533008 gazette JP 2010-250449 A

  In the RDBMS, the table structure (column name or data type) may be referred to as “schema”. In RDBMS, a schema is defined before data is inserted (stored) in a table.

  On the other hand, in the document DB, document data in the JSON format, XML format, or the like can be inserted into the DB without requiring prior schema definition before the data is inserted into the DB. For example, in MongoDB, data in the JSON format (referred to as “document” in MongoDB) such as the following program 1 is inserted into the collection.

For example, the following program 1 includes four documents. Here, in the following program, one document is shown in a range surrounded by a pair of “{” and “}”. In these four programs, the format of the other fields varies from document to document, except that the "name" field is common. However, the first and fourth documents have the same format.
Program 1
{
"name": "Horny",
"date": "1992-02-13",
"gender": "m",
"weight": 600
},
{
"name": ”SO43659”,
"date": ”2011-05-31”,
"account": “AWS29825”,
"price": 59.99,
"tags": “Sales”
},
{
"name": "sue",
"age": 26,
"status": "A",
"groups": "news"
},
{
"name": "Aurora",
"date": "1991-01-24",
"gender": "f",
"weight": 450
},
On the other hand, in MongoDB, a search is performed by specifying a field at the time of search. For example, “name field is“ Merry ”and date field is a document from 2015 to 2016”, but MongoDB parses each document and confirms what fields are in it. Search for. Therefore, the processing speed of MongoDB is low compared to the RDBMS in which each column is stored with a predetermined schema from the beginning.

  In one aspect, the present invention provides a storage technique that enables a high-speed search for a document having an undefined data type and including a field name and a value corresponding to the field name.

  In one aspect, an information processing apparatus disclosed in the present application includes an acquisition unit, a holding unit, a generation unit, and a storage control unit. The obtaining unit obtains a document whose data type is not determined and includes a field name and a value corresponding to the field name, and the holding unit obtains field name identification information for identifying the field name described in the document, Are stored in the order of field name description. The generation unit generates description order type relationship information that associates the description order with the document type information specified according to the description order. The storage control unit stores type-value relationship information in which the type information is associated with the value corresponding to the field name corresponding to the field name specifying information held in the description order in the storage area.

  In one aspect of the information processing method disclosed in the present application, a computer performs the following processing. That is, the computer obtains a document whose data type is not determined and includes a field name and a value corresponding to the field name. The computer holds field name specifying information for specifying field names described in the document in the order in which the field names are described in the document. The computer generates description order type relationship information that associates the description order with the document type information specified in accordance with the description order. The computer stores type-value relationship information in which the type information and the value corresponding to the field name corresponding to the field name specifying information held in the description order are related to each other in the storage area.

  In one aspect of the program disclosed in the present application, the computer performs the following processing. That is, the computer obtains a document whose data type is not determined and includes a field name and a value corresponding to the field name. The computer holds field name specifying information for specifying field names described in the document in the order in which the field names are described in the document. The computer generates description order type relationship information that associates the description order with the document type information specified in accordance with the description order. The computer stores type-value relationship information in which the type information and the value corresponding to the field name corresponding to the field name specifying information held in the description order are related to each other in the storage area.

  According to the present invention, in one aspect, a data type is not determined, and a document including a field name and a value corresponding to the field name can be stored so as to be searched at high speed.

It is a figure for demonstrating the storing method of simple JSON data. It is a figure for demonstrating the storage method of JSON data by a BSON format. 3A is a diagram for explaining a method of reading data stored by the method of FIG. 1, and FIG. 3B is a diagram for explaining a method of reading data stored by the method of FIG. FIG. An example of the information processing apparatus in this embodiment is shown. The block diagram of the information processing apparatus in this embodiment is shown. An example of the program P used in the present embodiment is shown. An example of the document stored in the store area | region in this embodiment is shown. An example of the field ID management tree in this embodiment is shown. An example of the field name management table in this embodiment is shown. An example of the type ID management tree in this embodiment is shown. An example of the type management table in this embodiment is shown. It is a figure for demonstrating the production | generation of the field ID arrangement | sequence which stored field ID from JSON in the order of description in JSON in this embodiment. It is a flowchart (the 1) of the insertion of the document in the collection based on JSON in this embodiment. It is a flowchart (the 2) of the insertion of the document in the collection based on JSON in this embodiment. It is the data used in FIG. 13A and FIG. 13B. It is a flowchart of the search process of the document in this embodiment. It is a figure explaining the acquisition method of the designated field value at the time of using this embodiment. An example of the hardware constitutions of the computer which executes the program in this embodiment is shown.

In order to improve the search performance, the structure is analyzed at the time of document insertion, and the same field may be collected or an index may be set. In this case, the search is speeded up, but the insertion speed is reduced.
(I) For example, the simplest JSON storage method is a method of storing JSON as it is as a character string. However, extra blanks are deleted from the character string constituting JSON. In MongoDB, if the program A1 is stored as it is, the data in the storage and memory is saved in the format of FIG.

Program A1
{
"name:" Horny ",
"date": "1992-02-13",
"gender": "m",
"weight": 600
},

  In this way, the insertion of the JSON format document is added after the previous JSON. Since input data is only described in JSON, it is fast.

In the search, a specified field name is designated for the stored document, and the field value is extracted. In this case, the data stored in JSON are read sequentially from the top. For example, when extracting a field value of a specific field name ““ gender ””, JSON data is sequentially read from the top as shown in FIG. Therefore, the search takes time and the search speed is low.
(Ii) Next, a case where data is stored in the BSON format will be described. The MongoDB storage format is described in the BSON (binary type JSON) format. For example, when the program A1 is stored in the BSON format, it is saved in the format of FIG.

  FIG. 2 is a diagram for explaining a method of storing JSON data in the BSON format. As indicated by reference numeral 1, the byte length of the entire BSON is stored as an integer value in the first 4 bytes of the BSON. As indicated by reference numeral 9, the end of BSON ends with a NULL termination character (expressed as “¥ 0”).

  Each field in the BSON includes a field name (symbol 3) and a field name data type (symbol 2), a field value (symbol 5), and a field value data length (symbol 4).

  If the data type of the field name is a character string, “2” is set (reference numeral 2). If the data type of the field name is a 4-byte integer, “16” is set (reference 2), and the field name (reference 3) is set to a field name with NULL at the end.

  The field value differs depending on the designated data type (reference numeral 2). As indicated by reference numeral 2 ', if the data type = character string (2), the length of the character string is first stored as a 4-byte integer in the field value, followed by a NULL termination character string. The stored data length is included in the data length of the field value. As indicated by reference numeral 2 ″, if the data type = integer type (16), a 4-byte integer is stored as it is. The data length of the field value is the stored data length.

  As described above, in the BSON storage method, since the JSON is analyzed and built into the BSON as shown in FIG. 2, it takes more time than storing the JSON as it is, and the processing speed is reduced.

  In the search, a specified field name is designated for the stored document, and the field value is extracted. In the case of the BSON format, since the fields are divided in advance, it is possible to search by skipping the field unit. For example, as shown in FIG. 3B, when a field value of a specific field name ““ gender ”” is extracted, the search is performed by skipping the reading in field units. Therefore, the search time is shortened compared to the JSON format, but it is still necessary to read the field names sequentially from the top.

  As described above, MongoDB has a structured data format that parses JSON called BSON, but field names are stored as character strings, and matching is necessary by character comparison. MessagePack is the same as MongoDB.

  In Oracle's JSON, field names are matched with field IDs, but the type structure of the entire JSON is not identified.

  Therefore, in the present embodiment, a storage technique is provided that enables high-speed search for a document whose data type is not determined and that includes a field name and a value corresponding to the field name.

  FIG. 4 shows an example of an information processing apparatus in the present embodiment. The information processing apparatus 11 includes an acquisition unit 12, a holding unit 13, a generation unit 14, and a storage control unit 15.

  The acquisition unit 12 acquires a document whose data type is not determined and includes a field name and a value corresponding to the field name. As an example of the acquisition unit 2, processing of an acquisition unit 23 and S <b> 1 described later can be given.

  The holding unit 13 holds field name specifying information for specifying field names described in the document in the order of description of the field names in the document. As an example of the holding unit 13, processing of a storage search control unit 24 and S <b> 9 described later is given.

  The generation unit 14 generates description order type relationship information that associates the description order with the document type information specified according to the description order. As an example of the generation unit 14, processing of a storage search control unit 24 and S <b> 10 to S <b> 13 described later can be given. As an example of the description order type relation information, a type management table 32 to be described later is cited.

  The storage control unit 15 stores, in the storage area, type-value relation information that associates type information with values corresponding to field names corresponding to field name specifying information held in the order of description. As an example of the storage control unit 15, processing of a storage search control unit 24, which will be described later, and S <b> 14 to S <b> 15 is given. An example of the type-value relationship information is a document stored in a store area 28 described later.

  The holding unit 13 acquires field name specifying information corresponding to the field name from the field name based on the first tree structure information holding the field name using the first tree structure. The holding unit 13 holds the field name specifying information in the order of description of field names in the document.

  The storage control unit 15 uses the second tree structure to store the field name specifying information in the description order, and based on the second tree structure information, specifies the document type specified according to the description order from the description order of the field name specifying information. Get information. The storage control unit 15 stores the type-value relationship information in which the description order is associated with the type information in the storage area.

  The information processing apparatus 11 further includes a search unit 16. When the acquisition unit 12 acquires the search condition, the search unit 16 associates the description order type relationship information and the type-value relationship information based on the type information. The search unit 16 performs a search based on whether or not the value corresponding to the field name included in the search condition satisfies the condition from the related description order type relationship information and the type-value relationship information. As an example of the search unit 16, processing of a storage search control unit 24 and S <b> 23 to S <b> 30 which will be described later is given.

  With this configuration, a document whose data type is not determined and which includes a field name and a value corresponding to the field name can be stored so as to be searched at high speed. In other words, by separating the component elements (meta information) that make up the document and the values corresponding to each component from the schemeless document, and managing and storing the meta information in a format that allows efficient search, Search speed can be improved.

Details of this embodiment will be described below.
Here, the schema definition is described. The “Schema” of the RDBMS indicates a column format definition with a table definition command. In the case of PostgreSQL, which is one of the RDBMSs, the table definition is performed by a CREATE TABLE command such as program 2, which is a “schema”.

(Program 2)
CREATE TABLE tablename (
name TEXT,
date DATE,
gender CHAR (1),
weight INTGER
):

  The “schema” of the JSON format document indicates a structure in which only the field names are left. If there is a JSON document of the program 3, the program 4 becomes the “schema” of the document.

(Program 3)
{
"name": "Horny",
"date": "1992-02-13",
"gender": "m",
"weight": 600
},

(Program 4)
{
"name" :,
"date" :,
"gender" :,
"weight":
},

  In the present embodiment, the schema is conscious of the order of field names, and even if the same field name is included, the schema is different if the order is different. Therefore, program 4 and program 5 have different schemas.

(Program 5)
{
"date" :,
"name" :,
"gender" :,
"weight":
},

  However, there may be variations in which the program 4 and the program 5 are regarded as the same schema within the scope of the present embodiment.

The present embodiment provides a technique that enables high-speed search without sacrificing the document insertion speed. In the present embodiment, the following is realized.
-The JSON meta information is analyzed when a JSON format document is inserted into the document DB, and a unique ID is assigned to the JSON structure. Use this to speed up searches.
-Optimization of meta-information when inserting a JSON-formatted document into the document DB minimizes a decrease in insertion speed.
-Reduce the amount of data to be stored by removing meta information (fields, etc.) from JSON. This reduces the amount of input and output to the storage and speeds up overall performance.

  FIG. 5 is a block diagram of the information processing apparatus according to this embodiment. The information processing device 21 is, for example, a computer. The information processing apparatus 21 includes a control unit 22 and a storage unit 27.

  The control unit 22 includes an acquisition unit 23, a storage search control unit 24, and an output unit 25. The acquisition unit 23 acquires command and JSON format document information from the request source. The request source includes, for example, an external terminal or an internal program.

  The storage search control unit 24 stores, in the storage unit 27, a JSON-formatted document (hereinafter referred to as “document”) received in a format to be described later in accordance with the acquired command. Further, the storage search control unit 24 reads a document from the storage unit 27 in a format to be described later in accordance with the acquired command. The output unit 25 outputs the document read from the storage unit 27 to the request source in accordance with the acquired command.

  Here, in the present embodiment, each collection has the following components. A unique number is assigned to the field name that appears for each collection. The unique number is a field ID (Field-ID).

  The data structure of JSON is considered to be an array of field IDs contained therein. A unique number is assigned to the “array of field IDs” in the order of appearance. The unique number is defined as a type ID (Type-ID).

  The storage unit 27 includes a store area 28, a field ID management tree 29, a field name management table 30, a type ID management tree 31, and a type management table 32.

  The store area 28 is a main part of the document, that is, an area for mainly storing field values.

  The field ID management tree 29 has a tree data structure in which a field ID can be searched from a field name. As an example, the field ID management tree 29 is assumed to be configured with a data structure called a trie tree or a prefix tree, but is not limited to this.

  The field name management table 30 has a data structure that forms a pair with the field ID management tree 29 and is used when a field name is acquired from a field ID. As an example, it is assumed that the field name management table 30 is configured by an array or a B-tree structure, but the field name management table 30 is not limited to this.

  The type ID management tree 31 has a data structure for searching for a type ID from an array of field IDs appearing in JSON. The type ID management tree 31 manages the JSON schema.

  The type management table 32 is an area for managing information for each type corresponding to the JSON schema.

  FIG. 6 shows an example of the program P used in this embodiment. Since the program P is the same as the program 1 described above, its description is omitted. In FIG. 6, the program P includes four programs, which are referred to as a program P1, a program P2, a program P3, and a program P4.

  FIG. 7 shows an example of a document stored in the store area 28 in the present embodiment. In the store area 28, field values included in the program P of FIG. 6 are stored as type IDs.

  FIG. 8 shows an example of a field ID management tree in the present embodiment. In FIG. 8, the tree structure of the field ID management tree 29 is described assuming a prefix tree.

  FIG. 9 shows an example of a field name management table in the present embodiment. The field name management table 30 includes items of “field ID” and “field name”. “Field ID” is identification information for identifying a field name. The “field name” stores the name of the field to be managed.

  FIG. 10 shows an example of a type ID management tree in the present embodiment. In FIG. 10, the tree structure of the type ID management tree 31 is described assuming a prefix tree.

  FIG. 11 shows an example of a type management table in the present embodiment. The type management table 32 includes items of “type ID” 32-1, “number of fields” 32-2, and “field” 32-3. The item “number of fields” 32-2 stores the number of fields constituting the document specified by the type ID. In the item “field” 32-3, field IDs of fields included in the document corresponding to the type ID are stored in the arrangement order 32-4 and the ascending order 32-5 according to the order.

  FIG. 12 is a diagram for explaining generation of a field ID array storing field IDs from JSON in the order of description in JSON in the present embodiment. FIG. 12 will be described later.

  Next, a process for inserting a document into the collection will be described with reference to FIGS. For example, it is assumed that the acquisition unit 23 acquires a document in JSON format (FIG. 6) together with a document insertion instruction to the collection.

  A plurality of fields are included in JSON (FIG. 6). The storage search control unit 24 extracts each field from the JSON (FIG. 6), searches the field ID management tree 29, and acquires a field ID corresponding to each field name.

  If the field name existing in the JSON does not exist in the field ID management tree 29, the storage search control unit 24 newly registers the field name in the field ID management tree 29. Here, the storage search control unit 24 assigns a new field ID to the registered field name. At the same time, the storage search control unit 24 registers a new field name / field ID pair in the field ID management tree 29.

  Up to this point, the field ID of each field of JSON can be acquired. For example, in the case of the JSON shown in FIG. 12, the field ID of “name” is “1”, the field ID of “date” is “7”, the field ID of “gender” is “5”, and the field ID of “weight” is “ 4 ". When these field IDs are arranged in the order of appearance, an array of [1, 7, 5, 4] is obtained. This is referred to as a field ID array.

  Next, the storage search control unit 24 searches the type ID management tree 31 (FIG. 10) using the obtained field ID array, and acquires the type ID indicating the JSON structure. As a result of the search, if the obtained field ID array does not hit in the type ID management tree 31, the obtained field ID array can be said to be an unregistered JSON structure. Therefore, the storage search control unit 24 newly assigns a type ID to the obtained field ID array and registers it in the type ID management tree 31 and the type management table 32.

  The storage search control unit 24 stores the field IDs in the “arrangement order” of the item “field” of the type management table 32 in the order of appearance of the fields in the document. In addition to this, the storage search control unit 24 sorts the field IDs in the normal order, and stores them in the type management table 32 in pairs with the appearance order.

  When the type ID of the JSON to be inserted is obtained, the storage search control unit 24 stores the field values in the store area 28 together with the type IDs in the order of appearance as shown in FIG.

  13A and 13B are flowcharts for inserting a document into a collection based on JSON in the present embodiment. FIG. 14 shows data used in FIGS. 13A and 13B.

  The acquisition unit 23 acquires a command and JSON from the request source (S1). The request source includes, for example, an external terminal or an internal program. JSON has the contents shown in FIG.

  The storage search control unit 24 determines the acquired command. If it is determined that the acquired command is a command for inserting a document into the collection, the storage search control unit 24 performs JSON lexical analysis (S2). In the JSON lexical analysis, the storage search control unit 24 reads the JSON in order from the head, and divides the JSON character string into fields (name / value pairs). As a lexical analysis result, the storage search control unit 24 obtains the contents of FIG. In the lexical analysis result, a set of field name (Name) and value (Value) is stored in the reading order.

  The storage search control unit 24 acquires one field from the top of the lexical analysis result, and designates this as F (S3). The storage search control unit 24 searches for “Name” (field name) of F from the field ID management tree 29 (S4).

  If the name of F does not exist in the field ID management tree 29 (“NO” in S5), the storage search control unit 24 assigns a new field ID to the name of F (S6). The storage search control unit 24 registers the name of F and the assigned field ID in the field ID management tree 29 and the field name management table 30 (S7). The field ID management tree 29 is constructed with a data structure such as a trie tree or a prefix tree, for example. After the process of S7, the process returns to S4.

  When the field name of F exists in the field ID management tree 29 (“YES” in S5), the storage search control unit 24 acquires the field ID of the searched field name from the field ID management tree 29. The storage search control unit 24 assigns the acquired field ID as I to the lexical analysis result as shown in FIG. 14C (S8).

  The storage search control unit 24 obtains one line at a time from the top of the lexical analysis result, and repeats the loop processing of S3 to S8 for the number of lines included in the lexical analysis result.

  The storage search control unit 24 creates an array from I of each field, and designates the created array as J (S9). For example, in the case of FIG. 14C, I is “1, 7, 5, 4” from the top. In this case, as shown in FIG. 14D, J is represented by [1, 7, 5, 4].

  The storage search control unit 24 searches J from the type ID management tree 31 (S10). When J does not exist in the type ID management tree 31 (“NO” in S11), the storage search control unit 24 newly assigns a type ID to J (S12).

  The storage search control unit 24 registers J and the assigned type ID in the type ID management tree 31 and the type management table 32 (S13). The type ID management tree 31 is constructed with a data structure such as a prefix tree, for example. After the process of S13, the process returns to S9.

  When J exists in the type ID management tree 31 (“YES” in S10), the storage search control unit 24 acquires the type ID of the searched J from the type ID management tree 31 (S14).

  The storage search control unit 24 registers JSON in the store area 28 (S15). Here, the storage search control unit 24 sequentially stores the acquired type ID and the field value (Value) corresponding to each I constituting J in the store area 28.

Next, a search process for documents included in the collection will be described.
FIG. 15 is a flowchart of document search processing in the present embodiment. As an example, consider extracting a document that meets a condition from a collection. For example, it is assumed that a search is performed under the condition “extract documents with“ age ”of 20 or more”.

  The acquisition unit 23 acquires a command and JSON from the request source (S21). The request source includes, for example, an external terminal or an internal program. In the JSON, a target field name (hereinafter referred to as “target field name”) is described.

  The storage search control unit 24 determines the acquired command. When it is determined that the acquired command is a command for searching the target field name from the field ID management tree 29 (field name management table 30), the storage search control unit 24 uses the field ID management tree 29 (field name management table 30). The target field name is searched from (S22).

  In the case of the above search conditions, the storage search control unit 24 searches the field ID management tree 29 (FIG. 8) (field name management table 30) for the field name “age” and acquires the field ID corresponding to the field name. (In this case, the field ID of “age” is “3”).

  If the target field name does not exist in the field ID management tree 29 (field name management table 30) (“NO” in S22), there is no document to be extracted in the field ID management tree 29 (field name management table 30). Perform the process. That is, the storage search control unit 24 notifies the output unit 25 that the target field name does not exist in the field ID management tree 29 (field name management table 30) as a search result. The output unit 25 outputs the search result to the request source (S31).

  When the target field name exists in the field ID management tree 29 (field name management table 30) (“YES” in S22), the storage search control unit 24 selects the target from the field ID management tree 29 (field name management table 30). The field ID of the field name is acquired and set as F (S23).

  The storage search control unit 24 acquires one line (one document) from the store area 28 and sets it as the document I (S24).

  The storage search control unit 24 extracts the type ID from the document I and sets it as S (S25). The storage search control unit 24 extracts a line (document) whose type ID matches S from the type management table 32, and sets this as a document T (S26).

  In each row (each document) of the type management table 32, fields included in the row (document) are arranged in order of field ID. The storage search control unit 24 searches whether or not F is included in the field ID in the document T (S27). The search can be a binary search or a SIMD (Single Instruction Multiple Data) instruction.

  When F is not included in the field ID in T (“NO” in S27), the storage search control unit 24 performs the first process (S24) of the loop process. That is, it can be seen that the field (document T) does not include the field “age”. In this case, the storage search control unit 24 returns to the process of S24 to take out the next document.

  When F is included in the field ID in T (“YES” in S27), the storage search control unit 24 extracts F in I (S28). In this case, in the above example, the storage search control unit 24 extracts the value of the field “age” from the store area 28.

  The storage search control unit 24 determines whether the search condition is satisfied (S29). In the above example, the storage search control unit 24 compares the value of the field “age” extracted from the store area 28 with the search condition (“age” is 20 or more ”).

  If the search condition is not satisfied (“NO” in S29), that is, if the value of the field extracted from the store area 28 does not satisfy the search condition, the storage search control unit 24 performs a loop process to extract the next document. The top process (S24) is performed.

  If the search condition is satisfied (“YES” in S29), that is, if the value of the field extracted from the store area 28 satisfies the search condition, the storage search control unit 24 extracts a document that satisfies the search condition (S30). Save it in the work area. Specifically, the storage search control unit 24 takes out the arrangement of field IDs in the arrangement order in the row of the type management table 32 acquired in S26, searches the field name management table 30, and sets the field name as a work area. Output. Similarly, the storage search control unit 24 outputs the field value from the store area 28 to the work area. When all the fields have been output, the storage / retrieval control unit 24 returns to the process of S24 to take out the next document.

  The storage search control unit 24 obtains one line at a time in order from the top of the store area, and repeats the loop processing of S24 to S30 for the number of lines included in the lexical analysis result.

  The storage search control unit 24 transmits to the output unit 25 the documents stored in the work area in S30 as search results. The output unit 25 outputs the search result to the request source (S31).

  For example, the program P1 in FIG. 6 is taken as an example. That is, the storage search control unit 24 takes out the document on the first line from the store area 28 and acquires type ID = 1 from the document. The storage search control unit 24 acquires a row corresponding to the acquired type ID = 1 from the type management table 32.

  Here, the field ID order registered in the row acquired from the type management table 32 is [1, 4, 5, 7]. Therefore, in S27, it is understood that 3 is not included in the field ID of “age”. In this case, the process proceeds to “NO” in S27, and loop processing is executed for the next document.

  Next, for example, the program P3 in FIG. 6 is taken as an example. In this case, the storage search control unit 24 takes out the document on the third line from the store area 28 and acquires type ID = 3 from the document. The storage search control unit 24 acquires a row corresponding to the acquired type ID = 3 from the type management table 32.

  Here, the field ID order registered in the row acquired from the type management table 32 is [1, 3, 9, 10]. In S27, it can be seen that 3 is included in the field ID of “age”, so the storage search control unit 24 performs the processes of S28 and S29.

  Here, the arrangement order of the field IDs registered in the row acquired from the type management table 32 is [1, 3, 10, 9]. The storage search control unit 24 searches the field name management table 30 based on the field ID arrangement order [1, 3, 10, 9], and the corresponding field names are “name”, “age”, “status”. "," Groups ". The storage search control unit 24 outputs it together with the value of the field extracted from the store area 28.

  FIG. 16 is a diagram for explaining a method for acquiring a designated field value when this embodiment is used. For example, it is assumed that the field “gender” is designated in the search. At this time, as shown in FIG. 8, the field ID of the field “gender” is “5”. In the case of a document of type ID = 1, referring to the type management table 32, the field ID “5” is stored in the third column. Based on this, the value “m” corresponding to the field ID = “5” is extracted from the store area 28 for the document of type ID = 1. Thereafter, this value is compared with the search condition.

  As described above, since the JSON schema is managed by the type ID, after searching the field ID from the field name, it can be searched only by comparing numerical values.

  In this way, by analyzing the JSON, it is possible to separate the field name excluding the value and information (meta information) regarding the structure of the document indicating which field name is included from the field value. The meta information is managed and stored using a tree structure such as a trie tree, a prefix tree, and a B-tree so that a field to be searched can be efficiently searched.

  Therefore, by using this embodiment, it is possible to analyze a schemeless document and manage meta information by ID, and manage it as a field value to store a schemeless document so that it can be searched at high speed. .

  FIG. 17 shows an example of the hardware configuration of a computer that executes a program according to this embodiment. The information processing apparatus is, for example, a computer 41.

  The computer 41 includes a central processing unit (CPU) 42, a memory 43, a hard disk drive (HDD) 44, a network interface card (NIC) 45, an input interface 46 and a video interface 47. Hereinafter, the interface is referred to as “I / F”.

  The CPU 42, memory 43, HDD 44, NIC 45, input I / F 46 and video I / F 47 are connected by a bus 48. An input device 49 such as a keyboard or a mouse is connected to the input I / F 46. A display 50 is connected to the video I / F 47.

  The CPU 42 is an example of a processor, and is a central processing unit that controls the overall operation of the computer 41. The memory 43 functions as a working area.

  The HDD 44 is a mass storage device that stores an operating system (OS) and a program according to the present embodiment. The HDD 44 includes a store area 28 and stores a field ID management tree 29, a field name management table 30, a type ID management tree 31, and a type management table 32. The HDD 44 is an example of a large-capacity storage device, and is not limited thereto. For example, the HDD 44 may be an SSD (Solid State Drive).

  The NIC 45 is an interface for connecting to a communication network such as the Internet or a local area network (LAN) by wire or wireless.

  The input I / F 46 is an interface that transmits a command input from the input device 18 to the CPU 42. The video I / F 47 is an interface that outputs an image to the display 50.

  The CPU 42 reads and executes a program according to the present embodiment, that is, a program for storing and searching a document with respect to the document DB stored in the HDD 44. Thereby, the CPU 42 functions as the acquisition unit 23, the storage search control unit 24, and the output unit 25.

  The program for realizing the processing described in the above embodiment may be stored in the storage device HDD, for example, via the communication network and the NIC 45 from the program provider side. Moreover, the program which implement | achieves the process demonstrated by the said embodiment may be stored in the portable storage medium marketed and distribute | circulated. In this case, the portable storage medium may be set in an external or built-in reader, and the program may be read out and executed by the CPU 42. As a portable storage medium, various types of storage media such as a CD-ROM, a DVD-ROM, a flexible disk, an optical disk, a magneto-optical disk, an IC card, and a USB memory device can be used. A program stored in such a storage medium is read by a reader.

  According to the present embodiment, the field name is mapped to a field ID that is an integer value, and a search is performed from a field ID array that has already been arranged in the field ID order, so that the search process can be speeded up.

  In addition, according to the present embodiment, it is possible to increase the speed as much as possible with a pattern in which a new JSON schema does not appear. For example, if the JSON to be inserted is parsed only once from the beginning to the end, data to be stored in the store area can be created. Thereby, the overhead of insertion processing is reduced.

  The present invention is not limited to the above-described embodiment, and various configurations or embodiments can be taken without departing from the gist of the present invention.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
An acquisition unit that acquires a document whose data type is not determined and includes a field name and a value corresponding to the field name;
A holding unit for holding field name specifying information for specifying a field name described in the document in the order of description of the field name in the document;
A generating unit that generates description order type relationship information that associates the description order with the type information of the document specified according to the description order;
A storage control unit for storing, in a storage area, type-value relationship information that associates the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
An information processing apparatus comprising:
(Appendix 2)
The holding unit obtains the field name specifying information corresponding to the field name from the field name based on the first tree structure information holding the field name using the first tree structure, and the field name Holding specific information in the order of description of the field names in the document;
The storage control unit specifies the field name specifying information according to the description order from the description order of the field name specifying information based on the second tree structure information in which the field name specifying information is held in the description order using the second tree structure. The information processing apparatus according to appendix 1, wherein the type information of the document to be acquired is acquired, and type-value relationship information in which the description order and the type information are related is stored in a storage area.
(Appendix 3)
The information processing apparatus further includes:
When acquiring the search condition by the acquisition unit, the description order type relationship information and the type-value that relate and relate the description order type relationship information and the type-value relationship information based on the type information. The information processing apparatus according to appendix 1 or 2, further comprising: a search unit that performs a search based on whether or not a value corresponding to a field name included in the search condition from the relationship information satisfies the condition.
(Appendix 4)
Computer
Retrieve a document with an undefined data type and a field name and a value corresponding to the field name,
Holding field name specifying information for specifying field names described in the document in the order of description of the field names in the document;
Generating description order type relationship information that associates the description order with the type information of the document specified according to the description order;
Storing in the storage area type-value relationship information associating the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
An information processing method characterized by the above.
(Appendix 5)
In the holding, the field name specifying information corresponding to the field name is acquired from the field name based on the first tree structure information holding the field name using the first tree structure, and the field name specifying Holding information in the order of description of the field names in the document;
In the storage, based on the second tree structure information in which the field name specifying information is held in the description order using the second tree structure, the field name specifying information is specified according to the description order from the description order. The information processing method according to appendix 4, wherein the type information of the document is acquired, and type-value relationship information in which the description order is associated with the type information is stored in a storage area.
(Appendix 6)
The information processing method further includes:
When acquiring the search condition, based on the type information, the description order type relation information and the type-value relation information are related, and the related description order type relation information and the type-value relation information are related. The information processing method according to appendix 4 or 5, wherein a search is performed based on whether or not a value corresponding to a field name included in the search condition satisfies the condition.
(Appendix 7)
On the computer,
Retrieve a document with an undefined data type and a field name and a value corresponding to the field name,
Holding field name specifying information for specifying field names described in the document in the order of description of the field names in the document;
Generating description order type relationship information that associates the description order with the type information of the document specified according to the description order;
Storing in the storage area type-value relationship information associating the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
A program that executes processing.
(Appendix 8)
In the holding, the field name specifying information corresponding to the field name is acquired from the field name based on the first tree structure information holding the field name using the first tree structure, and the field name specifying Holding information in the order of description of the field names in the document;
In the storage, based on the second tree structure information in which the field name specifying information is held in the description order using the second tree structure, the field name specifying information is specified according to the description order from the description order. The program according to appendix 7, wherein the type information of the document is acquired, and type-value relationship information that associates the description order with the type information is stored in a storage area.
(Appendix 9)
The program further includes:
When acquiring the search condition, based on the type information, the description order type relation information and the type-value relation information are related, and the related description order type relation information and the type-value relation information are related. The program according to appendix 7 or 8, wherein a search is performed based on whether or not a value corresponding to a field name included in the search condition satisfies the condition.

DESCRIPTION OF SYMBOLS 11 Information processing apparatus 12 Acquisition part 13 Holding part 14 Generation part 15 Storage control part 16 Search part 21 Information processing apparatus 22 Control part 23 Acquisition part 24 Storage search control part 25 Output part 27 Storage part 28 Store area 29 Field ID management tree 30 Field name management table 31 Type ID management tree 32 Type management table

Claims (5)

An acquisition unit that acquires a document whose data type is not determined and includes a field name and a value corresponding to the field name;
A holding unit for holding field name specifying information for specifying a field name described in the document in the order of description of the field name in the document;
A generating unit that generates description order type relationship information that associates the description order with the type information of the document specified according to the description order;
A storage control unit for storing, in a storage area, type-value relationship information that associates the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
An information processing apparatus comprising: The holding unit obtains the field name specifying information corresponding to the field name from the field name based on the first tree structure information holding the field name using the first tree structure, and the field name Holding specific information in the order of description of the field names in the document;
The storage control unit specifies the field name specifying information according to the description order from the description order of the field name specifying information based on the second tree structure information in which the field name specifying information is held in the description order using the second tree structure. 2. The information processing apparatus according to claim 1, wherein the type information of the document to be acquired is acquired, and type-value relationship information in which the description order is associated with the type information is stored in a storage area. The information processing apparatus further includes:
When acquiring the search condition by the acquisition unit, the description order type relationship information and the type-value that relate and relate the description order type relationship information and the type-value relationship information based on the type information. The information processing apparatus according to claim 1, further comprising: a search unit that performs a search based on whether a value corresponding to a field name included in the search condition satisfies the condition from relationship information. . Computer
Retrieve a document with an undefined data type and a field name and a value corresponding to the field name,
Holding field name specifying information for specifying field names described in the document in the order of description of the field names in the document;
Generating description order type relationship information that associates the description order with the type information of the document specified according to the description order;
Storing in the storage area type-value relationship information associating the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
An information processing method characterized by the above. On the computer,
Retrieve a document with an undefined data type and a field name and a value corresponding to the field name,
Holding field name specifying information for specifying field names described in the document in the order of description of the field names in the document;
Generating description order type relationship information that associates the description order with the type information of the document specified according to the description order;
Storing in the storage area type-value relationship information associating the type information with the value corresponding to the field name corresponding to the field name specifying information held in the description order;
A control program that executes processing.