WO2019181594A1 - パラメータ設定装置、演算装置、それらの方法、プログラム、および記録媒体 - Google Patents
パラメータ設定装置、演算装置、それらの方法、プログラム、および記録媒体 Download PDFInfo
- Publication number
- WO2019181594A1 WO2019181594A1 PCT/JP2019/009597 JP2019009597W WO2019181594A1 WO 2019181594 A1 WO2019181594 A1 WO 2019181594A1 JP 2019009597 W JP2019009597 W JP 2019009597W WO 2019181594 A1 WO2019181594 A1 WO 2019181594A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- character string
- unit
- size
- calculation
- thread
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0816—Key establishment, i.e. cryptographic processes or cryptographic protocols whereby a shared secret becomes available to two or more parties, for subsequent use
- H04L9/085—Secret sharing or secret splitting, e.g. threshold schemes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/131—Fragmentation of text files, e.g. creating reusable text-blocks; Linking to fragments, e.g. using XInclude; Namespaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/42—Data-driven translation
- G06F40/44—Statistical methods, e.g. probability models
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09C—CIPHERING OR DECIPHERING APPARATUS FOR CRYPTOGRAPHIC OR OTHER PURPOSES INVOLVING THE NEED FOR SECRECY
- G09C1/00—Apparatus or methods whereby a given sequence of signs, e.g. an intelligible text, is transformed into an unintelligible sequence of signs by transposing the signs or groups of signs or by replacing them by others according to a predetermined system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/36—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols with means for detecting characters not meant for transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/34—Encoding or coding, e.g. Huffman coding or error correction
Definitions
- the present invention relates to a calculation technique for a character string of a text file.
- the length of each cell of such a text file is arbitrary, and the text file often does not include information indicating the position and length of each cell. Therefore, in order to specify the position and length of each cell, the character string of the input text file must be read in order from the beginning. Therefore, considering the position and length of each record and each cell of the text file, it is not easy to specify the number of records handled in one unit process and the parallel number in the calculation process, and perform an efficient calculation process. .
- the present invention has been made in view of such a point.
- a text file including one or more records, each record including one or more cells of arbitrary length, and each cell including an arbitrary number of characters.
- An object is to efficiently perform arithmetic processing.
- a parameter setting device for arithmetic processing on a character string of a text file includes W records, each of which includes G cells having an arbitrary length, and each of the cells includes an arbitrary number of characters.
- W and G are integers of 1 or more, and G cells correspond to attribute information.
- C is the cache memory size, and M is the main memory size.
- the parameter setting device includes a maximum size setting unit, a minimum size setting unit, an encode size setting unit, a calculation size setting unit, a reference size setting unit, a processing unit calculation unit, and a parallel number calculation unit.
- the maximum size setting unit inputs the attribute information and sets the maximum value S csv of the size of the character string for one record of the text file.
- the minimum size setting unit sets the minimum value s csv of the character string size for one record with the attribute information as an input.
- the encoding size setting unit sets a maximum value Senc of the total size of encoding information obtained by encoding a character string for one record into a predetermined finite set.
- the calculation size setting unit sets the maximum value S ss of the total size of calculation values obtained by performing a specific calculation on the encoding information for one record. However, the encoding and the calculation are processes executed for each processing unit character string that is a character string of r records of the text file.
- the reference size setting unit sets a total size S ref of reference information indicating the position and length of each cell for one record.
- the processing unit calculation unit obtains a function value of C / (S csv + S enc + S ref ) as the number of records r.
- the parallel number calculation unit obtains a function value of f 0 / I ⁇ r ⁇ S csv as the parallel number n p in the arithmetic processing.
- I is the maximum value of the number of encoding and calculation iterations executed for each processing unit character string.
- each record including one or more cells having an arbitrary length, and each cell including an arbitrary number of characters.
- FIG. 1 is a block diagram illustrating a calculation system according to an embodiment.
- FIG. 2 is a block diagram illustrating a functional configuration of the parameter setting device according to the embodiment.
- FIG. 3 is a block diagram illustrating a functional configuration of the server device according to the embodiment.
- FIG. 4 is a block diagram illustrating a functional configuration of the processing unit according to the embodiment.
- FIG. 5 is a flowchart for illustrating the parameter setting process of the embodiment.
- FIG. 6 is a flowchart for illustrating the arithmetic processing according to the embodiment.
- FIG. 7 is a flowchart for illustrating the processing of the thread i according to the embodiment.
- FIG. 8 is a conceptual diagram for illustrating processing of each thread according to the embodiment.
- FIG. 1 is a block diagram illustrating a calculation system according to an embodiment.
- FIG. 2 is a block diagram illustrating a functional configuration of the parameter setting device according to the embodiment.
- FIG. 3 is a block diagram illustrating a functional configuration
- FIG. 9 is a conceptual diagram for illustrating a text file according to the embodiment.
- FIG. 10 is a conceptual diagram for illustrating a text file according to the embodiment.
- FIG. 11 is a conceptual diagram for illustrating a text file of the embodiment.
- FIG. 12 is a conceptual diagram for illustrating a text file according to the embodiment.
- FIG. 13 is a flowchart for illustrating the processing of the thread i according to the embodiment.
- FIG. 14 is a conceptual diagram for illustrating processing of each thread according to the embodiment.
- FIG. 15 is a conceptual diagram for illustrating the processing of the thread i according to the embodiment.
- FIG. 16 is a conceptual diagram for illustrating the processing of the thread i according to the embodiment.
- FIG. 17 is a conceptual diagram for illustrating the processing of the thread i according to the embodiment.
- FIG. 18 is a conceptual diagram for illustrating the processing of the thread i according to the embodiment.
- ⁇ Text file> In each embodiment, arithmetic processing is performed on a character string of a text file.
- This text file contains W records, each of which contains G cells of arbitrary length, and each of the cells contains an arbitrary number of characters. However, the length of each cell has an upper limit according to the attribute of each cell.
- W and G are integers of 1 or more. For example, at least one of W and G is an integer of 2 or more. W may be an integer of 2 or more, G may be an integer of 2 or more, and both W and G may be integers of 2 or more.
- W is an integer of 2 or more
- a line break exists between records adjacent to each other, and a plurality of records are separated from each other by the line break.
- G is an integer of 2 or more
- information for specifying a break between cells exists between adjacent cells.
- a delimiter or a line break exists between cells adjacent to each other, and a plurality of cells are delimited by the delimiter or line break.
- An example of a delimiter is a comma “,”.
- a tab or a line break may exist between cells adjacent to each other, and a half-width space or a line break may exist between cells adjacent to each other.
- the number G of cells included in each record is the same.
- G cells of each record correspond to attribute information (also called “schema”).
- the attribute information indicates what kind of attribute information each cell has, and information for specifying or estimating at least the maximum value and minimum value of the size (data amount) of the character string represented in each cell. Is included.
- the attribute information includes information indicating what kind of finite set the cell represents.
- the attribute information may indicate “a cell represents a mod (modulo p) modulo p (p is a positive integer)” or “a predetermined number of cells (for example, 10).
- a cell is an integer of a predetermined integer type (for example, a signed 32-bit integer),” or “a character string represented by an element of an extension field GF (2 8 )”. It may be a character string representing “)”.
- Each of the G pieces of attribute information may correspond to each of the G cells of each record on a one-to-one basis (that is, one piece of attribute information may represent an attribute of one cell).
- One piece of attribute information may correspond to a plurality of (for example, G) cells of each record (that is, one piece of attribute information may represent the attributes of a plurality of cells). In the former case, the attributes of a plurality of cells belonging to one record may be different from each other or the same.
- the attribute information may represent the type of information represented by the cell.
- the attribute information may be included in the text file (for example, the header of the text file may be attribute information) or may not be included. Examples of text files are CSV (Comma-Separated Values) files, TSV (tab-separated values) files, SSV (space-separated values), and the like. These are collectively called CSV (character-separated values) files and DSV (delimiter-separated values) files.
- the parameter setting device sets and outputs a parameter for “arithmetic processing” for the character string of the text file.
- This “arithmetic processing” may be anything. Examples of “arithmetic processing” include secret sharing processing, secret calculation processing (see, for example, Non-Patent Documents 1 and 2), encryption processing, signature generation processing, and the like.
- the parameters set by the parameter setting device are the number of records handled in one unit process and the parallel number in the arithmetic process.
- the parameter setting device further sets a file buffer size of data read collectively from the text file.
- the cache memory size (cache memory storage capacity) of the cache memory of the arithmetic device that performs arithmetic processing on the character string of the text file is denoted as C
- the main memory size (main memory storage capacity) of the main memory is denoted by M. Is written.
- the parameter setting device includes a maximum size setting unit, a minimum size setting unit, an encode size setting unit, a calculation size setting unit, a reference size setting unit, a processing unit calculation unit, and a parallel number calculation unit.
- the parameter setting device further includes a buffer size calculation unit.
- the maximum size setting unit receives the attribute information and sets and outputs a maximum value S csv of the size of the character string for one record of the text file.
- the maximum value S csv is a record size obtained by estimating the size of the character string of each record of the text file. That is, the maximum value S csv is the sum of the maximum value (or estimated value) of the size of each cell represented by the attribute information for one record.
- the attribute information includes information for specifying or estimating the maximum value of the size of the character string represented by each cell, and the maximum size setting unit sets the maximum value S csv using this information. For example, when the attribute information indicates that the cell is a character string representing a signed 32-bit integer, the maximum size of the cell is 11 bytes (1 byte for code + 10 for an integer of 11 digits). Byte).
- the minimum size setting unit inputs attribute information and sets and outputs a minimum value s csv of the size of a character string for one record.
- the minimum value s csv is a record size obtained by estimating the size of the character string of each record in the text file. That is, the minimum value s csv is the sum of the minimum value (or its estimated value) of the size of each cell represented by the attribute information for one record.
- the attribute information includes information for specifying or estimating the minimum value of the size of the character string represented by each cell, and the minimum size setting unit sets the minimum value s csv using this information. For example, when the attribute information indicates that the cell is a character string representing a signed 32-bit integer, the minimum value of the size of the cell is 1 byte.
- the encoding size setting unit sets and outputs a maximum value Senc of the total size of encoding information obtained by encoding (converting) a character string for one record based on a predetermined finite set.
- the maximum value Senc is a record size that is estimated by enlarging the total size of encoding information for one record.
- Examples of the “predetermined finite set” to which the encoding information belongs are a finite set represented by a remainder (mod p) modulo p, a finite set of values represented by predetermined bits, and a predetermined integer of a predetermined number of bits. For example, a finite set represented by an integer of type. This “predetermined finite set” is determined in advance according to the content of “calculation” described below.
- the maximum value Senc is specified from, for example, a predetermined finite set to which the encoding information belongs and attribute information.
- the attribute information represents a "that represents a remainder cells modulo 2 61 (mod 2 61)", in the remainder of the string of this cell modulo 2 61 (mod 2 61)
- the maximum size of the encoding information corresponding to the cell is 8 bytes.
- the predetermined finite set to which the encoding information belongs is determined in advance, for example.
- the calculation size setting unit sets and outputs the maximum value S ss of the total size of calculation values obtained by performing a specific “calculation” on the encoding information for one record.
- the maximum value S ss is a record size obtained by estimating the total size of calculated values for one record to be larger. Examples of this “calculation” are secret sharing, secret calculation, encryption, signature generation, and the like. “Operation” may be performed using each cell as an operator, or may be performed using a plurality of cells as an operator.
- the maximum value S ss is specified from, for example, a predetermined finite set to which the encoding information belongs, the content of “calculation”, and attribute information.
- the attribute information represents a "that represents a remainder cells modulo 2 61 (mod 2 61)", in the remainder of the string of this cell modulo 2 61 (mod 2 61) Encoded in the encoded information to be expressed, and “calculation” secretly distributes the value of each cell to N (where N is a positive integer) parties using the Shamir secret sharing scheme (eg, Non-Patent Document 3).
- the maximum size of the operation value corresponding to the cell is 8N bytes.
- the reference size setting unit sets and outputs the total size S ref of the reference information indicating the position and length of each cell for one record in the text file.
- the “cell position” may be, for example, the position of the first character of the cell, the position of the last character of the cell, or the position of a character in another cell.
- the “information indicating the cell position” may be, for example, the number of characters from the first character of the character string of the text file to the “cell position” character, or may be a function value of this number of characters.
- the “information indicating the cell length” may be, for example, the number of characters in the cell or a function value of this number of characters.
- the reference information is specified from attribute information, for example.
- the data size required to represent the position and length of each cell is determined by the format for representing it. For example, when the position and length of each cell is represented by an unsigned 64-bit integer, the reference information for each cell is 16 bytes.
- the above-mentioned “encoding” and “calculation” are executed for each processing unit character string that is a character string for r records of the text file. Processing executed for each processing unit character string is referred to as “unit processing”.
- the processing unit calculating section represents the record number r to be processed in unit process once to obtain the "C / function value of (S csv + S enc + S ref) " (C / (S csv + S as a record number r enc + S get the function value of ref ) and output.
- the cache memory size C may be determined in advance or may be input.
- the maximum value S csv is obtained by the maximum size setting unit
- the maximum value S enc is obtained by the encode size setting unit
- the total size S ref is obtained by the reference size setting unit.
- the “function value of ⁇ ” may be ⁇ itself, or may be another value corresponding to ⁇ . Examples of “function value of ⁇ ” include a minimum integer equal to or larger than ⁇ , a maximum integer equal to or smaller than ⁇ , and an integer closest to ⁇ .
- r C / (S csv + S enc + S ref )
- r ROUNDUP (C / (S csv + S enc + S ref ))
- ROUNDUP ( ⁇ ) is a round-up function that rounds up ⁇ to an integer value
- ROUNDDOWN ( ⁇ ) is a round-down function that rounds down ⁇ to an integer value
- ROUND ( ⁇ ) is a rounded integer that rounds ⁇ to the function closest to ⁇ . It is.
- S csv + S enc + S ref is until the processing unit reads a character string for one record from a text file, encodes it into encoded information while referring to the reference information, and performs “calculation” such as secret sharing.
- Represents the memory size required for the above process hereinafter referred to as “a series of processes for one record”). If this memory size is equal to or smaller than the cache memory size, a series of processing for one record can be executed at high speed without reading data from the main memory in the middle.
- C / (S csv + S enc + S ref ) represents how many times the memory size (S csv + S enc + S ref ) necessary for “a series of processing for one record” can be secured in the cache memory.
- f 0 / I ⁇ r ⁇ S csv function values a give (function value of f 0 / I ⁇ r ⁇ S csv as a parallel number n p representing the number of parallel n p in the processing parallel number calculating section Output).
- I is the maximum number of repetitions of “encoding” and “calculation” executed for each processing unit character string.
- I is the number of repetitions of “encoding” and “calculation” executed for each processing unit character string.
- the main memory size M may be determined in advance or may be input.
- the minimum value s csv is set by the minimum size setting unit, the maximum value S ncc is set by the encode size setting unit, and the maximum value S ss is set by the calculation size setting unit.
- the total size S ref is set by the reference size setting unit, and the maximum value I of the number of repetitions is predetermined.
- the maximum value I of the number of repetitions is the total processing amount of the preprocessing (the number of operations) with respect to the total processing amount for performing “encoding” and “calculation” of the processing unit character string that is a character string for r records. (Total) is set to be a predetermined value or less. r may be obtained by the processing unit calculation unit, or may be obtained from C, S csv , S enc , S ref .
- X (s csv + S enc + max (S ref , S ss )) / s csv is a character string for one record in which a memory size necessary for a series of processing for one record is read from a text file Represents the maximum memory size.
- S csv ⁇ X represents the maximum value of the memory size necessary for a series of processing for one record
- I ⁇ r ⁇ S csv ⁇ X is a memory required for repeating the series of processing for r records I times. Represents the maximum size.
- the buffer size calculation unit obtains “function value of f 0 / n p ” representing the file buffer size f of data read collectively from the character string of the text file during the arithmetic processing (f 0 as the file buffer size f / N p function value is obtained) and output.
- f f 0 / n p
- f ROUNDUP (f 0 / n p )
- f ROUNDDOWN (f 0 / n p )
- It may be f ROUND (f 0 / n p ).
- n p may be obtained by the parallel number calculation unit, or may be obtained from f 0 , I, r, S csv . That is, as long as a function value of f 0 / n p can be obtained, n p obtained by the parallel number calculation unit does not necessarily have to be used to generate a function value of f 0 / n p .
- n p f 0 / I ⁇ r ⁇ S csv
- f 0 / n p I ⁇ r ⁇ S csv .
- file buffer size f of the character string read from the text file in order to repeat r record processing I times.
- a larger file buffer size f is closer to a sequential accelerator and is faster, but requires a larger storage capacity of the main memory.
- the file buffer size f obtained as described above realizes high-speed processing within the constraints of a predetermined main memory size M.
- the arithmetic device performs arithmetic processing on the character string of the input text file.
- the arithmetic unit uses the record number r and the parallel number np obtained by the parameter setting device in this arithmetic processing.
- the arithmetic device uses the file buffer size f obtained by the parameter setting device in this arithmetic processing.
- the arithmetic unit may use a predetermined value as the file buffer size f or may use another file buffer size f corresponding to the attribute information. Good.
- the text file may contain characters that are not subject to “calculation” such as secret sharing.
- a comma in a CSV file is a character representing a cell delimiter and is not a target of “calculation”.
- a character string of a cell may be surrounded by double quotations ““ ”“ ””, but double quotations are not subject to “calculation”.
- a character representing a line feed (for example, ⁇ n) or the like is not a target of “calculation” (these double quotations ““ ”and“ ”” are not included in the cell).
- Such characters that are not subject to “calculation” are called “special characters”.
- the division between cells cannot be determined unless the cells “123” and “456” are read in order from the top. For example, when only “456” in the latter half is read, it cannot be determined whether this is one cell representing “456” or a part of a cell representing a value including “456”.
- ⁇ is further added as an escape character when a character “ ⁇ n” representing a line feed is used as an object of “calculation”. For example, when a value “123 ⁇ n456” of a cell is a target of “calculation”, the cell may be expressed as “123 ⁇ n456”.
- the separation between cells cannot be determined unless the cells “123 ⁇ n456” are read in order from the top.
- the process of specifying the position and length of each cell in the text file cannot be performed in parallel, and this process must be performed in order from the beginning of the text file.
- An arithmetic device capable of handling such a text file has a main memory, a cache memory, and a plurality of processing units.
- Each processing unit includes a reading unit, a file reading lock releasing unit, a parsing unit, a buffer boundary lock releasing unit, an encoding unit, an arithmetic unit, and a parallelism lock releasing unit.
- the plurality of processing units are assigned to processing of any thread.
- the processing unit that performs processing of thread i performs the following processing. Note that i represents each thread, i ⁇ ⁇ 0,..., T ⁇ 1 ⁇ , T is a positive integer representing the number of threads corresponding to the text file size TS, and 1 ⁇ n p ⁇ T.
- TS TS 0 +... + TS T-1 or TS ⁇ TS 0 +... + TS T-1 is satisfied for the size TS i of the character string read from the text file in each thread i, or f ⁇ T ⁇ TS. Fulfill. It is assumed that the file read lock and buffer boundary lock of thread 0 and the parallelism lock of threads 0,..., N p ⁇ 1 are released in the initial state.
- the reading unit After the file reading lock and the parallelism lock of the thread i are released, the reading unit reads a character string S i that can be stored in the area of the file buffer size f from the character string of the text file and stores it in the main memory.
- the character string S 0 is “a character string having a data amount that can be stored in the area of the file buffer size f” with the first character of the text file as the head.
- the character string S i is “the amount of data that can be stored in the area of the file buffer size f starting from the character immediately after the last character of the character string S i-1 read by the thread i-1. Character string ".
- the “character string that can be stored in the area of the file buffer size f” may be, for example, the longest character string that can be stored in the area of the file buffer size f, or a size obtained by subtracting a constant from the file buffer size f. It may be the longest character string that can be stored in the area.
- the file reading lock releasing unit releases the file reading lock of the thread i + 1. Thereby, it is possible to prevent the accesses to the main memory among a plurality of threads from competing with each other. However, there is no thread corresponding to i + 1> T, and the file read lock of the thread that does not exist is not released.
- the character terminating in a character string S 0 is not the end of the cell, it can not identify the cell containing the thread 0 at the end of the character, not even be calculated the reference information.
- the character at the start of the character string S i may be the start of the cell or may not be the start of the cell, and the character at the end included in the character string S i may be the end of the cell. It may not be a termination.
- starting character of the string S i is not a beginning of a cell, can not identify the cell containing only string S i of the start of the string S i characters.
- the parsing unit uses a character string S i-1 which is not included in the cell specified by the thread i-1 and the character string S i , and uses the character string S i to include the start character of the character string S i. Is identified.
- the terminal character included in the character string S i is not the terminal of the cell, the thread i cannot specify the cell including the terminal character and cannot calculate the reference information.
- the terminal character included in the character string ST-1 at the end of the text data is the end of the cell.
- the parsing unit When the area for storing the reference information is insufficient in the main memory, the parsing unit collectively secures a buffer area in the main memory for storing the reference information for r records. Predetermined processing (overhead) is required to secure the buffer area. Rather than securing a buffer area for each record, by securing a buffer area for each r record corresponding to unit processing, a variable-length record can be processed while suppressing overhead.
- the buffer boundary unlocking unit releases the buffer boundary lock of the thread i + 1. There is no thread corresponding to i + 1> T, and the buffer boundary lock of the thread that does not exist is not released.
- the encoding unit After the buffer boundary lock of thread i + 1 is released, the encoding unit performs processing unit characters that are character strings corresponding to r records to be processed included in the combined character string CS i from the text data based on the information specified by the reference information.
- column PS i, select j, selected processing unit character string PS i, encoding information E i, j is a predetermined original finite set, the process for encoding the j, performed using a cache memory.
- the beginning of the processing unit character string PS i, j is the beginning of any record, and the end of the processing unit character string PS i, j is the end of any record.
- the combined character string CS 0 when i 0 is S 0
- the combined character string CS i when i ⁇ 1 is a character string S i combined immediately after the combined character string CS i ⁇ 1 .
- J is a positive integer
- j 0,..., J ⁇ 1.
- the encoding unit, the string S i from the processing unit character string PS i choose a j, or string S i and the string
- the processing unit character string PS i, j is selected from the character string obtained by combining S i-1 .
- the encoding unit selects the processing unit character string PS i, j from the character string obtained by combining the character string S i and the character string S i ′ . However, 0 ⁇ i ′ ⁇ i ⁇ 1.
- the encoding unit selects a processing unit character string PS 0,0 ,..., PS 0, J ⁇ 1 from the beginning of the character string S 0 .
- the encoding unit selects a processing unit character string PS i, 0 ,..., PS i, J-1 starting from a character not selected as a processing unit character string in the character string S i-1 To do. For J ⁇ 2, PS i, j followed by PS i, immediately after the j-1.
- the encoding unit performs encoding for each processing unit character string PS i, j .
- data is arranged in a code-oriented manner (data is arranged in the order of record 1, record 2,..., Record W), and all records correspond to the same “attribute set”.
- the processing speed is higher when the same kind of data is continuously processed than when different types of data are continuously processed. Therefore, it is desirable that the encoding unit continuously encodes r cells for r records corresponding to the same attribute information in the processing unit character string PS i, j .
- Encoding section processing unit character string based on the information specified by the reference information PS i, select j, encoding selected processing unit character string PS i, a j encoding information E i, the j. It is possible to perform high-speed processing by storing reference information for r records, processing unit character strings PS i, j , and encoding information E i, j required in the course of the processing in a cache memory and performing calculations. As described above, such a process can be performed by using the record number r corresponding to C / (S csv + S enc + S ref ).
- the calculation unit performs a process of performing a specific “calculation” on the encoding information E i, j to obtain a calculation value SS i, j and storing it in the main memory using the cache memory. It is desirable that the calculation unit continuously performs “calculation” corresponding to r cells for r records corresponding to the same attribute information. Even during this process, it is possible to perform high-speed processing by storing the reference information for r records, the processing unit character string PS i, j , and the encoding information E i, j in the cache memory and performing calculations.
- the parallelism lock release unit releases the parallelism lock of the thread i + n p .
- the parallelism lock of the thread that does not exist is not released.
- the processing unit that has been processing the thread i is released, and the processing unit can perform processing of another thread.
- ⁇ Calculation device (high-speed version)>
- the process of specifying the position and length of each cell in the text file can be performed in parallel. Higher-speed calculation is possible. For example, in the case of a text file that does not use escape characters, such parallel processing becomes possible.
- An arithmetic device capable of handling such a text file has a main memory, a cache memory, and a plurality of processing units. Each processing unit includes a reading unit, a file reading unlocking unit, a parsing unit, a cell specifying unit, a buffer boundary unlocking unit, an encoding unit, a computing unit, and a parallelism unlocking unit.
- the plurality of processing units are assigned to processing of any thread.
- the processing unit that performs processing of thread i performs the following processing. Assume that the file read lock of thread 0 and the buffer boundary lock and the parallelism lock of threads 0,..., N p ⁇ 1 are released in the initial state.
- the reading unit After the file reading lock and the parallelism lock of the thread i are released, the reading unit reads a character string S i that can be stored in the area of the file buffer size f from the character string of the text file and stores it in the main memory.
- the details are the same as those of the arithmetic unit (general version).
- the file reading lock releasing unit releases the file reading lock of the thread i + 1.
- the details are the same as those of the arithmetic unit (general version).
- the parsing unit calculates reference information representing the position and length of each cell included in the character string S i and stores it in the main memory.
- the parsing unit can start this process before the buffer boundary lock of thread i is released. In other words, the parsing unit calculates the position of each cell included in the character string S i and the calculation of the reference information indicating the position and length of each cell included in the character string S i ⁇ 1 when i ⁇ 1. Calculation of reference information representing the length can be started. For example, the parsing unit specifies the cell of the character string S i , calculates the reference information of each specified cell, and stores it in the main memory.
- the parsing unit specifies a cell based on information (for example, a delimiter or a line feed) located at the cell boundary, calculates reference information for each specified cell, and stores it in the main memory.
- information for example, a delimiter or a line feed
- the terminal character included in the character string S 0 may be the end of the cell or may not be the end of the cell. If the character terminating in a character string S 0 is not the end of the cell, parsing unit can not identify the cell containing the end character, not even be calculated the reference information.
- i ⁇ 1 the character at the start of the character string S i may be the start of the cell or may not be the start of the cell, and the character at the end included in the character string S i may be the end of the cell.
- the parsing unit cannot identify the cell that includes the character at the start of the character string S i and cannot calculate the reference information.
- the parsing unit cannot identify the cell including the terminal character and cannot calculate the reference information.
- the terminal character included in the terminal character string ST-1 is the end of the cell.
- the parsing unit collectively secures a buffer area in the main memory for storing reference information for r records when there is not enough area for storing the reference information in the main memory. Thus, variable length records can be processed while suppressing overhead.
- the character string S i is a character string that immediately follows the character string S i-1 .
- the cell specifying unit uses the reference information, the character string S i-1, and the character string S i after the buffer boundary lock of the thread i is released, and is included in the character string S i-1.
- Information A i corresponding to the position of the cell immediately following the last cell is obtained and stored in the main memory.
- the information A i includes, for example, information indicating a record to which a cell immediately following the last cell included in the character string S i-1 belongs and information indicating an attribute corresponding to the cell (for example, what number from the beginning of the record) Or information indicating the position and length of the cell immediately following the last cell included in the character string S i-1 .
- the end of the character string S i-1 is the end of the cell
- the first cell of the character string S i is “the cell immediately following the last cell included in the character string S i-1 ”.
- the storage of the information A i in the main memory may be omitted.
- the character string S i uses the character string S i-1 and the character string S i, and “the last cell included in the character string S i-1 ” is used.
- information corresponding to the position of the cell for which the parse portion could not be specified is obtained.
- the record to which each cell of the text file belongs and the attribute corresponding to the cell can be specified by the reference information and the information Ai .
- the cell specifying unit does nothing.
- the buffer boundary lock release unit releases the buffer boundary lock of the thread i + 1 after the information A i is obtained.
- the details are the same as those of the arithmetic unit (general version). Thereafter, the processing unit that has been processing the thread i is released, and the processing unit can perform processing of another thread.
- the encoding unit selects a processing unit character string PS i, j that is a character string for r records to be processed included in the combined character string CS i based on the reference information and information specified by the information A i .
- the processing unit character string PS i, j is encoded using the cache memory to encode the encoded information E i, j that is the element of a predetermined finite set.
- the details are the same as those of the arithmetic unit (general version) except that information Ai is used in addition to the reference information.
- the calculation unit performs a process of performing a specific “calculation” on the encoding information E i, j to obtain a calculation value SS i, j and storing it in the main memory using the cache memory.
- the details are the same as those of the arithmetic unit (general version).
- the parallelism lock release unit releases the parallelism lock of the thread i + n p .
- the details are the same as those of the arithmetic unit (general version).
- the parameter setting device sets the record number r, the parallel number n p , and the file buffer size f handled by one unit process, and the arithmetic device (general version) can use an escape character CSV ( An example of secret sharing (computation) of a Comma-Separated Values) file (text file) will be described. Below, it demonstrates centering on difference with the matter demonstrated so far, and description may be abbreviate
- the computing system 1 of this embodiment includes a parameter setting device 11, a computing device 12, and N server devices 13-1 to 13-N.
- N is a positive integer of 2 or more.
- Information can be transmitted from the parameter setting device 11 to the arithmetic device 12, and information can be transmitted from the arithmetic device 12 to the server devices 13-1 to 13-N.
- Information transmission may be performed via a network, may be performed using other communication means, or may be performed via a portable recording medium.
- the parameter setting device 11 includes an input unit 111a, an output unit 111b, a storage unit 112, a control unit 113, a maximum size setting unit 114a, a minimum size setting unit 114b, an encode size setting unit 114c, a calculation size. It has a setting unit 114d, a reference size setting unit 114e, a processing unit calculation unit 114f, a parallel number calculation unit 114g, and a buffer size calculation unit 114h.
- the parameter setting device 11 executes each process under the control of the control unit 113. Each value obtained by the parameter setting device 11 is stored in the storage unit 112, read out from the storage unit 112 as necessary, and used for other processing.
- the arithmetic device 12 includes an input unit 121a, an output unit 121b, an auxiliary storage unit 122, a main memory 123, a control unit 125, and processing units 126-1 to 126-Q.
- Q is an integer of 2 or more.
- the arithmetic device 12 executes each process under the control of the control unit 125.
- the parameter setting process of the parameter setting device 11 will be described with reference to FIG.
- the attribute information of the text data to be processed is input to the input unit 111a of the parameter setting device 11 (FIG. 2) and stored in the storage unit 112.
- the attribute information may be read from text data or may be given from other than text data (step S111a).
- the maximum size setting unit 114a receives the attribute information read from the storage unit 112 as input and sets and outputs the maximum value S csv of the character string size for one record of the text file (step S114a).
- the minimum size setting unit 114b receives the attribute information read from the storage unit 112 as an input, sets and outputs the minimum value s csv of the character string size for one record (step S114b).
- the encode size setting unit 114c receives the attribute information read from the storage unit 112 as an input, and based on the information indicating the “predetermined finite set” to which the encode information belongs, the character string for one record is generated based on the predetermined finite set.
- a maximum value Senc of the total size of encoding information obtained by encoding (conversion) is set and output.
- the “predetermined finite set” to which the encoding information of this embodiment belongs is a finite set to which secret sharing is performed, and is determined in advance (step S114c).
- the operation size setting unit 114d receives attribute information read from the storage unit 112 as an input, and is obtained by secret sharing (calculation) of encoded information for one record based on a predetermined finite set to which the encoded information belongs and a secret sharing scheme.
- the maximum value S ss of the total size of secret sharing values (calculated values) is set and output.
- the secret sharing scheme of this embodiment is predetermined (step S114d).
- the reference size setting unit 114e receives the attribute information read from the storage unit 112 as an input, sets and outputs the total size S ref of reference information indicating the position and length of each cell for one record in the text file ( Step S114e).
- the processing unit calculation unit 114f receives S csv, S enc, and S ref and inputs a function value r of C / (S csv + S enc + S ref ) (the number r of records processed in one unit processing, that is, a processing unit) The number of records r) contained in the character string is obtained and output.
- the cache memory size C may be determined in advance or may be input (step S114f).
- the parallel number calculation unit 114g receives S csv, s csv , S ref , S enc , I and r and inputs the function value n p (parallel number n p in the arithmetic processing) of f 0 / I ⁇ r ⁇ S csv. Output.
- f 0 is a function value of s csv ⁇ M / (s csv + S enc + max (S ref , S ss )).
- the main memory size M may be determined in advance or may be input (step S114g).
- the buffer size calculation unit 114h receives f 0 and n p as input, and obtains a function value f of f 0 / n p (file buffer size f of data read collectively from a character string of a text file during arithmetic processing). (Step S114h).
- the output unit 111b outputs r, n p , f obtained as described above (step S111b).
- FIG. 9 to FIG. 12 illustrate text data.
- the text data illustrated in FIG. 9 is a CSV file in which each cell is surrounded by double quotations. When double quotation ““ ”is used as a cell value,“ “” is added as an escape character before that.
- the cell cell (w, g) corresponds to the g-th attribute att (g) from the beginning of the record rec (w) (step S111ab).
- the file read lock and buffer boundary lock of thread 0 and the parallelism lock of threads 0,..., N p ⁇ 1 are released.
- the control unit 125 assigns an unused processing unit 126-q among the processing units 126-1 to 126-Q to the thread i, and a plurality of threads execute processing of each thread i in parallel as much as possible (step S126). ).
- Each secret sharing value obtained in this way is output from the output unit 121b, sent to each server device 13-1 to 13-N, and stored in each server device 13-1 to 13-N (step S111b). . Details of the processing of the thread i will be described below.
- the reading unit 1261-q of the processing unit 126-q that performs processing of the thread i determines whether both the file reading lock and the parallelism lock of the thread i are released.
- the file read lock and the parallelism lock of thread 0 are released in the initial state (steps S1261a-q).
- the determination in step S1261a-q is repeated.
- the reading unit 1261-q reads the file buffer size f from the main memory 123, and stores the area of the file buffer size f in the main memory 123. Secure. Furthermore, the reading unit 1261-q reads a character string S i that can be stored in the area of the file buffer size f from the character string of the text file stored in the auxiliary storage unit 122. In the example of FIG. 11, the following are read as a string S 0.
- the reading unit 1261-q stores the character string S i in the area of the file buffer size f secured in the main memory 123 (step S1261b-q in FIG. 7, R i in FIG. 8).
- the file reading lock releasing unit 1267-q releases the file reading lock of the thread i + 1 (step S1267-q in FIG. 7, UR i + 1 in FIG. 8).
- the parsing unit 1262-q determines whether or not the buffer boundary lock of the thread i has been released.
- the buffer boundary lock of thread 0 is released in the initial state (step S1262a-q). If the buffer boundary lock of thread i has not been released, the determination in step S1262a-q is repeated.
- the parsing unit 1262-q parses the character string S 0 to generate the cells “Ishida” “Taro” “1990/2/8” “100-0002” “sjeifdfgjrrf”. "45dkfjkejdf5""Ishida""Jiro”"1985/5/2""111-0112""25df4d4ed”"1s4dlccclseed""Ishida""Hanako" 2001/4/8 "" 111-2222 "” 5d4e4d4ffg “” skekdjjfaae "” Sato "” Taro "” 1992/7/11 “” 111-0345 “” dlekd4f3e "are specified, and their reference information is calculated.
- the parsing unit 1262-q reads r from the main memory 123 when there is not enough area for storing the reference information in the main memory 123, and sets a buffer area in the main memory 123 for storing reference information for r records. Secure together. Thereafter, the process proceeds to step S1268-q (step S1262c-q in FIG. 7, P i in FIG. 8).
- the parsing unit 1262-q sends the parsing result (information specifying each cell reference information and information not included in the cell) from the main memory 123 in the thread i-1. Read and specify a character not included in the cell specified by the thread i-1 in the character string S i-1 .
- the end of the character string S i-1 is the end of the cell, there is no character that is not included in the cell specified by the thread i-1 in the character string S i-1 (step S1262d-q).
- the parsing unit 1262-q reads the character string S i from the main memory 123, parses the character string obtained by combining the character string S i and the character not included in the cell specified by the thread i-1. Reference information representing the position and length of each cell included in the character string is calculated and stored in the main memory 123. Reference information when the end of the string S i-1 is the end of the cell, parsing unit 1262-q is to parse the string S i, representing the position and length of each cell in the string S i Is calculated and stored in the main memory 123. For example, in the case of the character strings S 0 and S 1 illustrated in FIG. 11 and FIG.
- the parsing unit 1262-q includes the character “that is not included in the cell specified by the thread i-1 in the character string S i ⁇ 1.
- Character string “4selddks” “k304kdkk400-03d” combining “4selddks“ ”and character string S i-1 “Sato”, “Jiro”, “1989/8/21”, “123-0434”, “dkesopd445e”, “4ssjdejdoseae3230dds” “Sato”, “Hanako”, “1995/2/3”, “145-0234”, “skdeofl4s3d3”, “skek94kdskd4dc” “Tanaka”, “Taro”, “1992/3/23”, “134-0134”, “dj394949495kf”, “47s52 ⁇ n5412485d” “Tanaka”, “Jiro”, “1979/4/21”, “11 Each cell contained in this string “4selddks“
- the parsing unit 1262-q reads r from the main memory 123 when there is not enough area for storing the reference information in the main memory 123, and sets a buffer area in the main memory 123 for storing reference information for r records. Secure together. Thereafter, the process proceeds to step S1268-q (step S1262e-q in FIG. 7, P i in FIG. 8).
- step S1268-q the buffer boundary lock releasing unit 1268-q releases the buffer boundary lock of the thread i + 1. However, there is no thread corresponding to i + 1> T, and the buffer boundary lock of the thread that does not exist is not released (step S1268-q in FIG. 7, UB i + 1 in FIG. 8).
- a character string PS i, j that is a character string for r records to be processed included in the combined character string CS i from the text data.
- the encoding unit 1265-q is using the processing unit character string PS i, j and the reference information of the cache memory 1260-q
- the processing unit character string PS i, encode information j is a predetermined original finite set E i, j
- the encoded information E i, j is stored in the cache memory 1260-q (step S1265-q in FIG. 7, E i in FIG. 8).
- the calculation unit 1266-q performs secret sharing of the encoded information E i, j read from the cache memory 1260-q to obtain a secret sharing value (calculation value) SS i, j and stores it in the main memory 123.
- the secret sharing values SS i, 0 ,..., SS i, J-1 May be overwritten in the area where the reference information for r records is stored (step S1266-q in FIG. 7, SS i in FIG. 8).
- the parallelism lock releasing unit 1269-q reads n p from the main memory 123 and releases the parallelism lock of the thread i + n p . However, there is no thread corresponding to i + n p > T, and the parallelism lock of the thread that does not exist is not released. Thereafter, the control unit 125 releases the allocation of the processing unit 126-q to the thread i. As a result, the processing unit 126-q can be assigned to another thread (step S1269-q in FIG. 7, UP i + np in FIG. 8).
- the parameter setting device sets the record number r, the parallel number n p , and the file buffer size f handled by one unit process, and the arithmetic device (high-speed version) is prohibited from using escape characters.
- the arithmetic device high-speed version
- the computing system 2 of this embodiment includes a parameter setting device 11, a computing device 22, and N server devices 13-1 to 13-N.
- Information can be transmitted from the parameter setting device 11 to the arithmetic device 22, and information can be transmitted from the arithmetic device 22 to the server devices 13-1 to 13-N.
- the parameter setting device 11 includes an input unit 111a, an output unit 111b, a storage unit 112, a control unit 113, a maximum size setting unit 114a, a minimum size setting unit 114b, an encode size setting unit 114c, a calculation size. It has a setting unit 114d, a reference size setting unit 114e, a processing unit calculation unit 114f, a parallel number calculation unit 114g, and a buffer size calculation unit 114h.
- the parameter setting device 11 executes each process under the control of the control unit 113. Each value obtained by the parameter setting device 11 is stored in the storage unit 112, read out from the storage unit 112 as necessary, and used for other processing.
- the arithmetic device 12 includes an input unit 121a, an output unit 121b, an auxiliary storage unit 122, a main memory 123, a control unit 125, and processing units 226-1 to 226-Q.
- Q is an integer of 2 or more.
- the arithmetic device 12 executes each process under the control of the control unit 125.
- FIG. 15 to FIG. 18 illustrate text data.
- the text data illustrated in FIG. 15 is a CSV file in which each cell is not surrounded by double quotations.
- each cell contains only characters that can be used to determine whether or not each cell represents a secret sharing (calculation) target.
- the cell cell (w, g) corresponds to the g-th attribute att (g) from the beginning of the record rec (w) (step S211ab).
- the file read lock of thread 0 and the parallelism lock of threads 0,..., N p ⁇ 1 are released.
- the control unit 125 assigns an unused processing unit 226-q among the processing units 226-1 to 226-Q to the thread i, and a plurality of threads execute processing of each thread i in parallel as much as possible (step S226). ).
- Each secret sharing value obtained in this way is output from the output unit 121b, sent to each server device 13-1 to 13-N, and stored in each server device 13-1 to 13-N (step S111b). . Details of the processing of the thread i will be described below.
- the reading unit 1261-q of the processing unit 226-q that performs processing of the thread i determines whether both the file reading lock and the parallelism lock of the thread i are released.
- the file read lock and the parallelism lock of thread 0 are released in the initial state (steps S1261a-q).
- the determination in step S1261a-q is repeated.
- the reading unit 1261-q reads the file buffer size f from the main memory 123, and stores the area of the file buffer size f in the main memory 123. Secure. Furthermore, the reading unit 1261-q reads a character string S i that can be stored in the area of the file buffer size f from the character string of the text file stored in the auxiliary storage unit 122. In the example of FIG. 17, the following are read as a string S 0.
- the file reading lock releasing unit 1267-q releases the file reading lock of the thread i + 1 (step S1267-q in FIG. 13, UR i + 1 in FIG. 14).
- the parsing unit 2262-q parses the character string S i read from the main memory 123, calculates reference information representing the position and length of each cell included in the character string S i , and stores the reference information in the main memory 123. For example, in the case of the character string S 0 illustrated in FIG. 17, the parsing unit 2262-q parses the character string S 0 to generate the cells “Ishida” “Taro” “1990/2/8” “100-0002” “Tokyo”.
- the parsing unit 2262-q parses the character string S 1 to generate the cells “2001/6/2”, “274-16”, “Enoshima, Fujisawa City, Kanagawa Prefecture ⁇ ”“ 03-9999-9999 ”“ Sato ”“ Hanako ”“ 2002/7/2 ”“ 352-725 ”“ Shinjuku Shinjuku-ku Tokyo ”“ 03-1111-9999 ”“ Tanaka ”“ Taro ” “2001/1/1”, “100-0002”, “Tokyo Chiyoda-ku 00” “03-1234-5678” “Tanaka” “Jiro” “2001/1/2” “251-0036” Calculate their reference information.
- the parsing unit 2262-q reads r from the main memory 123 when there is not enough area for storing reference information in the main memory 123, and sets a buffer area for storing reference information for r records to the main memory 123. 123 is secured together. The parsing unit 2262-q can start this process before the buffer boundary lock of the thread i is released.
- the parsing unit 2262-q calculates the reference information of each cell included in the character string S i before the calculation of the reference information of each cell included in the character string S i-1 when i ⁇ 1.
- the process can be started (step S2262-q in FIG. 13, P i in FIG. 14).
- the cell specifying unit 2264-q determines whether or not the buffer boundary lock of the thread i has been released.
- the buffer boundary lock of thread 0 is released in the initial state (step S2264a-q). If the buffer boundary lock of the thread i has not been released, the determination in step S2264-q is repeated.
- the cell specifying unit 2264-q uses the reference information, the character string S i-1, and the character string S i.
- Information A i corresponding to the position of the cell immediately following the last cell included in column S i-1 is obtained and stored in the main memory.
- the cell specifying unit 2264-q does nothing (step S2264b-q).
- the buffer boundary lock releasing unit 1268-q releases the buffer boundary lock of the thread i + 1. However, there is no thread corresponding to i + 1> T, and the buffer boundary lock of the thread that does not exist is not released (step S1268-q in FIG. 13, UB i + 1 in FIG. 14).
- this invention is not limited to the above-mentioned embodiment.
- the parameter setting device sets the record number r, the parallel number n p , and the file buffer size f handled by one unit process, but the parameter setting device sets the file buffer size f.
- the embodiment may not be set.
- a CSV (Comma-Separated Values) file is exemplified as a text file.
- the processing for other text files described above may be performed.
- the example in which secret sharing is performed as “calculation” has been described, but other computations may be performed as “calculation”.
- Each of the above devices is a general-purpose or dedicated computer including a processor (hardware processor) such as a CPU (central processing unit) and a memory such as a random-access memory (RAM) and a read-only memory (ROM). Is configured by executing a predetermined program.
- the computer may include a single processor and memory, or may include a plurality of processors and memory.
- This program may be installed in a computer, or may be recorded in a ROM or the like in advance.
- some or all of the processing units are configured using an electronic circuit that realizes a processing function without using a program, instead of an electronic circuit (circuitry) that realizes a functional configuration by reading a program like a CPU. May be.
- An electronic circuit constituting one device may include a plurality of CPUs.
- a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.
- This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.
- a computer that executes such a program first stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device.
- the computer reads a program stored in its own storage device, and executes a process according to the read program.
- the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer.
- the processing according to the received program may be executed sequentially.
- the above-described processing may be executed by a so-called ASP (Application Service Provider) type service that does not transfer a program from the server computer to the computer but implements a processing function only by the execution instruction and result acquisition. Good.
- ASP Application Service Provider
- the processing functions of this apparatus are not realized by executing a predetermined program on a computer, but at least a part of these processing functions may be realized by hardware.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
Description
[概要]
まず概要を説明する。
<テキストファイル>
各実施形態ではテキストファイルの文字列に対する演算処理を行う。このテキストファイルはW個のレコードを含み、レコードのそれぞれは任意長のG個のセルを含み、セルのそれぞれは任意個の文字を含む。ただし、各セルの長さには各セルの属性に応じた上限がある。WおよびGが1以上の整数である。例えば、WおよびGの少なくとも一方は2以上の整数である。Wが2以上の整数であってもよいし、Gが2以上の整数であってもよいし、WおよびGの両方が2以上の整数であってもよい。Wが2以上の整数である場合、互いに隣接するレコードの間にはレコードの区切りを特定するための情報が存在する。例えば、互いに隣接するレコードの間に改行が存在し、複数のレコードは改行によって互いに区切られている。また、Gが2以上の整数である場合、互いに隣接するセルの間にはセル間の区切りを特定するための情報が存在する。例えば、互いに隣接するセルの間に区切り文字または改行が存在し、複数のセルは区切り文字または改行によって互いに区切られている。区切り文字の例はカンマ「,」である。その他の例として、互いに隣接するセルの間にタブまたは改行が存在してもよいし、互いに隣接するセルの間に半角スペースまたは改行が存在してもよい。Wが2以上の整数である場合、各レコードに含まれるセルの個数Gは互いに同一である。各レコードのG個のセルは属性情報(「スキーマ」とも呼ぶ)に対応している。属性情報は各セルがどのような属性の情報であるかを表しており、少なくとも各セルで表される文字列のサイズ(データ量)の最大値と最小値とを特定または推定するための情報を含んでいる。例えば、属性情報はセルがどのような有限集合の元を表しているのか示す情報を含んでいる。例えば、属性情報は「セルがpを法とした剰余(mod p)を表していること(pは正整数)」を表していてもよいし、「セルが所定個(例えば10個)の所定の有限体(例えば拡大体GF(28))の要素で表現される文字列であること」を表していてもよいし、「セルが所定の整数型の整数(例えば、符号付き32ビット整数)を表す文字列であること」を表していてもよい。G個の属性情報のそれぞれが各レコードのG個のセルのそれぞれに一対一で対応していてもよいし(すなわち、1個の属性情報が1個のセルの属性を表していてもよい)、1個の属性情報が各レコードの複数個(例えばG個)のセルに対応していてもよい(すなわち、1個の属性情報が複数個のセルの属性を表していてもよい)。前者の場合、1つのレコードに属する複数のセルの属性が互いに異なっていてもよいし、互いに同一であってもよい。また、Wが2以上の整数である場合、すべてのレコードのG個のセルに対応する「G個の属性の組」は互いに同一である。すなわち、すべてのレコードが有するg番目(ただし、g=1,…,G)のセルの属性att(g)は互いに同一である。その他、属性情報がセルが表す情報の種別を表現していてもよい。また属性情報はテキストファイルに含まれていてもよいし(例えば、テキストファイルのヘッダが属性情報あってもよい)、含まれていなくてもよい。テキストファイルの例は、CSV(Comma-Separated Values)ファイル、TSV(tab-separated values)ファイル、SSV(space-separated values)などである。これらはCSV(character-separated values)ファイルやDSV(delimiter-separated values)ファイルとして総称される。
パラメータ設定装置は、テキストファイルの文字列に対する「演算処理」のためのパラメータを設定して出力する。この「演算処理」はどのようなものであってもよい。「演算処理」の例は秘密分散処理、秘密計算処理(例えば、非特許文献1,2等参照)、暗号化処理、署名生成処理などである。パラメータ設定装置が設定するパラメータは、1つの単位処理で扱われるレコード数および演算処理における並列数である。好ましくは、パラメータ設定装置は、さらにテキストファイルからまとめて読み込まれるデータのファイルバッファサイズも設定する。以下では、テキストファイルの文字列に対する演算処理を行う演算装置のキャッシュメモリのキャッシュメモリサイズ(キャッシュメモリの記憶容量)をCと表記し、メインメモリのメインメモリサイズ(メインメモリの記憶容量)をMと表記する。
演算装置は入力されたテキストファイルの文字列に対する演算処理を行う。演算装置はこの演算処理でパラメータ設定装置で得られたレコード数rおよび並列数npを用いる。パラメータ設定装置でさらにファイルバッファサイズfが得られる場合には、演算装置はこの演算処理でさらにパラメータ設定装置で得られたファイルバッファサイズfを用いる。パラメータ設定装置でファイルバッファサイズfが得られない場合、演算装置はファイルバッファサイズfとして予め定められた値を用いてもよいし、属性情報に対応するその他のファイルバッファサイズfが用いられてもよい。
テキストファイルの各セルが、単独で「演算」の対象を表すか否かを判別可能な文字のみを含む場合、テキストファイルの各セルの位置および長さを特定する処理を並列に行うことができ、さらに高速な演算が可能となる。例えば、エスケープ文字を使用していないテキストファイルの場合にはこのような並列処理が可能になる。このようなテキストファイルに対応可能な演算装置はメインメモリとキャッシュメモリと複数の処理部とを有する。各処理部は、読み込み部とファイル読み込みロック解除部とパース部とセル特定部とバッファ境界ロック解除部とエンコード部と演算部と並列性ロック解除部とを有する。これら複数の処理部は何れかのスレッドの処理に割り当てられる。スレッドiの処理を行う処理部は以下の処理を行う。初期状態でスレッド0のファイル読み込みロックならびにバッファ境界ロックおよびスレッド0,…,np-1の並列性ロックが解除されているものとする。
図面を用いて第1実施形態を説明する。第1実施形態では、パラメータ設定装置が1つの単位処理で扱われるレコード数r、並列数np、ファイルバッファサイズfを設定し、演算装置(一般版)がエスケープ文字の使用が可能なCSV(Comma-Separated Values)ファイル(テキストファイル)の秘密分散(演算)を行う例を説明する。以下では、これまで説明した事項との相違点を中心に説明し、既に説明した事項については説明を省略する場合がある。
図1に例示するように、本実施形態の演算システム1は、パラメータ設定装置11、演算装置12、およびN個のサーバ装置13-1~13-Nを有する。ただし、Nは2以上の正整数である。パラメータ設定装置11から演算装置12への情報の伝達が可能であり、演算装置12からサーバ装置13-1~13-Nへの情報の伝達が可能である。なお、情報の伝達はネットワークを介して行われてもよいし、その他の通信手段を用いて行われてもよいし、可搬型の記録媒体を介して行われてもよい。
図5を用いて、パラメータ設定装置11のパラメータ設定処理を説明する。
演算処理対象のテキストデータの属性情報がパラメータ設定装置11(図2)の入力部111aに入力され、記憶部112に格納される。属性情報はテキストデータから読み込まれたものであってもよいし、テキストデータ以外から与えられたものであってもよい(ステップS111a)。
図6から図12を用い、演算装置12の演算処理を説明する。
図6に例示するように、パラメータ設定装置11から出力されたr,np,fとが演算装置12(図3)の入力部121aに入力され、補助記憶部122に格納される(ステップS111aa)。また演算処理対象のテキストデータが入力部121aに入力され、補助記憶部122に格納される。図9から図12にテキストデータを例示する。図9に例示するテキストデータは、各セルがダブルクォーテーションで囲まれたCSVファイルである。セルの値としてダブルクォーテーション「“」を用いる場合にはその前にエスケープ文字として「“」が付加される。例えば、“4selddks““k304kdkk400-03d”は、「4selddks“k304kdkk400-03d」という秘密分散対象の値を表している。図10に例示するように、このテキストファイルはW個のレコードrec(1),…,rec(W)を含み、レコードrec(w)(ただし、w=1,…,W)のそれぞれは任意長のG個のセルcell(w,g)を含み(ただし、g=1,…,G)、セルcell(w,g)のそれぞれは任意個の文字を含む。セルcell(w,g)はレコードrec(w)の最初からg番目の属性att(g)に対応する(ステップS111ab)。
図7および図8に例示するように、スレッドiの処理を行う処理部126-qの読み込み部1261-qは、スレッドiのファイル読み込みロックおよび並列性ロックの両方が解除されたかを判定する。スレッド0のファイル読み込みロックおよび並列性ロックは初期状態で解除されている(ステップS1261a-q)。スレッドiのファイル読み込みロックおよび並列性ロックの両方が解除されていない場合にはステップS1261a-qの判定が繰り返される。
“石田”,“太郎”,“1990/2/8”,“100-0002”,“sjeifdfgjrrf”,“45dkfjkejdf5”
“石田”,“次郎”,“1985/5/2”,“111-0112”,“25df4d4ed”,“1s4dlccclseed”
“石田”,“花子”,“2001/4/8”,“111-2222”,“5d4e4d4ffg”,“skekdjjfaae”
“佐藤”,“太郎”,“1992/7/11”,“111-0345”,“dlekd4f3e”,“4selddks“
図12の例では、文字列S1として以下が読み込まれる。
“k304kdkk400-03d”
“佐藤”,“次郎”,“1989/8/21”,“123-0434”,“dkesopd445e”,“4ssjdejdoseae3230dds”
“佐藤”,“花子”,“1995/2/3”,“145-0234”,“skdeofl4s3d3”,“skek94kdskd4dc”
“田中”,“太郎”,“1992/3/23”,“134-0134”,“dj394949495kf”,“47s52\n5412485d”
“田中”,“次郎”,“1979/4/21”,“11
読み込み部1261-qは、メインメモリ123に確保したファイルバッファサイズfの領域に文字列Siを格納する(図7のステップS1261b-q、図8のRi)。
“4selddks““k304kdkk400-03d”
“佐藤”,“次郎”,“1989/8/21”,“123-0434”,“dkesopd445e”,“4ssjdejdoseae3230dds”
“佐藤”,“花子”,“1995/2/3”,“145-0234”,“skdeofl4s3d3”,“skek94kdskd4dc”
“田中”,“太郎”,“1992/3/23”,“134-0134”,“dj394949495kf”,“47s52\n5412485d”
“田中”,“次郎”,“1979/4/21”,“11
をパースし、この文字列に含まれる各セル「4selddks““k304kdkk400-03d」「佐藤」「次郎」「1989/8/21」「123-0434」「dkesopd445e」「4ssjdejdoseae3230dds」「佐藤」「花子」「1995/2/3」「145-0234」「skdeofl4s3d3」「skek94kdskd4dc」「田中」「太郎」「1992/3/23」「134-0134」「dj394949495kf」「47s52\n5412485d」「田中」「次郎」「1979/4/21」の位置および長さを表す参照情報を計算してメインメモリ123に格納する。最後の“11の終端はセルの終端ではないため、スレッド1では“11の参照情報は計算されない。パース部1262-qは、メインメモリ123に参照情報を格納する領域が足りなくなったときに、メインメモリ123からrを読み込み、rレコード分の参照情報を格納するためのバッファ領域をメインメモリ123にまとめて確保する。その後、処理がステップS1268-qに進む(図7のステップS1262e-q、図8のPi)。
第2実施形態では、パラメータ設定装置が1つの単位処理で扱われるレコード数r、並列数np、ファイルバッファサイズfを設定し、演算装置(高速版)がエスケープ文字の使用が禁止されたCSV(Comma-Separated Values)ファイル(テキストファイル)の秘密分散(演算)を行う例を説明する。
図1に例示するように、本実施形態の演算システム2は、パラメータ設定装置11、演算装置22、およびN個のサーバ装置13-1~13-Nを有する。パラメータ設定装置11から演算装置22への情報の伝達が可能であり、演算装置22からサーバ装置13-1~13-Nへの情報の伝達が可能である。
第1実施形態と同一である。
図6および図13から図18を用い、演算装置22の演算処理を説明する。
図6に例示するように、パラメータ設定装置11から出力されたr,np,fとが演算装置22(図3)の入力部121aに入力され、補助記憶部122に格納される(ステップS111aa)。また演算処理対象のテキストデータが入力部121aに入力され、補助記憶部122に格納される。図15から図18にテキストデータを例示する。図15に例示するテキストデータは、各セルがダブルクォーテーションで囲まれていないCSVファイルである。本実施形態のテキストデータでのエスケープ文字の使用は許可されておらず、各セルは単独で秘密分散(演算)の対象を表すか否かを判別可能な文字のみを含む。図16に例示するように、このテキストファイルはW個のレコードrec(1),…,rec(W)を含み、レコードrec(w)(ただし、w=1,…,W)のそれぞれは任意長のG個のセルcell(w,g)を含み(ただし、g=1,…,G)、セルcell(w,g)のそれぞれは任意個の文字を含む。セルcell(w,g)はレコードrec(w)の最初からg番目の属性att(g)に対応する(ステップS211ab)。
図13および図14に例示するように、スレッドiの処理を行う処理部226-qの読み込み部1261-qは、スレッドiのファイル読み込みロックおよび並列性ロックの両方が解除されたかを判定する。スレッド0のファイル読み込みロックおよび並列性ロックは初期状態で解除されている(ステップS1261a-q)。スレッドiのファイル読み込みロックおよび並列性ロックの両方が解除されていない場合にはステップS1261a-qの判定が繰り返される。
石田,太郎,1990/2/8,100-0002,東京都渋谷区〇〇〇,03-3234-5678
石田,次郎,2000/4/2,274-16,神奈川県藤沢市江の島〇〇〇,03-9999-9999
石田,花子,1985/6/2,352-725,東京都港区区〇〇〇,03-1111-9999
佐藤,太郎,2001/5/1,100-0002,東京都千代田区〇〇〇,03-3234-5678
佐藤,次
図18の例では、文字列S1として以下が読み込まれる。
郎,2001/6/2,274-16,神奈川県藤沢市江の島〇〇〇,03-9999-9999
佐藤,花子,2002/7/2,352-725,東京都新宿区新宿〇〇〇,03-1111-9999
田中,太郎,2001/1/1,100-0002,東京都千代田区〇〇〇,03-1234-5678
田中,次郎,2001/1/2,251-0036,神奈川県藤沢市江の島〇〇〇
読み込み部1261-qは、メインメモリ123に確保したファイルバッファサイズfの領域に文字列Siを格納する(図13のステップS1261b-q、図14のRi)。
11 パラメータ設定装置
12,22 演算装置
13-1~13-N サーバ装置
Claims (11)
- テキストファイルの文字列に対する演算処理のためのパラメータ設定装置であって、
前記テキストファイルはW個のレコードを含み、前記レコードのそれぞれは任意長のG個のセルを含み、前記セルのそれぞれは任意個の文字を含み、WおよびGが1以上の整数であり、前記G個のセルは属性情報に対応しており、
Cがキャッシュメモリサイズ、Mがメインメモリサイズであり、
前記属性情報を入力として前記テキストファイルの1レコード分の文字列のサイズの最大値Scsvを設定する最大サイズ設定部と、
前記属性情報を入力として前記1レコード分の文字列のサイズの最小値scsvを設定する最小サイズ設定部と、
前記1レコード分の文字列を所定の有限集合の元にエンコードして得られるエンコード情報の合計サイズの最大値Sencを設定するエンコードサイズ設定部と、
前記1レコード分の前記エンコード情報に特定の演算を行って得られる演算値の合計サイズの最大値Sssを設定する演算サイズ設定部と、
前記1レコード分の前記セルそれぞれの位置および長さを表す参照情報の合計サイズSrefを設定する参照サイズ設定部と、
前記エンコードおよび前記演算は前記テキストファイルのrレコード分の文字列である処理単位文字列ごとに実行される処理であり、レコード数rとしてC/(Scsv+Senc+Sref)の関数値を得る処理単位算出部と、
Iが前記処理単位文字列ごとに実行される前記エンコードおよび前記演算の繰り返し回数の最大値であり、Sref≧Sssのときmax(Sref,Sss)=Srefであり、Sref<Sssのときmax(Sref,Sss)=Sssであり、f0がscsv・M/(scsv+Senc+max(Sref,Sss))の関数値であり、前記演算処理における並列数npとしてf0/I・r・Scsvの関数値を得る並列数算出部と、
を有するパラメータ設定装置。 - 請求項1のパラメータ設定装置であって、
前記演算処理の際に前記テキストファイルの文字列からまとめて読み込まれるデータのファイルバッファサイズfとしてf0/npの関数値を得るバッファサイズ算出部をさらに有するパラメータ設定装置。 - 請求項1または2のパラメータ設定装置であって、
前記繰り返し回数の最大値Iは、前記処理単位文字列の前記エンコードおよび前記演算を行うための合計処理量に対する前処理の合計処理量の比率が所定値以下となるように定められている、パラメータ設定装置。 - テキストファイルの文字列に対する演算処理を行う演算装置であって、
前記テキストファイルはW個のレコードを含み、前記レコードのそれぞれは任意長のG個のセルを含み、前記セルのそれぞれは任意個の文字を含み、WおよびGが1以上の整数であり、fが定められたファイルバッファサイズであり、npが並列数であり、rがレコード数を表す正整数であり、iが各スレッドを表し、i∈{0,…,T-1}であり、Tが前記テキストファイルの文字列のサイズに対応するスレッド数を表す正整数であり、1≦np≦Tであり、初期状態でスレッド0のファイル読み込みロックおよびバッファ境界ロックならびにスレッド0,…,np-1の並列性ロックが解除されており、
メインメモリとキャッシュメモリと複数の処理部とを有し、
前記複数の処理部のうちスレッドiの処理を行う処理部は、
前記スレッドiのファイル読み込みロックおよび並列性ロックが解除された後、前記テキストファイルの文字列から前記ファイルバッファサイズfの領域に格納可能な文字列Siを読み込んで前記メインメモリに格納する読み込み部と、
前記文字列Siが前記メインメモリに格納された後にスレッドi+1のファイル読み込みロックを解除するファイル読み込みロック解除部と、
スレッドiのバッファ境界ロックが解除された後、前記文字列Siに含まれる各セルの位置および長さを表す参照情報を計算して前記メインメモリに格納するパース部と、
前記文字列Siに含まれる各セルの位置および長さを表す参照情報が計算された後にスレッドi+1のバッファ境界ロックを解除するバッファ境界ロック解除部と、
i=0の場合の結合文字列CS0がS0であり、i≧1の場合の結合文字列CSiが結合文字列CSi‐1の直後に前記文字列Siを結合したものであり、Jが正整数であり、j=0,…,J-1であり、前記参照情報によって特定される情報に基づいて前記結合文字列CSiに含まれる処理対象のrレコード分の文字列である処理単位文字列PSi,jを選択し、前記処理単位文字列PSi,jを所定の有限集合の元であるエンコード情報Ei,jにエンコードする処理を、前記キャッシュメモリを利用して行うエンコード部と、
前記エンコード情報Ei,jに特定の演算を行って演算値SSi,jを得て前記メインメモリに格納する処理を、前記キャッシュメモリを利用して行う演算部と、
前記演算値SSi,jが得られた後にスレッドi+npの並列性ロックを解除する並列性ロック解除部と、
を有する演算装置。 - テキストファイルの文字列に対する演算処理を行う演算装置であって、
前記テキストファイルはW個のレコードを含み、前記レコードのそれぞれは任意長のG個のセルを含み、前記セルのそれぞれは任意個の文字を含み、WおよびGが1以上の整数であり、fが定められたファイルバッファサイズであり、npが並列数であり、rがレコード数を表す正整数であり、iが各スレッドを表し、i∈{0,…,T-1}であり、Tが前記テキストファイルの文字列のサイズに対応するスレッド数を表す正整数であり、1≦np≦Tであり、初期状態でスレッド0のファイル読み込みロックおよびスレッド0,…,np-1の並列性ロックが解除されており、
メインメモリとキャッシュメモリと複数の処理部とを有し、
前記複数の処理部のうちスレッドiの処理を行う処理部は、
前記スレッドiのファイル読み込みロックおよび並列性ロックが解除された後、前記テキストファイルの文字列から前記ファイルバッファサイズfの領域に格納可能な文字列Siを読み込んで前記メインメモリに格納する読み込み部と、
前記文字列Siが前記メインメモリに格納された後にスレッドi+1のファイル読み込みロックを解除するファイル読み込みロック解除部と、
前記文字列Siに含まれる各セルの位置および長さを表す参照情報を計算して前記メインメモリに格納するパース部と、
i≧1の場合、前記文字列Siは文字列Si‐1の直後に続く文字列であり、スレッドiのバッファ境界ロックが解除された後、前記参照情報と前記文字列Si‐1と前記文字列Siとを用い、前記文字列Si‐1に含まれる最後のセルの直後に続くセルの位置に対応する情報Aiを得るセル特定部と、
前記情報Aiが得られた後にスレッドi+1のバッファ境界ロックを解除するバッファ境界ロック解除部と、
i=0の場合の結合文字列CS0がS0であり、i≧1の場合の結合文字列CSiが結合文字列CSi‐1の直後に前記文字列Siを結合したものであり、Jが正整数であり、j=0,…,J-1であり、前記参照情報および前記情報Aiによって特定される情報に基づいて、前記結合文字列CSiに含まれる処理対象のrレコード分の文字列である処理単位文字列PSi,jを選択し、前記処理単位文字列PSi,jを所定の有限集合の元であるエンコード情報Ei,jにエンコードする処理を、前記キャッシュメモリを利用して行うエンコード部と、
前記エンコード情報Ei,jに特定の演算を行って演算値SSi,jを得て前記メインメモリに格納する処理を、前記キャッシュメモリを利用して行う演算部と、
前記演算値SSi,jが得られた後にスレッドi+npの並列性ロックを解除する並列性ロック解除部と、
を有する演算装置。 - 請求項5の演算装置であって、
前記セルは単独で前記演算の対象を表すか否かを判別可能な文字のみを含み、
前記パース部は、i≧1において、前記文字列Si-1に含まれる各セルの位置および長さを表す参照情報の計算が終わる前に、前記文字列Siに含まれる各セルの位置および長さを表す参照情報の計算を開始する、演算装置。 - 請求項4から6の何れかの演算装置であって、
Cが前記キャッシュメモリのキャッシュメモリサイズ、Mが前記メインメモリのメインメモリサイズであり、
Scsvが前記テキストファイルの1レコード分の文字列のサイズの最大値であり、
scsvが前記1レコード分の文字列のサイズの最小値であり、
Sencが前記1レコード分の文字列を前記有限集合の元にエンコードして得られるエンコード情報の合計サイズの最大値であり、
Sssが前記1レコード分の前記エンコード情報に前記演算を行って得られる演算値の合計サイズの最大値であり、
Srefが前記1レコード分の前記セルそれぞれの位置および長さを表す参照情報の合計サイズであり、
前記レコード数rがC/(Scsv+Senc+Sref)の関数値であり、
Iが前記処理単位文字列PSi,jごとに実行される前記エンコードおよび前記演算の繰り返し回数の最大値であり、Sref≧Sssのときmax(Sref,Sss)=Srefであり、Sref<Sssのときmax(Sref,Sss)=Sssであり、f0がscsv・M/(scsv+Senc+max(Sref,Sss))の関数値であり、前記並列数npがf0/I・r・Scsvの関数値であり、
前記ファイルバッファサイズfがf0/npの関数値である、演算装置。 - 請求項4から7の何れかの演算装置であって、
前記パース部は、前記メインメモリに前記参照情報を格納する領域が足りなくなったときに、前記rレコード分の前記参照情報を格納するためのバッファ領域を前記メインメモリにまとめて確保する、演算装置。 - 請求項1から8の何れかの装置の各部の処理を実行する方法。
- 請求項1から8の何れかの装置としてコンピュータを機能させるためのプログラム。
- 請求項1から8の何れかの装置としてコンピュータを機能させるためのプログラムを格納したコンピュータ読み取り可能な記録媒体。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2020508217A JP7010365B2 (ja) | 2018-03-19 | 2019-03-11 | パラメータ設定装置、演算装置、それらの方法、およびプログラム |
EP19771212.8A EP3770889B1 (en) | 2018-03-19 | 2019-03-11 | Parameter setting apparatus, calculation apparatus, methods therefor, program, and recording medium |
US16/981,064 US11907641B2 (en) | 2018-03-19 | 2019-03-11 | Parameter setting apparatus, computation apparatus, method, program, and recoding medium thereof |
AU2019238219A AU2019238219B2 (en) | 2018-03-19 | 2019-03-11 | Parameter setting apparatus, computation apparatus, method, program, and recoding medium thereof |
CN201980019927.0A CN111868805B (zh) | 2018-03-19 | 2019-03-11 | 参数设定装置、运算装置、它们的方法、以及记录介质 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018-051009 | 2018-03-19 | ||
JP2018051009 | 2018-03-19 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019181594A1 true WO2019181594A1 (ja) | 2019-09-26 |
Family
ID=67986173
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2019/009597 WO2019181594A1 (ja) | 2018-03-19 | 2019-03-11 | パラメータ設定装置、演算装置、それらの方法、プログラム、および記録媒体 |
Country Status (6)
Country | Link |
---|---|
US (1) | US11907641B2 (ja) |
EP (1) | EP3770889B1 (ja) |
JP (1) | JP7010365B2 (ja) |
CN (1) | CN111868805B (ja) |
AU (1) | AU2019238219B2 (ja) |
WO (1) | WO2019181594A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2021070322A1 (ja) * | 2019-10-10 | 2021-04-15 | ||
JP7464555B2 (ja) | 2021-03-12 | 2024-04-09 | Kddi株式会社 | データ提供装置、集計システム及びデータ提供プログラム |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001076014A (ja) * | 1999-09-03 | 2001-03-23 | Kyowa Exeo Corp | Cadデータファイル変換方法およびcadデータファイル変換プログラムを記録したコンピュータ読み取り可能な記録媒体 |
JP2013101324A (ja) * | 2011-10-11 | 2013-05-23 | Nippon Telegr & Teleph Corp <Ntt> | データベース撹乱パラメータ設定装置、方法及びプログラム並びにデータベース撹乱システム |
KR20140055093A (ko) * | 2012-10-30 | 2014-05-09 | 동국대학교 산학협력단 | 분산 데이터 저장, 복원 시스템 및 방법 |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040199537A1 (en) * | 2003-04-03 | 2004-10-07 | Duff Robert Cory | System for storing and retrieving database information |
CN1853181A (zh) * | 2003-09-15 | 2006-10-25 | Ab开元软件公司 | 数据归档 |
CA2584525C (en) * | 2004-10-25 | 2012-09-25 | Rick L. Orsini | Secure data parser method and system |
US20080049930A1 (en) * | 2006-08-25 | 2008-02-28 | Samsung Electronics Co., Ltd. | Method and apparatus for encrypting data |
US10515141B2 (en) * | 2012-07-18 | 2019-12-24 | Software Ag Usa, Inc. | Systems and/or methods for delayed encoding of XML information sets |
CN104615585B (zh) * | 2014-01-06 | 2017-07-21 | 腾讯科技(深圳)有限公司 | 处理文本信息的方法及装置 |
JP6367959B2 (ja) * | 2014-10-08 | 2018-08-01 | 日本電信電話株式会社 | 部分文字列位置検出装置、部分文字列位置検出方法及びプログラム |
US9959069B2 (en) * | 2015-02-12 | 2018-05-01 | Microsoft Technology Licensing, Llc | Externalized execution of input method editor |
WO2017017738A1 (ja) * | 2015-07-24 | 2017-02-02 | 富士通株式会社 | 符号化プログラム、符号化装置、及び符号化方法 |
JP2017059132A (ja) * | 2015-09-18 | 2017-03-23 | 株式会社デンソーウェーブ | 端末装置及び端末装置のためのコンピュータプログラム |
US10928992B2 (en) * | 2019-03-11 | 2021-02-23 | International Business Machines Corporation | HTML editing operations |
-
2019
- 2019-03-11 EP EP19771212.8A patent/EP3770889B1/en active Active
- 2019-03-11 JP JP2020508217A patent/JP7010365B2/ja active Active
- 2019-03-11 US US16/981,064 patent/US11907641B2/en active Active
- 2019-03-11 CN CN201980019927.0A patent/CN111868805B/zh active Active
- 2019-03-11 WO PCT/JP2019/009597 patent/WO2019181594A1/ja unknown
- 2019-03-11 AU AU2019238219A patent/AU2019238219B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2001076014A (ja) * | 1999-09-03 | 2001-03-23 | Kyowa Exeo Corp | Cadデータファイル変換方法およびcadデータファイル変換プログラムを記録したコンピュータ読み取り可能な記録媒体 |
JP2013101324A (ja) * | 2011-10-11 | 2013-05-23 | Nippon Telegr & Teleph Corp <Ntt> | データベース撹乱パラメータ設定装置、方法及びプログラム並びにデータベース撹乱システム |
KR20140055093A (ko) * | 2012-10-30 | 2014-05-09 | 동국대학교 산학협력단 | 분산 데이터 저장, 복원 시스템 및 방법 |
Non-Patent Citations (4)
Title |
---|
A. SHAMIR: "How to Share a Secret", COMMUNICATIONS OF THE ACM, vol. 22, no. 11, November 1979 (1979-11-01), pages 612 - 613, XP000565227, DOI: 10.1145/359168.359176 |
DAI IKARASHIKOJI CHIDAKOKI HAMADAKATSUMI TAKAHASHI: "Secure Database Operations Using An Improved 3-party Verifiable Secure Function Evaluation", SCIS 2011, 2011 |
Y. SHAFRANOVICH: "RFC4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files", SOLIDMATRIX TECHNOLOGIES, INC., October 2005 (2005-10-01), Retrieved from the Internet <URL:Internet<http://www.ietf.org/rfc/rfc4180.txt>> |
YONAMINE, YUTAKA ET AL.: "The wide area network storage system based on secret sharing threshold scheme", TECHN. REPORT OF IEICE, vol. 105, no. 290, 9 September 2005 (2005-09-09), pages 37 - 44, XP008183776 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2021070322A1 (ja) * | 2019-10-10 | 2021-04-15 | ||
WO2021070322A1 (ja) * | 2019-10-10 | 2021-04-15 | 日本電信電話株式会社 | 秘密多重反復計算装置、方法及びプログラム |
CN114514569A (zh) * | 2019-10-10 | 2022-05-17 | 日本电信电话株式会社 | 秘密多重迭代计算装置、方法以及程序 |
AU2019469404B2 (en) * | 2019-10-10 | 2023-02-02 | Nippon Telegraph And Telephone Corporation | Secret multiple repetition calculation apparatus, method and program |
JP7351343B2 (ja) | 2019-10-10 | 2023-09-27 | 日本電信電話株式会社 | 秘密多重反復計算装置、方法及びプログラム |
CN114514569B (zh) * | 2019-10-10 | 2024-03-01 | 日本电信电话株式会社 | 秘密多重迭代计算装置、方法以及记录介质 |
JP7464555B2 (ja) | 2021-03-12 | 2024-04-09 | Kddi株式会社 | データ提供装置、集計システム及びデータ提供プログラム |
Also Published As
Publication number | Publication date |
---|---|
US11907641B2 (en) | 2024-02-20 |
CN111868805A (zh) | 2020-10-30 |
EP3770889A4 (en) | 2021-12-15 |
CN111868805B (zh) | 2023-08-29 |
AU2019238219B2 (en) | 2021-06-24 |
EP3770889A1 (en) | 2021-01-27 |
JP7010365B2 (ja) | 2022-01-26 |
US20210027009A1 (en) | 2021-01-28 |
EP3770889B1 (en) | 2023-08-23 |
JPWO2019181594A1 (ja) | 2021-03-11 |
AU2019238219A1 (en) | 2020-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110348231B (zh) | 实现隐私保护的数据同态加解密方法及装置 | |
Wang et al. | Circuit oram: On tightness of the goldreich-ostrovsky lower bound | |
JP5957126B1 (ja) | 秘密計算装置、秘密計算方法、およびプログラム | |
US10153894B2 (en) | Homomorphic encryption with optimized encoding | |
US10075289B2 (en) | Homomorphic encryption with optimized parameter selection | |
US9900147B2 (en) | Homomorphic encryption with optimized homomorphic operations | |
JP5826934B2 (ja) | 秘密分散システム、データ分散装置、分散データ変換装置、秘密分散方法、およびプログラム | |
US10965448B1 (en) | Dynamic distributed storage for scaling blockchain | |
US9300471B2 (en) | Information processing apparatus, information processing method, and program | |
WO2019181594A1 (ja) | パラメータ設定装置、演算装置、それらの方法、プログラム、および記録媒体 | |
KR101825838B1 (ko) | 데이터의 부분 암호화 방법, 데이터의 부분 복호화 방법 및 부분 암호화된 데이터를 복호화하는 프로그램을 저장하는 저장매체 | |
Lee et al. | TensorCrypto: High throughput acceleration of lattice-based cryptography using tensor core on GPU | |
KR20240096558A (ko) | 분산된 블록체인 기능들을 위한 방법들 및 시스템들 | |
JP2013156720A (ja) | 匿名データ提供システム、匿名データ装置、及びそれらが実行する方法 | |
Balmany et al. | Dynamic proof of retrievability based on public auditing for coded secure cloud storage | |
WO2013153628A1 (ja) | 演算処理システムおよび演算結果認証方法 | |
JP6273224B2 (ja) | 暗号化システム、暗号化装置、復号装置、暗号化方法 | |
KR101503730B1 (ko) | 데이터 복원 장치 및 데이터 복원 방법 | |
Kleinjung et al. | Using a grid platform for solving large sparse linear systems over GF (2) | |
Lee et al. | TensorCrypto | |
US20240184760A1 (en) | Method of storing blockchain transaction data using fountain codes and apparatus for the same | |
Chen et al. | Practical cloud storage auditing using serverless computing | |
CN110764698B (zh) | 信息收发方法及设备 | |
US20240184577A1 (en) | Secret calculation system, apparatus, method and program | |
JP6273225B2 (ja) | 暗号化システム、暗号化装置、復号装置、暗号化方法、暗号化プログラム、復号プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19771212 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2020508217 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2019238219 Country of ref document: AU Date of ref document: 20190311 Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2019771212 Country of ref document: EP Effective date: 20201019 |