JP4334955B2 - Biological information lossless encoder - Google Patents

Description

  The present invention uses bioinformatics, genomic drug discovery, development of new biomaterials, and other fields to construct and search biological information databases, CG animation video production using computer graphics, visualization video production in science and technology simulation, and CG It relates to the field of visualization of polymer structure and behavior.

  In recent years, with the rapid development of bioinformatics (bioinformatics) such as the Human Genome Project, a huge biological information database is being built. In particular, the completeness of DNA sequences is increasing, and accumulation of proteome information is progressing at a rapid pace. In order to utilize such a large-capacity database accumulated and applied to drug development, new material development, etc., it is important to handle the database smoothly via a network. In other words, how to efficiently compress and search efficiently is important.

  Since the biological information sequence leads to a fatal defect even with a single character error, when compression is performed, lossy compression or near lossless compression such as MPEG cannot be applied, and is limited to lossless compression. Fortunately, the biological information sequence is in ASCII text format, so it can be compressed to some extent with general-purpose lossless compression tools (ZIP, LZH, etc.) for text, and the ZIP technology can be applied to currently accumulated databases. Has been.

Several other techniques have been proposed for encoding such biological information (see, for example, Patent Document 1 and Patent Document 2).
JP 2003-188735 A JP-A-2003-101485 Further, analysis of biological information requires analysis of a three-dimensional model such as a protein three-dimensional structure, and a method for compressing such a three-dimensional model has also been proposed (for example, And Patent Document 3). Japanese Patent Laid-Open No. 10-320583

  However, the general-purpose compression tool (universal compression method) as described above or the technology shown in the above-mentioned patent document cannot limit the characteristics of biological information, and therefore has a limit on the compression rate. For example, since a DNA sequence is composed of 4 characters, it can theoretically be encoded with 2 bits per character. However, in the FASTA format, which is a recording format used in the DNA homologous search engine FASTA, an annotation is used. Since information is mixed, compression is possible only up to about 3 bits. Further, DNA has a unique repetitive pattern, and if this is utilized, there is a possibility that it can be compressed to less than 2 bits.

  Therefore, the present invention makes use of the characteristics of the biological information sequence, and is a lossless encoding device for biological information that can compress biological information with an optimal code length even when annotation information is mixed, and completely decodes the compressed biological information sequence. It is an object of the present invention to provide a biological information retrieval apparatus that can be retrieved with a small amount of memory, and a three-dimensional information lossless encoding apparatus that can perform lossless compression even for a three-dimensional model.

  In order to solve the above-described problem, in the present invention, a lossless encoding device for biological information is composed of character sequence information defined within a predetermined range and annotation information for annotating information in a specific range of the sequence information. The annotation information and the sequence data body are separated from the biological information file to form the annotation data and the sequence data body, and the biological data file can be restored, and the annotation data is linked to the sequence data body. Data separating means for adding information, fixed length encoding means for obtaining intermediate array data by performing data compression by assigning a fixed bit length to each character recorded in the array data body, and A configuration having variable length encoding means for compressing data with a variable bit length for each of the intermediate sequence data compressed at a fixed length and the annotation data; Characterized in that was.

  Further, in the present invention, a search key input for inputting a sequence using a search device for biological information that searches for a target sequence from sequence data for search in which one base or amino acid is recorded in less than 1 byte. And a search pattern creation means for creating a plurality of search patterns by moving the input search key by one base or one amino acid recording unit and adding arbitrary bits so as to be a whole byte unit, It is characterized by having a collating means for performing collation by comparing the created search pattern with the search array data in units of 1 byte.

  In the present invention, the lossless encoding device for three-dimensional information is a three-dimensional information composed of character information including numerical values defined within a predetermined range and annotation information for annotating information in a specific range of the character information. A run-length encoding unit that extracts a blank character code indicating an information delimiter from the information file, performs run-length encoding, and converts a blank character part in the three-dimensional information file into a predetermined run-length code; The numerical value included in the character information is separated into a numerical data body, and the separated other is used as annotation data, so that the three-dimensional information file can be restored in the annotation data with link information to the numerical data body. Data separation means for adding, variable length encoding means for compressing data with variable bit length for each of the numeric data body and the annotation data Characterized by being configured to include.

  According to the biological information lossless encoding apparatus of the present invention, for a biological information file in which annotation information and sequence information are mixed, the annotation information and the sequence information are separated into an annotation data and an array data body, respectively. In addition, since the link information to the sequence data main body is added and then encoded, the biological information can be compressed with the optimum code length even if the annotation information is mixed. .

  According to the biological information search apparatus of the present invention, the input search key is moved one character (one base or amino acid) at a time to create a plurality of search patterns in byte units as a whole. Since the sequence data is searched by using it, there is an effect that the search can be performed with a small amount of memory.

  According to the lossless encoding apparatus for three-dimensional information of the present invention, for a three-dimensional information file in which annotation information and numerical information are mixed, after extracting a blank character code indicating a delimiter of information and performing run-length encoding, Annotation information and numeric information are separated into annotation data and numeric data body, respectively, and after adding link information to the numeric data body in the annotation data, each is encoded, so the 3D model As a result, the lossless compression can be performed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(Lossless encoding device for biological information)
FIG. 1 is a functional block diagram showing a configuration of a biological information lossless encoding apparatus according to the present invention. In FIG. 1, 1 is data separation means, 2 is fixed length coding means, and 3 is variable length coding means. The data separating means 1 has a function of separating annotation information and sequence information recorded in a biological information file to obtain annotation data and a sequence data body. The fixed-length encoding unit 2 has a function of encoding one array data body separated by the data separating unit 1 by assigning a fixed bit length to each character regardless of each array character. The variable length encoding means 3 has a function of encoding one annotation data separated by the data separation means 1 and the array data main body encoded by the fixed length encoding means 2 with variable lengths, respectively. .

  Here, the structure of biological information to be compressed in the present invention will be described. In this embodiment, a base sequence and an amino acid sequence can be used as biological information. Here, first, the base sequence will be described. FIG. 2A is a diagram showing an original base sequence file expressed in the FASTA format which is a typical data format. In FIG. 1, t, c, a, and g represent four types of bases, thymine, cytonin, adenine, and guanine, respectively. Here, the annotation information other than the four characters indicating the base is omitted as <ANNOTATION>, but actually, the annotation information for explaining the base sequence is described. The characters that make up the annotation information and each base are recorded in ASCII code, and 8 bits are required to record one character.

  Next, the processing operation of the apparatus shown in FIG. 1 will be described. First, when the original base sequence file as shown in FIG. 2 (a) is input, first, the data separating means 1 separates the annotation information and the sequence information in the original base sequence file to obtain the annotation data and the sequence data body. And Specifically, the original base sequence file as shown in FIG. 2 (a) is decoded in order from the top, and the data is a text format composed only of ASCII character data of t, c, a, and g. In this case, it is determined that the data is an array data body. If the text format includes ASCII character data other than t, c, a, and g, it is determined that the data is annotation data and separated. At this time, the number of bases separated as the sequence data body is counted, and the number of recorded bases is recorded after each annotation information. For example, in the example of FIG. 2A, since 67 bases are recorded after <ANNOTATION2>, information indicating that 67 bases should be inserted is recorded in the annotation data. However, in the present embodiment, the annotation information is recorded in the ASCII code, and values from 0 to 127 are recognized as character information. Therefore, the base value 67 is added to the maximum value 127 used as character information and recorded. Therefore, as shown in FIG. 2B, “194” is recorded after <ANNOTATION2>.

  The information that can be recorded in 1 byte is from 0 to 255. As described above, 0 to 127 are used as character information. Therefore, the number of bases that can be recorded in 1 byte is up to 128. Therefore, when the number of bases is 129 or more, it is recorded with 2 bytes. For example, in the example of FIG. 2A, since 136 bases are recorded after <ANNOTATION1>, information indicating that 136 bases should be inserted is recorded in the annotation data. In this case, 136 is divided into 128 and 8, and 127 is added to the first byte and the second byte for recording. Therefore, as shown in FIG. 2B, “255” and “135” are recorded after <ANNOTATION1>. Thus, by recording the number of bases to be inserted in the annotation data, it becomes possible to establish a link with the sequence data body at the time of decoding.

  The sequence data body is obtained by removing the annotation information from the original base sequence file and arranging the bases continuously. For this reason, when 136 bases and 67 bases are recorded as shown in FIG. 2A, 203 bases are continuously recorded as shown in FIG. 2C.

  Subsequently, the fixed length encoding means 2 performs fixed length encoding on the array data body to obtain intermediate array data. Specifically, encoding is performed by replacing each base recorded in 8 bits with 2 bits. Specifically, replacement is performed using the base conversion table shown in FIG. As a result, what was recorded in 8 bits per base is recorded in 2 bits, and the amount of data is greatly reduced.

  On the other hand, the variable length encoding means 3 encodes the annotation data with a variable length. Here, an outline of the processing by the variable length coding means 3 is shown in the flowchart of FIG. First, the read annotation data is run-length compressed byte by byte (step S1). Next, a byte data frequency table is created (step S2). Specifically, a frequency table is created in which bit arrays having a small bit length are associated in the order of byte data having a high appearance frequency. The created frequency table is saved for later use. Next, the run-length compressed data is converted using the created frequency table (step S3). As a result, the higher the frequency, the smaller the value. Subsequently, the data converted by the frequency table is subjected to variable length coding (step S4). For the variable length encoding process in step S4, a known method such as Golomb-Rice can be used. As a result, compressed annotation data is obtained.

  After processing the annotation data, the variable length encoding means 3 performs variable length encoding on the intermediate sequence data that has been fixed length encoded by the fixed length encoding means 2. This process is the same as the process in steps S1 to S4. As a result, compressed array data is obtained.

  Through the above processing, a compressed file (including compressed annotation data, compressed sequence data, annotation frequency table, sequence frequency table, and base conversion table) is obtained. By storing this compressed file in a predetermined storage device, the compressed file can be distributed. For example, if these are stored in a predetermined directory of a computer that is open to the Internet, the user can download a small amount of data, so that data can be acquired quickly. Become.

  Next, the compressed file decoding process will be described. In the decoding process, the annotation data is restored from the compressed annotation data and the annotation frequency table, the intermediate sequence data is restored from the compressed sequence data and the sequence frequency table, the sequence data body is restored from the intermediate sequence data and the base conversion table, and finally The original biological information file is obtained by integrating the annotation data and the sequence data body. Specifically, first, the processing opposite to the processing shown in the flowchart of FIG. 4 is performed on the compressed annotation data, and the annotation data is restored using the annotation frequency table. Also, the compressed array data is subjected to a process reverse to the process shown in the flowchart of FIG. 4, and the intermediate array data is restored using the array frequency table. In the intermediate sequence data, each base is represented by 2 bits. Therefore, the sequence data body is restored by returning to 8 bits for each base using the base conversion table. Next, the annotation data and the sequence data body are integrated. This is done by reading the annotation information <ANNOTATION> of the annotation data and adding the number of bases corresponding to the number of inserted characters immediately after that from the sequence data body. Read and insert after annotation information. By performing this process on each piece of annotation information, the biological information file is restored.

(Biological information search device)
Next, a biological information search apparatus according to the present invention will be described. FIG. 5 is a functional block diagram showing the configuration of the biological information search apparatus according to the present invention. In FIG. 5, 11 is a search key input means, 12 is a search pattern creation means, and 13 is a collation means. The search key input means 1 has a function of inputting a search key that is an array to be searched. The search pattern creation means 12 has a function of creating a plurality of search patterns by moving the input search key character by character. The collation means 13 has a function of collating the created search pattern with the array in the intermediate array data.

  Next, the processing operation of the search device shown in FIG. 5 will be described. The structure of the intermediate sequence data is shown in FIG. As described above, in the intermediate sequence data, each base is recorded with 2 bits. In FIG. 6, the data is divided in units of 1 byte (4 bases). Consider a case where such an intermediate sequence data is used to search for an array “tatagc”. In this case, when a search key “tatagc” is input from the search key input means 11, the search pattern creation means 12 displays A “tatagc **”, B “* tagtagc *”, C, as shown in FIG. Four types of search patterns, “** tagtagc” and D “*** tagagc ***”, are created. Here, “*” is an arbitrary array of 2 bits. This search pattern is an integer byte. Here, the search patterns A, B, and C are 2 bytes, and the search pattern D is 3 bytes. Next, the collation means 13 searches in byte units from the top of the search pattern. For example, first, “data” of the first byte of the A pattern is used to perform matching with the intermediate array data in units of 1 byte. If there is a matching array, “gc **” in the second byte is Perform matching. By doing so, it is not necessary to perform matching of all the sequences to be searched suddenly, and only when the first byte is matched, it is only necessary to perform matching after the second byte. Shortened. If no matching sequence is found in the A pattern, the search is attempted in all patterns in the order of the B pattern, the C pattern, and the D pattern.

(Example of amino acid sequence)
In the above-described examples of the lossless encoding device and search device for biological information, the DNA base sequence has been described as an example, but the same applies to amino acid sequences. Here, differences in the case of compressing and searching for amino acid sequences from the case of the DNA base sequence will be described. In the case of an amino acid sequence, after processing by the data separation unit 1, each fixed-length encoding unit 2 converts each amino acid represented by 8 bits into 4 bits. However, since there are 20 types of amino acids, they cannot be expressed in 4 bits, so 5 types with relatively low appearance frequency are expressed in 8 bits, and the other 15 types are expressed in 4 bits. Specifically, the conversion is performed using the amino acid conversion table shown in FIG.

  Next, the search for amino acid sequences will be described. The structure of the intermediate sequence data in the case of amino acids is shown in FIG. In the intermediate sequence data, each amino acid is recorded in 4 bits or 8 bits as described above. In FIG. 8, the data is shown in units of 1 byte (1 or 2 amino acids). Consider a case where such an intermediate sequence data is used to search for an array “EKAR”. In this case, two patterns E “EKAR” and F “* EKAR *” as shown in FIG. 8B are created and searched in byte units. Here, “*” is an arbitrary array of 4 bits. For example, first, “EK” in the first 1 byte of the E pattern is used to perform matching with intermediate array data in units of 1 byte. If there is a matching array, matching with “AR” in the second byte I do. By doing so, it is not necessary to perform matching of all the sequences to be searched suddenly, and only when the first byte is matched, it is only necessary to perform matching after the second byte. Shortened. If no matching sequence is found in the E pattern, a search is attempted using the F pattern.

(Lossless encoding device for 3D information)
FIG. 9 is a functional block diagram showing the configuration of the lossless encoding apparatus for three-dimensional information according to the present invention. In FIG. 9, 21 is a run length encoding means, 22 is a fixed tag encoding means, 23 is a data separation means, and 24 is a variable length encoding means. The run-length encoding means 21 has a function of performing run-length encoding of blank characters in the three-dimensional information file. The fixed tag encoding means 22 has a function of converting a fixed tag in the three-dimensional information file into a corresponding bit string. The data separation means 23 has a function of obtaining annotation data and a numerical data body by separating the annotation information and the numerical information recorded in the three-dimensional information file. The variable length encoding unit 24 has a function of encoding the annotation data and the numerical data main body separated by the data separation unit 23 with variable lengths.

  The structure of the three-dimensional information file to be compressed in the present invention will be described. FIG. 10A is a diagram showing a three-dimensional CG file expressed in the VRML format, which is a typical data format. In FIG. 10A, the underline indicates “space”. Here, annotation information other than numerical values is omitted as <ANNOTATION> in the same manner as in FIG. 1, but actually, annotation information for explaining numerical values is described.

  Subsequently, compression of the three-dimensional information file will be described. First, when three-dimensional data is read, the run-length encoding means 11 performs run-length encoding of space (blank) information. Next, the fixed tag encoding means 12 encodes the fixed tag. Specifically, encoding is performed using a fixed tag conversion table as shown in FIG. Next, the data separating means 13 separates the numerical values included in the character information into a numerical data body, and sets the other separated as annotation data. Specifically, the original three-dimensional CG file as shown in FIG. 10A is sequentially decoded from the top, and the data is 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 If the text format is composed only of ASCII character data with a minus sign and a decimal symbol, it is determined that the data is a numeric data body, and if the text format includes ASCII character data other than the above, It is determined that it is annotation data and is separated. At this time, the number of numerical values separated as the numerical data body is counted, and information about the recorded numerical values is recorded after each annotation information. At this time, a numerical value is assigned to the character read in byte units according to the following [Conversion rule 1].

[Conversion rule 1]
0 to 127: ASCII character string 128 to 191: Inserted numerical value length +127
192 to 223: Fixed tag code +192
224 to 255: insertion space length + 223
For example, the space of the first five characters shown in FIG. 10A is recorded in the annotation data shown in FIG. 10B as “228” by adding 223 to the insertion space length “5”. Similarly, a two-character space is recorded in the annotation data as “225” by adding 223 to the insertion space length “2”. Further, the standard tag “POINT” shown in FIG. 10A is added to “21” obtained from the table shown in FIG. 9 by adding 192 to “213” to the annotation data shown in FIG. 10B. To be recorded. The numerical value is recorded as a value obtained by adding 127 to the number of consecutive numerical values including “.”. That is, when there are 8 such as “0.000000”, it is recorded as “135”, and when 9 such as “−0.000100”, it is recorded as “136”.

  The numerical data body is obtained by removing the annotation information from the original three-dimensional CG data and arranging the numerical values continuously. Therefore, as shown in FIG. 1C, numerical values are continuously recorded.

  The variable length encoding means 24 encodes the annotation data and the numerical data main body with a variable length. Specifically, the processing shown in the flowchart of FIG. 4 is executed. As a result, a compressed file (compressed annotation data, compressed numerical data, annotation frequency table, numerical frequency table, fixed tag conversion table) is obtained.

  Subsequently, the decoding process will be described. The decryption process restores the annotation data from the compressed annotation data and the annotation frequency table, restores the numeric data body from the compressed numeric data body and the numeric frequency table, and finally integrates the annotation data and the numeric data body into the original array. You will get data. Specifically, first, the processing opposite to the processing shown in the flowchart of FIG. 4 is performed on the compressed annotation data, and the annotation data is restored using the annotation frequency table. Also, the compressed numeric data body is processed in reverse to the processing shown in the flowchart of FIG. 4, and the numeric data body is restored using the numeric frequency table. Next, the annotation data and the numerical data body are integrated. This is done by reading the annotation information <ANNOTATION> of the annotation data, and converting the numerical value recorded immediately after that according to the above-mentioned [Conversion Rule 1]. The corresponding number of numerical values are read from the numerical data body and inserted after the annotation information. By performing this process for each piece of annotation information, the three-dimensional information file is restored.

  Note that each of the devices shown in FIGS. 1, 5, and 8 is specifically realized by installing a dedicated software program in hardware such as a computer.

It is a functional block diagram which shows the structure of the lossless encoding apparatus of the biological information which concerns on this invention. It is a figure which shows the mode of the process by the data separation means. It is a figure which shows an example of a base conversion table. It is a flowchart which shows the process outline | summary by a variable-length encoding means. It is a functional block diagram which shows the structure of the search apparatus of the biological information which concerns on this invention. It is a figure which shows the intermediate sequence data and search pattern in the case of a base sequence. It is a figure which shows an example of an amino acid conversion table. It is a figure which shows the intermediate sequence data in the case of an amino acid sequence, and a search pattern. It is a functional block diagram which shows the structure of the lossless encoding apparatus of the three-dimensional information which concerns on this invention. It is a figure which shows the mode of the process by the data separation means. It is a figure which shows an example of a fixed tag conversion table.

Explanation of symbols

1, 23 ... Data separation means 2 ... Fixed length encoding means 3, 24 ... Variable length encoding means 11 ... Search key input means 12 ... Search pattern creation means 13 ... Verification Means 14 ... Archive execution means 21 ... Run length encoding means 22 ... Fixed tag encoding means

Claims (4)

For a biological information file composed of character sequence information defined within a predetermined range and annotation information that annotates information of a specific range of the sequence information,
Data separation for adding link information to the sequence data body to the annotation data so that the annotation information and the sequence data body can be separated into the annotation data and the sequence data body and the biological information file can be restored. Means,
Fixed-length encoding means for obtaining intermediate array data by performing data compression by assigning a fixed bit length to each character recorded in the array data body;
Variable length encoding means for compressing data with a variable bit length for each of the intermediate array data compressed with the fixed length and the annotation data;
A lossless encoding apparatus for biological information, comprising: In claim 1,
The variable-length encoding means performs run-length compression on each byte array unit of the annotation data or array data body, and performs encoding by assigning bits having a short length in order of occurrence frequency of each byte data. A lossless encoding device for biological information, characterized in that it exists. In claim 1,
The sequence data body is composed of four types of characters a, g, c, and t (capital letters are acceptable), each sequence data is recorded in 8 bits, and the fixed-length encoding means includes: A lossless encoding device for biological information, wherein each character is encoded with a fixed length of 2 bits. In claim 1,
The array data body is L, A, S, G, V, E, K, I, T, D, R, P, N, F, Q, Y, M, H, C, W (lower case is also acceptable) The fixed-length encoding means includes L, A, S, G, V, E, K, I, T, the amino acid sequence data in which each character is recorded in 8 bits. D, R, P, N, F, and Q characters are encoded with a fixed length of 4 bits, and Y, M, H, C, and W characters are encoded with a fixed length of 8 bits. A lossless encoding apparatus for biological information, characterized in that it performs the conversion.

Images