RU2747625C1 - Method for joint data compression and encryption in genome alignment - Google Patents

Abstract

FIELD: computational technology.

SUBSTANCE: invention relates to the area of computational technology, namely to processing of genome data. A computer-implemented method of joint data compression and encryption in genome alignment generating a compressed representation of a genome sequence in form of a sequence of variants obtained based at a comparison with a reference genome, and accepting unprocessed sequencing data (NGS) as input using an alignment processor, the original data is received by a genome data encoder from the alignment processor, the data is transposed by the processor into the original data file based on the alignment position, then the processor performs encryption into a data file considering the position allowing selective search over the encrypted and compressed alignment reference map, and the resulting encrypted data file is recorded from the encoder data in the SECRAM format with compressed alignment in the biobank storage unit.

EFFECT: increased reliability of data storage.

1 cl, 7 dwg

Description

The present invention relates to the field of genomic data processing in general and, more specifically, to next generation sequencing applications.

A known method and system for compressing genome sequences using graphic processing units WO / 2016/125154, IL).

The disadvantage of this method is the possibility of accidental leakage of confidential genomic information in the process of data retrieval. Due to the lack of encryption, in the event of a leak, genomic information can be used in various ways, for example, to refuse employment and health insurance, blackmail or even genetic discrimination.

There is a known method for managing raw genomic data with a method for maintaining confidentiality in a biobank. (see, for example, patent (WO 2014202615) Method to manage raw genomic data in a privacy preserving manner in a biobank No. WO / 2014/202615, C0048)

The disadvantage of this method is the need to use additional costs for storing and processing a large amount of non-compressed information. Due to the lack of preliminary compression of genomic data, when using this method, additional economic and technological requirements arise, which makes the method impractical for use in clinical genomic applications.

A compression / decompression method and apparatus for genomic variant call data No. H03M 7/30, GB) is adopted as a prototype (see, for example, patent (EP 3430551) Compression / decompression method and apparatus for genomic variant call data No. H03M 7/30, GB), which consists in one diagram generates a compressed sequence representation of at least a portion of an individual's genome. The method includes obtaining an input file containing a sequence representation of at least part of the genome of an individual in the form of a sequence of variants determined by comparison with a reference genome; the method allows you to access the biodata reference database containing many lists of links of genetic variants of different people; the list of links contains a sequence of genetic variants from one haplotype; the list of references is a double mosaic that corresponds to a part of an individual's genome up to a threshold value; each mosaic represents one of the two haplotypes of the individual's genome for which a condensed representation is to be created; a compressed representation is created by encoding the tiles themselves and encoding the deviations from those tiles.

The disadvantage of the prototype method is that it only compresses data and does not provide for their encryption. This does not ensure the confidentiality of data, which can lead to accidental or deliberate leakage of information about the genetic characteristics of an individual from bio-databases.

The technical problem of the invention is the combined compression and encryption of genomic data alignment information to reduce the amount of information, which reduces the economic and technological requirements for storing and processing information, and also ensures the preservation of the confidentiality of individual characteristics of human genomic data at all stages of interaction with bio-databases.

The technical problem is achieved in that, in the method for compressing genomic data, the alignment information of the genomic data organized as a read-based alignment data stream is converted into a position-based alignment data stream. Position-based alignment information is encoded into a link-based alignment data stream. The link-based alignment data stream is encrypted by a combination of order-preserving genomic position information and symmetric encryption of the link-based delta alignment data. The resulting compressed and encrypted stream is indexed and stored in the biobank storage unit in the database.

The essence of the invention is as follows. FIG. 1 shows a genomic data processing scheme containing a genomic data encoder 6, a next generation sequencer (NGS) 1 and a biobank 7 storage unit 7. In a next generation sequencer 1 with a processor, raw NGS sequence data is generated into one or more data files. The raw sequencing data file is generated in FASTQ format.

An alignment processor (module 2) accepts raw NGS sequencing data as input, aligns the short reads to the reference genome, and generates a raw alignment data file. The raw alignment data file is in SAM format or BAM format, the binary equivalent of SAM format.

Alignment module 2 is programmed to implement various methods for aligning genomic data. Equalizer 2 is a computer system or part of a computer system including a central processing unit (CPU, "processor" or "computer processor"), memory such as RAM, and storage modules such as a hard disk, and communication interfaces for communicating with other computer systems via a communications network, such as the Internet or a local area network.

The genomic data encoder 6 receives, as input, the original alignment data from the alignment unit 2; transpose it with the transpose unit 3 into the original alignment data file based on the position; compressing by the compression unit 4 the position-based alignment raw data file into the reference-based position compressed data file; and encrypted, in the encryption module 5, into a data file, which allows a selective search on the encrypted and compressed alignment reference card (file format "SECRAM"). From the genomic data encoder 6, the resulting encrypted SECRAM data file with compressed alignment is written to the biobank storage unit 6.

The transposition unit 3 converts genomic alignment information from a read-based data structure to a position-based data structure as shown in FIG. 2. In a read-based format, the aligned data represented by short reads is stored sequentially as generated by sequencer 1, read-read-read, in a raw alignment data file.

Read-based formats include SAM format, BAM format, and CRAM format. In the position-based format (FIG. 2), information about one position is grouped together in continuous storage, therefore, the aligned data is stored by position in the file.

FIG. 3 shows the structure of a position-based data file for 5 reads (read 1, read 2, read 3, read 4, read 5) overlapping 9 positions with indices from 0 to 8. At position 0, the initial marker of read 1 and read 2 is written As shown in FIG. 3, the * character is used as a start marker for each start of a short read in a text data file, followed by metadata information related to the short read, such as its name or identifier, string and / or display quality, and the nucleotide base identified in this position (A, T, C or G, denoted ". *" in Fig. 3) with a corresponding quality score. At position +1, the continuation of Read1 and Read2, that is, the next nucleotide base identified at that position (A, T, C, or G, denoted ". *" In Fig. 3), as well as the beginning of Read3, are recorded. At position +3, the last base Read1 and Read2, followed by the end marker Read1 and Read2, and the continuation of Read3, Read4, and Read5, respectively, that is, the next nucleotide base identified at that position (A, T, C, or G, denoted " . * "In Fig. 3) for Read3, Read4 and Read5, respectively, are written. In a possible embodiment, as shown in FIG. 3, the character ". * $" Is used as an ending marker for each short end of a read in a text data file, but other embodiments are possible. For purposes of illustration, only ultrashort reads spanning positions 3 (read 4) to 7 (read 5) are shown in FIG. 3, but the proposed data structure is applicable to short reads from 100 bps and above, which is usually output by NGS sequencers. The longer the read, the shorter the RInfo metadata overhead in the resulting position-based data file structure. In an exemplary embodiment (not shown), the read information Rinfo may also include the read length so that the end marker does not need to be written.

Compression of data is performed in a compression unit 4. In a compression unit 4, the position-based alignment raw data file is compressed into the compressed position-based data file.

Correctly aligned short reads should have significant redundancy, since most of the reads will match the link. For example, in FIG. 2 Data Read1 and Read2 should be very similar. In the proposed embodiment of the reference compression, the position value (POS) is extracted from the original alignment data based on the position. For a given position, all reads that span that position can be ordered by their starting position, then each read is appended with a unique order. One reading can be assigned a different order for different positions because the corresponding lists of covered readings in those positions are different. At any single position, the lines POS captures one or more of three different primitive delta alignment operators, such as:

1. REPLACEMENT - Order // 'S' // [A | T | C | G]: read (specified by Order) to replace the specified letter versus reference.

2. INSERT - Order // 'I' // i // {A, T, C, G} i: read contains an insert of i letters that are listed.

3. DELETE - Order // 'D': read to delete.

1. REPLACEMENT - Order // 'S' // [A | T | C | G]: A read (indicated by Order) has substitution with the indicated letter compared to a reference.

2. INSERT - Order // 'I' // i // {A, T, C, G} i: read contains an insert of i letters that are listed.

3. DELETE - Order // 'D': read has delete.

For example, PosC, which looks like "9I4ATIG ... 23SA ... 57D" means:

9I4ATG: Insert 4 letters "ATTG" in 9

23SA: replacement with the letter "A" in 23

57D: deleting at 57

9I4ATG: insert 4 letters "ATTG" 9

23SA: replacement with the letter "A" in 23

57D: deleting at 57

A simple example of a proposed reference compression data structure is shown in FIG. 4 where:

The pos at position 7 refers to 1SG (replacing the base 'G' in Read1, ordered as read # 1 at that position)

The pos at position 12 refers to 1D3 (deleting 3 bases in Read1, ordered as read # 1 at that position)

PosC at position 23 refers to 1IAT (insertion of two bases 'A', 'T' in Read3 is ordered as read # 1 at that position)

PosC at position 25 refers to 1IC (insert base 'C' in Read3 is ordered as read # 1 at that position)

The pos at position 7 refers to 1SG (replacing the base 'G' in Read1, ordered as read # 1 at that position)

The pos at position 12 refers to 1D3 (deleting 3 bases in Read1, ordered as read # 1 at that position)

PosC at position 23 refers to 1IAT (insertion of two bases 'A', 'T' in Read3 is ordered as read # 1 at that position)

PosC at position 25 refers to 1IC (insert base 'C' in Read3 is ordered as read # 1 at that position)

More complex operators of difference alignment (eg, soft constraint, hard constraint, skip ...) can also be encoded by the above primitive operators or a combination thereof, as will be obvious to those skilled in the art.

The read headers list contains a list of reads that start at this position. It expands as (Order // RInfo) *, where "*" means an arbitrary number of such headers. In an exemplary embodiment, the read information RInfo may also include the read length so that we do not need to store the end marker.

Quality scores record the quality scores for the foundations of that position.

PosC records information about variants relative to the reference sequence.

The line size is the length (measured in bytes) of the position line;

The read headers list contains a list of reads that start at this position. It expands as (Order // RInfo) *, where "*" means an arbitrary number of such headers. In an exemplary embodiment, the read information RInfo may also include the read length so that we do not need to store the end marker.

Quality scores record the quality scores for the foundations of that position.

PosC records information about variants relative to the reference sequence.

Once the original position-based alignment data has been converted into a compressed position reference data structure, the data coding specialist can apply additional data coding techniques such as entropy coding and / or text coding algorithms to further compress the data into a compact binary reference file. compressed location data. In an exemplary embodiment, variable length coding can be used to further compress differences found in reference compression as well as read metadata such as display quality metrics.

The data is encrypted in the encryption module 5. The encryption module assigns a master key Km to each patient, which can be used to obtain different encryption keys for different encryption steps. The encryption module 5 independently encrypts the option information for each position, that is, each row in the data structure in FIG. 3 to provide granular privacy control by partial extraction of genomic alignment data while addressing common threats of genomic alignment information leakage. Thus, data retrieval is limited to only the positions of interest from the resulting data file (for example, in the "SECRAM" file format) without leaking any information from positions outside the region of interest, even if the original read alignment data (for example, in the SAM / BAM file format ) cover both relevant and irrelevant positions.

The encryption module 5 encodes the compressed genomic data file format of FIG. 5) into an encrypted compressed SECRAM file format as shown in FIG. 5) in two stages. At the first stage, the order-preserving encryption key is extracted from the encryption module 5 from the patient's master key Km and the position fields Pos1, Pos2, Pos3 are encrypted into the order-preserving encrypted block of positions 5.3 from the block 5.0 of compressed genomic data files using an order-preserving encryption scheme (OPE ) with an encryption key with preserving the order of OPE. This order-preserving encryption scheme allows the retrieval of the resulting encrypted data 5.3 in a given string corresponding to a specific position (OPE (Pos1), OPE (Pos2) or OPE (Pos3) ... in FIG. 5) without requiring decryption of the entire data block 5.3 (for example , block of 50,000 data lines) at the decoding stage.

In a second step, with reference to the format of FIG. 5, the encryption module 5 encrypts confidential information in each position, such as the PSC data block 5.1 SG, D-3 I-AT ... encoded short difference reads relative to the reference sequence into the encrypted PSC data block 5.4 using the modern SE security encryption method. The encryption module 5 derives the key Ksc from the patient's master key Km. For the i-th block 5.1, covering several position lines in its input compressed position data file associated with patient m, the encryption module 5 generates a random number Ri. For each row of a position in the compressed block i, the encryption module 5 encrypts the concatenated data 5.1 POC using a stream cipher using a symmetric encryption key Ksc and a random value Ri to generate symmetrically encrypted data 5.4 POC. In the proposed embodiment, the XOR stream cipher mode is used for encryption. In an exemplary embodiment, AES is used in the CTR stream encryption mode. In an exemplary embodiment, the encryption unit 5 stores the random salt Ri in an index file (not shown). In another possible embodiment (not shown), the encryption module 5 stores the random salt Ri in the header of the encrypted data block.

The overall design for security and privacy is highly dependent on the underlying key management system.

Thus, the developed method of joint compression and encryption of data in genomic alignment, generating a compressed representation of the genome sequence in the form of a sequence of variants obtained on the basis of comparison with the reference genome, which reduces the economic and technological requirements for storing and processing information, and also increases the confidentiality of individual genomic data. data by eliminating the time interval between compression of genomic data after sequencing and encryption, which increases data security.

Claims (1)

A computer-implemented method of jointly compressing and encrypting data in genomic alignment, which generates a compressed representation of the genome sequence in the form of a sequence of variants obtained on the basis of comparison with a reference genome, including what is taken as input data raw sequencing data (NGS) using an alignment processor, align the short reads with the reference genome and generate a raw alignment data file, while the raw data file is in SAM or BAM format, the original data from the alignment processor is sent to the genomic data encoder, transposed using processor compresses the raw data file based on the alignment position, then the processor compresses the positional alignment raw data file based on the alignment position, then the processor encrypts the data file based on the position, which allows selective search on encrypted compressed and compressed reference alignment card, and from the data encoder write the resulting encrypted data file in SECRAM format with compressed alignment to the biobank storage unit.

Images