CN117194488A - Cell expression data processing method and device and electronic equipment - Google Patents
Cell expression data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN117194488A CN117194488A CN202311246891.4A CN202311246891A CN117194488A CN 117194488 A CN117194488 A CN 117194488A CN 202311246891 A CN202311246891 A CN 202311246891A CN 117194488 A CN117194488 A CN 117194488A
- Authority
- CN
- China
- Prior art keywords
- cell expression
- data
- matrix data
- expression matrix
- hash index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000014509 gene expression Effects 0.000 title claims abstract description 148
- 238000003672 processing method Methods 0.000 title claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims abstract description 90
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000007906 compression Methods 0.000 claims description 34
- 230000006835 compression Effects 0.000 claims description 34
- 108090000623 proteins and genes Proteins 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 16
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006837 decompression Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000003766 bioinformatics method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000011712 cell development Effects 0.000 description 1
- 230000024245 cell differentiation Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 239000012895 dilution Substances 0.000 description 1
- 238000010790 dilution Methods 0.000 description 1
- 238000000684 flow cytometry Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000010839 reverse transcription Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a cell expression data processing method, a device and electronic equipment, and relates to the technical field of data processing, wherein the method comprises the steps of obtaining cell expression data; processing the cell expression data to obtain cell expression matrix data in a specific format; compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data; and establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index. The application reduces the complexity of cell expression data processing and simultaneously reduces the occupied space of storage.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for processing cell expression data, and an electronic device.
Background
Cell expression data refers to data obtained by measuring and analyzing the expression level of a cellular gene by certain bioinformatics methods and techniques. These data can be used to study problems with cell type, cell heterogeneity, cell development and differentiation, etc.
Cell expression data typically exists in the form of a gene expression matrix, where each row represents a gene and each column represents the expression level of a cell or tissue sample. The gene expression matrix is an oversized sparse matrix in nature, and in order to realize random reading and writing of a certain element, the gene expression matrix needs to be split into a plurality of parts to form a set of complex indexing mechanisms, such as a multi-section index or a tree index. And because the gene expression matrix cannot be subjected to convolution calculation, the gene expression matrix can only be read according to the whole line. Therefore, the processing method of the conventional sparse matrix/oversized matrix is used, the complexity of processing the cell expression data is high, and the storage occupied space is large.
Therefore, a cell expression data processing method, a cell expression data processing device and electronic equipment are provided.
Disclosure of Invention
The specification provides a cell expression data processing method, a cell expression data processing device and electronic equipment, which reduce the complexity of cell expression data processing and simultaneously reduce the occupied space of storage.
The present specification provides a cell expression data processing method comprising:
obtaining cell expression data;
processing the cell expression data to obtain cell expression matrix data in a specific format;
compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
and establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index.
Optionally, the cell expression matrix data in the specific format includes cell expression matrix data based on rows and based on columns.
Optionally, the compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data includes:
respectively compressing the cell expression matrix data in the specific format according to rows through a lossless compression model to obtain binary cell expression matrix data;
wherein the lossless compression model comprises LZMA, GZIP, BZIP.
Optionally, the establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index includes:
traversing each element in the binary cell expression matrix data, and determining a hash value of each element;
establishing a hash index based on each element and the corresponding hash value, wherein each row of storage information is a gene name, a starting point lean amount and an end point offset amount;
and sequentially storing each element in the binary cell expression matrix data to the hash index, and storing the hash index.
Optionally, the method for determining the hash value of each element includes MD5 and SHA-1.
Optionally, after establishing a hash index based on the compressed cell expression matrix data and storing the compressed cell expression data according to the hash index, the method includes:
acquiring elements to be queried;
determining hash values of the elements to be queried by using a target hash value determining mode based on the elements to be queried, wherein the target hash value determining mode comprises the same hash value determining mode of each element as that of establishing a hash index;
and inquiring cell expression data corresponding to the element to be inquired in the hash index based on the hash value of the element to be inquired.
The present specification provides a cell expression data processing apparatus comprising:
the acquisition module is used for acquiring cell expression data;
the processing module is used for processing the cell expression data to obtain cell expression matrix data in a specific format;
the compression module is used for compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
and the indexing module is used for establishing a hash index based on the compressed cell expression matrix data and storing the compressed cell expression data according to the hash index.
Optionally, the cell expression matrix data in the specific format includes cell expression matrix data based on rows and based on columns.
Optionally, the compression module includes:
respectively compressing the cell expression matrix data in the specific format according to rows through a lossless compression model to obtain binary cell expression matrix data;
wherein the lossless compression model comprises LZMA, GZIP, BZIP.
Optionally, the indexing module includes:
traversing each element in the binary cell expression matrix data, and determining a hash value of each element;
establishing a hash index based on each element and the corresponding hash value, wherein each row of storage information is a gene name, a starting point lean amount and an end point offset amount;
and sequentially storing each element in the binary cell expression matrix data to the hash index, and storing the hash index.
Optionally, the method for determining the hash value of each element includes MD5 and SHA-1.
Optionally, after the indexing module, the indexing module includes:
acquiring elements to be queried;
determining hash values of the elements to be queried by using a target hash value determining mode based on the elements to be queried, wherein the target hash value determining mode comprises the same hash value determining mode of each element as that of establishing a hash index;
and inquiring cell expression data corresponding to the element to be inquired in the hash index based on the hash value of the element to be inquired.
The specification also provides an electronic device, wherein the electronic device includes:
a processor; the method comprises the steps of,
a memory storing processor-executable instructions that, when executed, cause the processor to perform the method of any of the above.
The present specification also provides a computer readable storage medium storing one or more programs which when executed by a processor implement any of the methods described above.
The application has at least the following advantages:
(1) The usability is sacrificed, the high random query efficiency under the condition of reading the cell expression matrix data is replaced, and the time complexity is reduced;
(2) On the premise of not losing random query efficiency, the compression rate is high, and the storage occupied space is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a method for processing cell expression data according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of cell expression matrix data in a specific format provided in the embodiments of the present disclosure;
FIG. 3 is a schematic diagram of binary cell expression matrix data provided in the embodiments of the present disclosure;
FIG. 4 is a schematic diagram of a hash index provided by an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a cell expression data processing apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of a computer readable medium according to an embodiment of the present disclosure.
Detailed Description
The following description is presented to enable one of ordinary skill in the art to make and use the application. The preferred embodiments in the following description are by way of example only and other obvious variations will occur to those skilled in the art. The basic principles of the application defined in the following description may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the application.
Exemplary embodiments of the present application are described more fully below in connection with fig. 1-7. However, the exemplary embodiments can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the application to those skilled in the art. The same reference numerals in the drawings denote the same or similar elements, components or portions, and thus a repetitive description thereof will be omitted.
The features, structures, characteristics or other details described in a particular embodiment do not exclude that may be combined in one or more other embodiments in a suitable manner, without departing from the technical idea of the application.
In the description of specific embodiments, features, structures, characteristics, or other details described in the present application are provided to enable one skilled in the art to fully understand the embodiments. However, it is not excluded that one skilled in the art may practice the present application without one or more of the specific features, structures, characteristics, or other details.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The term "and/or" and/or "includes all combinations of any one or more of the associated listed items.
Fig. 1 is a schematic diagram of a cell expression data processing method according to an embodiment of the present disclosure, where the method may include:
s110: obtaining cell expression data;
in the specific embodiment of the present specification, single cells are separated from a sample by flow cytometry, microfluidic technology, or manual operation, and RNA of the single cells is extracted for reverse transcription and amplification. Sequencing the amplified RNA fragments to obtain sequencing data, namely cell expression data.
S120: processing the cell expression data to obtain cell expression matrix data in a specific format;
optionally, the cell expression matrix data in the specific format includes cell expression matrix data based on rows and based on columns.
In the specific embodiment of the present disclosure, fig. 2 is a schematic diagram of cell expression matrix data in a specific format provided in the examples of the present disclosure, wherein the cell expression matrix data is based on rows and columns of cells, and shows the expression level of each gene in each cell, and can be used to analyze the gene expression and the difference between cells.
S130: compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
optionally, the S130 includes:
respectively compressing the cell expression matrix data in the specific format according to rows through a lossless compression model to obtain binary cell expression matrix data;
wherein the lossless compression model comprises LZMA, GZIP, BZIP.
In the specific embodiment of the present disclosure, fig. 3 is a schematic diagram of binary cell expression matrix data provided in the embodiment of the present disclosure, where LZMA, GZIP and BZIP are all commonly used lossless compression models, which can be used to compress cell expression matrix data, and the lossless compression model can restore 100% of an original file completely without any data loss.
An LZMA (Lempel-Ziv-Markov chain-Algorithm) model is an improved compression/decompression tool based on an LZ77 compression Algorithm, and has the characteristics of high compression rate, high decompression speed, low memory consumption and the like. When compressing cell expression matrix data, the LZMA algorithm can find repeated data patterns and encode the repeated data patterns, so that data compression is realized.
The GZIP model is also a common compression model, and has higher compression rate and decompression speed. The GZIP model adopts a DEFLATE compression algorithm, and the algorithm combines LZ77 and Huffman coding, so that a higher compression effect can be realized.
BZIP model, which is a compression model based on Burows-Wheeler Transform (BWT) and Huffman coding, has higher compression rate and better compression/decompression speed. The BZIP model achieves high compression rates by encoding repeated patterns in the data block.
When compressing cell expression matrix data in a particular format, one or more of the compression models may be selected for trial to find the most appropriate compression model. At the same time, the proper compression level and parameters can be selected according to the characteristics and the requirements of the data so as to balance the trade-off between the compression rate and the decompression speed.
S140: and establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index.
Optionally, the establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index includes:
traversing each element in the binary cell expression matrix data, and determining a hash value of each element;
establishing a hash index based on each element and the corresponding hash value, wherein each row of storage information is a gene name, a starting point lean amount and an end point offset amount;
and sequentially storing each element in the binary cell expression matrix data to the hash index, and storing the hash index.
In the specific embodiment of the present disclosure, fig. 4 is a schematic diagram of the hash index provided in the embodiment of the present disclosure, specifically, first, each element in binary cell expression matrix data needs to be traversed, which may be accomplished through one cycle, where in each cycle, one element is fetched, and then the next step is performed. Then, for each fetched element, a hash value thereof is calculated. Then, a hash index is created based on each element and its corresponding hash value by creating a new data structure, which may be a list, where each element is a tuple, the first element of the tuple is the hash value of the element, the second element is the name of the gene, the third element is the starting point argument, and the fourth element is the ending point offset. Next, each element in the binary cell expression matrix data is saved to the hash index in turn, which can be done by adding the element and its hash value, the gene name, the starting point dilution, the ending point offset to the hash index in each cycle. Finally, by writing the hash index into a file and saving it.
Optionally, the method for determining the hash value of each element includes MD5 and SHA-1.
Optionally, after establishing a hash index based on the compressed cell expression matrix data and storing the compressed cell expression data according to the hash index, the method includes:
acquiring elements to be queried;
determining hash values of the elements to be queried by using a target hash value determining mode based on the elements to be queried, wherein the target hash value determining mode comprises the same hash value determining mode of each element as that of establishing a hash index;
and inquiring cell expression data corresponding to the element to be inquired in the hash index based on the hash value of the element to be inquired.
The application has at least the following advantages:
(1) The usability is sacrificed, the high random query efficiency under the condition of reading the cell expression matrix data is replaced, and the time complexity is reduced;
(2) On the premise of not losing random query efficiency, the compression rate is high, and the storage occupied space is reduced.
Fig. 5 is a schematic structural diagram of a cell expression data processing apparatus according to an embodiment of the present disclosure, where the apparatus may include:
an acquisition module 10 for acquiring cell expression data;
the processing module 20 is configured to process the cell expression data to obtain cell expression matrix data in a specific format;
the compression module 30 is configured to compress the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
an indexing module 40, configured to establish a hash index based on the compressed cell expression matrix data, and store the compressed cell expression data according to the hash index.
Optionally, the cell expression matrix data in the specific format includes cell expression matrix data based on rows and based on columns.
Optionally, the compression module 30 includes:
respectively compressing the cell expression matrix data in the specific format according to rows through a lossless compression model to obtain binary cell expression matrix data;
wherein the lossless compression model comprises LZMA, GZIP, BZIP.
Optionally, the indexing module 40 includes:
traversing each element in the binary cell expression matrix data, and determining a hash value of each element;
establishing a hash index based on each element and the corresponding hash value, wherein each row of storage information is a gene name, a starting point lean amount and an end point offset amount;
and sequentially storing each element in the binary cell expression matrix data to the hash index, and storing the hash index.
Optionally, the method for determining the hash value of each element includes MD5 and SHA-1.
Optionally, after the indexing module 40, the method includes:
acquiring elements to be queried;
determining hash values of the elements to be queried by using a target hash value determining mode based on the elements to be queried, wherein the target hash value determining mode comprises the same hash value determining mode of each element as that of establishing a hash index;
and inquiring cell expression data corresponding to the element to be inquired in the hash index based on the hash value of the element to be inquired.
The functions of the apparatus according to the embodiments of the present application have been described in the foregoing method embodiments, so that the descriptions of the embodiments are not exhaustive, and reference may be made to the related descriptions in the foregoing embodiments, which are not repeated herein.
Based on the same inventive concept, the embodiments of the present specification also provide an electronic device.
The following describes an embodiment of an electronic device according to the present application, which may be regarded as a specific physical implementation of the above-described embodiment of the method and apparatus according to the present application. Details described in relation to the embodiments of the electronic device of the present application should be considered as additions to the embodiments of the method or apparatus described above; for details not disclosed in the embodiments of the electronic device of the present application, reference may be made to the above-described method or apparatus embodiments.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. An electronic device 300 according to this embodiment of the present application is described below with reference to fig. 6. The electronic device 300 shown in fig. 6 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 6, the electronic device 300 is embodied in the form of a general purpose computing device. Components of electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one memory unit 320, a bus 330 connecting the different system components (including the memory unit 320 and the processing unit 310), a display unit 340, and the like.
Wherein the storage unit stores program code that is executable by the processing unit 310 such that the processing unit 310 performs the steps according to various exemplary embodiments of the application described in the above processing method section of the present specification. For example, the processing unit 310 may perform the steps shown in fig. 1.
The memory unit 320 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 3201 and/or cache memory 3202, and may further include Read Only Memory (ROM) 3203.
The storage unit 320 may also include a program/utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 330 may be one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a viewer to interact with the electronic device 300, and/or any device (e.g., router, modem, etc.) that enables the electronic device 300 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 350. Also, electronic device 300 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 360. The network adapter 360 may communicate with other modules of the electronic device 300 via the bus 330. It should be appreciated that although not shown in fig. 6, other hardware and/or software modules may be used in connection with electronic device 300, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
Those skilled in the art will readily understand from the description of the embodiments above. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a computer readable storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, or a network device, etc.) to perform the above-mentioned method according to the present application. The computer program, when executed by a data processing device, enables the computer readable medium to carry out the above-described method of the present application, namely: such as the method shown in fig. 1.
Fig. 7 is a schematic diagram of a computer readable medium according to an embodiment of the present disclosure.
A computer program implementing the method shown in fig. 1 may be stored on one or more computer readable media. The computer readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the spectator computing device, partly on the spectator device, as a stand-alone software package, partly on the spectator computing device, partly on a remote computing device, or entirely on a remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the spectator computing device through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the internet using an internet service provider).
In summary, the application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functionality of some or all of the components in accordance with embodiments of the present application may be implemented in practice using a general purpose data processing device such as a microprocessor or Digital Signal Processor (DSP). The present application can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present application may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.
The above-described specific embodiments further describe the objects, technical solutions and advantageous effects of the present application in detail, and it should be understood that the present application is not inherently related to any particular computer, virtual device or electronic apparatus, and various general-purpose devices may also implement the present application. The foregoing description of the embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the application.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. A method for processing cell expression data, comprising:
obtaining cell expression data;
processing the cell expression data to obtain cell expression matrix data in a specific format;
compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
and establishing a hash index based on the compressed cell expression matrix data, and storing the compressed cell expression data according to the hash index.
2. The method of claim 1, wherein the cell expression matrix data in a specific format includes cell expression matrix data based on rows and columns.
3. The method for processing cell expression data according to claim 2, wherein compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data comprises:
respectively compressing the cell expression matrix data in the specific format according to rows through a lossless compression model to obtain binary cell expression matrix data;
wherein the lossless compression model comprises LZMA, GZIP, BZIP.
4. The cell expression data processing method of claim 3, wherein the creating a hash index based on the compressed cell expression matrix data and saving the compressed cell expression data according to the hash index comprises:
traversing each element in the binary cell expression matrix data, and determining a hash value of each element;
establishing a hash index based on each element and the corresponding hash value, wherein each row of storage information is a gene name, a starting point lean amount and an end point offset amount;
sequentially storing each element in the binary cell expression matrix data to the hash index ， And saving the hash index.
5. The method of claim 4, wherein the means for determining the hash value of each element comprises MD5, SHA-1.
6. The method of claim 5, wherein after creating a hash index based on the compressed cell expression matrix data and storing the compressed cell expression data according to the hash index, comprising:
acquiring elements to be queried;
determining hash values of the elements to be queried by using a target hash value determining mode based on the elements to be queried, wherein the target hash value determining mode comprises the same hash value determining mode of each element as that of establishing a hash index;
and inquiring cell expression data corresponding to the element to be inquired in the hash index based on the hash value of the element to be inquired.
7. A cell expression data processing apparatus comprising:
the acquisition module is used for acquiring cell expression data;
the processing module is used for processing the cell expression data to obtain cell expression matrix data in a specific format;
the compression module is used for compressing the cell expression matrix data in the specific format to obtain compressed cell expression matrix data;
and the indexing module is used for establishing a hash index based on the compressed cell expression matrix data and storing the compressed cell expression data according to the hash index.
8. The cell expression data processing apparatus of claim 7, wherein the cell expression matrix data in the specific format includes cell expression matrix data based on rows and columns of cells.
9. An electronic device, wherein the electronic device comprises:
a processor; and a memory storing processor-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-6.
10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311246891.4A CN117194488A (en) | 2023-09-25 | 2023-09-25 | Cell expression data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311246891.4A CN117194488A (en) | 2023-09-25 | 2023-09-25 | Cell expression data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117194488A true CN117194488A (en) | 2023-12-08 |
Family
ID=88996082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311246891.4A Pending CN117194488A (en) | 2023-09-25 | 2023-09-25 | Cell expression data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117194488A (en) |
-
2023
- 2023-09-25 CN CN202311246891.4A patent/CN117194488A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7924183B2 (en) | Method and system for reducing required storage during decompression of a compressed file | |
| CN106549673B (en) | Data compression method and device | |
| Janin et al. | BEETL-fastq: a searchable compressed archive for DNA reads | |
| Grigoras et al. | Accelerating SpMV on FPGAs by compressing nonzero values | |
| US11704286B2 (en) | High-density compression method and computing system | |
| US8972200B2 (en) | Compression of genomic data | |
| US20220382717A1 (en) | Content-based dynamic hybrid data compression | |
| JP2024009919A (en) | Device for storing storage object data | |
| WO2019080670A1 (en) | Gene sequencing data compression method and decompression method, system, and computer readable medium | |
| CN110995273B (en) | Data compression method, device, equipment and medium for power database | |
| US8515882B2 (en) | Efficient storage of individuals for optimization simulation | |
| US20180041224A1 (en) | Data value suffix bit level compression | |
| CN102270262A (en) | Method and device for compressing and decompressing analog waveform of integrated circuit | |
| JP5549177B2 (en) | Compression program, method and apparatus, and decompression program, method and apparatus | |
| CN113687773A (en) | Data compression model training method and device and storage medium | |
| CN117194488A (en) | Cell expression data processing method and device and electronic equipment | |
| WO2023159820A1 (en) | Image compression method, image decompression method, and apparatuses | |
| CN108829872A (en) | Immediate processing method, equipment, system and the storage medium of lossless compression file | |
| Li et al. | Erasing-based lossless compression method for streaming floating-point time series | |
| CN116418348A (en) | Data compression method, device, equipment and storage medium | |
| CN114282141A (en) | Processing method and device for compression format data, electronic equipment and readable storage medium | |
| Yan et al. | RabbitQCPlus 2.0: More efficient and versatile quality control for sequencing data | |
| Kumar et al. | Effective bitstream compression approaches for high speed digital systems | |
| US8819093B2 (en) | Systems and methods to reduce I/O and/or speed up out-of-core linear solvers | |
| Liu et al. | A versatile compression method for floating-point data stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |