KR102494833B1 - Preprocessing and convolutional operation apparatus for clinical decision-making artificial intelligence development using hypercubic shapes based on bio data - Google Patents

Abstract

The embodiments of the present invention provide a data processing device and method, which converts multi-dimensional initial data into a table-type data structure, and can apply a neural network model to hypercubic data through a method of calculating between data matching a table and a designed filter. The data processing method includes the steps of: preprocessing initial data into table-based conversion data; and applying the filter of the neural network model to table-based transformed data.

Description

Preprocessing and convolutional operation device for clinical decision-making artificial intelligence development using hypercubic shape based on biometric data

The technical field to which the present invention belongs relates to biometric data preprocessing and machine learning.

The contents described in this part merely provide background information on the present embodiment and do not constitute prior art.

FCS (flow cytometry standard) data originating from medical and biological analysis equipment detects the optical/electromagnetic properties of individual cells (or particles with similar physical, hydrodynamic, and optical properties; hereinafter referred to as cells) in a sample and biomarker data showing the results of flow cytometry/image cytometry that quantitatively analyzes the number and properties of cells. These data are used as indicators to find correlations with various disease groups.

Existing FCS data analysis is a process of selecting/isolating and counting specific cells (groups) or extracting cell properties (size, structure, antigenic phenotype) associated with optical measurements.

There are no known examples of applying machine learning to comprehensively analyze the overall morphological characteristics of FCS data and find the biological/clinical meaning of each sample.

Among machine learning, the CNN (Convolutional Neural Network) algorithm performs an operation to apply a filter while moving an image data area in a convolution step.

KR 10-1857624 (2018.05.08)

The main purpose of the embodiments of the present invention is to apply a neural network model to hypercubic data by converting multi-dimensional initial data into a table-type data structure and operating between data matching the table and a designed filter.

Other non-specified objects of the present invention may be additionally considered within the scope that can be easily inferred from the following detailed description and effects thereof.

According to an aspect of the present embodiment, a data processing method includes pre-processing initial data into table-based conversion data; It provides a data processing method comprising applying a filter of a neural network model to the table-based conversion data.

According to another aspect of the present embodiment, in a data processing device including a processor, the processor preprocesses initial data into table-based conversion data, and applies a filter of a neural network model to the table-based conversion data. A data processing device is provided.

According to another aspect of the present embodiment, in the disease diagnosis method performed by a computing device including one or more processors and a memory storing one or more programs executed by the processors, the computing device includes: a data acquisition step of acquiring extracted biometric extraction data; a data preprocessing step of transforming initial data generated based on a plurality of parameters included in the biometric data into coordinate values for each of a plurality of channels and reconstructing the transformed data into training data; a data learning step of extracting feature values from the reconstructed training data and performing learning by classifying the feature values; and a disease diagnosis step of diagnosing a specific disease using the learned feature value.

As described above, according to the embodiments of the present invention, a neural network model is formed on hypercubic data through a method of converting multi-dimensional initial data into a table-type data structure and operating between data matching the table and a designed filter. There are effects that can be applied.

Even if the effects are not explicitly mentioned here, the effects described in the following specification expected by the technical features of the present invention and their provisional effects are treated as described in the specification of the present invention.

1 is a diagram illustrating a data processing apparatus according to an embodiment of the present invention.
2 is a diagram illustrating table-based conversion data output by a data processing apparatus according to an embodiment of the present invention.
3 is a diagram illustrating 2D data and 2D filters that can be processed by the data processing apparatus according to an embodiment of the present invention.
4 is a diagram illustrating conversion data in the form of a table for two-dimensional data processed by a data processing apparatus according to an embodiment of the present invention.
5 is a diagram illustrating an operation between 2D data and a 2D filter that can be processed by a data processing apparatus according to an embodiment of the present invention.
6 is a diagram illustrating an operation in which a data processing apparatus according to an embodiment of the present invention performs an operation based on converted data in the form of a table for two-dimensional data.
7 is a diagram illustrating 3D data and 3D filters that can be processed by the data processing apparatus according to an embodiment of the present invention.
8 is a diagram illustrating conversion data in the form of a table for 3D data processed by a data processing apparatus according to an embodiment of the present invention.
9 is a diagram illustrating an operation between 3D data and a 3D filter that can be processed by the data processing apparatus according to an embodiment of the present invention.
10 is a diagram illustrating an operation in which a data processing apparatus according to an embodiment of the present invention performs an operation based on conversion data in the form of a table for 3D data.
11 is a diagram illustrating an operation of designing and arranging a filter frame according to a data processing apparatus according to an embodiment of the present invention.
12 is a diagram illustrating an operation in which a data processing apparatus according to an embodiment of the present invention expands and designs a filter frame according to an increase in dimension.
13 is a diagram illustrating an operation of fractal expansion of a filter frame according to a dimension increase of a data processing apparatus according to an embodiment of the present invention.
14 is an exemplary view in which a data processing apparatus according to an embodiment of the present invention expresses movement of a convolution filter in a 3D cubic space as an operation of performing downward shift and skip of a row group in a cubic table.
15 and 16 are diagrams illustrating a data processing method according to another embodiment of the present invention.
17 is an exemplary diagram for explaining a conventional analysis operation of biological extraction data.
18 is a schematic block diagram of an apparatus for diagnosing a disease based on biological extraction data according to another embodiment of the present invention.
19 is a block diagram schematically illustrating an operational configuration of a processor in an apparatus for diagnosing a disease according to another embodiment of the present invention.
20 is a flowchart illustrating a method for diagnosing a disease based on biological extraction data according to another embodiment of the present invention.
21 is an exemplary diagram for explaining an operation of diagnosing a disease using patient information and biometric extraction data according to another embodiment of the present invention.
22 is a block diagram for explaining an operation of diagnosing a disease using a neural network according to another embodiment of the present invention.
23 is an exemplary diagram for explaining an operating process of a diagnosis device in a computer according to another embodiment of the present invention.
24 to 25 are exemplary diagrams for explaining an operation of generating initial data based on biologically extracted data according to another embodiment of the present invention.
26 to 29 are exemplary diagrams illustrating initial data of each of a plurality of channels according to another embodiment of the present invention.
30 and 31 are exemplary diagrams for explaining an operation of modifying basic data based on biologically extracted data according to another embodiment of the present invention.
32 is a diagram for explaining an operation of reconstructing data based on biologically extracted data according to another embodiment of the present invention by way of example.

Hereinafter, in the description of the present invention, if it is determined that a related known function may unnecessarily obscure the subject matter of the present invention as an obvious matter to those skilled in the art, the detailed description thereof will be omitted, and some embodiments of the present invention will be described. It will be described in detail through exemplary drawings.

The present invention relates to a method for diagnosing a disease by pre-processing bio-extracted data and an apparatus therefor.

The present invention relates to a device developed as a module of diagnostic equipment to utilize raw data in the form of FCS (Flow Cytometry Standard) originating from biomedical analysis equipment for clinical decision-making using a visual recognition artificial intelligence algorithm.

The present invention shapes high-dimensional FCS data into a supercubic space and applies it to existing visual recognition artificial intelligence algorithms. Preprocessing is performed to apply the data converted into a supercubic shape to the visual recognition CNN algorithm.

Existing CNN algorithms apply convolution filters to two-dimensional image data regions having height, width, and color, so it is not easy to apply convolution filters to high-dimensional hypercubic shape data with only existing CNN algorithms.

The present invention applies a supercube convolution filter across the entire high-dimensional supercube, and combines it with high-dimensional FCS raw data hypercube conversion technology to be used as a visual recognition artificial intelligence-based clinical decision-making diagnosis equipment.

FCS data originating from medical and biological analysis equipment detects the optical/electromagnetic properties of individual cells (or particles with similar physical, hydrodynamic, and optical properties) in a sample and quantitatively analyzes the number and properties of cells therefrom. It is biomarker data showing the results of flow cytometry/image cytometry. These data are used as indicators to find correlations with various disease groups.

However, there is no known case of applying machine learning to comprehensively analyze the overall morphological characteristics of FCS data and find the biological/clinical meaning of each sample.

The present invention converts FCS data, which is analysis data generated as a result of clinical information or biological experiments generated in the process of observing patients' diseases and progress, into hypercubic data to enable image analysis machine learning, and converts the hypercubic data into convolution ( It enables visual recognition machine learning by applying convolution preprocessing, and provides a device that finds patterns related to various diseases [eg, blood cancer] or biological characteristics.

There is no known technology for converting FCS data into a supercubic shape, performing convolution processing on 4-dimensional or higher-dimensional data for this data, and applying CNN. Medical/biological analysis FCS data conversion Even when hypercubic shapes are not considered, there are no known examples of applying convolution processing and CNN machine learning to general multivariate data corresponding to 4-dimensional or higher-dimensional shapes.

By promoting the development of FCS data machine learning models for clinical prediction through the present invention, it is possible to break away from the conventional disease diagnosis method based on piecemeal numerical comparison and to enable contextual and integrated interpretation of the results of automated blood analysis and flow cytometry analysis. It can help in accurate disease diagnosis and understanding the clinical situation.

As the FCS data patterns having clinical usefulness are discovered through the present invention, medical innovation is possible to quickly diagnose or identify patients by discovering abnormalities in patients that doctors did not recognize at an early stage. Compared to disease-specific tests, automated blood analysis can be used to track disease progression and changes in patients' conditions, which can contribute to improving the efficiency of medical resource distribution.

The present invention facilitates biological and medical research by facilitating the development of new algorithms that automate the reading of existing flow cytometry test results, which mainly rely on handwriting by analysts.

FCS data in the medical field is being produced stably and in large quantities due to the implementation of automated blood analysis tests, which are common tests. In addition, due to the well-established regional and international quality control system of clinical pathology tests, the mechanical performance is being maintained with a very high level of standardization achieved.

Therefore, it is clear that flow cytometry-derived FCS data, including automated blood analysis tests, are very suitable base data for the development of machine learning algorithms aimed at clinical use.

The FCS data conversion method, which is the content of the present invention, has great industrial and academic value in that it can open the door to a new field of medical machine learning. Furthermore, machine learning can be made possible for other high-dimensional data.

1 is a diagram illustrating a data processing apparatus according to an embodiment of the present invention.

The device 11 includes at least one processor 120 , a computer readable storage medium 13 and a communication bus 17 .

The processor 120 may control the device 11 to operate. For example, processor 12 may execute one or more programs stored in computer readable storage medium 130 . The one or more programs may include one or more computer executable instructions, which when executed by processor 12 may cause device 11 to perform operations in accordance with an illustrative embodiment. .

Computer readable storage medium 13 is configured to store computer executable instructions or program code, program data and/or other suitable form of information. Computer executable instructions or program code, program data and/or other suitable forms of information may also be provided via input/output interface 15 or communication interface 16. The program 14 stored on the computer readable storage medium 13 includes a set of instructions executable by the processor 12 . In one embodiment, computer readable storage medium 13 includes memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other types of storage media that can be accessed by the data processing apparatus 11 and store desired information, or suitable combinations thereof.

Communication bus 17 interconnects various other components of data processing device 11, including processor 12 and computer readable storage medium 13.

Device 11 may also include one or more input/output interfaces 15 and one or more communication interfaces 16 providing interfaces for one or more input/output devices. Input/output interface 15 and communication interface 16 are connected to communication bus 17 . An input/output device (not shown) may be connected to other components of the device 11 via an input/output interface 15 .

The processor 12 of the data processing device 11 preprocesses the initial data into table-based conversion data, and applies a filter of the neural network model to the table-based conversion data.

The processor 12 converts a first data structure formed of N-dimensional data by N axes (where N is a natural number equal to or greater than 2) into a second data structure formed in the form of a table.

The first data structure may include a hypercube having depth information of 4 dimensions or more. The first data structure may include flow cytometry of a clinical sample such as blood or a biological analysis sample, and bio-extracted data representing a measurement result of an analysis technique using flow cytometry. The biological extraction data may be expressed in a pre-set standardized format or a flow cytometry standard (FCS) format.

In the second data structure, (i) coordinate information corresponding to the N axes and (ii) value information matching the coordinate information may be arranged in a row direction or a column direction. The second data structure may merge measurement values of some parameters of biometric extraction data, transform them into data including coordinate values for channels, and include the transformed data and count values.

In order to apply the neural network model to the first data structure, the processor designs a second data structure and a arithmetic filter frame structure in which dimensions are expressed. The processor may set the start position of the filter frame structure by arranging the filter center of the filter frame structure based on preset coordinates. The processor may expand the filter weight elements of the filter frame structure in a dimension-dependent fractal method based on a row direction or a column direction in consideration of the dimension of the first data structure.

The processor may perform an operation between matching elements by moving the filter frame structure in a row direction or column direction of the table of the second data structure. The processor may skip the operation when the filter center of the filter frame structure satisfies a preset row condition or column condition.

2 is a diagram illustrating table-based conversion data output by a data processing apparatus according to an embodiment of the present invention.

In this embodiment, a table having two types of columns (or rows) for coordinates and grayscale density values of supercubic voxels expresses the format of a supercubic space of a certain standard. A convolution filter has the same dimensions as the hypercubic space to be convolved, but a much smaller size.

Referring to FIG. 2, it represents a 6-dimensional hypercubic space. The size of each dimension is 5 (including arbitrary units for size). Assign 6 axes to each of the 6 dimensions. The location of each voxel is indicated by coordinates. Coordinates have 6 components, each component being a projection position in that dimension. Each voxel with its coordinates and density values occupies a row (or column) of this table. 2, they are arranged in a row direction, but may be arranged in a column direction according to design. That is, the row data and the column data may be arranged in a permuted state (diagonal movement).

All voxels constituting the hypercubic space are arranged in a specific order. First, in the top 5 rows of the table, the positions (or coordinate values) of the 1st to 5th dimensions are 0 and the positions (or coordinate values) of the 6th dimension are 0 to 4 ((0,0,0,0, 0,0), ... ,(0,0,0,0,0,4)) shows grayscale density values of voxels. The position (coordinate value) of the 5th dimension of the voxels in the next row changes to 1, and the position (coordinate value) of the 6th dimension moves from 0 to 4 in the same way as before ((0,0,0,0,1 ,0), ... , (0,0,0,0,1,4)). Next, for the 5th dimension positions (coordinate values) 2 to 4, after adding rows with the 6th dimension positions (coordinate values) 0 to 4, respectively, the 4th dimension positions (coordinate values) 1 to 4 ((0, Repeat the entire process described above for 0,0,1,0,0), ... , (0,0,0,4,0,0)). Add voxels to table rows in this way (move their position in each dimension). The table-type data structure can express information about all 5 ⁶ voxels from (0,0,0,0,0,0) to (4,4,4,4,4,4) at the location of the supercube. there is.

Convolution assigns a weight to each filter voxel, multiplies and sums the grayscale density values of the filter voxel and the overlapped hypercube voxel. Since each row of the table representing the hypercubic space corresponds to a voxel, voxels overlapping the filter can be designated as a row group. When the filter passes through the supercube in a scanning manner, it is overlapped with voxels in other hypercube spaces at each movement step, and the overlapped voxels form a separate row group at each step. At each movement step, the row group corresponding to the supercubic space voxels overlapped with the filter shows a specific topology or pattern in the table, and its position is changed along with the movement of the convolution filter. You can start with images and cubes and progress to higher-dimensional hypercubes.

3 is a diagram illustrating 2D data and a 2D filter that can be processed by a data processing device according to an embodiment of the present invention, and FIG. 4 is a 2D data processing device processed by the data processing device according to an embodiment of the present invention. It is a diagram illustrating conversion data in the form of a table for data.

The first example is a 5 x 5 image. Columns labeled Axis-1 and Axis-2 show the projected positions (or coordinate values) of pixels in dimension-1 and dimension-2. The order of listing the rows with location and concentration information is as described above. Next, we show the application of a 3 x 3 convolutional filter to an image.

5 is a diagram illustrating an operation between 2D data and a 2D filter that can be processed by a data processing apparatus according to an embodiment of the present invention, and FIG. 6 is a data processing apparatus according to an embodiment of the present invention. It is a diagram illustrating an operation of performing an operation based on conversion data in the form of a table for data.

The embodiment follows the rule of no special treatment for edges. First, set the center of the filter to the (1,1) position of the image and perform convolution. Filters and images are represented as superimposed matrices. The filter then moves in the axis-1 direction for the next convolution step. Convolution is simply multiplying the weight of a filter pixel by the intensity of the overlapping image pixels and summing the products.

The first convolution produces 10 (=0x0 + 1x1 + 0x2 + 1x0 + 2x2 + 1x4 + 0x1 + 1x1 + 0x6). The second convolution convolution is 21 (=0x1 + 1x2 + 0x1 + 1x2 + 2x4 + 1x3 + 0x1 + 1x6 + 0x2). Adds a filter column to the original table and displays the weights assigned to the filter pixels. In the filter column, specify the row group corresponding to the image pixels that overlap the filter pixels.

Rows 1, 2, 3, 6, 7, 8, 11, 12, and 13 in the table represent the image pixels overlapping the filter in the first convolution step. In the table representation of the convolution, each filter weight is multiplied by the concentration of the same row.

The numbers in the filter column, labeled FW (filter weights)1 and FW2 respectively, indicate that the filters are two-dimensional. Some of the filters in column FW2 may extend beyond two dimensions.

7 is a diagram illustrating 3D data and a 3D filter that can be processed by a data processing device according to an embodiment of the present invention, and FIG. 8 is a 3D data processing device processed by the data processing device according to an embodiment of the present invention. It is a diagram illustrating conversion data in the form of a table for data.

The second example is a cube measuring 5 x 5 x 5. Apply a 3 x 3 x 3 filter to the cube and show how it can be represented in a table.

9 is a diagram illustrating an operation between 3D data and a 3D filter that can be processed by a data processing apparatus according to an embodiment of the present invention, and FIG. 10 is a diagram illustrating a data processing apparatus according to an embodiment of the present invention It is a diagram illustrating an operation of performing an operation based on conversion data in the form of a table for data.

The filter center is located at (1,1,1) in the cubic space and the convolution is performed. The filter then moves in the axis-1 direction for the next convolution step. The filter center is now at (2,1,1) in cubic space.

The value of the first convolution is 39(= 0x2 + 0x0 + 0x1 + 0x1 + 1x4 + 0x6 + 0x2 + 0x13 + 0x18 + 0x3 + 1x0 + 0x1 + 1x1 + 2x4 + 1x6 + 0x2 + 1x16 + 0x24 + 0x1 + 0x0 + 0x1 + 0x0 + 1x4 + 0x8 + 0x1 + 0x12 + 0x36).

The value of the second convolution is 61 (= 0x0 + 0x1 + 0x1 + 0x4 + 1x6 + 0x4 + 0x13 + 0x18 + 0x8 + 0x0 + 1x1 + 0x1 + 1x4 + 2x6 + 1x6 + 0x16 + 1x24 + 0x8 + 0x1 + 0x1 + 0x1 + 0x4 + 1x8 + 0x10 + 0x12 + 0x36 + 0x12).

Adds a filter column to the original table and displays the weights assigned to the filter voxels in the column. Cubic voxels (and corresponding rows) overlapping the filter voxel are designated in the filter column. Rows 1, 2, 3, 6, 7, 8, 11, 12, and 13 in the table represent the voxels of the cube that overlap the filter in the first step of the convolution. In the table representation of the convolution, each filter weight is multiplied by the concentration of the same row.

11 is a diagram illustrating an operation of designing and arranging a filter frame according to a data processing apparatus according to an embodiment of the present invention.

We present an example of convolution in n-dimensional hypercubic space. Referring to FIG. 11 , an example of a convolution filter framework (rows/columns of filter voxels) in a table can be seen.

A simple method for determining the coordinates of a filter voxel in a hypercubic space of a certain standard is as follows. In the example, the number of voxels in each dimension is set to an odd number in order to easily find the filter center position. It also describes how to use convolutional filters that have the same size (expressed in number of voxels) in all dimensions. For example, consider an n-dimensional hypercubic space where each dimension is k _i (i = 1, ... , n) and a 3 ⁿ hypercubic convolution filter where each dimension is 3.

The following is a method of selecting coordinates of convolution filter voxels in hypercube space. An arbitrary point (x ₁ , ... , x _n ) in hypercube space is set as the filter center. If the coordinates of 3 ⁿ filter voxels are expressed as (X ₁ , ... , X _n ), X ₁ is x ₁ -1, x ₁ , x ₁ +1, X ₂ is x ₂ -1, x ₂ , x ₂ +1, ... , X _n can have the values x _n -1, x _n , x _n +1. Since the coordinates for each dimension have three values, a total of 3 ⁿ coordinates are possible, and each represents the position of a filter voxel and supercubic voxels overlapping the filter. Selecting the hypercube table rows having these coordinates results in a row group corresponding to the filter overlapping hypercube voxel. If the initial convolution stage filter center coordinates are (1, ... , 1), the row with coordinates (0 or 1 or 2, ... , 0 or 1 or 2) becomes the coordinates of the remaining filter voxels. By selecting these rows, you can determine the filter frame.

The following is a method of directly obtaining the filter superposition row group from the row (central row) to which the filter center is assigned in the hypercube space table.

First get the two rows of the table adjacent to the center row. These three selected rows correspond to a 3-unit-long line segment along the axis of a particular dimension (dimension-1 in this case). Next, a total of nine rows are selected by adding three rows spaced apart by k ₁ above and below the three rows in the previous step. This operation creates a 3 ² -sized square within the hypercube that shares its centroid with the filter. Next, add 9 rows k ₁ xk ₂ above and below the previous 9 rows to select a total of 27 rows. This operation creates a 3 ³ -sized cube concentric with the convolution filter. In the next step, a 3 ⁴ -size 4-dimensional hypercube is created by selecting 27 rows k ₁ xk ₂ xk ₃ above and below each of the 27 rows.

The same operation on the table is repeated until 3 ⁿ rows corresponding to a 3 ⁿ -size convolutional hypercubic filter are selected.

12 is a diagram illustrating an operation in which a data processing apparatus according to an embodiment of the present invention expands and designs a filter frame according to an increase in dimension.

Referring to FIG. 12, it shows how the framework structure of the convolution filter expands in the table as the dimension increases. Here, the size of the convolution filter is 3 for each dimension and the topological structure of the filter (nested) rows in the table is extended in the same pattern in a fractal way. As the dimension increases by 1, it adds the same current structure above and below itself.

13 is a diagram illustrating an operation of fractal expansion of a filter frame according to a dimension increase of a data processing apparatus according to an embodiment of the present invention. In the case of a 2D filter, a row is added at the center of the filter, 3 rows above and below the 3 rows, and a row separated by k ₁ above and below the 3 rows. In the case of a 3D filter, the same frame rows as the 2D filter are added at a distance of k ₁ xk ₂ rows above and below. In the case of 4D, 3D filter frames are added at a spacing of k ₁ xk ₂ xk ₃ rows.

Expressed in general terms, in the case of n-dimensional, (n-1)-dimensional filter frames are added above and below the (n-1)-dimensional filter frame itself at a distance of k ₁ x ... xk _n-1 rows above and below it. . 3 ⁿ rows can be selected for an n-dimensional convolution filter in a fractal method according to dimension expansion.

The calculation for the convolution is the same as for the low-dimensional space and filter, multiplying the grayscale density of the supercubic voxel by the weight of the overlapping filter voxel and summing the products.

The following describes how the filter frame of the table is changed when the filter scans the supercubic space.

First, we need to determine the starting point of the convolution, that is, the initial location of the center of the filter.

For a convolution filter of size ⁿ 3, the starting point expressed as coordinates is (1, ... , 1). The row with these coordinates in the table is the starting point for the center of the filter.

The 3 ⁿ rows corresponding to the filter frame maintain the framework topology within the table as the filter moves in hypercubic space. One voxel movement of the filter in hypercube space equals one row downward movement of the filter frame in the table representation. The table can be configured so that the filter frame continues to move down the table as it scans the hypercubic space. You can also organize tables in other directions. If data is recorded in the column direction of the table, one voxel movement of the filter in the hypercubic space may be configured to be the same as lateral movement.

14 is an exemplary view in which a data processing apparatus according to an embodiment of the present invention expresses movement of a convolution filter in a 3D cubic space as an operation of performing downward shift and skip of a row group in a cubic table.

The default action is to shift down one line (or to the side of one column).

When the filter reaches the edge of a certain dimension, the next scan move is special. The filter moves the position one by one in the next higher dimension. And the filter returns to the starting position of the current dimension. This type of scanning movement is represented by moving the filter frame down one or more rows in the table. If no edge processing is applied, there is no convolutional product when any part of the filter crosses the end of the space. Non-productive locations of filters should not appear in the final convolution result.

Therefore, it is necessary to skip a certain number of filter frame positions in the table, which results in a downward movement of the filter frame greater than one row.

A filter can still be at the end of a dimension in successive moves. For example, when scanning along the surface of a room. In this case a much more significant movement of the filter frame position is required.

In a simple way, we can set the movement pattern to the coordinates of the center of the filter frame. You can pre-determine the row corresponding to the unproductive filter center position (coordinates). Rows whose coordinates contain one of 0, k ₁ -1, ... , k _n -1 are listed. The filter frame skips all convolution steps centered on this row. Frame movement continues through a regular skipping operation until the filter center reaches an end point (k ₁ -2, ... , k _n -2).

For example, if the filter center of dimension 1 is 0 or k ₁ -1, one row is skipped. If the filter centers of dimensions 1 and 2 are (0 or k ₁ -1) and (0 or k ₂ -1), then k ₁ rows are skipped. Skip k ₁ xk ₂ rows if the filter centers of dimensions 1, 2, and 3 are (0 or k ₁ -1), (0 or k ₂ -1), and (0 or k ₃ -1), respectively.

Referring to Fig. 14, it shows intermittent skipping of the convolution as the filter frame moves down one or more rows. Large columns with numbers indicate convolution steps.

15 and 16 are diagrams illustrating a data processing method according to another embodiment of the present invention.

The data processing method may be performed by a data processing device.

The data processing method includes pre-processing initial data into table-based conversion data (S21) and applying a neural network model filter to the table-based conversion data (S22).

The preprocessing step (S21) may include converting a first data structure formed of N-dimensional data by N (N is a natural number of 2 or more) axes into a second data structure formed in the form of a table (S31). there is.

The first data structure may include a hypercube having depth information of 4 dimensions or more. The first data structure includes bio-extracted data representing a result of flow cytometry of blood, and the bio-extracted data may be expressed in a pre-set standardized format or a flow cytometry standard (FCS) format.

In the second data structure, (i) coordinate information corresponding to the N axes and (ii) value information matching the coordinate information may be arranged in a row direction or a column direction. The second data structure may merge measurement values of some parameters of biometric extraction data, transform them into data including coordinate values for channels, and include the transformed data and count values.

The preprocessing step (S21) may include designing a filter frame structure in which dimensions are expressed and operable with the second data structure in order to apply the neural network model to the first data structure (S32).

In the step of designing the filter frame structure ( S32 ), the start position of the filter frame structure may be set by arranging the filter center of the filter frame structure based on preset coordinates.

In the step of designing the filter frame structure ( S32 ), the filter weight elements of the filter frame structure may be expanded in a fractal method according to a dimension in a row direction or a column direction in consideration of the dimension of the first data structure.

The step of applying the filter (S22) may include a step (S33) of performing an operation between matching elements by moving the filter frame structure based on the row direction or column direction of the table of the second data structure. .

In the step of applying the filter ( S22 ), if the filter center of the filter frame structure satisfies a preset row condition or column condition, the operation may be skipped.

17 is an exemplary diagram for explaining a conventional analysis operation of biological extraction data.

In general, the FCS data analysis method, as shown in FIG. 17, carefully selects/separates cells (clusters) to be analyzed based on the analyst's academic knowledge, counts the selected cells, or measures optical properties (e.g., : light scattering intensity, fluorescence emission) and related biological properties (eg size, structure, antigenic phenotype) are extracted.

18 is a schematic block diagram of an apparatus for diagnosing diseases based on biological extraction data according to an embodiment of the present invention.

The disease diagnosis apparatus 100 according to the present embodiment includes an input unit 110, an output unit 120, a processor 200, a memory 300, and a database 400. The disease diagnosis apparatus 100 of FIG. 2 is according to an embodiment, and all blocks shown in FIG. 18 are not essential components, and some blocks included in the disease diagnosis apparatus 100 in another embodiment are added or changed. or can be deleted. Meanwhile, each component included in the disease diagnosis apparatus 100 may be implemented as a separate software device or as a separate hardware device combined with software.

The disease diagnosis apparatus 100 automatically preprocesses FCS (Flow Cytometry Standard) data as learning data, uses the preprocessed data as data for machine learning and artificial intelligence diagnosis models, and through machine learning, the feature values of various diseases ( feature), and identify the correlation between feature values and disease to generate a predictive diagnosis model or to diagnose a specific disease.

The input unit 110 means means for inputting or obtaining data for controlling the disease diagnosis apparatus 100 . The input unit 110 may interwork with the processor 200 to input various types of control signals, or may directly acquire data by interworking with an external device and transmit it to the processor 200 .

The output unit 120 may display various information such as data preprocessing results, learning results, and diagnosis results in conjunction with the processor 200 . The output unit 120 preferably displays various information through a display (not shown) provided in the disease diagnosis apparatus 100, but is not necessarily limited thereto.

The processor 200 performs a function of executing at least one instruction or program included in the memory 300 .

The processor 200 according to the present embodiment performs data pre-processing based on biometric extraction data obtained from the input unit 110 or the database 400, and performs machine learning for disease diagnosis based on the pre-processed data. . Also, the processor 200 may diagnose a disease of a diagnosis target based on a pre-learned learning result. A detailed operation of the processor 200 according to this embodiment will be described with reference to FIG. 3 . Here, the bio-extracted data is preferably bio-extracted FCS (Flow Cytometry Standard) raw data, but is not necessarily limited thereto.

The memory 300 includes at least one instruction or program executable by the processor 200 . The memory 300 may include instructions or programs for preprocessing data based on biometric data. Also, the memory 300 may include instructions or programs for performing machine learning based on preprocessed data. Also, the memory 300 may include instructions or programs for an operation of diagnosing a disease of a diagnosis target based on a learning result.

The database 400 refers to a general data structure implemented in the storage space (hard disk or memory) of a computer system using a database management program (DBMS), such as data search (extraction), deletion, editing, addition, etc. It means a data storage format that can freely perform, Oracle, Informix, Sybase, Relational Database Management System (RDBMS) such as DB2, Gemston, Orion ( Orion), O2, etc., and XML Native Databases, such as Excelon, Tamino, Sekaiju, etc. It can be implemented according to the purpose and has appropriate fields or elements to achieve its function.

The database 400 according to this embodiment may store information related to bio-extracted data and provide bio-extracted data and information related to bio-extracted data. The biological extraction data stored in the database 400 may be data representing a result of flow cytometry of blood. The biological extraction data is preferably data in a predetermined standardized format or FCS (Flow Cytometry Standard) format, but is not necessarily limited thereto.

The database 140 is described as being implemented in the disease diagnosis device 100, but is not necessarily limited thereto, and may be implemented as a separate data storage device.

19 is a block diagram schematically illustrating an operational configuration of a processor in a disease diagnosis apparatus according to an embodiment of the present invention.

The processor 200 included in the disease diagnosis apparatus 100 according to the present embodiment includes a data acquisition unit 210, a data pre-processing unit 220, a data learning unit 230, and a disease diagnosis unit 240. The processor 200 of FIG. 19 is according to one embodiment, and all blocks shown in FIG. 19 are not essential components, and some blocks included in the processor 200 may be added, changed, or deleted in other embodiments. there is. Meanwhile, each component included in the processor 200 may be implemented as a separate software device or a separate hardware device combined with software.

The data acquisition unit 210 performs an operation of obtaining biological extraction data extracted from the blood of a subject to be diagnosed. Here, the biological extraction data may be data representing a result of flow cytometry of blood. The biological extraction data is preferably data in a predetermined standardized format or FCS (Flow Cytometry Standard) format, but is not necessarily limited thereto.

The data acquisition unit 210 may obtain biological extraction data through the input unit 110 or the database 400 that interoperates with the processor 200 . Here, when the data acquisition unit 210 acquires bio-extracted data from the database 400 interlocking with the processor 200, the bio-extracted data is automatically collected at predetermined intervals or inputted through the input unit 110. A data request signal may be transmitted to the database 400 to collect biological extraction data.

The data pre-processing unit 220 transforms initial data generated based on a plurality of parameters included in biometric data into coordinate values for each of a plurality of channels, and reconstructs the transformed data into training data. The data pre-processing unit 220 according to the present embodiment includes an initial data generating unit 222, a data transforming unit 224, and a data reconstruction unit 226.

The initial data generation unit 222 generates initial data using measurement values of all or some parameters of a plurality of parameters of a test item channel included in the biometric extraction data.

The initial data generator 222 may generate the initial data using the measurement values for each of at least two or more parameters among a plurality of parameters.

The data transformation unit 224 merges all or partial parameter measurement values included in the initial data without processing, transforms them into data including coordinate values for each of the test item channels, and transforms each of the transformed data and the transformed data. Create a data table containing count values for .

In addition, the data transforming unit 224 transforms the data by replacing the measured values of all or some parameters included in the initial data with a quotient divided by a predetermined value (eg, a specific value such as 4, 8, 32, etc.) image depth conversion), and a method of adding a predetermined value (eg, 10) to each quotient to prevent loss of data that occurs at this time.

A data table including the transformed data and count values for each of the transformed data is created.

The data transforming unit 224 transforms the measured values of some parameters into transformed data including coordinate values generated by merging sequentially or in a predetermined order.

In addition, the data transforming unit 224 deletes the same coordinate value when the same coordinate value as the coordinate value included in the transformed data exists, and increases the count value for the coordinate value by a preset unit to obtain a count value and generate the data table including modified data and the updated count value.

The data reconstruction unit 226 performs an operation of reconstructing the data table for machine learning using the transformed data included in the data table.

(n_i는 기 설정된 기준 크기값 이상의 자연수) 형태의 기계학습용 이미지(데이터 테이블)로 재구성할 수 있다. 여기서, 재구성된 기계학습용 이미지(데이터 테이블)는 2 차원 또는 3 차원의 형태일 수 있다. The data restructuring unit 226 configures the coordinate values included in the transformed data into 1-dimensional coordinate values, and in the process of constructing the 1-dimensional coordinate values, fills the part where the coordinate values do not exist with 0 values, or the coordinate values using a method to display only the parts that exist

It can be reconstructed into an image (data table) for machine learning in the form of (n _i is a natural number equal to or greater than a predetermined standard size value). Here, the reconstructed image for machine learning (data table) may be in a two-dimensional or three-dimensional form.

Although the data pre-processing unit 220 according to this embodiment is described as being included in the disease diagnosis apparatus 100, it is not necessarily limited thereto, and may be implemented as a separate device from the disease diagnosis apparatus 100. For example, the data pre-processing unit 220 may be implemented as a separate device such as a data pre-processing device (not shown) that converts biological extraction data into machine learning data for diagnosis, and the data pre-processing device (not shown) It can be linked with a device that diagnoses a disease by performing learning in various forms.

The data learning unit 230 extracts feature values from the reconstructed learning data, classifies the extracted feature values, and performs learning for disease diagnosis. The data learning unit 230 according to the present embodiment includes a feature extraction unit 232 and a feature classification unit 234.

The feature extraction unit 232 extracts feature values from the reconstructed data included in the data table for machine learning using a synthetic network algorithm.

The feature classification unit 234 classifies feature values for each specific disease and performs learning.

The disease diagnosis unit 240 performs an operation of diagnosing a specific disease using the learned feature value. When new information about a diagnosis target is input, the disease diagnosis unit 240 diagnoses the disease by comparing the new information with feature values of a specific disease.

20 is a flowchart illustrating a method for diagnosing a disease based on biological extraction data according to an embodiment of the present invention.

The disease diagnosis apparatus 100 obtains biological extraction data extracted from the blood of a diagnosis subject (S310). Here, the biological extraction data may be data representing a result of flow cytometry of blood. The biological extraction data is preferably data in a predetermined standardized format or FCS (Flow Cytometry Standard) format, but is not necessarily limited thereto.

The disease diagnosis apparatus 100 generates initial data based on biologically extracted data (S320). The disease diagnosis apparatus 100 generates initial data using measurement values of all or some parameters of a plurality of parameters of a test item channel included in bio-extracted data.

The disease diagnosis apparatus 100 generates a data table by transforming data included in the initial data (S330). The disease diagnosis apparatus 100 merges measurement values of some parameters included in the initial data and transforms them into data including coordinate values for each of the test item channels, and transforms the transformed data and count values for each of the transformed data Create a data table containing

The disease diagnosis apparatus 100 generates a data table for machine learning by reconstructing the transformed data included in the data table (S340).

(n_i는 기 설정된 기준 크기값 이상의 자연수) 형태의 기계학습용 이미지(데이터 테이블)로 재구성할 수 있다. The disease diagnosis apparatus 100 configures the coordinate values included in the transformed data included in the data table as 1-dimensional coordinate values, and fills the part where the coordinate values do not exist with 0 values in the process of configuring the 1-dimensional coordinate values. or by using a method of displaying only the part where coordinate values exist

It can be reconstructed into an image (data table) for machine learning in the form of (n _i is a natural number equal to or greater than a predetermined standard size value).

The disease diagnosis apparatus 100 extracts feature values from the reconstructed data included in the data table for machine learning using a synthetic network algorithm (S350).

The disease diagnosis apparatus 100 performs learning based on the feature values and classifies the feature values for each specific disease (S360).

The disease diagnosis apparatus 100 diagnoses a specific disease using the learned feature value (S370). When new information about a diagnosis target is input, the disease diagnosis apparatus 100 diagnoses the disease by comparing the new information with feature values for a specific disease.

In FIG. 20, each step is described as sequentially executed, but is not necessarily limited thereto. In other words, since it can be applied by changing and executing the steps described in FIG. 20 or by executing one or more steps in parallel, FIG. 20 is not limited to a time-series sequence.

The disease diagnosis method according to the present embodiment described in FIG. 20 may be implemented as an application (or program) and recorded on a recording medium readable by a terminal device (or computer). A recording medium in which an application (or program) for implementing the method for diagnosing a disease according to the present embodiment is recorded and which is readable by a terminal device (or computer) is any type of recording device storing data that can be read by a computing system, or includes media

21 is an exemplary diagram for explaining an operation of diagnosing a disease using patient information and biometric extraction data according to an embodiment of the present invention. Specifically, FIG. 21 is an exemplary diagram for explaining a data pre-processing process of converting patient information and biometric FCS raw data according to an embodiment of the present invention into a hypercubic shape that can be applied to visual recognition machine learning.

In the disease diagnosis apparatus 100, the data pre-processing unit 220 performs data pre-processing for machine learning.

Patient information capable of distinguishing diagnosis subjects is anonymized, and clinical examination results of the anonymized information are input to the data pre-processing unit 220 .

The data pre-processing unit 220 obtains bio-extracted data in a preset Excel format or FCS format, and expresses measurement values of a plurality of parameters included in the bio-extracted data as vector-based coordinate values to generate initial data.

The data pre-processing unit 220 merges the coordinate values of a plurality of parameters included in the initial data and transforms them into one coordinate value, and generates a data table (data frame) through counting the transformed data and each merged coordinate value. create The data pre-processing unit 220 updates the data table by READ or WRITE data stored in the database.

(n_i는 기 설정된 기준 크기값 이상의 자연수) 형태의 기계학습용 이미지(데이터 테이블)로 재구성할 수 있다. The data pre-processing unit 220 reconstructs and transforms the transformed data included in the data table. The data pre-processing unit 220 configures the coordinate values included in the transformed data included in the data table in the disease diagnosis device 100 as 1-dimensional coordinate values, and in the process of configuring the 1-dimensional coordinate values, the coordinate values do not exist. Fill in the missing part with 0 values, or use a method of displaying only the parts where coordinate values exist

It can be reconstructed into an image (data table) for machine learning in the form of (n _i is a natural number equal to or greater than a predetermined standard size value).

The data pre-processing unit 220 transfers the converted machine learning data or machine learning data table to the data learning unit 230 so that learning for diagnosing a specific disease is performed.

22 is a block diagram for explaining an operation of diagnosing a disease using a neural network according to an embodiment of the present invention.

The data learning unit 230 uses the data for machine learning configured in the data preprocessing unit 220 as input data to perform a process of image learning.

The data learning unit 230 performs an operation of detecting a feature value (Feature) from input data through a process of image learning. Here, the data learning unit 230 may detect feature values of the input data using a synthetic network algorithm based on a plurality of convolutional layers and other advanced machine learning algorithms.

The data learning unit 230 performs learning based on the detected feature values to classify feature values of a specific disease.

The disease diagnosis unit 240 may diagnose a disease based on the learning result of the data learning unit 230 . The disease diagnosis unit 240 analyzes the presence or absence of feature values extracted from a patient group with a specific disease (eg, blood cancer, etc.) pre-learned from the data when new data or data prior to machine learning for the diagnosis target is input, Depending on the presence or absence of feature values, a specific disease can be diagnosed.

23 is an exemplary diagram for explaining an operating process of a diagnosis device in a computer according to an embodiment of the present invention.

The disease diagnosis apparatus 100 according to the present embodiment may be implemented as a diagnosis apparatus 700 in a computer. The diagnosis device 700 in the computer may include a data processing unit 710, a characteristic value generator 720, an artificial intelligence unit 730, and a diagnosis unit 740.

The data processing unit 710 transforms initial data generated based on a plurality of parameters included in the biometric data into coordinate values for each of a plurality of channels, and reconstructs the transformed data into data for machine learning. Here, the data processing unit 710 may be implemented in a form including all or part of the functions of the data pre-processing unit 220 .

The feature value generation unit 720 may generate feature values extracted from the reconstructed data included in the data table for machine learning using a synthetic network algorithm and other improved machine learning algorithms. Here, the feature value generation unit 720 may be implemented in a form including some functions of the data learning unit 230 .

The artificial intelligence unit 730 performs learning based on the extracted feature values, and classifies feature values for each specific disease according to the learning result. Here, the artificial intelligence unit 730 may be implemented in a form including some functions of the data learning unit 230 .

The diagnosis unit 740 diagnoses a specific disease using the learned feature value. When new information on a diagnosis target is input, the diagnosis unit 740 diagnoses a disease by comparing the new information with a feature value for a specific disease. Here, the diagnosis unit 740 may be implemented in a form including all or some functions of the disease diagnosis unit 240 .

24 and 25 are exemplary diagrams for explaining an operation of generating initial data based on biologically extracted data according to an embodiment of the present invention.

Referring to FIG. 24 , bio-extracted data extracted from the blood of a diagnosis subject includes a plurality of parameters, and each of the plurality of parameters includes a measured value. For example, biological extraction data extracted through an automatic blood cell analyzer is divided into two to four files for each patient, sample, and analysis module of the analysis equipment, and each file lists the measured values for each analysis parameter as shown in FIG. 24. It can be implemented in the form of a table.

For example, bio-extracted data may be a set of points composed of 4-dimensional coordinates using 4 analysis parameters. However, in order to help understanding through image representation, three parameters among the four parameters included in the biological extraction data were selected, and three-dimensional coordinates were expressed as shown in FIG. 25 using the selected parameters. Here, the disease diagnosis apparatus 100 may generate initial data for data pre-processing through the selected parameters.

26 to 29 are exemplary diagrams illustrating initial data of each of a plurality of channels according to an embodiment of the present invention. 26 to 29 are exemplary diagrams showing initial data of each of a plurality of parameters (three parameters in this example) included in CBC-derived FCS data according to an embodiment of the present invention in a shape in a 3-dimensional (super)cube. The shapes in each of the 10 cubes illustrated in FIGS. 26 to 29 are visualizations of data derived from 10 specimens or 10 patients, and have similar but different morphological characteristics.

Based on the living body extraction data, three-dimensional coordinate points may be graphed as plots as shown in FIGS. 26 to 29 . The plot patterns of these coordinate points are similar for each patient/specimen, but show subtle differences. For example, since the automatic blood analysis equipment simultaneously performs individual analysis through 2 to 4 channels (or modules), it can generate 2 to 4 FCS data per sample.

26 to 29 enumerate three parameters among four parameters (FCS, FCSW, SSC, SFL: 4 dimensions) of FCS data for each channel of CBC collected from 10 patients in 3-dimensional coordinates. it has been done Ten FCS data plots for each channel are enumerated and displayed for visual comparison.

FIG. 26 shows plots for the WDF channel (one of the white blood cell analysis channels of an automatic blood cell analyzer), and FIG. 27 shows plots for the WPC channel (one of the white blood cell analysis channels of an automatic blood cell analyzer). FIG. 28 is plots for the WNR channel (automated blood cell analyzer leukocyte analysis channel), and FIG. 29 shows plots for the PLT-F channel (one of the automated blood cell analyzer platelet analysis channels). Each plot shown in FIGS. 26 to 29 shows a similar clustering aspect, but shows subtle differences in detailed distribution patterns.

30 and 31 are exemplary diagrams for explaining an operation of modifying basic data based on biologically extracted data according to an embodiment of the present invention.

30 is an exemplary diagram for explaining that FCS data can be expressed in a supercubic space (in this example, a 3D cube corresponding to three parameters). The hypercube space is composed of a set of hypercube pixels, and the coordinates representing the position of each pixel become the measured value of each corresponding parameter. The gray scale intensity of each pixel is determined by the number of cells or particles having a combination of parameter values corresponding to the location of each pixel.

31 shows a data table for explaining an operation of transforming initial data. 31 is an exemplary description of a table displaying shading intensities (Count column) for each pixel according to the relationship between parameter values and supercubic pixel coordinates and the shading intensity definition of each pixel, sorted and displayed according to pixel coordinates.

The disease diagnosis apparatus 100 merges measurement values of each parameter of initial data (FCS data) and transforms each test item value into one coordinate value.

In addition, the diagnostic device 100 transforms the data (image depth conversion), and a method of adding a predetermined value (eg, 10) to each quotient to prevent loss of data that occurs at this time.

Also, the disease diagnosis apparatus 100 creates a data table including modified data and count values for each of the modified data.

When the same coordinate value as the coordinate value included in the transformed data exists, the disease diagnosis apparatus 100 deletes the same coordinate value, increases the count value for the coordinate value by a preset unit, updates the count value, and transforms the Creates a data table containing updated data and updated count values. For example, the disease diagnosis apparatus 100 assigns a count value of 1 when the coordinate value of the transformed data is 1, and assigns a count value of 2 to the corresponding coordinate value when the same coordinate value exists. You can create data tables.

The disease diagnosis apparatus 100 may calculate the number of coordinate points corresponding to each pixel in the coordinate space through the data table. The example of FIG. 10A shows a graph of coordinate values included in transformed data, and FIG. 10B shows an operation of counting coordinate points corresponding to each pixel in a coordinate space through a data table.

32 is a diagram for explaining an operation of reconstructing data based on biologically extracted data according to an embodiment of the present invention by way of example. This is an example of FCS data being first converted into a table representing the shape in the hypercube as in the above method, and then rearranged and secondly converted into a 2D image format.

The disease diagnosis apparatus 100 may represent the count values displayed in the coordinate order of the data table as a one-dimensional array in the same order, and reconstruct them into a two-dimensional array (image format) for machine learning.

(n_i는 기 설정된 기준 크기값 이상의 자연수) 형태의 기계학습용 데이터 테이블로 재구성할 수 있다. 예를 들어, 질병 진단 장치(100)는 도 11에 도시된 바와 같이, 12 * 12 크기의 기계학습용 데이터 테이블과 같이 데이터를 재구성할 수 있다. 여기서, 하나의 행은 하나의 좌표값 및 카운트값을 의미한다. The disease diagnosis apparatus 100 configures the coordinate values included in the transformed data as one-dimensional coordinate values, and in the process of constructing the one-dimensional coordinate values, fills a part where no coordinate values exist with a value of 0, or coordinate values using a method to display only the parts that exist

It can be reconstructed into a data table for machine learning in the form of (n _i is a natural number equal to or greater than a predetermined standard size value). For example, as shown in FIG. 11 , the disease diagnosis apparatus 100 may reconstruct data such as a 12*12 machine learning data table. Here, one row means one coordinate value and one count value.

The device may be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, and may be implemented using a general-purpose or special-purpose computer. The device may be implemented using a hardwired device, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like. Also, the device may be implemented as a System on Chip (SoC) including one or more processors and controllers.

The device may be mounted in the form of software, hardware, or a combination thereof in a computing device or server equipped with hardware elements. A computing device or server includes all or part of a communication device such as a communication modem for communicating with various devices or wired/wireless communication networks, a memory for storing data for executing a program, and a microprocessor for executing calculations and commands by executing a program. It can mean a variety of devices, including

Operations according to the present embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer readable medium. Computer readable medium refers to any medium that participates in providing instructions to a processor for execution. A computer readable medium may include program instructions, data files, data structures, or combinations thereof. For example, there may be a magnetic medium, an optical recording medium, a memory, and the like. The computer program may be distributed over networked computer systems so that computer readable codes are stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing this embodiment may be easily inferred by programmers in the art to which this embodiment belongs.

These embodiments are for explaining the technical idea of this embodiment, and the scope of the technical idea of this embodiment is not limited by these embodiments. The scope of protection of this embodiment should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

Claims (20)

In the data processing method for the convolution operation,
The data processing method is performed by a data processing apparatus including a processor and a memory in which a program for execution related to the data processing method is stored, and the processor, according to the execution of the program,
pre-processing the initial data into table-based conversion data; and
Performing operations including applying a filter of a neural network model to the table-based conversion data;
In the preprocessing step,
For the convolutional product operation, the initial data having a first data structure formed of N-dimensional data by N (where N is a natural number of 2 or more) axes is converted into table-based conversion data having a second data structure in the form of a table. Converting to;
In order to apply the neural network model to the first data structure, designing a structure of a filter frame that is operable with the second data structure and represents a dimension,
The second data structure is a structure in which weights of filters for performing a composite product operation of coordinate values for the N axes, values matching the coordinate information, and values matching the coordinate information are arranged in the same row or column. Characterized in that having a, data processing method. delete According to claim 1,
The data processing method of claim 1 , wherein the first data structure includes a hypercube having depth information of 4 dimensions or more including 2 dimensions and 3 dimensions. delete According to claim 1,
The first data structure includes flow cytometry of a blood clinical specimen or biological analysis sample and bio-extracted data representing measurement results of an analysis technique using flow cytometry. format or expressed in Flow Cytometry Standard (FCS) format,
Wherein the second data structure merges measurement values of some parameters of the biometric extraction data, transforms them into data including coordinate values for channels, and includes the transformed data and count values. delete delete According to claim 1,
Wherein N is a natural number of 4 or more,
Designing the filter frame structure by the processor determines a start coordinate corresponding to a filter center position according to the filter frame structure;
and extending the filter weight elements of the filter frame structure in a row direction or a column direction in consideration of the dimension of the first data structure. According to claim 1,
The step of applying the filter is,
and performing an operation between matched elements by moving the filter frame structure based on a row direction or column direction of a table of the second data structure. According to claim 9,
The step of applying the filter is,
and skipping the operation if a filter center of the filter frame structure satisfies a preset row condition or column condition. In the data processing device for the convolution operation,
It includes a processor and a memory in which a program for executing operations related to data processing is stored, and according to the execution of the program, the processor:
pre-processing the initial data into table-based conversion data; and
Performing operations including applying a filter of a neural network model to the table-based conversion data;
In the pre-processing, the initial data having a first data structure formed of N-dimensional data by N axes (where N is a natural number of 2 or more) is converted into a second data structure in the form of a table for a convolution operation. Converting into table-based conversion data having
and designing a structure of a filter frame in which a dimension is expressed and operable with the second data structure in order to apply a neural network model to the first data structure, wherein the second data structure is on the N axes A data processing device characterized by having a structure in which weights of a filter for performing a composite product operation of a coordinate value for , a value matching the coordinate information, and a value matching the coordinate information are arranged in the same row or the same column. delete According to claim 11,
The first data structure includes a hypercube having depth information of 4 dimensions or more including 2 dimensions and 3 dimensions. According to claim 11,
In the second data structure, (i) coordinate information corresponding to the N axes and (ii) value information matching the coordinate information are arranged in a row direction or a column direction. According to claim 11,
The first data structure includes flow cytometry of a blood clinical specimen or biological analysis sample and bio-extracted data representing measurement results of an analysis technique using flow cytometry. format or expressed in Flow Cytometry Standard (FCS) format,
The second data structure merges measurement values of some parameters of the biometric extraction data, transforms them into data including coordinate values for channels, and includes the transformed data and count values. delete delete According to claim 11,
Wherein N is a natural number of 4 or more,
Designing the filter frame structure by the processor determines a start coordinate corresponding to a filter center position according to the filter frame structure;
and extending the filter weight elements of the filter frame structure in a row direction or a column direction in consideration of the dimension of the first data structure. According to claim 11,
the processor,
and performing an operation between matching elements by moving the filter frame structure based on a row direction or column direction of a table of the second data structure. According to claim 19,
the processor,
The data processing apparatus of claim 1 , wherein the operation is skipped when a filter center of the filter frame structure satisfies a preset row condition or column condition.

Images