TWI808785B - Data splitting system and method for validating machine learning - Google Patents

Abstract

A data splitting method for validating machine learning is adapted to a blood pressure dataset including blood pressure data of a plurality of subjects. The method includes dividing measurement ranges of the systolic blood pressure and the diastolic blood pressure into a plurality of first intervals and a plurality of second intervals respectively, generating a plurality of classes according to the plurality of first intervals and the plurality of second intervals, with each class including one of the plurality of first intervals and one of the plurality of second intervals, determining and recording a matching condition of each subject and the plurality of classes, with each matching condition includes a plurality of marks corresponding to the plurality of classes, each mark has one of the a first state and the second state, the first state representing that the blood pressure data matches the class corresponding to the mark and the second state representing that the blood pressure does not match the class corresponding to the mark, and distributing the plurality of subjects into a plurality of subsets by performing a distribution procedure.

Description

Data splitting system and method for validating machine learning

The present invention relates to machine learning, in particular to a data splitting system and method for verifying machine learning.

Validation strategies are integral to tuning hyperparameters of machine learning and deep learning models. For training and validation purposes, data splitting or cross-validation (CV) techniques can be used to divide the data into training data sets, validation data sets, and test data sets. These techniques provide insights about the accuracy and generalizability of models on independent and new datasets. However, if the data splitting strategy is not appropriate, these techniques can also lead to the problem of over-fitting or bias of the model.

Existing cross-validation methods such as Leave-One-Out, Holdout and K-fold CV are common practices in standard machine learning problems (such as regression problems). However, blood pressure estimation is not a standard regression problem. For example, the data points of the blood pressure dataset are not completely independent of each other. That is, these data sets often have many segments from the same record or subject, which may contain very similar information. Furthermore, there are two targets in the blood pressure estimation problem, systolic blood pressure (SBP) and diastolic blood pressure (DBP), which are more similar to multi-task or multi-output regression problems. Finally, the systolic and diastolic distributions are often skewed, and since extreme blood pressures are rare, blood pressure estimation becomes an unbalanced regression problem.

In order to correctly partition the data during cross-validation, the above differences must be taken into account. For example, random partitioning of the data may result in segments of the same subject appearing in the training, validation, and test sets simultaneously. Since data segments from the same subject carry similar information, this will lead to a breakdown of independence between sets and lead to overly optimistic results. In addition, due to distribution skew and imbalance, random data partitioning may cause problems such as different distributions or data shifts between each set, or even lack of minority cases in the test data set.

In view of this, the present invention proposes a data splitting system and method for verifying machine learning to avoid the above-mentioned problems. The cross-validation of the blood pressure estimation task can keep the data of the same subject in the same set, and keep the distribution of the systolic and diastolic blood pressure of the original data set in different data sets as much as possible. In other words, in multiple data sets generated by applying the present invention, whether it is systolic blood pressure or diastolic blood pressure, there is a high similarity among the multiple trends presented in multiple data sets.

A data splitting method for verifying machine learning according to an embodiment of the present invention is applicable to a blood pressure data set. The blood pressure data set includes a plurality of blood pressure data of a plurality of subjects respectively, and types of the blood pressure data include systolic blood pressure and diastolic blood pressure. The method includes the following steps performed by an arithmetic device: dividing the measurement range of the systolic blood pressure into a plurality of first intervals, and dividing the measurement range of the diastolic blood pressure into a plurality of second intervals, generating a plurality of categories according to the first interval and the second interval, each category including a first interval and a second interval, judging and recording the matching status of the blood pressure data of each subject and the category, and then generating a plurality of matching status corresponding to the plurality of testing subjects, each matching status includes a plurality of marks corresponding to the plurality of classes, each tag has one of the first state and the second state, and the first state represents the matching status of the blood pressure data. Category, the second state represents the category corresponding to the blood pressure data does not match the flag, and according to the matching status to perform the allocation process to allocate more Subjects were assigned to multiple sets.

A data splitting system for verifying machine learning according to an embodiment of the present invention includes a measurement device, a storage device, and a computing device. The measuring device is used to generate a blood pressure data set, wherein the blood pressure data set includes a plurality of blood pressure data of a plurality of subjects respectively, and types of the blood pressure data include systolic blood pressure and diastolic blood pressure. The storage device communicates with the measurement device to receive and store the blood pressure data set, and stores a computer-readable recording medium. The computing device communicates with the storage device, the computing device is used to run the computer and can read the recording medium to perform the following steps: divide the measurement range of the systolic blood pressure into a plurality of first intervals, and divide the measurement range of the diastolic blood pressure into a plurality of second intervals, generate a plurality of categories according to the first interval and the second interval, each category includes a first interval and a second interval, judge and record the matching status of the blood pressure data of each subject and the category, and then generate a plurality of matching status corresponding to a plurality of testing subjects, each matching status includes a plurality of marks corresponding to the plurality of classes, and each tag has one of the first state and the second state. Or, the first state represents the category corresponding to the blood pressure data matching flag, the second state represents the category corresponding to the blood pressure data unmatching flag, and the allocation procedure is executed according to the matching status to allocate multiple subjects into multiple sets.

In summary, the data splitting system and method for verifying machine learning proposed by the present invention have the following contributions or effects: first, the proposed method stores all samples from the same subject in the same set (training data set, verification data set or test data set); second, the proposed method can achieve similar blood pressure distributions on the training data set, verification data set and test data set; The blood pressure distributions of the different datasets are maintained while maintaining the blood pressure distributions of the different datasets, which means that the systolic blood pressure distributions of the training/validation datasets and the test datasets are similar, and the diastolic blood pressure distributions of these datasets are also similar.

The above description of the disclosure and the following description of the implementation are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the patent application scope of the present invention.

100: Data splitting system

10: Measuring device

30: storage device

50: computing device

S1~S6,S61~S69: steps

D0: blood pressure data set

D1: training data set

D2: Validation dataset

D3: Test data set

1 is a schematic diagram of the present invention applied to machine learning; FIG. 2 is a block diagram of a data splitting system for verifying machine learning according to an embodiment of the present invention; FIG. 3 is a flowchart of a data splitting method for verifying machine learning according to an embodiment of the present invention; FIG. 4 is a schematic diagram of a blood pressure data set; Schematic diagram of diastolic blood pressure distribution.

The detailed features and characteristics of the present invention are described in detail below in the embodiments, the content of which is sufficient for any person familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings, any person familiar with the relevant art can easily understand the concept and characteristics of the present invention. The following examples are to further describe the concept of the present invention in detail, but not to limit the scope of the present invention in any way.

Fig. 1 is a schematic diagram of the present invention applied to machine learning. As shown in Figure 1, after the blood pressure data set D0 has been pre-processed, the data splitting system and method proposed by the present invention can be applied to split the blood pressure data set D0 into a training data set D1, a verification data set D2 and a test data set D3. training materials Data set D1 and verification data set D2 are used to train and verify the blood pressure prediction model. The test data set D3 is used to test the blood pressure prediction model. In other application scenarios, the blood pressure data set may be generated after the original data set undergoes data preprocessing, and then the blood pressure data set is split. In other words, the present invention does not limit the execution sequence of data pre-processing.

FIG. 2 is a block diagram of a data splitting system for verifying machine learning according to an embodiment of the present invention. As shown in FIG. 2 , the system 100 includes a measuring device 10 , a storage device 30 and a computing device 50 .

The measurement device 10 is used for generating a blood pressure data set, wherein the blood pressure data set includes blood pressure data of a plurality of subjects. Each subject includes multiple pieces of blood pressure data, and the types of each blood pressure data include systolic blood pressure (SBP) and diastolic blood pressure (DBP). In one embodiment, the measuring device 10 is, for example, a wearable device with a pulse oximeter (pulse oximetry), which uses photoplethysmography (PPG) to obtain the PPG signal, and then converts the blood pressure data through the built-in microprocessor of the wearable device. In another embodiment, the measurement device 10 is, for example, a wearable device with electrodes, which uses electrocardiography (ECG) technology to obtain ECG signals, and then converts the blood pressure data through the built-in microprocessor of the wearable device. In yet another embodiment, the measuring device 10 is, for example, an electronic sphygmomanometer or an armband sphygmomanometer.

The storage device 30 is communicatively connected to the measuring device 10 to receive and store the blood pressure data set, and store a computer-readable recording medium. In one embodiment, the storage device 30 is, for example, a volatile memory and/or a non-volatile memory. Non-volatile memory includes read-only memory (ROM), programmable ROM (programmable ROM, PROM), electrically programmable ROM (electrically programmable ROM, EPROM), electrically erasable and programmable ROM (electrically erasable and programmable ROM, EEPROM), flash memory, phase-change random access memory (phase-change) random access memory, PRAM), magnetic RAM (magnetic RAM, MRAM), Resistive RAM (resistive RAM, RRAM) and/or ferroelectric RAM (ferroelectric RAM, FRAM). The volatile memory may include dynamic RAM (dynamic RAM, DRAM), static RAM (static RAM, SRAM) and/or synchronous DRAM (synchronous DRAM, SDRAM). In another embodiment, the storage device 30 is, for example, at least one of a hard disk drive (HDD), a solid-state drive (solid-state drive, SSD), a compact flash (CF) card, a secure digital (SD) card, a micro SD card, a mini SD card, an extreme digital (xD) card, and a memory stick.

The computing device 50 is communicatively connected to the storage device 30 . The computing device 50 is used to run the computer-readable recording medium to execute the data splitting method for verifying machine learning according to an embodiment of the present invention. In one embodiment, the computing device 50 is, for example: a microprocessor, such as a central processor unit (central processor unit, CPU), a graphics processing unit (graphic processing unit) and/or an application processor (application processor, AP); a logic chip, such as a field programmable gate array (field programmable gate array, FPGA) and a specific application IC (application-specific integrated circuit, ASIC).

Please refer to FIG. 3 and FIG. 4 . FIG. 3 is a flowchart of a data splitting method for verifying machine learning according to an embodiment of the present invention, and FIG. 4 is a schematic diagram of a blood pressure data set. The approach shown in Figure 3 was applied to a blood pressure dataset. In one embodiment, the blood pressure data set includes the respective blood pressure data of a plurality of subjects, and the types of the blood pressure data include systolic blood pressure and diastolic blood pressure, but not limited thereto.

Taking the actual values as an example, the data structure of the blood pressure data set is described below: the blood pressure of 500 people is measured for 60 minutes by the measuring device 10 (such as an electronic sphygmomanometer). Assuming a sampling frequency of 1 piece of blood pressure data is captured in 2 minutes, and blood pressure data is generated based on the original measurement data, each person can contribute 30 pieces of blood pressure data (60/2), and each blood pressure data package Include systolic and/or diastolic values. The blood pressure data set of the present invention is a collection of blood pressure data of these 500 individuals, totaling 15,000 blood pressure data (500*30).

The schematic diagram of the blood pressure data set is shown in Figure 4. It can be seen from Figure 4 that both the systolic blood pressure and the diastolic blood pressure have skewed distribution characteristics. For example, there are more data items in a certain value range such as 60-70 mmHg, and there are fewer data items in another value range such as 150-170 mmHg.

In step S1 , the computing device 50 obtains a blood pressure data set from the storage device 30 .

In step S2, the computing device 50 divides the measurement range of the systolic blood pressure into a plurality of first intervals. In step S3, the computing device 50 divides the measurement range of the diastolic pressure into a plurality of second intervals. The present invention does not limit the order in which steps S2 and S3 are performed.

In one embodiment, an actual numerical value is taken as an example to illustrate the division method of the first interval and the second interval. For example, the systolic blood pressure can be divided into four first intervals, and the four first intervals are: (1) lower than 100 millimeters of mercury (mmHg), (2) between 100 mmHg and 140 mmHg, (3) between 140 mmHg and 160 mmHg, and (4) exceeding 160 mmHg. For example diastolic blood pressure may be divided into four second intervals: (1) below 60mmHg, (2) between 60mmHg and 80mmHg, (3) between 80mmHg and 100mmHg, and (4) above 100mmHg. The above numerical values are for illustration only and not intended to limit the present invention. In other words, the present invention does not limit the size of the range of each first/second interval, the number of first/second intervals, and the like. In other embodiments, in addition to the two blood pressure constraints shown in FIG. 3: systolic blood pressure and diastolic blood pressure, the method of the present invention can further add a third blood pressure constraint, such as pulse pressure difference, and the computing device 50 divides the measurement range of the third blood pressure constraint into multiple third intervals.

SBP<140以及60

SBP<80。 In step S4, the computing device 50 generates a plurality of categories according to the plurality of first intervals and the plurality of second intervals. The number of categories is the number of combinations of the number of the first interval and the number of the second interval. Following the previous example, 16 categories (4*4) can be generated based on four first intervals and four second intervals, as shown in Table 1 below. Each of the 16 categories includes one of four first intervals and one of four second intervals. For example, category 6 represents 100

SBP<140 and 60

SBP<80.

In step S5, the computing device 50 judges and records the matching status of each subject's blood pressure data and each category, and then generates a plurality of matching statuses corresponding to all the subjects. Each matching condition includes a plurality of marks corresponding to a plurality of categories, and each mark has one of a first state and a second state. The first state represents that among the multiple pieces of blood pressure data of a subject, at least one piece of systolic blood pressure data is located in the first interval included in the category corresponding to the mark, and among the multiple pieces of blood pressure data of the same subject, at least one piece of diastolic blood pressure data is located in the second interval included in the category corresponding to the mark. The second state represents that there is no matching category in all the blood pressure data. Table 2 below uses actual values as an example to illustrate multiple matching conditions of multiple testees in multiple categories.

In the example shown in Table 2, the matching status of nine subjects in three categories is included. In Table 2, each column represents a matching condition, where a mark of 1 represents the first state, and a mark of 0 represents the second state. For example: the matching status of subject A is (1,0,1), which means that among the multiple blood pressure data of subject A, at least one blood pressure data matches category 1, no blood pressure data matches category 2, and at least one blood pressure data matches category 3. In general, step S5 is used to generate the blood pressure category distribution of multiple subjects. The data structure of this distribution is a 0-1 matrix composed of multiple matching conditions. The number of columns in the matrix is equal to the number of subjects, and the number of rows in the matrix is equal to the number of categories.

In step S6, the computing device 50 executes an allocation procedure according to the multiple matching conditions of all the subjects, so as to allocate all the subjects into multiple sets. In one embodiment, each set corresponds to a fold in K-fold cross-validation. The present invention does not limit the number and size of sets. The allocation procedure must take into account that each category is distributed as evenly as possible into each set, so that the distribution of SBP and DBP among the different sets is maintained. In addition, since the allocation procedure is based on the subject as the allocation unit, multiple blood pressure data from the same subject are avoided from being allocated to different sets, resulting in the collapse of data independence.

Please refer to FIG. 5 , which is a detailed flowchart of step S6 in FIG. 3 .

In step S61 , the computing device 50 calculates a plurality of required quantities of subjects corresponding to the sets according to the number of subjects and the number of sets. Continuing from the previous example, assume that the number of sets is 3, which are marked as set 1, set 2, and set 3. The number of subjects was 9, as shown in Table 2 for an example distribution. Therefore, set 1 corresponds to 3 subjects' demands, set 2 corresponds to 3 subjects' demands, and set 3 corresponds to 3 subjects' demands. In other words, the number of subjects required is the number of subjects divided by the number of sets. If there is no divisibility, the remaining unassigned subjects will be assigned to certain sets according to the procedure described later.

In step S62 , for each category, the computing device 50 calculates the matching numbers of the category having the first status among the multiple subjects, and then obtains a plurality of matching numbers corresponding to the multiple categories. This step S62 can be regarded as a loop procedure. The computing device 50 processes one category at a time (depending on the parallel processing capability of the computing device 50 , it can also process N categories at a time), until all the categories are processed. Based on the example of the matching status in Table 2, the calculation device 50 can obtain the results in Table 3 below after executing step S62.

In step S63, the computing device 50 calculates a plurality of required quantities of matching categories according to the matching quantity and the set quantity. In an embodiment, the required number of matching categories is an average value obtained by dividing the number of matches by the number of collections. Based on the matching quantity example in Table 3, the calculation device 50 can obtain the results in Table 4 below after executing step S63. For example, the number of matches corresponding to category 1 is 4, so the required number of matching categories for each set is 1.3 (4/3 is rounded to one decimal place).

In step S64 , the computing device 50 determines whether there are subjects in the blood pressure data set that have not been assigned to the set. If it is judged as yes, then execute the process of steps S65-S69, and then return to step S64 for re-judgment. The process of steps S65-S69 will be repeated until all the subjects are assigned to a certain set, then the judgment of step S64 will become negative, and the data splitting method according to an embodiment of the present invention will end.

In step S65 , the computing device 50 selects one of a plurality of categories as the designated category. In one embodiment, the number of matches for a given category is a minimum. For example, in Table 4, category 2 corresponds to the smallest number of matching category requirements (2.3>1.3>1), so in the first round of iterative process, the computing device 50 selects category 2 as the specified category. In other embodiments, the computing device 50 randomly selects the designated category.

In step S66, the computing device 50 selects a designated subject from the blood pressure dataset. The flag of the specified category of the specified subject is the first state. Following the previous example, in the first round of iterative process, 9 subjects including A~I in the blood pressure data set have not been assigned yet. When the designated category is category 2, subjects C, E, and F are selected as specified subjects, because the marks of these three subjects C, E, and F in category 2 are all value 1 representing the first state. In addition, since the three subjects C, E, and F are selected as designated subjects, the process from step S66 to step S69 will be repeated three times until all designated subjects are assigned, and step S66 will select a new designated subject in the next iteration.

In step S67, the computing device 50 selects one of the multiple sets as the designated set. In detail, the computing device 50 judges whether the first condition is satisfied. If the judgment is yes, then generate the specified set. If the determination is negative, the computing device 50 determines whether the second condition is satisfied. If the judgment is yes, then generate the specified set. If the determination is negative, the computing device 50 randomly selects one of the multiple sets meeting the third condition as the specified set.

The first condition is: find the first maximum value among all demand quantities of the matching category covered by the specified category, and the quantity of the first maximum value is equal to 1. If the first condition is satisfied, the set corresponding to the first maximum value is the designated set.

The second condition is: find the first maximum value among all the demand quantities of the matching category covered by the specified category, and the number of the first maximum value is greater than 1; If the second condition is satisfied, the set corresponding to the second maximum value is the specified set.

The third condition is: find the first maximum value among all the demand quantities of the matching category covered by the specified category, and the quantity of the first maximum value is greater than 1;

The following uses Table 4 as an example to illustrate the execution flow of step S67: the known designated category is category 2 generated in step S66. The number of all matching categories covered by category 2 is (1,1,1) respectively. Since the maximum value is 1, and the number of this maximum value is 3, the first condition is not satisfied. The number of subjects demanded by set 1~set 3 is (3,3,3) respectively. Since the maximum value is 3, and the number of this maximum value is 3, the second condition is not satisfied. The sets meeting the third condition include set 1, set 2, and set 3, so the computing device 50 randomly selects one of these three sets as the designated set.

In step S68, the computing device 50 assigns the specified subject to the specified set, and removes the specified subject from the blood pressure data set. Continuing from the previous example, the specified subjects generated in step S66 include subjects C, E, and F, and the specified sets generated in step S67 are sets 1, 2, and 3. In one embodiment, when step S66 generates a plurality of designated subjects, the computing device 50 may randomly select one of them for step S68. For example: in step S68, the computing device 50 assigns the subject C to the set 1, and then removes the blood pressure data of the subject C from the blood pressure data set.

In step S69 , the computing device 50 updates the required quantity of the matching category and the required quantity of the subject. Following the previous example, the updated results are shown in Table 5 below. Note that the number of matching category requirements of set 1 in category 2 is reduced from 1 to 0 (1-1), because set 1 has been assigned a subject C whose category 2 is 1. In addition, the required number of subjects in set 1 is also reduced from 3 to 2 (3-1), because set 1 has been assigned a subject C.

After step S69 is completed, it will return to step S64. Because there are still subjects A~B, D~I in the blood pressure data set. Therefore, the judgment result of step S64 is yes, and the second iterative process of step S65 to step S69 is executed.

In step S65, the number of matching category requirements corresponding to category 2 is still the smallest (note: the matching category requirement quantity lower than 1 is not considered), so in the second round of iterative process, the computing device 50 still selects category 2 as the designated category.

In step S66, the specified subjects selected by the computing device 50 include E and F.

In step S67, the number of matching category requirements in category 2 of set 2 and set 3 is (1, 1) respectively. Since the maximum value is 1, and the number of this maximum value is 2, the first condition is not satisfied. The number of subjects demanded from set 1 to set 3 is (2, 3, 3) respectively. Since the maximum value is 3, and the number of this maximum value is 2, the second condition is not satisfied. The sets meeting the third condition include set 2 and set 3, so the computing device 50 randomly selects one of the two sets as the specified set.

In step S68, for example, the designated subject E is assigned to the designated set 2, and the designated subject E is removed from the blood pressure data set.

In step S69, the updated results are shown in Table 6 below. Note that the matching category requirements of set 2 will decrease correspondingly according to the matching status (0, 1, 1) of subject E.

By analogy with the above process, each execution of the iterative process from step S65 to step S69 will assign a specified subject to a specified set, therefore, the total number of iterations is equal to the number of subjects in the blood pressure data set. Table 7, Table 8, and Table 9 show examples of the third iteration, sixth iteration, and last iteration (ninth iteration) respectively. For ease of understanding, this example assumes that "random selection" is implemented in alphabetical order and Arabic numeral order.

Please refer to FIG. 6 and FIG. 7. FIG. 6 is a schematic diagram of the systolic blood pressure distribution and diastolic blood pressure distribution generated according to the traditional data splitting method, and FIG. In Figure 6, the distributions of the training data set, validation data set, and test data set are not consistent. Taking the schematic diagram of systolic blood pressure distribution in Figure 6 as an example, the training data set has the largest amount of data at 130 mmHg, but the validation data set has the largest amount of data at 125 mmHg. In addition, both the validation and test datasets contain a small amount of data at about 190 mmHg, but the training dataset has almost no data there. This will lead to the inability of the trained blood pressure estimation model to predict systolic blood pressure values above 190 mmHg. In Figure 7, no matter the systolic blood pressure distribution or the diastolic blood pressure distribution, the three data sets of training data set, verification data set and test data set all have similar data distribution trends. An example is as follows: if a data set has more data in blood pressure interval A and less data in blood pressure interval B, then other data sets will also have the same characteristics. Therefore, such a splitting method of the data set helps to improve the generality and accuracy of the blood pressure prediction model.

In summary, the data splitting system and method for verifying machine learning proposed by the present invention have the following contributions or effects: one is that the proposed method combines all data from the same subject The samples are stored in the same set (training data set or test data set); secondly, the proposed method can achieve similar blood pressure distributions on the training data set, verification data set and test data set; thirdly, the proposed method can maintain the blood pressure distribution of different data sets when there are multiple constraints (such as systolic blood pressure and diastolic blood pressure, and can also include pulse rate, heartbeat and other constraints related to blood pressure that affect model training), which means that the systolic blood pressure distributions of the training/validation data set and the test data set are similar, and the diastolic blood pressure distribution of these data sets The pressure distribution is also similar.

Although the present invention is disclosed by the aforementioned embodiments, they are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, all changes and modifications are within the scope of patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the appended scope of patent application.

S1~S6: steps

Claims (5)

A data splitting system for verifying machine learning, comprising: a measurement device used to generate a blood pressure data set, wherein the blood pressure data set includes a plurality of subjects, the blood pressure data of each of the subjects, and the types of the blood pressure data include systolic blood pressure and diastolic blood pressure; a storage device communicates with the measurement device to receive and store the blood pressure data set, and stores a computer-readable recording medium; and a computing device communicates with the storage device. dividing the measurement range of the diastolic blood pressure into a plurality of second intervals; generating a plurality of categories according to the first intervals and the second intervals, each of the categories including one of the first intervals and one of the second intervals; judging and recording the matching status of the blood pressure data of each of the subjects with the categories, and then generating a plurality of matching conditions corresponding to the subjects, each of the matching conditions includes a plurality of marks corresponding to the categories, and each of the marks has a first One of a state and a second state, the first state represents that the systolic blood pressure data in the blood pressure data is located in the first interval included in the category corresponding to the mark, and the diastolic blood pressure data in the blood pressure data is located in the second interval included in the class corresponding to the mark, and the second state represents that the blood pressure data do not match the class corresponding to the mark; The data splitting system for verifying machine learning as described in claim 1, wherein the computing device running the computer-readable recording medium is further used to perform the following steps: calculate the number of required subjects corresponding to the sets according to the number of the subjects and the number of the sets; for each of the categories, calculate the number of matches in the category with the first state in the subjects, and then obtain the number of matches corresponding to the categories; calculate the number of needs for multiple matching categories according to the number of matches and the number of sets; When there are one or more of the subjects, the following steps are performed: select one of the categories as the specified category, and the matching number of the specified category is a minimum value; select a specified subject from the blood pressure dataset, and the flag of the specified category of the specified subject is the first state; according to the specified category, select one of the sets as a specified set; assign the specified subject to the specified set, and remove the specified subject from the blood pressure data set. The data splitting system for verifying machine learning as described in claim 2, wherein when the computing device running the computer-readable recording medium selects one of the sets as the specified set, it includes: finding the maximum number of matching categories among the numbers of matching categories corresponding to the sets; and when the maximum number of matching categories is equal to 1, using the set corresponding to the maximum value as the specified set. The data splitting system for verifying machine learning as described in claim item 3, wherein the computing device running the computer-readable recording medium is further used to perform the following steps: when the number of the maximum value is greater than 1, find the maximum number of testees in the number of testees corresponding to the sets; and when the maximum number of testees is equal to 1, use the set corresponding to the maximum number of testees as the specified set. The data splitting system for verifying machine learning as described in claim 4, wherein the computing device running the computer-readable recording medium is further used to perform the following steps: when the maximum number of subjects required is greater than 1, randomly select one from multiple sets corresponding to the maximum number of subjects required as the specified set.

Images