CN113380414B - Data acquisition method and system based on big data - Google Patents

Abstract

The invention discloses a data collection method and system based on big data, which relates to the technical field of medical data collection; the method includes: obtaining a variety of medical data through a collection and dispatch center, wherein the collection and dispatch center includes a plurality of different collectors , the different collectors obtain unstructured medical data through corresponding collection channels; summarize the unstructured medical data; process the medical data; locally store and/or store the processed medical data Cloud storage. This invention integrates and stores a variety of relevant medical data after collecting them, and provides two dirty data processing methods. During the processing process, dirty data can be accurately filtered, identified, collected, and displayed, and has strong reliability and high security. , and can also handle duplicate data in medical data.

Description

Data collection methods and systems based on big data

Technical field

The present invention relates to the technical field of medical data collection, and specifically to a data collection method and system based on big data.

Background technique

At this stage, my country's medical data mainly comes from diseases recorded in information systems and equipment such as hospital information system HIS, electronic medical record system EMR, image acquisition and transmission system PACS, laboratory examination information system LIS, pathology system PS, and medical equipment. Physical data. It also includes data generated by hospital material management and hospital operation systems. Surveys show that currently more than 70% of hospitals have implemented medical informatization, but less than 3% of hospitals have data exchanges. Medical big data is relatively scattered, and information islands need to be broken down. Sometimes two doctors will have different interpretations of the same medical record. Therefore, if the information between hospitals cannot be exchanged, it will be a great loss to the patients. Information islands also bring great inconvenience to doctors and hospital managers who need to use data and information.

Information islands are a historical problem left in the process of my country's health informatization construction. Due to the lack of relevant standards, various hospitals lack standard guidance when building medical information systems. There is no top-level design and fragmentation, resulting in the creation of information islands. Therefore, establishing a medical data collection center is an important means to improve medical technology, break up information islands, and realize interconnection between hospitals.

Due to the wide variety, large volume and fast update speed of medical data, the existing medical data collection system cannot handle a large amount of diverse data well, cannot guarantee the reliability of the collected data, and cannot handle repeated data.

Contents of the invention

In view of the above problems in the existing technology, the present invention proposes a data collection method and system based on big data, which can process a large amount of diverse data, has strong reliability and high security, and can also handle duplication in the collected data. data.

A data collection method based on big data according to the first embodiment of the present application includes:

Acquire a variety of medical data through a collection and dispatch center, where the collection and dispatch center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels;

Aggregate said unstructured medical data;

Process the medical data;

The processed medical data is stored locally and/or in the cloud.

According to some embodiments of this application, before obtaining medical data through multiple collection methods, it also includes:

Perform basic configuration on the services corresponding to the yml type file, and transfer medical data between each service through queues.

According to some embodiments of the present application, processing the medical data includes:

Verify the quality of medical data;

Label the verified medical data;

Create an index for tagged medical data.

According to some embodiments of the present application, verifying the quality of medical data includes:

Verify the accuracy of medical data;

De-duplicate the medical data through neural networks;

Encrypt the deduplicated medical data.

According to some embodiments of the present application, labeling the verified medical data includes:

Input the verified medical data into the BERT neural network to obtain the text vector V;

Randomly select multiple text vectors V as the cluster center point a;

Obtain the distance between other medical data and each cluster center point a, classify the other medical data into the nearest text vector V, and obtain the cluster center point b of the multi-category text vector V after the classification is completed;

Obtain the distance between other medical data and each cluster center point b, classify the other medical data into the nearest text vector V, and then obtain the cluster center point c of the multi-category text vector V after the classification is completed, and repeat In this step, multiple types of text are obtained;

Label the text described in each category with a central word;

The newly acquired medical data is classified according to the similarity with the center word.

According to some embodiments of the present application, labeling the verified medical data includes:

Classify existing medical data into multiple types;

The existing medical data is trained through bert+bilstm+cnn+attention+crf neural network until the accuracy is greater than the threshold;

Use the trained bert+bilstm+cnn+attention+crf neural network to classify the newly acquired medical data and assign it to the corresponding type.

According to some embodiments of the present application, storing the processed medical data locally includes:

Get the proxy service and port where the attribute table is located;

The proxy service scans the starting row of each attribute configuration in the attribute table, determines which attribute range the current medical data is in, and then stores it in the database;

The database stores the corresponding relationship between attributes and proxy services.

According to some embodiments of the present application, managing the database includes:

Read the medical data and translate it into an internal unified data format;

Perform addition, deletion, modification and check operations on the collection sources of medical data;

After obtaining the query results from the database, perform data format conversion on them.

A data collection system based on big data according to the second embodiment of the present application includes:

The collection module acquires a variety of medical data through a collection and dispatching center, where the collection and dispatching center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels;

A summary module to summarize the unstructured medical data;

A processing module, used to process the medical data;

A storage module is used to store the processed medical data locally and/or in the cloud.

According to some embodiments of the present application, the processing module includes:

Verification module to verify the quality of medical data;

The labeling module labels the verified medical data;

Create an index module to create an index for the tagged medical data.

Through the above technical solution, the technical effect obtained is: the present invention collects and integrates storage of a variety of relevant medical data, provides two dirty data processing methods, and can achieve accurate filtering, identification, and collection of dirty data during the processing process. , showing that it has strong reliability, high security, and can also handle repeated data in medical data.

Description of the drawings

Figure 1 is a hardware structural block diagram of a data collection computer disclosed in the embodiment of the present application;

Figure 2 is a flow chart of the data collection method disclosed in the embodiment of the present application;

Figure 3 is a flow chart of quality verification and processing of the medical data disclosed in the embodiment of the present application;

Figure 4 is a flow chart for verifying the quality of medical data disclosed in the embodiment of the present application;

Figure 5 is a flow chart of local storage of processed medical data disclosed in the embodiment of the present application.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the exemplary embodiments. be communicated to those skilled in the art.

The current medical quality management and control indicator framework for tertiary general hospitals includes 7 major categories of indicators, 44 categories of quality evaluation indicators, 730 individual indicators, 2610 composite indicators, and more than 400 monitoring data. The indicator categories include in-hospital death indicators, Return indicators, hospital infection indicators, surgical complications indicators, patient safety indicators, rational drug use indicators in medical institutions and hospital operation and management indicators. Its management system is huge and difficult to monitor. In addition, the database servers of each business system run DBMS. Issues such as whether there is manual data, the amount of manual data, and whether there is unstructured data are all technical barriers to medical interoperability. Therefore, the embodiment of this application provides data collection based on big data. Methods and systems. The data collection method can be executed in a server, computer or similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a data acquisition computer disclosed in an embodiment of the present application. As shown in FIG. 1 , the computer 10 may include one or more (only one is shown in FIG. 1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above-mentioned computer may also include a transmission device 106 and an input and output device 108 for communication functions. Persons of ordinary skill in the art can understand that the structure shown in Figure 1 is only illustrative and does not limit the structure of the above-mentioned computer. For example, computer 10 may also include more or fewer components than shown in FIG. 1 , or have a different configuration than shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data collection method in the embodiment of the present invention. The processor 102 executes various tasks by running the computer program stored in the memory 104. A functional application and data processing, that is, to implement the above method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

Transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the computer 10 . In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet wirelessly.

As shown in Figure 2, in some embodiments, a data collection method based on big data includes:

S1. Acquire a variety of medical data through a collection and dispatch center, where the collection and dispatch center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels;

Specifically, different collection methods can be used to obtain medical data through corresponding collection channels. The task scheduling center can manage different data (logs, database data, etc.) collection tasks. The collection method can be:

(1) There are various development data sets on the Internet. As long as you find the corresponding URL and get the download link, you can get data sets in the medical field. These data sets can help the medical system improve internal information, configure the collector to use crawlers and Rule matching and other methods are used to crawl and organize medical data.

(2) For the log service of the medical system, relevant log collection solutions can be adopted. Some of the more common log collection tools include Logstash, Filebeat, Flume, Fluentd, Logagent, rsyslog, and syslog-ng. Configure the collector to read and collect this type of medical data by reading log text information.

(3) Obtain corresponding medical data through social surveys, which can improve the data content of the medical system. Configure the collector to obtain medical data from social survey results.

(4) The medical system will have daily operations and business department modules, and various related data will be recorded in certain files or systems, such as common medical system databases. A large amount of medical data is stored in the database, and the collector is configured to obtain data in different ways for different types of databases.

(5) A medical sensor is a detection device that can sense the information being measured, and can convert the sensed information into electrical signals or other required forms of information output according to certain rules, and obtain medical data through the medical sensor. After uploading, configure the collector to collect these medical data.

S2. Aggregate the unstructured medical data;

Specifically, medical data is collected through a unified microservice interface. That is, after the collector obtains the data, it sends the data to the microservice through the Restful interface, and then places the data in the redis distributed cache platform for temporary storage.

Unstructured medical data obtained through different channels are summarized and then uniformly delivered to data quality verification for processing. This application adopts different collector methods and can process a variety of data. It solves the problem that traditional systems cannot handle a variety of data well.

S3. Process the medical data;

Specifically, after the medical data is obtained, the quality of the medical data is first verified, which includes verifying the accuracy of various related medical information, deduplicating and encrypting the medical data, etc. After further verification, the deduplicated data is labeled and the source of the information is monitored.

S4. Store the processed medical data locally and/or in the cloud.

Specifically, there are certain risks associated with local storage of medical data, such as local device damage leading to the disappearance of some or all of the data. Therefore, medical data can be backed up in the cloud. The medical data to be obtained is stored locally and a copy of the same data information is sent to the cloud for storage. Or every once in a while (for example: half a month, a month, etc.), the updated part of the locally stored medical data is compressed and then backed up to the cloud. Ensure data security.

The invention can extract medical data from distributed and heterogeneous data sources in hospital information system HIS, electronic medical record system EMR, image acquisition and transmission system PACS, laboratory examination information system LIS, pathology system PS and other hospital information systems. After reaching the temporary middle layer, it is cleaned, converted, integrated, and finally loaded into the database, which becomes the basis for online analysis and processing of medical data and medical data mining. The medical service supervision information data collection platform established through the above method has a high architecture security and It is easy to expand, can support various mainstream development languages, and provides rich interfaces. It can also support the storage and application of structured and unstructured data.

In some embodiments, before obtaining medical data through multiple collection methods, it also includes:

Perform basic configuration on the services corresponding to the yml type file, and transfer medical data between each service through queues.

Specifically, all configuration files during the data collection process are stored in nacos. There are many yml-type files in nacos, which are configuration files for each service, such as: collector service, data quality verification service, and user center service. , gateway services, data management services, etc. First, perform basic configuration on the yml type file such as service port, database address, startup method, etc. Each service automatically obtains the corresponding configuration from nacos when it is started.

It should be noted that the transfer of medical data in various services uses middleware tools, that is, using queues to solve concurrency problems and reduce server pressure.

As shown in Figure 3, in some embodiments, the medical data is subjected to quality checksum processing, including:

S31. Verify the quality of medical data;

Specifically, quality verification includes verifying the accuracy of medical data through comparison principles, deduplicating medical data through neural networks, and encrypting the deduplicated data with the TripleDES algorithm.

S32. Label the verified medical data;

Specifically, it can be achieved in two ways. The first way is to cluster the obtained medical data; the second way is to use neural networks for classification. After the classification is completed, the data source of the data is also stored as an attribute in the overall information of the data.

S33. Create an index for the tagged medical data.

Specifically, each piece of medical data is summarized to obtain the title of the data. There are two ways to obtain the title: one is to directly obtain the first 10-20 characters of the data as the title of the data, and the other is to obtain the summary of the data through the encoder and decoder in the Seq2Seq architecture. Create an index in Elasticsearch for the summary of the data, generation time and other important attributes to facilitate users to quickly query the data.

As shown in Figure 4, in some embodiments, checking the quality of medical data includes:

S311. Verify the accuracy of medical data;

Specifically, medical data accuracy verification can be implemented in a variety of ways. For example, the first method is to compare the MD5 encoding of the medical data sent by the middleware with the MD5 code carried by the data. If they are the same, it means that the transmitted data does not exist. problem; the second is to compare the similarity through multiple data sources of the data. If the difference is large, there is a problem with the data; the third is to determine whether there is a large error before and after data transmission. If the average value of the same indicator When there is a huge difference and it is not logical, it means that there is something wrong with the transmission process and the data obtained is inaccurate.

S312. Perform deduplication processing on the medical data through neural network;

Specifically, a neural network is used to deduplicate medical data text, and the obtained verified complete data is compared with the corresponding module information in the existing system. For example, a person's current condition information is obtained and compared with the person's condition information in the existing system. The judgment method can be to use bert in the neural network to obtain the corresponding sentence vector and then calculate the similarity. A similarity greater than 90% defines the texts as substantially the same, a similarity greater than 80% as roughly the same, and a similarity less than 50% as different. Filter out the same medical data and save different medical data.

S313. Encrypt the deduplicated medical data.

Specifically, the TripleDES algorithm can convert a 64-bit plaintext input block into a ciphertext output block with a data length of 64 bits, of which 8 bits are parity bits and the other 56 bits are used as the length of the password.

In some embodiments, labeling the verified medical data includes:

Input the verified medical data into the BERT neural network to obtain the text vector V;

Randomly select multiple (for example, 10) text vectors V as the cluster center point a;

Obtain the distance between other medical data and each cluster center point a (judging the distance between other medical data and text through similarity calculation), classify the other medical data into the nearest text vector V, After the classification is completed, the cluster center point b of the text vector V of multiple categories (for example, 10 categories) is obtained;

Obtain the distance between other medical data and each cluster center point b (judging the distance between other medical data and text through similarity calculation), classify the other medical data into the nearest text vector V, After the classification is completed, the cluster center point c of the text vector V of multiple categories (for example, 10 categories) is obtained. Repeat this step N times to obtain multiple (for example, 10) categories of text for storage;

Label the text described in each category with a central word;

The newly acquired medical data is classified according to the similarity with the center word.

In some embodiments, labeling the verified medical data includes:

Classify existing medical data into multiple types;

Specifically, the plurality of types may be 10 types, and the number is determined based on the amount of existing medical data.

The existing medical data is trained through bert+bilstm+cnn+attention+crf neural network until the accuracy is greater than the threshold;

Specifically, the graphics card memory of the device used for training should be greater than 10G, and the training accuracy rate should be greater than 90%.

Use the trained bert+bilstm+cnn+attention+crf neural network to classify the newly acquired medical data and assign it to the corresponding type.

It should be noted that this embodiment provides two or more processing methods for a variety of dirty data. When a certain data processing method fails, it can automatically identify and return the processing failure signal 0, and immediately start another processing method. Process the data to ensure the stability of data processing. For example: when the rule classification of medical data fails, the neural network model will be used to classify the medical data immediately to obtain accurate classification results.

As shown in Figure 5, in some embodiments, the processed medical data is stored locally, including:

S41. Obtain the proxy service and port where the attribute table is located;

Specifically, after obtaining medical data containing complete attributes. Connect to zookeeper through the client, and find the proxy service and port where the attribute table is located from the zookeeper node.

S42. The proxy service scans the starting row of each attribute configuration in the attribute table, determines which attribute range the current medical data is in, and then stores it in the database;

S43. The database stores the corresponding relationship between attributes and proxy services.

Specifically, the client directly requests the corresponding proxy service; after receiving the request from the client, the proxy service writes the medical data into the attributes.

In some embodiments, managing the database includes:

Read the medical data and translate it into an internal unified data format;

Specifically, this step enables the resources in the database to be fully managed and enables control over the data;

Perform addition, deletion, modification and check operations on the collection sources of medical data;

Specifically, the website is monitored in real time based on the source status, regular status, etc.; for keyword search collection, it is convenient to add/delete, start/stop collection in real time; and adjust the collection strategy in real time based on the actual collection situation. Such as adding/deleting collectors, etc.;

After obtaining the query results from the database, perform data format conversion on them.

Specifically, the user's data request (high-level instructions) is converted into complex machine code (low-level instructions) to implement the query operation on the database and obtain the query results; the query results are processed (format conversion) and returned to the user.

This embodiment also discloses a data collection system based on big data, including:

The collection module acquires a variety of medical data through a collection and dispatching center, where the collection and dispatching center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels;

An acquisition module that summarizes the unstructured medical data;

A processing module to process the medical data;

A storage module is used to store the processed medical data locally and/or in the cloud.

The system completes the storage operation of a single medical data, called a Job. After receiving a Job, it will start a process to complete the entire storage process. The system Job module is the central management node of a single job, responsible for functions such as data cleaning, sub-task segmentation (converting single job calculations into multiple sub-tasks), and TaskGroup management. After the system job is started, the job will be divided into multiple small tasks (subtasks) according to different source-end segmentation strategies to facilitate concurrent execution. Task is the smallest unit of system operations, and each Task is responsible for storing part of the data. After splitting multiple tasks, the system job will call the Scheduler module to recombine the split tasks into a TaskGroup according to the configured concurrent data volume. Each TaskGroup is responsible for running all assigned tasks with a certain concurrency. The default number of concurrencies for a single task group can be 10. Each Task is started by TaskGroup. After the Task is started, the Reader->Channel->Writer thread will be started to complete the data storage work. After the system job is running, the Job monitors and waits for the completion of multiple TaskGroup module tasks, and waits for the completion of all TaskGroup tasks before the Job successfully exits.

In some embodiments, the data collection system based on big data also includes:

The configuration module is used to perform basic configuration of the services corresponding to the yml type file, and transfer medical data between each service through queues.

In some embodiments, the processing module includes:

Verification module to verify the quality of medical data;

The labeling module labels the verified medical data;

Create an index module to create an index for the tagged medical data.

In some embodiments, the verification module includes:

Accuracy verification module to verify the accuracy of medical data;

A deduplication module that performs deduplication processing on the medical data through a neural network;

The encryption module encrypts the deduplicated medical data.

In some embodiments, the specific implementation of the labeling module includes:

Input the verified medical data into the BERT neural network to obtain the text vector V;

Randomly select multiple text vectors V as the cluster center point a;

Obtain the distance between other medical data and each cluster center point a, classify the other medical data into the nearest text vector V, and obtain the cluster center point b of the multi-category text vector V after the classification is completed;

Obtain the distance between other medical data and each cluster center point b, classify the other medical data into the nearest text vector V, and then obtain the cluster center point c of the multi-category text vector V after the classification is completed, and repeat In this step, multiple types of text are obtained;

Label the text described in each category with a central word;

The newly acquired medical data is classified according to the similarity with the center word.

In some embodiments, the specific implementation of the labeling module includes:

Classify existing medical data into multiple types;

The existing medical data is trained through bert+bilstm+cnn+attention+crf neural network until the accuracy is greater than the threshold;

Use the trained bert+bilstm+cnn+attention+crf neural network to classify the newly acquired medical data and assign it to the corresponding type.

In some embodiments, the specific implementation of the storage module includes:

Get the proxy service and port where the attribute table is located;

The proxy service scans the starting row of each attribute configuration in the attribute table, determines which attribute range the current medical data is in, and then stores it in the database;

The database stores the corresponding relationship between attributes and proxy services.

In some embodiments, managing the database includes:

Read the medical data and translate it into an internal unified data format;

Perform addition, deletion, modification and check operations on the collection sources of medical data;

After obtaining the query results from the database, perform data format conversion on them.

Since the problem-solving principle of the data collection system based on big data is similar to the above-mentioned data collection method, the implementation of the data collection system based on big data can be found in the implementation of the method, and will not be described again here.

Embodiments of the present invention also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the above data collection method are executed.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing the specified logical functions or steps of the process. , and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order, depending on the functionality involved, which shall It should be understood by those skilled in the art to which embodiments of the present invention belong.

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium, For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wires (electronic device), portable computer disk cartridges (magnetic device), random access memory (RAM), Read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, and subsequently edited, interpreted, or otherwise suitable as necessary. process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: a logic gate circuit with a logic gate circuit for implementing a logic function on a data signal. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

In addition, each functional module in various embodiments of the present invention can be integrated into one processing module, each module can exist physically alone, or two or more modules can be integrated together. The above integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention and are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although reference is made to the foregoing implementations The present invention has been described in detail in the examples. Those of ordinary skill in the art should understand that any person familiar with the technical field can still modify or modify the technical solutions recorded in the foregoing embodiments within the technical scope disclosed in the present invention. Changes can be easily imagined, or equivalent substitutions can be made to some of the technical features; and these modifications, changes or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the present invention. within the scope of protection. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims above.

Claims (7)

1. A data collection method based on big data, characterized by including: Acquire a variety of medical data through a collection and dispatch center, where the collection and dispatch center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels; Aggregate said unstructured medical data; Process the medical data; Store the processed medical data locally and/or in the cloud; The medical data will be processed, including: Verify the quality of medical data; Label the verified medical data; Create an index for tagged medical data; Label the verified medical data through method 1 or method 2, The first method includes: Input the verified medical data into the BERT neural network to obtain the text vector V; Randomly select multiple text vectors V as the cluster center point a; Obtain the distance between other medical data and each cluster center point a, classify the other medical data into the nearest text vector V, and obtain the cluster center point b of the multi-category text vector V after the classification is completed; Obtain the distance between other medical data and each cluster center point b, classify the other medical data into the nearest text vector V, and then obtain the cluster center point c of the multi-category text vector V after the classification is completed, and repeat In this step, multiple types of text are obtained; Label the text described in each category with a central word; The newly acquired medical data is classified according to the similarity with the central word; The second method includes: Classify existing medical data into multiple types; The existing medical data is trained through bert+bilstm+cnn+attention+crf neural network until the accuracy is greater than the threshold; Use the trained bert+bilstm+cnn+attention+crf neural network to classify the newly acquired medical data and assign it to the corresponding type. 2. A data collection method based on big data according to claim 1, characterized in that before obtaining medical data through multiple collection methods, it also includes: Perform basic configuration on the services corresponding to the yml type file, and transfer medical data between each service through queues. 3. A data collection method based on big data according to claim 1, characterized in that verifying the quality of medical data includes: Verify the accuracy of medical data; De-duplicate the medical data through neural networks; Encrypt the deduplicated medical data. 4. A data collection method based on big data according to claim 1, characterized in that local storage of the processed medical data includes: Get the proxy service and port where the attribute table is located; The proxy service scans the starting row of each attribute configuration in the attribute table, determines which attribute range the current medical data is in, and then stores it in the database; The database stores the corresponding relationship between attributes and proxy services. 5. A data collection method based on big data according to claim 4, characterized in that managing the database includes: Read the medical data and translate it into an internal unified data format; Perform addition, deletion, modification and check operations on the collection sources of medical data; After obtaining the query results from the database, perform data format conversion on them. 6. A data collection system based on big data, used to implement the data collection method according to any one of claims 1-5, characterized in that it includes: The collection module acquires a variety of medical data through a collection and dispatching center, where the collection and dispatching center includes a plurality of different collectors, and the different collectors obtain unstructured medical data through corresponding collection channels; A summary module to summarize the unstructured medical data; A processing module, used to process the medical data; A storage module is used to store the processed medical data locally and/or in the cloud. 7. A data collection system based on big data according to claim 6, characterized in that the processing module includes: Verification module to verify the quality of medical data; The labeling module labels the verified medical data; Create an index module to create an index for the tagged medical data.