CN114328262A - Monitoring method and device for sequencing data processing flow - Google Patents

Abstract

The invention discloses a monitoring method and device for a sequencing data processing flow. The method includes: acquiring a target tag of the sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow; executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow; The execution result monitors the predetermined processing flow. The method uses the target tag to identify the running state of the sequencing data in the predetermined processing flow, and executes the predetermined processing flow according to the target tag, which can minimize the communication between the operation manager and the information personnel during the execution of the predetermined processing flow, thereby solving the problem of In the prior art, the sequencing data analysis mode in which the operation manager interacts with the information personnel is adopted, which has the technical problems of long labor time, low labor efficiency and long cycle.

Description

Monitoring method and device for sequencing data processing flow

technical field

The invention relates to the field of software development, in particular, to a monitoring method and device for a sequencing data processing flow.

Background technique

With the increasing maturity of next-generation sequencing, single-cell sequencing and other sequencing technologies, the throughput is increasing, but more samples and a larger amount of data also increase the analysis cycle and labor time of sequencing data. Therefore, how to reduce the cycle of sequencing data analysis and the time consumption of personnel has become a problem.

In a sequencing company, as shown in Figure 1, the sequencing data analysis process is generally initiated by the company's operations manager, who fills out forms according to customer needs, and then informs the bioinformatics analysts in charge of the analysis. After understanding the project situation, start to create a project directory on the Linux cluster, configure the relevant files, start the pre-built analysis process, and finally check the report content and submit the results after the process is completed. In the above process, there are mainly the following problems that hinder the implementation of project automation, which in turn leads to the slowdown of the project cycle and the increase of manpower:

At the start, operation and end nodes of the project, the bioinformatics engineer and the operation manager need to communicate and communicate frequently, and report the progress and status of the project in real time, such as the need to confirm the data disconnection before the project starts, project requirements, whether to start Run the project; during the operation of the project, you need to report in real time whether it is running normally and the running time; at the end of the project, you need to submit a report, upload data, and reply whether the project is completed normally. These communication processes often cause the project to be interrupted for a period of time and not be able to transition to the next phase in real time.

For the above problems, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a method and device for monitoring a sequencing data processing flow, so as to at least solve the sequencing data analysis mode in the prior art that adopts the interaction between operation managers and information personnel, which has long labor time, low labor efficiency, and long cycle. technical problem.

According to an aspect of the embodiments of the present invention, a method for monitoring a sequencing data processing flow is provided, including: acquiring a target tag of a sequencing data processing flow, wherein the target tag is used to identify that the sequencing data is in a predetermined processing flow Execute the predetermined processing flow according to the target tag, and generate an execution result of the predetermined processing flow; monitor the predetermined processing flow according to the execution result.

Optionally, the target tag includes at least one of the following: a conventional test tag, a test-plus-test tag, and a compatibility tag, wherein the conventional test tag is used to identify the running state of the sequencing data in a normal processing flow , the test-plus-test label is used to identify the running state of the sequencing data in the test-plus-test processing flow, and the compatibility label is used to identify the running status of the sequencing data in the human-machine compatible processing flow.

Optionally, when the target label includes the conventional test label, executing the predetermined processing flow according to the target label, and generating an execution result of the predetermined processing flow, comprising: executing the conventional test label as waiting for data Get off the computer, check whether the sequencing data is off the computer, and send the detection result to the target object; after the target object is confirmed, execute the routine test tag if all samples have data off the computer and are waiting for execution, then start Analyze the sequencing data, and send a start notification to the target object; after the target object is confirmed, execute the routine test and label it as running, then analyze the sequencing data, and send the analysis result to The target object; after the target object is confirmed, the execution of the routine test label is completed, the sequencing data is uploaded, and the completion result is sent to the target object, or the execution of the routine test label is abnormally interrupted , the exception result is sent to the target object.

Optionally, before executing the routine test labeling waiting for data to be off the computer, the method further includes: synchronizing the sequencing data to a preset database.

Optionally, when the target label includes the test-plus-test label, executing the predetermined processing flow according to the target label, and generating an execution result of the predetermined processing flow, includes: executing the test-plus-test If the label is waiting for the test data to be off, then check whether the sequencing data of the test is off, and send the test result to the target object; after the target object is confirmed, execute the test and add the test label as All test samples have data offline and are waiting to be executed, then start to analyze the sequencing data of the test, and send a test start notification to the target object; after the target object is confirmed, execute the test test If the test tag is in the test run, analyze the sequencing data of the test test, and send the test analysis result to the target object; after the target object is confirmed, execute the test test and add the test tag In order to wait for the additional test data to go off the machine, check whether the added test sequencing data is off the machine, and send the additional test detection result to the target object; after the target object is confirmed, execute the test test and add the test tag As all the additional test samples have data off the machine and are waiting for execution, start the analysis of the sequencing data for additional testing, and send an additional testing start notification to the target object; after the target object is confirmed, execute the test. If the label of test plus test indicates that the test is in progress, the sequencing data of the test is analyzed, and the analysis result of the test is sent to the target object; after the target object is confirmed, the test plus test is performed. If the tag indicates that the operation is complete, the sequence data added to the test is uploaded, and the completed result is sent to the target object.

Optionally, after sending the test start notification to the target object or sending the additional test analysis result to the target object, the method further includes: after the target object is confirmed, executing the test test and adding the test label as abnormal. If interrupted, the abnormal result is sent to the target object.

Optionally, when the target label includes the compatibility label, executing the predetermined processing flow according to the target label, and generating an execution result of the predetermined processing flow, comprising: executing the compatibility label as having been If the analysis is performed manually, the feedback result will be sent to the target object; or, if the compatibility label is the historical information collection table, the reminder information will not be sent to the target object.

According to another aspect of the embodiments of the present invention, a monitoring device for a sequencing data processing flow is also provided, including an obtaining unit, a generating unit, and a monitoring unit, wherein the obtaining unit is used to obtain a target tag of the sequencing data processing flow, Wherein, the target tag is used to identify the running state of the sequencing data in the predetermined processing flow; the generating unit is configured to execute the predetermined processing flow according to the target tag, and generate an execution result of the predetermined processing flow; The monitoring unit is configured to monitor the predetermined processing flow according to the execution result.

According to another aspect of the embodiments of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium includes a stored program, wherein when the program is executed, the computer-readable storage medium is controlled The device executes any one of the monitoring methods for the sequencing data processing flow.

According to another aspect of the embodiments of the present invention, a processor is also provided, and the processor is used for running a program, wherein, when the program is running, any one of the monitoring methods for the sequencing data processing flow is executed.

In the embodiment of the present invention, the method for monitoring the sequencing data processing flow, first, obtains the target tag of the sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow; Then, the predetermined processing flow is executed according to the target tag, and an execution result of the predetermined processing flow is generated; finally, the predetermined processing flow is monitored according to the execution result. The method uses the target tag to identify the running status of the sequencing data in the predetermined processing flow, and executes the predetermined processing flow according to the target tag, which can minimize the communication between the operation manager and the information personnel during the project execution process, thereby solving the problem of existing problems. The technology adopts the sequencing data analysis mode of interaction between the operation manager and the information personnel, which has the technical problems of long labor time, low labor efficiency and long cycle.

Description of drawings

The accompanying drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached image:

Fig. 1 is the logic diagram of the monitoring method of sequencing data processing flow in the prior art;

2 is a flowchart of a monitoring method for a sequencing data processing flow according to an embodiment of the present application;

FIG. 3 is a logic diagram of a routine test tag execution predetermined processing flow according to an embodiment of the present application;

FIG. 4 is a logic diagram of a predetermined processing flow for testing and adding a testing tag according to an embodiment of the present application;

FIG. 5 is a logic diagram of a predetermined processing flow for executing a compatibility tag according to an embodiment of the present application;

6 is a schematic diagram of a monitoring device for a sequencing data processing flow according to an embodiment of the present application;

FIG. 7 is a logic diagram of a monitoring method for a sequencing data processing flow in the prior art according to an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

As mentioned in the background art, the prior art adopts the sequencing data analysis mode of interaction between operation managers and information personnel, which has the technical problems of long labor time, low labor efficiency and long cycle. In order to solve the above problems, a method of the present application In a typical embodiment, a method and apparatus for monitoring a sequencing data processing flow are provided.

According to an embodiment of the present invention, a method for monitoring a sequencing data processing flow is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and, Although a logical order is shown in the flowcharts, in some cases steps shown or described may be performed in an order different from that herein.

FIG. 2 is a flowchart of a monitoring method for a sequencing data processing flow according to an embodiment of the present application. As shown in FIG. 2 , the method includes the following steps:

Step S101, obtaining a target tag of the sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow;

Step S102: Execute the above-mentioned predetermined processing flow according to the above-mentioned target tag, and generate an execution result of the above-mentioned predetermined processing flow;

Step S103, monitoring the above-mentioned predetermined processing flow according to the above-mentioned execution result.

In the monitoring method for the above sequencing data processing flow, first, a target tag of the sequencing data processing flow is obtained, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow; then, the predetermined processing is performed according to the target tag process, generating the execution result of the above-mentioned predetermined processing flow; finally, monitoring the above-mentioned predetermined processing flow according to the above-mentioned execution result. The method uses the target tag to identify the running state of the sequencing data in the predetermined processing flow, and executes the predetermined processing flow according to the target tag, which can minimize the communication between the operation manager and the information personnel during the execution of the predetermined processing flow, thereby solving the problem of In the prior art, the sequencing data analysis mode in which the operation manager interacts with the information personnel is adopted, which has the technical problems of long labor time, low labor efficiency and long cycle.

In practical applications, the above method needs to be used in the Python environment of the Linux system, where the Python version can be Python 3.8.

In order to monitor the status of the sequencing data processing process in real time, and to automatically assign the predetermined processing flow according to different stages of the analysis flow of the sequencing data, in an embodiment of the present application, the target tag includes at least the following: One: a conventional test label, a test-plus-test label, and a compatibility label, wherein the above-mentioned conventional test label is used to identify the running state of the above-mentioned sequencing data in the normal processing flow, and the above-mentioned test-added test label is used to identify the above-mentioned sequencing data. In the running state of the test-plus-test processing flow, the compatibility label is used to identify the running status of the sequencing data in the human-machine compatible processing flow.

In practical applications, there are more than a dozen tags available. Relying on this tag algorithm, the sequencing data processing process can be well monitored, and the algorithm has good scalability, and can be increased or decreased at any time according to needs, which greatly increases the number of tags. Added flexibility. For common conventional test labels, test-plus-test labels, and compatibility labels, the corresponding meaning of each status is shown in the following table:

In another embodiment of the present application, as shown in FIG. 3 , when the target label includes the conventional test label, executing the predetermined processing flow according to the target label, and generating an execution result of the predetermined processing flow, includes: executing the above The routine test label is to wait for the data to go off the machine, then check whether the above-mentioned sequencing data is off the machine, and send the test results to the target object; after the above-mentioned target object is confirmed, execute the above-mentioned routine test label that all samples have data to go off the machine and wait for execution , then start to analyze the above-mentioned sequencing data, and send the start notification to the above-mentioned target object; after the above-mentioned target object is confirmed, execute the above-mentioned routine test tag is running, then analyze the above-mentioned sequencing data, and send the analysis result to the above-mentioned target object; after the above target object is confirmed, if the execution of the above routine test label is completed, the above sequencing data will be uploaded, and the completion result will be sent to the above target object, or, if the execution of the routine test label is abnormally interrupted, the abnormal result will be sent to the above target object. The above-mentioned routine test labels are mainly used to solve routine testing needs.

Specifically, the target object may be an operation manager, and the target object sent by the abnormal result may also be a biological information engineer.

In order to further reduce the time of the sequencing data processing flow, in another embodiment of the present application, before executing the routine test tag is to wait for the data to be offline, the method further includes: synchronizing the sequencing data to a preset database.

Specifically, the above-mentioned preset database may be, but is not limited to, a database based on mongoDB, which collects and saves detailed information of the predetermined processing flow, data that has been logged off, and the sample conditions and requirements of the predetermined processing flow submitted by the operation manager. These information will be stored in the form of predetermined processing flow, and some predetermined processing flow judgments will be made in combination with the saved information, such as whether to keep the bam file, whether to test and so on.

In a specific embodiment of the present application, in order to obtain the latest predetermined processing flow information and de-installation sample information in real time, and to query and extract the information in the database in time, an API software suite for interacting with the core database is written, The suite provides database information update, database scheduled processing flow data query, extraction, visualization and statistics functions.

In still another embodiment of the present application, as shown in FIG. 4 , when the target label includes the test-testing label, the predetermined processing flow is executed according to the target label, and an execution result of the predetermined processing flow is generated, including: Execute the above-mentioned test test and add the test label to wait for the test data to go off the machine, then check whether the above-mentioned sequencing data of the test test is off the machine, and send the test test result to the target object; after the above-mentioned target object is confirmed, execute the above test test and add. The test label is that all test samples have data off the machine and are waiting for execution, then start to analyze the above sequencing data of the test test, and send a test start notification to the above target object; after the above target object is confirmed, execute the above test test plus. If the test tag is in the test run, the above-mentioned sequencing data of the test test is analyzed, and the test analysis result is sent to the above-mentioned target object; after the above-mentioned target object is confirmed, execute the above-mentioned test and add the test tag to wait for the test data to be added Get off the machine, check whether the above-mentioned sequencing data for additional testing is off the machine, and send the additional testing results to the above-mentioned target object; after the above-mentioned target object is confirmed, execute the above-mentioned test and add the test label to ensure that all the additional test samples have data download. After the above-mentioned target object is confirmed, the execution of the above-mentioned test and the addition of the test tag indicates that the additional test is running, then the additional test start notification is sent to the above-mentioned target object. The above-mentioned sequencing data of the test is analyzed, and the analysis result of the additional test is sent to the above-mentioned target object; after the above-mentioned target object is confirmed, execute the above-mentioned test and add the test label to run complete, then upload the above-mentioned sequencing data of the additional test, and will complete The result is sent to the above target object.

In practical applications, when the above-mentioned target tags include the above-mentioned test and test tags, the difficulty lies in how to solve the time point when the test test starts after the test test is completed. In the prior art, this time point often needs to be determined by the operation and the customer, and then Notifying the bioinformatics engineer to carry out, the whole communication process takes a long time, and this application uses the python programming language to automatically start the additional testing of the predetermined processing process according to the label in the email sent by the operations manager, without notifying the bioinformatics engineer, after getting The additional test analysis is started immediately after the reply, which greatly reduces the labor time and increases the labor efficiency.

Specifically, in order to replace the interaction between bioinformatics analysts and operation managers, this application has developed the function of interacting by email. The operation manager will receive data at key nodes in the sequencing data analysis process, such as data logoff, analysis result report upload, etc. to the email reminder, and select to reply to specific fields according to the description of the running status of the sequencing data in the normal processing flow in the email. In the monitoring method of the sequencing data processing flow of this application, the inbox will be checked regularly and according to the operation manager's instructions Feedback to perform the next step in the analysis process, for example, whether to start data analysis, whether to wait for additional data testing or directly upload the report to end the question, etc.

In order to determine the state of executing the above-mentioned predetermined processing flow according to the above-mentioned test-adding label in time, in another embodiment of the present application, as shown in FIG. After the result is sent to the target object, the method further includes: after the target object is confirmed, executing the test test and adding the test tag as an abnormal interruption, then sending the abnormal result to the target object.

In practical applications, the execution of the above-mentioned test and test label is abnormal interruption, and the information can be synchronized to the database to ensure the efficiency of monitoring.

In yet another embodiment of the present application, as shown in FIG. 5 , when the target label includes the compatibility label, executing the predetermined processing flow according to the target label, and generating an execution result of the predetermined processing flow, includes: executing the above If the compatibility label has been analyzed manually, the feedback result is sent to the target object; or, if the compatibility label is a historical information collection table, the reminder information will not be sent to the target object. The above compatibility label is mainly to solve the compatibility problem between automatic execution and manual execution in a series of special cases. For example, the processing flow that has been analyzed by humans should not be executed automatically.

In order to increase the application scope of the monitoring method for the sequencing data processing flow of the present application, in a specific embodiment of the present application, an interface is provided for running a specified sequencing data analysis flow. In theory, as long as an effective interface is provided for the interface Function, you can connect most of the data analysis process, so as to realize the monitoring and automatic control of the data analysis process.

The embodiment of the present application also provides a monitoring device for the sequencing data processing flow. It should be noted that the monitoring device for the sequencing data processing flow of the embodiment of the present application can be used to execute the sequencing data processing provided by the embodiment of the present application. Process monitoring method. The following describes the monitoring device for the sequencing data processing flow provided by the embodiments of the present application.

FIG. 6 is a schematic diagram of a monitoring device for a sequencing data processing flow according to an embodiment of the present application. As shown in FIG. 6 , the device includes:

The acquiring unit 10 is configured to acquire a target tag of the sequencing data processing flow, wherein the above-mentioned target tag is used to identify the running state of the above-mentioned sequencing data in the predetermined processing flow;

A generating unit 20, configured to execute the above-mentioned predetermined processing flow according to the above-mentioned target tag, and generate an execution result of the above-mentioned predetermined processing flow;

The monitoring unit 30 is configured to monitor the above-mentioned predetermined processing flow according to the above-mentioned execution result.

The monitoring device for the above-mentioned sequencing data processing flow first obtains the target label of the sequencing data processing flow through the acquisition unit 10, wherein the above-mentioned target label is used to identify the running state of the above-mentioned sequencing data in the predetermined processing flow; The target tag executes the above-mentioned predetermined processing flow, and generates an execution result of the above-mentioned predetermined processing flow; finally, the monitoring unit 30 monitors the above-mentioned predetermined processing flow according to the above-mentioned execution result. The device uses the target tag to identify the running state of the sequencing data in the predetermined processing flow, and executes the predetermined processing flow according to the target tag, which can minimize the communication between the operation manager and the information personnel during the execution of the predetermined processing flow, thereby solving the problem of In the prior art, the sequencing data analysis mode in which the operation manager interacts with the information personnel is adopted, which has the technical problems of long labor time, low labor efficiency and long cycle.

In practical applications, the above device needs to be used in the Python environment of the Linux system, where the Python version may be Python 3.8.

In order to monitor the status of the sequencing data processing process in real time, and to automatically assign the predetermined processing flow according to different stages of the analysis flow of the sequencing data, in an embodiment of the present application, the target tag includes at least the following: One: a conventional test label, a test-plus-test label, and a compatibility label, wherein the above-mentioned conventional test label is used to identify the running state of the above-mentioned sequencing data in the normal processing flow, and the above-mentioned test-added test label is used to identify the above-mentioned sequencing data. In the running state of the test-plus-test processing flow, the compatibility label is used to identify the running status of the sequencing data in the human-machine compatible processing flow.

In practical applications, there are more than a dozen tags available. Relying on this tag algorithm, the sequencing data processing process can be well monitored, and the algorithm has good scalability, and can be increased or decreased at any time according to needs, which greatly increases the number of tags. Added flexibility. For common conventional test labels, test-plus-test labels, and compatibility labels, the corresponding meaning of each status is shown in the following table:

normal-waiting Waiting for data to drop off test-waiting Waiting for the test data to get off the machine add-waiting Waiting for the test data to get off the machine normal-ready All samples have data offline and are waiting to be executed test-ready All test samples have data offline and are waiting to be executed add-ready All additional test samples have data offline and are waiting to be executed normal-running running test-running test running add-running test running failed abort completed run complete manual-analysis has been analyzed by humans history-info Historical Information Collection Form

In another embodiment of the present application, the generation unit includes a first detection module, a first startup module, a first analysis module, and a first upload module, wherein the first detection module is used for the above-mentioned target tag including the above-mentioned conventional When testing the tags, the execution of the above-mentioned conventional test tags is to wait for the data to go off the computer, then to detect whether the above-mentioned sequencing data is off the computer, and to send the detection results to the target object; the above-mentioned first startup module is used to perform the above-mentioned routine after the above-mentioned target object is confirmed. The test tag is that all samples have data off the machine and are waiting to be executed, then start to analyze the above-mentioned sequencing data, and send a start notification to the above-mentioned target object; the above-mentioned first analysis module is used to execute the above-mentioned conventional test tag after the above-mentioned target object is confirmed In operation, the above-mentioned sequencing data is analyzed, and the analysis result is sent to the above-mentioned target object; the above-mentioned first uploading module is used to execute the above-mentioned conventional test tag after the above-mentioned target object is confirmed, and then upload the above-mentioned sequencing data when the operation is completed. , and send the completion result to the above target object, or, if the execution routine test is marked as abnormal interruption, the abnormal result is sent to the above target object. The above-mentioned routine test labels are mainly used to address routine testing needs.

Specifically, the target object may be an operation manager, and the target object sent by the abnormal result may also be a biological information engineer.

In order to further reduce the processing time of the sequencing data, in another embodiment of the present application, the device further includes a synchronization unit, and the synchronization unit synchronizes the sequencing data to a preset value before executing the routine test label to wait for the data to be offline. database.

Specifically, the above-mentioned preset database may be, but is not limited to, a mongoDB-based database, which collects and saves detailed information of the predetermined processing flow, data that has been logged off, and the sample conditions and requirements of the predetermined processing flow submitted by the operation manager. These information will be stored in the form of predetermined processing flow, and some predetermined processing flow judgments will be made in combination with the saved information, such as whether to keep the bam file, whether to test and so on.

In a specific embodiment of the present application, in order to obtain the latest predetermined processing flow information and de-installation sample information in real time, and to query and extract the information in the database in time, an API software suite for interacting with the core database is written, The suite provides database information update, database scheduled processing flow data query, extraction, visualization and statistics functions.

In yet another embodiment of the present application, the above-mentioned generating unit includes a second detection module, a second activation module, a second analysis module, a third detection module, a third activation module, a third analysis module, and a second uploading module, wherein , the second detection module is used to execute the test and add the test label to wait for the test data to go off the machine when the target label includes the test and test label, then detect whether the test-tested sequencing data is off the machine, and to The test test result is sent to the target object; the second startup module is used to execute the test test and add the test label after the confirmation of the target object, so that all test samples have data and are waiting to be executed, and then start the analysis test. the above-mentioned sequencing data, and send a test start notification to the above-mentioned target object; the above-mentioned second analysis module is used to execute the above-mentioned test test and add the test label after the confirmation of the above-mentioned target object. The data is analyzed, and the test analysis result is sent to the above-mentioned target object; the above-mentioned third detection module is used to execute the above-mentioned test test and add the test label after the confirmation of the above-mentioned target object. Whether the above-mentioned sequencing data is off the computer, and send the additional test result to the above-mentioned target object; the above-mentioned third startup module is used to execute the above-mentioned test and add-test label after the above-mentioned target object is confirmed, so that all the additional test samples have data to go off the computer and wait for execution, start the analysis of the above-mentioned sequencing data for additional testing, and send the additional testing start notification to the above-mentioned target object; the above-mentioned third analysis module is used to execute the above-mentioned test after the above-mentioned target object is confirmed. During operation, the above-mentioned sequencing data of the additional test is analyzed, and the analysis result of the additional test is sent to the above-mentioned target object; the above-mentioned second upload module is used to execute the above-mentioned test test and add the test label after the confirmation of the above-mentioned target object to complete the operation. , upload the above-mentioned sequencing data for additional testing, and send the completed result to the above-mentioned target object.

In practical applications, when the above-mentioned target tags include the above-mentioned test and test tags, the difficulty lies in how to solve the time point when the test test starts after the test test is completed. In the prior art, this time point often needs to be determined by the operation and the customer, and then Notifying the bioinformatics engineer to carry out, the whole communication process takes a long time, and this application uses the python programming language to automatically start the additional testing of the predetermined processing process according to the label in the email sent by the operations manager, without notifying the bioinformatics engineer, after getting The additional test analysis is started immediately after the reply, which greatly reduces the labor time and increases the labor efficiency.

Specifically, in order to replace the interaction between bioinformatics analysts and operation managers, this application has developed the function of interacting by email. The operation manager will receive data at key nodes in the sequencing data analysis process, such as data logoff, analysis result report upload, etc. to the email reminder, and select to reply to specific fields according to the description of the running status of the sequencing data in the normal processing flow in the email. In the monitoring method of the sequencing data processing flow of this application, the inbox will be checked regularly and according to the operation manager's instructions Feedback to perform the next step in the analysis process, for example, whether to start data analysis, whether to wait for additional data testing or directly upload the report to end the question, etc.

In order to determine the state of executing the above-mentioned predetermined processing flow according to the above-mentioned test-testing label in time, in another embodiment of the present application, the above-mentioned apparatus further includes an execution unit, and the above-mentioned execution unit is configured to send the test-test start notification to the target After the object or the additional test analysis result is sent to the target object, after the above-mentioned target object is confirmed, the execution of the above-mentioned test test and the addition of the test tag is abnormally interrupted, and the abnormal result is sent to the above-mentioned target object.

In practical applications, the execution of the above-mentioned test and test label is abnormal interruption, and the information can be synchronized to the database to ensure the efficiency of monitoring.

In another embodiment of the present application, the generating unit includes an execution module, and the execution module is configured to, when the target tag includes the compatibility tag, execute the compatibility tag and send the feedback result if the analysis has been performed manually. to the above-mentioned target object; or, if the above-mentioned compatibility tag is a historical information collection table, the reminder information will not be sent to the above-mentioned target object. The above compatibility label is mainly to solve the compatibility problem between automatic execution and manual execution in a series of special cases. For example, the processing flow that has been analyzed by humans should not be executed automatically.

In order to increase the application scope of the monitoring method for the sequencing data processing flow of the present application, in a specific embodiment of the present application, an interface is provided for running a specified sequencing data analysis flow. In theory, as long as an effective interface is provided for the interface Function, you can connect most of the data analysis process, so as to realize the monitoring and automatic control of the data analysis process.

The monitoring device of the above-mentioned sequencing data processing flow includes a processor and a memory, and the above-mentioned acquisition unit, generation unit, monitoring unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. .

The processor includes a kernel, and the kernel calls the corresponding program unit from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the prior art sequencing data analysis mode that uses the interaction between operation managers and information personnel has the technical problems of long labor time, low labor efficiency, and long cycle.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), the memory including at least one memory chip.

An embodiment of the present invention provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the monitoring method for the above sequencing data processing flow is implemented.

An embodiment of the present invention provides a processor, where the processor is used for running a program, wherein the monitoring method for the sequencing data processing flow is executed when the program is running.

An embodiment of the present invention provides a device. The device includes a processor, a memory, and a program stored in the memory and running on the processor. The processor implements at least the following steps when executing the program:

Step S101, acquiring a target tag of the sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow;

Step S102: Execute the above-mentioned predetermined processing flow according to the above-mentioned target tag, and generate an execution result of the above-mentioned predetermined processing flow;

Step S103, monitoring the above-mentioned predetermined processing flow according to the above-mentioned execution result.

The devices in this article can be servers, PCs, PADs, mobile phones, and so on.

The present application also provides a computer program product that, when executed on a data processing device, is adapted to execute a program initialized with at least the following method steps:

Step S101, acquiring a target tag of the sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in the predetermined processing flow;

Step S102: Execute the above-mentioned predetermined processing flow according to the above-mentioned target tag, and generate an execution result of the above-mentioned predetermined processing flow;

Step S103, monitoring the above-mentioned predetermined processing flow according to the above-mentioned execution result.

In order to enable those skilled in the art to understand the technical solutions of the present application more clearly, the technical solutions of the present application will be described in detail below with reference to specific embodiments and comparative examples.

Example

This embodiment provides a method for monitoring a sequencing data processing flow, including: acquiring a target tag of a sequencing data processing flow, wherein the target tag is used to identify the running state of the sequencing data in a predetermined processing flow; according to the target tag Execute the above-mentioned predetermined processing flow, and generate an execution result of the above-mentioned predetermined processing flow; and monitor the above-mentioned predetermined processing flow according to the above-mentioned execution result.

As shown in FIG. 7 , the execution mode of the monitoring method of the sequencing data processing flow in this embodiment is as follows:

Executes a predetermined processing flow based on the target tag.

In the waiting stage of the scheduled processing process, that is, normal-waiting and test-waiting: check the requirements, synchronize the database; check whether there is any logoff data; send email notifications.

In the waiting stage of adding test data in the scheduled processing process, that is, during add-waiting: check whether the data is off the machine; send an email to notify the operation manager; start the analysis according to the reply of the operation manager.

When the data is offline and waiting for the execution stage, that is, normal-ready, test-ready, and add-ready: check whether the data is offline; send an email to notify the operation manager; start the analysis according to the operation manager's reply.

At the completion stage of the scheduled processing process, that is, when completed: upload data; send an email to notify the operations manager.

When the scheduled processing process fails, i.e. failed: send email to notify operations manager and bioinformatics engineer; interrupt project, synchronize information to database.

In the above-mentioned monitoring method for the processing flow of sequencing data, a special label is added to the predetermined processing flow, the predetermined processing flow is processed correspondingly according to the target label, and the automation of the execution of the predetermined processing flow is realized by the technical means of labeling the predetermined processing flow Operation and monitoring.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the above-mentioned units may be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , which includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (10)

1. A method for monitoring a sequencing data processing flow is characterized by comprising the following steps:

acquiring a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow;

executing the preset processing flow according to the target label to generate an execution result of the preset processing flow;

and monitoring the preset processing flow according to the execution result.

2. The method of claim 1, wherein the target tag comprises at least one of: the sequence data processing method comprises a conventional test tag, a test adding test tag and a compatibility tag, wherein the conventional test tag is used for identifying the running state of the sequence data in a normal processing flow, the test adding test tag is used for identifying the running state of the sequence data in the test adding test processing flow, and the compatibility tag is used for identifying the running state of the sequence data in a human-computer compatible processing flow.

3. The method of claim 2, wherein when the target tag includes the regular test tag, executing the predetermined process flow according to the target tag, and generating an execution result of the predetermined process flow comprises:

if the conventional test tag is executed to wait for data to be downloaded, detecting whether the sequencing data is downloaded, and sending a detection result to a target object;

after the target object is confirmed, if the conventional test tag is executed, all samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data, and sending a starting notice to the target object;

after the target object is confirmed, if the conventional test tag is in operation, analyzing the sequencing data, and sending an analysis result to the target object;

and after the target object is confirmed, if the execution of the conventional test tag is operation completion, uploading the sequencing data and sending a completion result to the target object, or if the execution of the conventional test tag is abnormal interruption, sending an abnormal result to the target object.

4. The method of claim 3, wherein prior to performing the regular test tag as a wait for data to be off-line, the method further comprises:

and synchronizing the sequencing data to a preset database.

5. The method according to claim 2, wherein when the target tag includes the test plus test tag, executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow includes:

if the test adding label is executed to wait for the test data to be downloaded, detecting whether the test sequencing data is downloaded or not, and sending a test detection result to a target object;

after the target object is confirmed, if the test and the test tag are executed, all test samples have data to be downloaded and wait for execution, starting to analyze the sequencing data of the test, and sending a test starting notice to the target object;

after the target object is confirmed, if the test label is added in the test operation, analyzing the sequencing data of the test and sending the test analysis result to the target object;

after the target object is confirmed, if the test adding and testing label is executed to wait for adding and testing data to get off, detecting whether the sequencing data added and tested get off, and sending an adding and detecting result to the target object;

after the target object is confirmed, if the test adding and testing label is executed, all the adding and testing samples have data to be downloaded and wait for execution, starting to analyze the sequencing data added and tested, and sending a adding and testing starting notice to the target object;

after the target object is confirmed, if the test adding and testing label is executed in the adding and testing operation, analyzing the added and tested sequencing data, and sending an adding and testing analysis result to the target object;

and after the target object is confirmed, if the test and test tag is executed to be completed, uploading the test and test sequencing data, and sending a completed result to the target object.

6. The method of claim 3, wherein after sending a test initiation notification to the target object or sending an additive analysis result to the target object, the method further comprises:

and after the target object is confirmed, if the test adding and testing label is executed to be abnormal interruption, an abnormal result is sent to the target object.

7. The method of claim 2, wherein when the target tag includes the compatibility tag, executing the predetermined process flow according to the target tag, and generating an execution result of the predetermined process flow comprises:

if the compatibility tag is analyzed manually, sending a feedback result to the target object; or, if the compatibility tag is executed as a history information collection table, the reminding information is not sent to the target object any more.

8. A monitoring device for sequencing data processing flow is characterized by comprising:

the sequencing data processing device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a target label of a sequencing data processing flow, and the target label is used for identifying the running state of the sequencing data in a preset processing flow;

the generating unit is used for executing the preset processing flow according to the target label and generating an execution result of the preset processing flow;

and the monitoring unit is used for monitoring the preset processing flow according to the execution result.

9. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method for monitoring sequencing data processing flow according to any one of claims 1 to 7.

10. A processor configured to execute a program, wherein the program executes the method for monitoring the sequencing data processing flow of any of claims 1 to 7.

Images