CN114328262A - Method and device for monitoring sequencing data processing flow - Google Patents
Method and device for monitoring sequencing data processing flow Download PDFInfo
- Publication number
- CN114328262A CN114328262A CN202111680008.3A CN202111680008A CN114328262A CN 114328262 A CN114328262 A CN 114328262A CN 202111680008 A CN202111680008 A CN 202111680008A CN 114328262 A CN114328262 A CN 114328262A
- Authority
- CN
- China
- Prior art keywords
- test
- processing flow
- tag
- target object
- sequencing data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Apparatus Associated With Microorganisms And Enzymes (AREA)
Abstract
The invention discloses a method and a device for monitoring a sequencing data processing flow. Wherein, the method comprises the following steps: acquiring a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of sequencing data in a preset processing flow; executing a preset processing flow according to the target label to generate an execution result of the preset processing flow; and monitoring the preset processing flow according to the execution result. The method utilizes the target tag to identify the running state of the sequencing data in the preset processing flow, and executes the preset processing flow according to the target tag, so that the communication between an operation manager and information personnel in the execution process of the preset processing flow can be reduced to the minimum, and the technical problems of long labor time, low labor efficiency and long period existing in the prior art due to the adoption of a sequencing data analysis mode of interaction between the operation manager and the information personnel are solved.
Description
Technical Field
The invention relates to the field of software development, in particular to a method and a device for monitoring a sequencing data processing flow.
Background
With the secondary sequencing, the sequencing technologies such as single cell sequencing and the like are mature day by day, and the throughput is increased day by day, but the analysis period and the labor time of sequencing data are increased due to more samples and larger data volume. Therefore, how to reduce the cycle of sequencing data analysis and the man-hour consumption become a problem.
In a sequencing company, as shown in fig. 1, a sequencing data analysis process is generally initiated by an operation manager of the company, the operation manager fills a form according to the requirements of a client, and then notifies the requirements to a biological information analyst responsible for analysis, after the biological information analyst fully knows the conditions of the project, the biological information analyst starts to create a project catalog on a Linux cluster, configures related files, starts a pre-established analysis process, and finally detects the report content and submits the result after waiting for the completion of the process. In the above process, there are mainly the following problems that prevent the project from being automatically executed, and further lead to the problem that the project cycle is slowed down and the man-hour of the personnel is increased:
at the beginning, operation and end nodes of a project, a biological information engineer and an operation manager need to communicate and exchange frequently, and report the progress and situation of the project in real time, such as the off-line situation of data needing to be confirmed before the project starts, project requirements and whether the project starts to be operated; in the running process of the project, whether the project normally runs or not needs to be reported in real time, and the running time is up; and in the project finishing stage, a report needs to be submitted, data is uploaded, whether a project is normally finished or not is replied, and the like. These communication processes often result in a project that needs to be interrupted for a period of time and unable to transition to the next stage in real time.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for monitoring a sequencing data processing flow, which are used for at least solving the technical problems of long labor time, low labor efficiency and long period in the prior art due to the adoption of a sequencing data analysis mode of interaction between an operation manager and information personnel.
According to an aspect of the embodiments of the present invention, there is provided a method for monitoring a sequencing data processing flow, including: acquiring a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow; executing the preset processing flow according to the target label to generate an execution result of the preset processing flow; and monitoring the preset processing flow according to the execution result.
Optionally, the target tag comprises at least one of: the sequence data processing method comprises a conventional test tag, a test adding test tag and a compatibility tag, wherein the conventional test tag is used for identifying the running state of the sequence data in a normal processing flow, the test adding test tag is used for identifying the running state of the sequence data in the test adding test processing flow, and the compatibility tag is used for identifying the running state of the sequence data in a human-computer compatible processing flow.
Optionally, when the target tag includes the conventional test tag, executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow, where the execution result includes: if the conventional test tag is executed to wait for data to be downloaded, detecting whether the sequencing data is downloaded, and sending a detection result to a target object; after the target object is confirmed, if the conventional test tag is executed, all samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data, and sending a starting notice to the target object; after the target object is confirmed, if the conventional test tag is in operation, analyzing the sequencing data, and sending an analysis result to the target object; and after the target object is confirmed, if the execution of the conventional test tag is operation completion, uploading the sequencing data and sending a completion result to the target object, or if the execution of the conventional test tag is abnormal interruption, sending an abnormal result to the target object.
Optionally, before executing that the regular test tag is off-line for waiting data, the method further comprises: and synchronizing the sequencing data to a preset database.
Optionally, when the target tag includes the test-plus-test tag, executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow, where the executing result includes: if the test adding label is executed to wait for the test data to be downloaded, detecting whether the test sequencing data is downloaded or not, and sending a test detection result to a target object; after the target object is confirmed, if the test and the test tag are executed, all test samples have data to be downloaded and wait for execution, starting to analyze the sequencing data of the test, and sending a test starting notice to the target object; after the target object is confirmed, if the test label is added in the test operation, analyzing the sequencing data of the test and sending the test analysis result to the target object; after the target object is confirmed, if the test adding and testing label is executed to wait for adding and testing data to get off, detecting whether the sequencing data added and tested get off, and sending an adding and detecting result to the target object; after the target object is confirmed, if the test adding and testing label is executed, all the adding and testing samples have data to be downloaded and wait for execution, starting to analyze the sequencing data added and tested, and sending a adding and testing starting notice to the target object; after the target object is confirmed, if the test adding and testing label is executed in the adding and testing operation, analyzing the added and tested sequencing data, and sending an adding and testing analysis result to the target object; and after the target object is confirmed, if the test and test tag is executed to be completed, uploading the test and test sequencing data, and sending a completed result to the target object.
Optionally, after sending the test initiation notification to the target object or sending the test analysis result to the target object, the method further includes: and after the target object is confirmed, if the test adding and testing label is executed to be abnormal interruption, an abnormal result is sent to the target object.
Optionally, when the target tag includes the compatibility tag, executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow, where the execution result includes: if the compatibility tag is analyzed manually, sending a feedback result to the target object; or, if the compatibility tag is executed as a history information collection table, the reminding information is not sent to the target object any more.
According to another aspect of the embodiments of the present invention, there is also provided a monitoring apparatus for a sequencing data processing flow, including an obtaining unit, a generating unit, and a monitoring unit, where the obtaining unit is configured to obtain a target tag of the sequencing data processing flow, where the target tag is configured to identify an operation state of the sequencing data in a predetermined processing flow; the generating unit is used for executing the preset processing flow according to the target label and generating an execution result of the preset processing flow; the monitoring unit is used for monitoring the preset processing flow according to the execution result.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, which includes a stored program, wherein when the program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute any one of the methods for monitoring sequencing data processing flow.
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, wherein the program executes any one of the methods for monitoring the sequencing data processing flow.
In the embodiment of the invention, the method for monitoring the sequencing data processing flow comprises the steps of firstly, obtaining a target label of the sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow; then, executing the preset processing flow according to the target label to generate an execution result of the preset processing flow; and finally, monitoring the preset processing flow according to the execution result. The method utilizes the target tag to identify the running state of the sequencing data in the preset processing flow, and executes the preset processing flow according to the target tag, so that the communication between an operation manager and information personnel in the project execution process can be reduced to the minimum, and the technical problems of long labor time, low labor efficiency and long period existing in the prior art due to the adoption of a sequencing data analysis mode of interaction between the operation manager and the information personnel are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a logic diagram of a prior art method of monitoring sequencing data processing flow;
FIG. 2 is a flow diagram of a method of monitoring sequencing data processing flow according to an embodiment of the present application;
FIG. 3 is a logic diagram of a conventional test tag performing a predetermined process flow according to an embodiment of the present application;
FIG. 4 is a logic diagram of a process flow for performing a predetermined process on a test plus test tag according to an embodiment of the present application;
FIG. 5 is a logic diagram of a compatibility tag executing a predetermined process flow according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a monitoring device for sequencing data processing flow according to an embodiment of the present application;
FIG. 7 is a logic diagram of a prior art method of monitoring sequencing data processing flow according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As mentioned in the background art, the prior art adopts a sequencing data analysis mode in which an operation manager interacts with information personnel, which has the technical problems of long labor time, low efficiency and long period.
In accordance with an embodiment of the present invention, there is provided a method for monitoring a sequencing data processing flow, it should be noted that the steps illustrated in the flow chart of the accompanying drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flow chart, in some cases the steps illustrated or described may be performed in an order different than that described herein.
Fig. 2 is a flowchart of a monitoring method of a sequencing data processing flow according to an embodiment of the present application, and as shown in fig. 2, the method includes the following steps:
step S101, obtaining a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow;
step S102, executing the preset processing flow according to the target label and generating an execution result of the preset processing flow;
step S103, monitoring the predetermined processing flow according to the execution result.
The monitoring method of the sequencing data processing flow comprises the following steps of firstly, obtaining a target label of the sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow; then, executing the preset processing flow according to the target label to generate an execution result of the preset processing flow; and finally, monitoring the preset processing flow according to the execution result. The method utilizes the target tag to identify the running state of the sequencing data in the preset processing flow, and executes the preset processing flow according to the target tag, so that the communication between an operation manager and information personnel in the execution process of the preset processing flow can be reduced to the minimum, and the technical problems of long labor time, low labor efficiency and long period existing in the prior art due to the adoption of a sequencing data analysis mode of interaction between the operation manager and the information personnel are solved.
In practical applications, the above method needs to be used in a Linux system Python environment, where the Python version may be Python 3.8.
In order to monitor the state of the sequencing data in the processing process in real time and automatically allocate the predetermined processing flow according to different stages of the analysis flow of the sequencing data, in an embodiment of the present application, the target tag includes at least one of the following: the system comprises a conventional test tag, a test plus test tag and a compatibility tag, wherein the conventional test tag is used for identifying the running state of the sequencing data in a normal processing flow, the test plus test tag is used for identifying the running state of the sequencing data in the test plus test processing flow, and the compatibility tag is used for identifying the running state of the sequencing data in a human-computer compatible processing flow.
In practical application, the number of available tags is dozens of tags, the sequencing data processing flow can be well monitored by means of the tag algorithm, the algorithm has good expandability, can be increased or decreased according to requirements at any time, and greatly improves flexibility. For common conventional test tags, test plus test tags, and compatibility tags, the corresponding meaning of each state is as follows:
in another embodiment of the present application, as shown in fig. 3, when the target tag includes the conventional test tag, the executing the predetermined processing flow according to the target tag to generate an execution result of the predetermined processing flow includes: if the conventional test tag is executed to wait for data to be downloaded, detecting whether the sequencing data is downloaded, and sending a detection result to a target object; after the target object is confirmed, if the conventional test tag is executed, all samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data, and sending a starting notice to the target object; after the target object is confirmed, if the conventional test tag is in operation, analyzing the sequencing data and sending an analysis result to the target object; after the target object is confirmed, if the execution of the conventional test tag is operation completion, the sequencing data is uploaded, and a completion result is sent to the target object, or if the execution of the conventional test tag is abnormal interruption, an abnormal result is sent to the target object. The conventional test labels described above are primarily used to address the conventional test requirements.
Specifically, the target object may be an operation manager, and the target object to which the abnormal result is transmitted may be a biological information engineer.
In order to further reduce the sequencing data processing flow time, in another embodiment of the present application, before the executing the conventional test tag as waiting data is off-line, the method further includes: and synchronizing the sequencing data to a preset database.
Specifically, the preset database may be, but is not limited to, a mongoDB-based database that collects and stores detailed information of the predetermined process flow, data that has been off-line, and predetermined process flow sample situations and requirements submitted by the operation manager. The information is stored in the form of a predetermined processing flow unit, and some predetermined processing flow judgment is made by combining the stored information, such as whether the bam file needs to be reserved or not, whether the test is needed or not, and the like.
In a specific embodiment of the present application, in order to obtain the latest scheduled processing flow information and the latest offline sample information in real time and query and extract information in the database in time, a set of API software suite for interacting with the core database is programmed, and the suite provides functions of updating database information, querying, extracting, visualizing, counting, and the like of the scheduled processing flow data of the database.
In another embodiment of the present application, as shown in fig. 4, when the target tag includes the test addition tag, the executing the predetermined processing flow according to the target tag to generate an execution result of the predetermined processing flow includes: if the test adding label is executed to wait for the test data to be downloaded, detecting whether the test sequencing data is downloaded or not, and sending a test detection result to a target object; after the target object is confirmed, if the test and the test tag are executed, all test samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data of the test, and sending a test starting notice to the target object; after the target object is confirmed, if the test label is added in the test operation, analyzing the sequencing data of the test and sending a test analysis result to the target object; after the target object is confirmed, if the test additional test tag is executed to wait for additional test data to be downloaded, whether the additional test data is downloaded is detected, and an additional test result is sent to the target object; after the target object is confirmed, if the test adding and testing label is executed, all the adding and testing samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data added and tested, and sending an adding and testing starting notice to the target object; after the target object is confirmed, if the test adding and testing label is executed in adding and testing operation, analyzing the added and tested sequencing data, and sending an adding and testing analysis result to the target object; and after the target object is confirmed, if the test and test tag is executed to be completed, uploading the sequencing data to be tested, and sending a completed result to the target object.
In practical application, when the target label comprises the test and test label, the difficulty lies in how to solve the time point when the test starts after the test is finished, in the prior art, the time point is often determined after operation and customer determination, and then the biological information engineer is informed to perform the test, the time consumption of the whole communication process is long, the test of the preset processing flow is automatically started according to the label in the mail sent by the operation manager through the python programming language, the biological information engineer is not required to be informed, the test analysis is started at the first time when the response is obtained, the manual time is greatly reduced, and the human efficiency is increased.
Specifically, in order to replace a biological information analyzer to interact with an operation manager, the function of interacting by means of mails is developed, the operation manager receives a mail prompt at key nodes in the sequencing data analysis process, such as data downloading, analysis result report uploading and the like, and selects and replies a specific field according to the running state description of the sequencing data in the normal processing flow in the mails, and in the sequencing data processing flow monitoring method, a mail receiving box is checked at regular time, and the next operation is performed on the analysis flow according to the feedback of the operation manager, such as whether to start data analysis, whether to wait for data to be added for measurement or directly upload a report end and the like.
In order to determine a state in which the predetermined processing flow is executed according to the test addition tag in time, in another embodiment of the present application, as shown in fig. 4, after sending a test start notification to a target object or sending an addition analysis result to the target object, the method further includes: and after the target object is confirmed, if the test and test tag is executed to be an abnormal interrupt, an abnormal result is sent to the target object.
In practical application, the test and test tags are executed as abnormal interrupts, information can be synchronized in a database, and monitoring efficiency is guaranteed.
In another embodiment of the present application, as shown in fig. 5, when the target tag includes the compatibility tag, the executing the predetermined processing flow according to the target tag to generate an execution result of the predetermined processing flow includes: if the compatibility tag is analyzed manually, sending a feedback result to the target object; or, if the compatibility tag is a history information collection table, the reminding information is not sent to the target object. The compatibility tag is mainly used for solving the compatibility problem of automatic execution and manual execution in a series of special cases, for example, the process flow which is analyzed by the human should not be automatically executed any more.
In order to increase the application range of the monitoring method for the sequencing data processing flow, in a specific embodiment of the present application, an interface is provided for running a specified sequencing data analysis flow, and theoretically, most data analysis flows can be interfaced as long as an effective interface function is provided for the interface, so that monitoring, automation control and the like of the data analysis flows are realized.
The embodiment of the present application further provides a monitoring device for a sequencing data processing flow, and it should be noted that the monitoring device for a sequencing data processing flow of the embodiment of the present application can be used for executing the monitoring method for a sequencing data processing flow provided by the embodiment of the present application. The following describes a monitoring apparatus for a sequencing data processing flow provided in an embodiment of the present application.
Fig. 6 is a schematic diagram of a monitoring apparatus for sequencing data processing flow according to an embodiment of the present application, as shown in fig. 6, the apparatus includes:
an obtaining unit 10, configured to obtain a target tag of a sequencing data processing flow, where the target tag is used to identify an operation state of the sequencing data in a predetermined processing flow;
a generating unit 20, configured to execute the predetermined processing flow according to the target tag, and generate an execution result of the predetermined processing flow;
and a monitoring unit 30, configured to monitor the predetermined processing flow according to the execution result.
The monitoring device for the sequencing data processing flow firstly acquires a target label of the sequencing data processing flow through an acquisition unit 10, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow; then, the generating unit 20 executes the predetermined processing flow according to the target tag to generate an execution result of the predetermined processing flow; and finally, the monitoring unit 30 monitors the predetermined processing flow according to the execution result. The device utilizes the target label to mark the running state of the sequencing data in the preset processing flow, and executes the preset processing flow according to the target label, so that the communication between an operation manager and information personnel in the execution process of the preset processing flow can be reduced to the minimum, and the technical problems of long labor time, low labor efficiency and long period in the prior art due to the adoption of the sequencing data analysis mode of interaction between the operation manager and the information personnel are solved.
In practical applications, the above device needs to be used in a Linux system Python environment, where the Python version may be Python 3.8.
In order to monitor the state of the sequencing data in the processing process in real time and automatically allocate the predetermined processing flow according to different stages of the analysis flow of the sequencing data, in an embodiment of the present application, the target tag includes at least one of the following: the system comprises a conventional test tag, a test plus test tag and a compatibility tag, wherein the conventional test tag is used for identifying the running state of the sequencing data in a normal processing flow, the test plus test tag is used for identifying the running state of the sequencing data in the test plus test processing flow, and the compatibility tag is used for identifying the running state of the sequencing data in a human-computer compatible processing flow.
In practical application, the number of available tags is dozens of tags, the sequencing data processing flow can be well monitored by means of the tag algorithm, the algorithm has good expandability, can be increased or decreased according to requirements at any time, and greatly improves flexibility. For common conventional test tags, test plus test tags, and compatibility tags, the corresponding meaning of each state is as follows:
| normal-waiting | waiting for data to get off |
| test-waiting | Waiting for test data to be sent out |
| add-waiting | Waiting additional measurement data unloading machine |
| normal-ready | All samples have data to get off-line and wait for execution |
| test-ready | All test samples have data to be downloaded and wait for execution |
| add-ready | All the tested samples have data to be downloaded and wait for execution |
| normal-running | In operation |
| test-running | In test operation |
| add-running | In the process of adding and measuring |
| failed | Abnormal interrupt |
| completed | Completion of operation |
| manual-analysis | Has been analyzed manually |
| history-info | Historical information collection table |
In another embodiment of the present application, the generating unit includes a first detecting module, a first starting module, a first analyzing module, and a first uploading module, where the first detecting module is configured to, when the target tag includes the conventional test tag, execute that the conventional test tag is waiting for data to get off, detect whether the sequencing data get off, and send a detection result to a target object; the first starting module is used for starting and analyzing the sequencing data and sending a starting notice to the target object when the conventional test tag is executed and all samples have data to be downloaded and wait to be executed after the target object is confirmed; the first analysis module is used for analyzing the sequencing data and sending an analysis result to the target object when the conventional test tag is in operation after the target object is confirmed; the first uploading module is used for uploading the sequencing data and sending a completion result to the target object when the execution of the conventional test tag is finished after the target object is confirmed, or sending an abnormal result to the target object when the execution of the conventional test tag is abnormal interruption. The conventional test labels described above are primarily used to address the conventional test requirements.
Specifically, the target object may be an operation manager, and the target object to which the abnormal result is transmitted may be a biological information engineer.
In order to further reduce the sequencing data processing flow time, in another embodiment of the present application, the apparatus further includes a synchronization unit, and the synchronization unit synchronizes the sequencing data to a predetermined database before the regular test tag is executed as the waiting data.
Specifically, the preset database may be, but is not limited to, a mongoDB-based database that collects and stores detailed information of the predetermined process flow, data that has been off-line, and predetermined process flow sample situations and requirements submitted by the operation manager. The information is stored in the form of a predetermined processing flow unit, and some predetermined processing flow judgment is made by combining the stored information, such as whether the bam file needs to be reserved or not, whether the test is needed or not, and the like.
In a specific embodiment of the present application, in order to obtain the latest scheduled processing flow information and the latest offline sample information in real time and query and extract information in the database in time, a set of API software suite for interacting with the core database is programmed, and the suite provides functions of updating database information, querying, extracting, visualizing, counting, and the like of the scheduled processing flow data of the database.
In another embodiment of the present application, the generating unit includes a second detecting module, a second starting module, a second analyzing module, a third detecting module, a third starting module, a third analyzing module, and a second uploading module, where the second detecting module is configured to execute the test-added tag as a wait test data downloading process when the target tag includes the test-added tag, and detect whether the test data is downloaded, and send a test detection result to the target object; the second starting module is used for executing the test and the test tag after the target object is confirmed, starting and analyzing the sequencing data of the test if all test samples have data to be downloaded and wait for execution, and sending a test starting notice to the target object; the second analysis module is configured to, after the target object is confirmed, execute the test-plus-test label as a test operation, analyze the test sequencing data, and send a test analysis result to the target object; the third detection module is configured to execute that the test loading tag waits for loading data to be loaded after the target object is confirmed, detect whether the loaded sequencing data is loaded and send a loading detection result to the target object; the third starting module is used for executing the test adding and testing label that all the adding and testing samples have data to be downloaded and wait for execution after the target object is confirmed, starting to analyze the added and tested sequencing data, and sending an adding and testing starting notice to the target object; the third analysis module is configured to, after the target object is confirmed, execute the test-plus-test operation with the test-plus-test tag, analyze the sequence data that is plus-tested, and send an plus-test analysis result to the target object; and the second uploading module is used for uploading the sequencing data of the test and test if the test and test tag is finished after the target object is confirmed, and sending a finished result to the target object.
In practical application, when the target label comprises the test and test label, the difficulty lies in how to solve the time point when the test starts after the test is finished, in the prior art, the time point is often determined after operation and customer determination, and then the biological information engineer is informed to perform the test, the time consumption of the whole communication process is long, the test of the preset processing flow is automatically started according to the label in the mail sent by the operation manager through the python programming language, the biological information engineer is not required to be informed, the test analysis is started at the first time when the response is obtained, the manual time is greatly reduced, and the human efficiency is increased.
Specifically, in order to replace a biological information analyzer to interact with an operation manager, the function of interacting by means of mails is developed, the operation manager receives a mail prompt at key nodes in the sequencing data analysis process, such as data downloading, analysis result report uploading and the like, and selects and replies a specific field according to the running state description of the sequencing data in the normal processing flow in the mails, and in the sequencing data processing flow monitoring method, a mail receiving box is checked at regular time, and the next operation is performed on the analysis flow according to the feedback of the operation manager, such as whether to start data analysis, whether to wait for data to be added for measurement or directly upload a report end and the like.
In order to determine a state in which the predetermined processing flow is executed according to the test addition tag in time, in another embodiment of the present application, the apparatus further includes an execution unit, where the execution unit is configured to, after sending a test start notification to a target object or sending an addition analysis result to the target object, execute that the test addition tag is an abnormal interrupt after the target object is confirmed, and send an abnormal result to the target object.
In practical application, the test and test tags are executed as abnormal interrupts, information can be synchronized in a database, and monitoring efficiency is guaranteed.
In another embodiment of the present application, the generating unit includes an executing module, where the executing module is configured to, when the target tag includes the compatibility tag, execute that the compatibility tag has been analyzed manually, and send a feedback result to the target object; or, if the compatibility tag is a history information collection table, the reminding information is not sent to the target object. The compatibility tag is mainly used for solving the compatibility problem of automatic execution and manual execution in a series of special cases, for example, the process flow which is analyzed by the human should not be automatically executed any more.
In order to increase the application range of the monitoring method for the sequencing data processing flow, in a specific embodiment of the present application, an interface is provided for running a specified sequencing data analysis flow, and theoretically, most data analysis flows can be interfaced as long as an effective interface function is provided for the interface, so that monitoring, automation control and the like of the data analysis flows are realized.
The monitoring device of the sequencing data processing flow comprises a processor and a memory, wherein the acquisition unit, the generation unit, the monitoring unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can set one or more than one, and the technical problems of long labor time, low labor efficiency and long period in the prior art due to the adoption of a sequencing data analysis mode of interaction between an operation manager and information personnel are solved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
The embodiment of the invention provides a computer readable storage medium, wherein a program is stored on the computer readable storage medium, and the program is used for realizing the monitoring method of the sequencing data processing flow when being executed by a processor.
The embodiment of the invention provides a processor, which is used for running a program, wherein the program executes the monitoring method of the sequencing data processing flow during running.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized:
step S101, obtaining a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow;
step S102, executing the preset processing flow according to the target label and generating an execution result of the preset processing flow;
step S103, monitoring the predetermined processing flow according to the execution result.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device:
step S101, obtaining a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow;
step S102, executing the preset processing flow according to the target label and generating an execution result of the preset processing flow;
step S103, monitoring the predetermined processing flow according to the execution result.
In order to make the technical solutions of the present application more clearly understood by those skilled in the art, the technical solutions of the present application will be described in detail below with reference to specific examples and comparative examples.
Examples
The embodiment provides a method for monitoring a sequencing data processing flow, which comprises the following steps: acquiring a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow; executing the preset processing flow according to the target label to generate an execution result of the preset processing flow; and monitoring the preset processing flow according to the execution result.
As shown in fig. 7, the execution mode of the monitoring method of the sequencing data processing flow in this embodiment is as follows:
and executing a preset processing flow according to the target label.
In the waiting phase of the preset processing flow, namely normal-waiting and test-waiting: checking the requirement, and synchronizing the database; checking whether the data of the off-line is available; and sending a mail notification.
In the waiting stage of the preset processing flow and the measured data, namely add-waiting: checking whether the data is off-line; sending a mail to inform an operation manager; starting analysis according to the reply of the operation manager.
When the data is downloaded and waiting for the execution phases, namely normal-ready, test-ready and add-ready: checking whether the data is off-line; sending a mail to inform an operation manager; starting analysis according to the reply of the operation manager.
When the predetermined processing flow is completed, namely completed: uploading data; and sending the mail to inform an operation manager.
When the predetermined processing flow fails to run, namely failed: sending a mail to inform an operation manager and a biological information engineer; the project is interrupted and the information is synchronized to the database.
According to the method for monitoring the sequencing data processing flow, the special label is added to the preset processing flow, the preset processing flow is correspondingly processed according to the target label, and automatic operation and monitoring of execution of the preset processing flow are realized through the technical means of labeling of the preset processing flow.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for monitoring a sequencing data processing flow is characterized by comprising the following steps:
acquiring a target label of a sequencing data processing flow, wherein the target label is used for identifying the running state of the sequencing data in a preset processing flow;
executing the preset processing flow according to the target label to generate an execution result of the preset processing flow;
and monitoring the preset processing flow according to the execution result.
2. The method of claim 1, wherein the target tag comprises at least one of: the sequence data processing method comprises a conventional test tag, a test adding test tag and a compatibility tag, wherein the conventional test tag is used for identifying the running state of the sequence data in a normal processing flow, the test adding test tag is used for identifying the running state of the sequence data in the test adding test processing flow, and the compatibility tag is used for identifying the running state of the sequence data in a human-computer compatible processing flow.
3. The method of claim 2, wherein when the target tag includes the regular test tag, executing the predetermined process flow according to the target tag, and generating an execution result of the predetermined process flow comprises:
if the conventional test tag is executed to wait for data to be downloaded, detecting whether the sequencing data is downloaded, and sending a detection result to a target object;
after the target object is confirmed, if the conventional test tag is executed, all samples have data to be downloaded and wait to be executed, starting to analyze the sequencing data, and sending a starting notice to the target object;
after the target object is confirmed, if the conventional test tag is in operation, analyzing the sequencing data, and sending an analysis result to the target object;
and after the target object is confirmed, if the execution of the conventional test tag is operation completion, uploading the sequencing data and sending a completion result to the target object, or if the execution of the conventional test tag is abnormal interruption, sending an abnormal result to the target object.
4. The method of claim 3, wherein prior to performing the regular test tag as a wait for data to be off-line, the method further comprises:
and synchronizing the sequencing data to a preset database.
5. The method according to claim 2, wherein when the target tag includes the test plus test tag, executing the predetermined processing flow according to the target tag, and generating an execution result of the predetermined processing flow includes:
if the test adding label is executed to wait for the test data to be downloaded, detecting whether the test sequencing data is downloaded or not, and sending a test detection result to a target object;
after the target object is confirmed, if the test and the test tag are executed, all test samples have data to be downloaded and wait for execution, starting to analyze the sequencing data of the test, and sending a test starting notice to the target object;
after the target object is confirmed, if the test label is added in the test operation, analyzing the sequencing data of the test and sending the test analysis result to the target object;
after the target object is confirmed, if the test adding and testing label is executed to wait for adding and testing data to get off, detecting whether the sequencing data added and tested get off, and sending an adding and detecting result to the target object;
after the target object is confirmed, if the test adding and testing label is executed, all the adding and testing samples have data to be downloaded and wait for execution, starting to analyze the sequencing data added and tested, and sending a adding and testing starting notice to the target object;
after the target object is confirmed, if the test adding and testing label is executed in the adding and testing operation, analyzing the added and tested sequencing data, and sending an adding and testing analysis result to the target object;
and after the target object is confirmed, if the test and test tag is executed to be completed, uploading the test and test sequencing data, and sending a completed result to the target object.
6. The method of claim 3, wherein after sending a test initiation notification to the target object or sending an additive analysis result to the target object, the method further comprises:
and after the target object is confirmed, if the test adding and testing label is executed to be abnormal interruption, an abnormal result is sent to the target object.
7. The method of claim 2, wherein when the target tag includes the compatibility tag, executing the predetermined process flow according to the target tag, and generating an execution result of the predetermined process flow comprises:
if the compatibility tag is analyzed manually, sending a feedback result to the target object; or, if the compatibility tag is executed as a history information collection table, the reminding information is not sent to the target object any more.
8. A monitoring device for sequencing data processing flow is characterized by comprising:
the sequencing data processing device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a target label of a sequencing data processing flow, and the target label is used for identifying the running state of the sequencing data in a preset processing flow;
the generating unit is used for executing the preset processing flow according to the target label and generating an execution result of the preset processing flow;
and the monitoring unit is used for monitoring the preset processing flow according to the execution result.
9. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method for monitoring sequencing data processing flow according to any one of claims 1 to 7.
10. A processor configured to execute a program, wherein the program executes the method for monitoring the sequencing data processing flow of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111680008.3A CN114328262A (en) | 2021-12-31 | 2021-12-31 | Method and device for monitoring sequencing data processing flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111680008.3A CN114328262A (en) | 2021-12-31 | 2021-12-31 | Method and device for monitoring sequencing data processing flow |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114328262A true CN114328262A (en) | 2022-04-12 |
Family
ID=81022681
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111680008.3A Pending CN114328262A (en) | 2021-12-31 | 2021-12-31 | Method and device for monitoring sequencing data processing flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114328262A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116024079A (en) * | 2023-03-16 | 2023-04-28 | 深圳市真迈生物科技有限公司 | Method, device, sequencing system and storage medium for controlling chip loading |
-
2021
- 2021-12-31 CN CN202111680008.3A patent/CN114328262A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116024079A (en) * | 2023-03-16 | 2023-04-28 | 深圳市真迈生物科技有限公司 | Method, device, sequencing system and storage medium for controlling chip loading |
| CN116024079B (en) * | 2023-03-16 | 2023-08-04 | 深圳市真迈生物科技有限公司 | Method, device, sequencing system and storage medium for controlling chip loading |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109361562B (en) | Automatic testing method based on associated network equipment access | |
| CN110058998B (en) | Software testing method and device | |
| CN112269697B (en) | Equipment storage performance testing method, system and related device | |
| CN102789405A (en) | Automated testing method and system for mainboard | |
| CN114328262A (en) | Method and device for monitoring sequencing data processing flow | |
| CN115147236A (en) | Processing method, processing device and electronic equipment | |
| CN109818825B (en) | Rack server intelligent test method and system | |
| CN110647453A (en) | Application performance comparison test method, system, equipment and computer readable storage medium | |
| CN105204929B (en) | Batch processing dispatching method, device and system | |
| CN107025166A (en) | A kind of log collecting method and system | |
| CN111835566A (en) | System fault management method, device and system | |
| CN111200654A (en) | Client request error processing method and device | |
| CN108710692A (en) | A kind of automobile part production line test system and method | |
| CN115292159A (en) | Test debugging method and system, corresponding computer equipment and storage medium | |
| CN114265735A (en) | Equipment testing method, system, device and computer readable storage medium | |
| CN112395119B (en) | Abnormal data processing method, device, server and storage medium | |
| CN112104497A (en) | Terminal management method, device, system, server, terminal and storage medium | |
| CN118245385B (en) | Test method, test platform, equipment, medium and product | |
| CN108761303A (en) | A kind of detection method and system based on pcb board | |
| CN114942623B (en) | Method and device for testing controller, vehicle and computer readable storage medium | |
| CN117453456A (en) | Firmware version security assessment method and related device | |
| CN115129610B (en) | Method and system for testing aircraft engineering algorithm | |
| CN109491845B (en) | Test method and system for storage product controller | |
| CN114138578B (en) | Server testing method and device | |
| CN114816938A (en) | Server aging program monitoring method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |