CN112235396B - Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium - Google Patents

Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium Download PDF

Info

Publication number
CN112235396B
CN112235396B CN202011091563.8A CN202011091563A CN112235396B CN 112235396 B CN112235396 B CN 112235396B CN 202011091563 A CN202011091563 A CN 202011091563A CN 112235396 B CN112235396 B CN 112235396B
Authority
CN
China
Prior art keywords
link
content
scheduling
task
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011091563.8A
Other languages
Chinese (zh)
Other versions
CN112235396A (en
Inventor
刘光林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202011091563.8A priority Critical patent/CN112235396B/en
Publication of CN112235396A publication Critical patent/CN112235396A/en
Application granted granted Critical
Publication of CN112235396B publication Critical patent/CN112235396B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/61Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1042Peer-to-peer [P2P] networks using topology management mechanisms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/55Push-based network services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/62Establishing a time schedule for servicing the requests

Abstract

The application relates to a content processing link adjustment method, a content processing link adjustment device, computer equipment and a storage medium. The method comprises the following steps: acquiring a first scheduling task of target content and a corresponding first content processing link; executing a first scheduling task according to the first content processing link; when the updated content attribute of the target content is detected, obtaining a target content attribute set according to the updated content attribute, and generating a second scheduling task of the target content; determining a second content processing link corresponding to the second scheduling task according to the target content attribute set; when the first content processing link and the second content processing link have conflict link processing nodes, stopping executing the first scheduling task, traversing the link processing nodes in the second content processing link according to the time sequence, and obtaining the output result identification of the currently traversed link processing node; and when the node output result is not inquired according to the output result identification, executing the scheduling subtask corresponding to the link processing node. The method can improve the content processing efficiency.

Description

Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for adjusting a content processing link, a computer device, and a storage medium.
Background
With the continuous development of computer technology and internet technology, more and more multimedia information is spread and presented by means of the internet, and people can conveniently and quickly acquire various multimedia information without going out. Multimedia information such as pictures, articles, and videos can be understood as contents. When a user publishes contents through various platforms, the scheduling server acquires the contents published by the user from the corresponding platforms, processes the acquired contents according to the pre-configured content processing link, and pushes the processed contents to the corresponding platforms for displaying. And if the content attribute corresponding to the content is updated, the content is processed again according to the pre-configured content processing link. However, this content processing method has a problem of low processing efficiency.
Disclosure of Invention
In view of the above, it is necessary to provide a content processing link adjusting method, apparatus, computer device, and storage medium capable of improving content processing efficiency in view of the above technical problems.
A content processing link adjustment method, the method comprising:
acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
executing the first scheduling task according to the first content processing link;
when the updated content attribute of the target content is detected, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content;
determining a second content processing link corresponding to the second scheduling task according to the target content attribute set;
when the first content processing link and the second content processing link have conflicted link processing nodes, stopping the execution of the first scheduling task, traversing the link processing nodes in the second content processing link according to a time sequence, and obtaining an output result identifier corresponding to the currently traversed link processing node;
and when the corresponding node output result is not inquired according to the currently obtained output result identifier, executing a scheduling subtask corresponding to the currently traversed link processing node.
A content processing link adjustment apparatus, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
a task execution module, configured to execute the first scheduling task according to the first content processing link;
the obtaining module is further configured to, when an updated content attribute of the target content is detected, obtain a target content attribute set of the target content according to the updated content attribute, and generate a second scheduling task corresponding to the target content;
the obtaining module is further configured to determine a second content processing link corresponding to the second scheduling task according to the target content attribute set;
a link adjustment module, configured to stop execution of the first scheduling task when a conflicting link processing node exists between the first content processing link and the second content processing link, traverse the link processing node in the second content processing link according to a time sequence, and obtain an output result identifier corresponding to the currently traversed link processing node;
and the task execution module is also used for executing the scheduling subtask corresponding to the currently traversed link processing node when the corresponding node output result is not inquired according to the currently obtained output result identifier.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
executing the first scheduling task according to the first content processing link;
when the updated content attribute of the target content is detected, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content;
determining a second content processing link corresponding to the second scheduling task according to the target content attribute set;
when the first content processing link and the second content processing link have conflicted link processing nodes, stopping the execution of the first scheduling task, traversing the link processing nodes in the second content processing link according to a time sequence, and obtaining an output result identifier corresponding to the currently traversed link processing node;
and when the corresponding node output result is not inquired according to the currently obtained output result identifier, executing a scheduling subtask corresponding to the currently traversed link processing node.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
executing the first scheduling task according to the first content processing link;
when the updated content attribute of the target content is detected, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content;
determining a second content processing link corresponding to the second scheduling task according to the target content attribute set;
when the first content processing link and the second content processing link have conflicted link processing nodes, stopping the execution of the first scheduling task, traversing the link processing nodes in the second content processing link according to a time sequence, and obtaining an output result identifier corresponding to the currently traversed link processing node;
and when the corresponding node output result is not inquired according to the currently obtained output result identifier, executing a scheduling subtask corresponding to the currently traversed link processing node.
In the method, the device, the computer equipment and the storage medium for adjusting the content processing link, during the process of executing the first scheduling task corresponding to the target content according to the first content processing link corresponding to the first scheduling task, or after the first scheduling task is executed, if the content attribute of the target content is detected to be updated, generating the second scheduling task corresponding to the target content, optimizing the second content processing link according to the target content attribute set after the content attribute is updated, if the conflicting link processing nodes exist in the first content processing link and the second content processing link and indicate that the first scheduling task conflicts with the second scheduling task, stopping executing the first scheduling task, traversing the link processing nodes in the second content processing link according to the time sequence, and identifying whether the node output result exists according to the output result currently corresponding to the sequentially traversed link processing nodes, and determining the scheduling subtasks to be executed in the second scheduling task, and implementing the processing of the target content after the content attribute is updated by executing each scheduling subtask to be executed, so that the processing resources can be saved and the content processing efficiency can be improved under the condition of ensuring the processing accuracy of the target content.
Drawings
FIG. 1 is a diagram of an exemplary implementation of a method for adjusting a content processing link;
FIG. 2 is a flow diagram illustrating a method for adjusting a content processing link according to one embodiment;
FIG. 3 is a schematic diagram of a link structure of a first content processing link determined based on an initial set of content attributes of target content in one embodiment;
FIG. 4 is an architecture diagram of a scheduling system in one embodiment;
FIG. 5 is a flow chart illustrating a method for adjusting a content processing link according to another embodiment;
FIG. 6 is a block diagram showing the structure of a content processing link adjustment apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The content processing link adjustment method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the dispatch server 104 over a network. The scheduling server 104 obtains a first scheduling task corresponding to the target content and a first content processing link corresponding to the first scheduling task, executes the first scheduling task according to the first content processing link, obtains a target content attribute set of the target content according to the update content attribute when the update content attribute of the target content is detected, generates a second scheduling task corresponding to the target content, determines a second content processing link corresponding to the second scheduling task according to the target content attribute set, stops the execution of the first scheduling task when the first content processing link and the second content processing link have conflicting link processing nodes, traverses the link processing nodes in the second content processing link according to the time sequence, obtains an output result identifier corresponding to the current link processing node, and when the corresponding node output result is not queried according to the output result identifier obtained by the traversal, and executing the scheduling subtask corresponding to the currently traversed link processing node. The scheduling server 104 may obtain the target content to be processed from the terminal 102, and may further display the processed target content through the terminal 102. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the scheduling server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, the scheduling server 104 corresponds to a scheduling system for implementing the content processing and scheduling functions of the scheduling system.
In one embodiment, as shown in fig. 2, a content processing link adjustment method is provided, which is described by taking the method as an example applied to the scheduling server in fig. 1, and includes the following steps:
step 202, a first scheduling task corresponding to the target content and a first content processing link corresponding to the first scheduling task are obtained.
The target content refers to content to be processed. The content refers to multimedia information such as pictures, articles, videos and the like. The first scheduling task refers to a scheduling task currently corresponding to the target content, and specifically may refer to a scheduling task that is currently executed or is to be executed and corresponds to the target content. The first content processing link refers to a processing link or an execution link referred to when the first scheduling task is executed, the first content processing link includes a plurality of link processing nodes arranged according to a time sequence, each link processing node corresponds to one scheduling subtask in the first scheduling task, and processing logic for executing the corresponding scheduling subtask can be acquired based on the link processing nodes.
Specifically, the scheduling server generates a first scheduling task corresponding to the target content, queries a content processing link matched with an initial content attribute set according to the initial content attribute set corresponding to the target content, and determines the queried content processing link as a first content processing link corresponding to the first scheduling task. It will be appreciated that the initial set of content attributes is a set of content attributes to which the target content currently corresponds. The initial set of content attributes corresponds to a target set of content attributes derived from the updated set of content attributes.
In one embodiment, the content attribute refers to attribute characteristics of the content, such as content type, content source, content author, popularity or awareness of the content author, content title, content body, content data volume, recommendation or sharing, and the like, which are not listed herein.
In one embodiment, when target content to be processed is acquired, the scheduling server generates a first scheduling task corresponding to the target content, acquires an initial content attribute set corresponding to the target content, and determines a content processing link pre-configured for the initial content attribute set as a first content processing link corresponding to the first scheduling task.
In one embodiment, the scheduling task corresponding to the target content corresponds to the content attribute set one to one, if the content attribute of the target content is updated, a new content attribute set is obtained according to the updated content attribute, and accordingly, a new scheduling task is generated for the target content with the updated content attribute, and the new scheduling task corresponds to the new content attribute set. The scheduling server is pre-configured with a plurality of content attribute sets, and is pre-configured with a matched content processing link for each content attribute set, so that the content processing link corresponding to the scheduling task can be determined based on the content attribute set corresponding to the scheduling task.
In an embodiment, if no matched content processing link is found according to the initial content attribute set of the target content, a content attribute set with the highest matching degree with the initial content attribute set is selected from a plurality of pre-configured content attribute sets, and the content processing link matched with the selected content attribute set is determined as the content processing link matched with the initial content attribute set, that is, the content processing link is determined as the first content processing link corresponding to the corresponding first scheduling task. It can be understood that the matching degree between the two content attribute sets can be determined based on the existing matching degree calculation method, and will not be described herein again.
Step 204, executing the first scheduling task according to the first content processing link.
Specifically, after acquiring a first content processing link corresponding to a first scheduling task corresponding to the target content, the scheduling server executes the corresponding first scheduling task according to the first content processing link, so as to implement processing of the target content. It is understood that the scheduling server executes the process of the first scheduling task according to the first content processing link, that is, processes the target content according to the first content processing link.
And step 206, when the updated content attribute of the target content is detected, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content.
Specifically, the scheduling server dynamically detects the update of the content attribute corresponding to the target content, and when it is detected that the content attribute of the target content is updated, acquires the updated content attribute as the updated content attribute. The scheduling server obtains a corresponding target content attribute set according to the updated content attribute corresponding to the target content and the initial content attribute set, that is, the corresponding content attribute in the initial content attribute set is updated according to the updated content attribute to obtain the target content attribute set. Correspondingly, the scheduling server generates a corresponding second scheduling task for the target content with the updated content attribute, so that the target content with the updated content attribute can be processed by executing the second scheduling task.
And step 208, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
Specifically, after obtaining a target content attribute set and a second scheduling task corresponding to the target content, the scheduling server queries a content processing link matched with the target content attribute set according to the target content attribute set, and determines the queried content processing link as a second content processing link corresponding to the second scheduling task.
It is to be understood that, since the initial content attribute set corresponding to the target content is not consistent with the target content attribute set, the first content processing link determined based on the initial content attribute set may be the same as or different from the second content processing link determined based on the target content attribute set, and is specifically determined according to the content processing links pre-configured for the initial content attribute set and the target content attribute set respectively.
For example, assuming that the target content is an article, determining that a first content processing link corresponding to a first scheduling task is "intra-picture linking, title processing, text understanding, and manual review" according to an initial content attribute set of the target content, and in a process of executing the first scheduling task according to the first content processing link, assuming that a title of the article is updated when the first scheduling task is executed to a scheduling subtask corresponding to a link processing node of "title processing", obtaining a target content attribute set according to an updated content attribute of "content title", generating a second scheduling task, and determining that a second content processing link corresponding to the second scheduling task is "intra-picture linking, title processing, text understanding, and manual review" according to the target content attribute set. It follows that when the content attribute of the target content is updated, the content processing links determined based on the content attribute sets before and after the update may be consistent.
For example, assuming that the updated content attribute corresponding to the target content is "no manual review is needed", the second content processing link is determined to be "intra-picture linking, title processing, and text understanding" according to the target content attribute set, and thus it can be seen that the first content processing link is inconsistent with the second content processing link, and the second content processing link is compared with the first content processing link, which reduces the call of link processing nodes, optimizes the content processing link, and can extract the content processing speed while saving resources.
Step 210, when the first content processing link and the second content processing link have conflicting link processing nodes, stopping the execution of the first scheduling task, traversing the link processing nodes in the second content processing link according to the time sequence, and obtaining the output result identifier currently corresponding to the currently traversed link processing node.
The second content processing link comprises a plurality of link processing nodes arranged according to time sequence, each link processing node corresponds to one scheduling subtask, each link processing node corresponds to a node input parameter and a node output result, and the corresponding scheduling subtask is executed according to the node input parameter to obtain the corresponding node output result. And the output result identification corresponding to the link processing node is used for identifying the node output result of the link processing node. The node input parameters of the link processing node include one or more of a content attribute on which the link processing node depends, a node version number of the link processing node, and a node output result of a previous link processing node of the link processing node in the second content processing link.
Specifically, the scheduling server compares a first content processing link corresponding to the target content with a second content processing link to determine whether the first content processing link and the second content processing link have a conflict according to a comparison result. When a conflicting link processing node exists in the first content processing link and the second content processing link, it is indicated that the first content processing link conflicts with the second content processing link, that is, it is indicated that a first scheduling task corresponding to the target content conflicts with a second scheduling task, the scheduling server stops executing the first scheduling task, and sequentially traverses the link processing nodes in the second content processing link according to a time sequence, so as to obtain an output result identifier currently corresponding to the currently traversed link processing node.
In one embodiment, when the same link processing node exists in the first content processing link and the second content processing link, then a determination is made that there is a conflicting link processing node in the first content processing link and the second content processing link.
In one embodiment, in the process of traversing the link processing node in the second content processing link according to the time sequence, the scheduling server determines the output result identifier currently corresponding to the currently traversed link processing node according to the currently corresponding node input parameter of the currently traversed link processing node.
And 212, when the corresponding node output result is not inquired according to the currently obtained output result identifier, executing a scheduling subtask corresponding to the currently traversed link processing node.
And each link processing node in the second content processing link corresponds to one scheduling subtask in the second scheduling task, and the processing of the target content after the content attribute is updated is realized by executing each scheduling subtask in the second scheduling task.
Specifically, after obtaining the output result identifier currently corresponding to the currently traversed link processing node, the scheduling server locally queries the node output result corresponding to the cache according to the obtained output result identifier. If the node output result is inquired, it is indicated that the scheduling sub-task corresponding to the currently traversed link processing node is executed, and the node output result corresponding to the currently traversed link processing node can be multiplexed, so that the scheduling sub-task corresponding to the currently traversed link processing node does not need to be executed repeatedly. If the node output result is not queried, it indicates that the scheduling sub-task corresponding to the currently traversed link processing node has not been executed, or if the scheduling sub-task corresponding to the currently traversed link processing node has been executed but the node output result corresponding to the currently traversed link processing node cannot be multiplexed, the scheduling sub-task corresponding to the currently traversed link processing node needs to be re-executed.
In one embodiment, if the currently traversed link processing node is executed before obtaining the second content processing link corresponding to the target content, the currently traversed link processing node has cached the corresponding output result identifier and node output result locally. If the current corresponding output result identifier of the currently traversed link processing node is not consistent with the cached output result identifier, the corresponding node output result cannot be queried locally according to the current corresponding output result identifier, that is, the cached node output result cannot be multiplexed. It can be understood that, since the output result identifier currently corresponding to the currently traversed link processing node is uniquely determined by the node input parameter currently corresponding to the currently traversed link processing node, if the node input parameter is updated, the scheduling sub-task corresponding to the currently traversed link processing node needs to be executed again.
In the method for adjusting the content processing link, during the process of executing the first scheduling task corresponding to the target content according to the first content processing link corresponding to the first scheduling task, or after the first scheduling task is executed, if the content attribute of the target content is detected to be updated, generating a second scheduling task corresponding to the target content, optimizing the second content processing link according to the target content attribute set after the content attribute is updated, if the conflict link processing node exists between the first content processing link and the second content processing link, indicating that the conflict exists between the first scheduling task and the second scheduling task, stopping executing the first scheduling task, traversing the link processing nodes in the second content processing link according to the time sequence, and identifying whether the node output result exists according to the output result currently corresponding to the sequentially traversed link processing nodes, and determining the scheduling subtasks to be executed in the second scheduling task, and implementing the processing of the target content after the content attribute is updated by executing each scheduling subtask to be executed, so that the processing resources can be saved and the processing efficiency can be improved under the condition of ensuring the processing accuracy of the target content.
In one embodiment, traversing the link processing node in the second content processing link according to a time sequence to obtain the output result identifier corresponding to the currently traversed link processing node includes: traversing link processing nodes in the second content processing link according to the time sequence; acquiring node input parameters corresponding to the currently traversed link processing nodes; the node input parameters comprise content attributes on which the currently traversed link processing node depends; and obtaining the output result identification currently corresponding to the currently traversed link processing node according to the node input parameters.
Specifically, the scheduling server sequentially traverses the link processing nodes in the second content processing link according to a time sequence, acquires attribute identifications corresponding to content attributes on which the currently traversed link processing nodes depend according to node identifications corresponding to the currently traversed link processing nodes, acquires content attributes corresponding to the attribute identifications from a target content attribute set corresponding to target content, and takes the acquired content attributes as node input parameters corresponding to the currently traversed link processing nodes. Further, the scheduling server obtains an output result identifier corresponding to the currently traversed link processing node according to the node input parameter corresponding to the currently traversed link processing node.
In one embodiment, the scheduling server obtains a corresponding output result identifier according to the acquired node input parameter and a preset mapping relation. The preset mapping relationship may be, for example, hash operation on the node input parameter, or convert the node input parameter into a binary string by using MD5 Algorithm (Message-Digest Algorithm 5), and use the converted binary string as a corresponding output result identifier.
In one embodiment, in the process of executing the corresponding scheduling task according to the content processing link corresponding to the target content, after the scheduling server finishes executing the scheduling subtask corresponding to a single link processing node, the node output result corresponding to the link processing node is cached locally in correspondence with the output result identifier determined by the node input parameter of the link processing node, so that the corresponding node output result can be queried locally based on the node input parameter.
In a specific embodiment, the preset mapping relationship is Key ═ MD5Sum (pa, pb, pc, …), where Key represents the output result identifier, pa, pb, pc, … represents the node input parameters, and MD5Sum () represents the MD5 algorithm used to convert the node input parameters into the output result identifier. It is to be understood that when the node input parameter includes a plurality of node input sub-parameters, the plurality of node input sub-parameters are ordered in a preset order. Correspondingly, the node output result corresponding to the output result identifier can be represented by Value, and therefore, if the node output result corresponding to the output result identifier is locally stored, the corresponding node output result can be queried according to the output result identifier.
In one embodiment, for each link processing node, the node input parameter uniquely corresponds to the output result identifier, and if the node input parameter changes, the corresponding output result identifier also changes inevitably. Therefore, if the content attribute depended by the link processing node is updated, the output result identifier corresponding to the link processing node at present is also updated correspondingly, and the corresponding node output result cannot be inquired locally according to the output result identifier corresponding to the link processing node at present, so that the scheduling subtask corresponding to the link processing node is indicated to be required to be executed again. Correspondingly, if the content attribute depended on by the link processing node is not updated, and the corresponding node output result can be locally inquired according to the output result identifier determined by the node input parameter currently corresponding to the link processing node, the node output result of the link processing node can be multiplexed, so that the scheduling subtask corresponding to the link processing node does not need to be re-executed, resources can be saved, and the processing performance can be improved.
For example, it is assumed that the target content is an article, a first content processing link corresponding to a first scheduling task corresponding to the target content is "intra-picture linking, title processing, text understanding, and manual review", and it is also assumed that when the first scheduling task is executed to a scheduling subtask corresponding to a link processing node of "title processing", a title of the article is updated, and a second content processing link corresponding to a second scheduling task corresponding to the target content is "intra-picture linking, title processing, text understanding, and manual review". Since the scheduling subtask corresponding to the link processing node "intra-picture chaining" is executed and the content attribute depended by the link processing node is not updated, the output result of the node corresponding to the link processing node can be multiplexed without repeated execution, but the scheduling subtask corresponding to the link processing node "title processing" is executed, but since the content attribute depended by the link processing node is updated, the output result of the node corresponding to the link processing node cannot be multiplexed and the scheduling subtask corresponding to the link processing node needs to be repeatedly executed.
In an embodiment, in the process of executing the scheduling task according to the content processing link, the node output result corresponding to the link processing node and the output result identifier are cached locally, and the content processing links respectively determined for the multiple scheduling tasks corresponding to the target content may include the same link processing node. Therefore, for each link processing node in the second content processing link, if the scheduling subtask corresponding to a certain link processing node has been executed and the corresponding node output result can be queried according to the output result identifier corresponding to the link processing node at present, it is characterized that the node output result can be multiplexed, and the scheduling subtask corresponding to the link processing node does not need to be executed.
In the foregoing embodiment, in the process of traversing the link processing node in the second content processing link according to the time sequence, according to the content attribute that the currently traversed link processing node depends on in the target content attribute set, the output result identifier corresponding to the link processing node is determined, so that when the dependent content attribute is updated, or when the scheduling sub-task corresponding to the link processing node is not executed yet for the target content, it is determined according to the corresponding output result identifier that the scheduling sub-task corresponding to the link processing node needs to be re-executed, and when the node output result corresponding to the link processing node can be reused, the node output result is reused, and the scheduling sub-task corresponding to the link processing node is not repeatedly executed any more, so that the repeated invocation of the link processing node can be reduced, while resources are saved, the processing capacity of the scheduling server and the processing efficiency of the target content are improved.
In one embodiment, the node input parameters further include a node output result currently corresponding to a previous link processing node of the currently traversed link processing node.
Specifically, the scheduling server obtains the content attribute that the currently traversed link processing node depends on according to the above-mentioned manner, determines the previous link processing node of the currently traversed link processing node from the second content processing link, obtains the node output result currently corresponding to the previous link processing node, and uses the content attribute that the currently traversed link processing node depends on and the node output result currently corresponding to the previous link processing node as the node input parameter corresponding to the currently traversed link processing node, and obtains the output result identifier currently corresponding to the currently traversed link processing node according to the node input parameter in the above-mentioned manner.
In the above embodiment, the output result identifier corresponding to the currently traversed link processing node is obtained according to the content attribute depended by the currently traversed link processing node and the node output result currently corresponding to the link processing node that is traversed before, so that, if the scheduling sub-task corresponding to each of the currently traversed link processing node and the previous link processing node of the link processing node is executed before the second content processing link is obtained, and after the corresponding scheduling sub-task is executed, the content attribute depended on by the currently traversed link processing node is not updated, and the node output result corresponding to the previous link processing node can be multiplexed, the scheduling sub-task corresponding to the currently traversed link processing node does not need to be executed again, and it is able to avoid additional repeated execution of the scheduling sub-task, thereby saving resources, the content processing efficiency is improved.
In one embodiment, the node input parameters further include a node version identification currently corresponding to the currently traversed link processing node.
Each link processing node in the content processing link corresponds to a node version identifier, and when the version of the link processing node is updated, the node version identifier of the link processing node is updated correspondingly. The node version identification is used to uniquely identify the version of the link processing node.
Specifically, in the process of traversing the link processing node in the second content processing link according to the time sequence, the scheduling server may further obtain a current corresponding node version identifier of the currently traversed link processing node, and use the content attribute depended on by the currently traversed link processing node, the current corresponding node version identifier, and the current corresponding node output result of the previous link processing node as the node input parameter corresponding to the currently traversed link processing node, and obtain the current corresponding output result identifier of the currently traversed link processing node according to the node input parameter in the manner described above.
In the above embodiment, the node input parameters corresponding to the currently traversed link processing node include the content attribute depended on by the currently traversed link processing node and the currently corresponding node version identifier, and the node output result currently corresponding to the previous link processing node, and according to the output node identifier obtained from the node input parameter, it is determined whether the scheduling sub-task corresponding to the currently traversed link processing node needs to be re-executed, so that when any one of the node input sub-parameters in the node input parameters changes, the scheduling sub-task corresponding to the currently traversed link processing node is re-executed, and the redundant execution of the scheduling sub-task can be reduced under the condition of ensuring the accuracy of content processing, so that the content processing efficiency can be improved.
In one embodiment, step 202 comprises: acquiring a first scheduling task corresponding to target content to be processed; determining the task priority of a first scheduling task according to the initial content attribute set of the target content; caching the first scheduling task to a waiting queue according to the task priority; and when the task execution condition is met, extracting the first scheduling task from the waiting queue, and determining a first content processing link corresponding to the first scheduling task according to the initial content attribute set.
The task priority refers to the priority of task processing. The waiting queue is used for buffering scheduling tasks which are waiting to be executed and are not executed, and the scheduling tasks buffered in the waiting queue are sorted according to the task priority. The task execution condition is a basis or condition for extracting the cached scheduling task from the waiting queue and triggering a task execution flow aiming at the extracted scheduling task. The task execution condition is that the first scheduled task is the scheduled task with the highest task priority in the waiting queue, and resources available for executing the first scheduled task exist in the scheduling server. The first scheduling task is the scheduling task with the highest task priority in the waiting queue, and means that the first scheduling task is currently located at the head position of the waiting queue. The resource for executing the first scheduled task refers to a processing resource or a computing resource, such as a task execution process or a task execution thread, allocated in the scheduling server for executing the scheduled task, and is not limited specifically herein.
Specifically, the scheduling server obtains target content to be processed, generates a first scheduling task corresponding to the target content, and obtains an initial content attribute set corresponding to the target content. The scheduling server inquires the task priority matched with the initial content attribute set, determines the inquired task priority as the task priority corresponding to the first scheduling task, and caches the first scheduling task to the waiting queue according to the determined task priority. The scheduling server dynamically detects whether the first scheduling task cached in the waiting queue meets a task execution condition. And when the task execution condition is judged to be met, the scheduling server extracts the first scheduling task from the waiting queue and determines a first content processing link corresponding to the first scheduling task according to the initial content attribute set corresponding to the corresponding target content.
In an embodiment, for each scheduling task cached in the waiting queue, the scheduling server dynamically detects whether the scheduling task satisfies a task execution condition, extracts the scheduling task satisfying the task execution condition from the waiting queue when it is determined that the task execution condition is satisfied, and determines a content processing link corresponding to the scheduling task according to an initial content attribute set of corresponding target content.
In one embodiment, the scheduling task corresponding to the target content corresponds to the content attribute set one to one, if the content attribute of the target content is updated, a new content attribute set is obtained according to the updated content attribute, and accordingly, a new scheduling task is generated for the target content with the updated content attribute, and the new scheduling task corresponds to the new content attribute set. The scheduling server is preconfigured with a plurality of content attribute sets, and is preconfigured with a matched task priority for each content attribute set, so that the task priority corresponding to the scheduling task can be determined based on the content attribute set corresponding to the scheduling task.
In the foregoing embodiment, for target content to be processed, according to a task priority determined by an initial content attribute set of the target content, a corresponding first scheduling task is cached to a waiting queue until a task execution condition is satisfied, the first scheduling task is extracted from the caching queue, and a first content processing link corresponding to the first scheduling task is determined according to the corresponding initial content attribute set, so that the first scheduling task is executed according to the first content processing link. Therefore, under the condition that the resource of the scheduling server is limited, the processing pressure of the scheduling server can be relieved, the processing reliability of the target content is ensured, and the processing efficiency of the single target content is improved.
In one embodiment, step 208 includes: determining task execution time of a second scheduling task according to the updated content attribute; when the task execution time is reached, acquiring a distributed lock corresponding to the target content; and when the distributed lock is acquired, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
The task execution time refers to a time for executing the scheduling task, such as executing immediately, or executing after a preset time period since the scheduling task is generated, or executing at nine nights, for example, and is not limited specifically herein. The distributed lock refers to a lock shared in a scheduling server, each target content corresponds to one distributed lock, and if the target content corresponds to a plurality of scheduling tasks, only the scheduling task of the distributed lock which acquires the target content can be executed.
In one embodiment, the scheduling server is preconfigured with a corresponding task execution timing for each content attribute, so that when the content attribute of the target content is updated, the task execution timing corresponding to the scheduling task triggered and generated based on the updated content attribute can be determined according to the updated content attribute, that is, the task execution timing preconfigured for the updated content attribute is determined as the task execution timing corresponding to the scheduling task triggered and generated based on the updated content attribute.
Specifically, when detecting the updated content attribute of the target content, the scheduling server determines the task execution timing of the second scheduling task according to the detected updated content attribute, and detects whether the task execution timing of the second scheduling task is reached in real time. And when the task execution time of the second scheduling task is judged to be reached, the scheduling server acquires the distributed lock corresponding to the target content aiming at the second scheduling task. And if the distributed lock corresponding to the target content is acquired, distributing the acquired distributed lock to a second scheduling task, and determining a second content processing link corresponding to the second scheduling task according to the target content attribute set of the target content.
In one embodiment, the processing of the target content is abstracted to the scheduling task, and the processing resources (i.e., the target content) of the scheduling task are in competition, and one target content being processed cannot be occupied by a plurality of scheduling tasks at the same time. In this way, if the target content corresponds to a plurality of second scheduling tasks, only the second scheduling task that acquires the distributed lock will be executed.
In one embodiment, the scheduling server includes a plurality of task scheduling units and distributed lock control units, and the task scheduling unit that acquires the scheduling task corresponding to the target content requests the distributed lock control unit for the distributed lock corresponding to the target content when the task execution time of the acquired scheduling task is reached, and allocates the requested distributed lock to the acquired scheduling task. It can be understood that, when initially acquiring target content to be processed, the distributed lock control unit generates a distributed lock corresponding to the target content.
In the above embodiment, a corresponding distributed lock is configured for each target content to be processed, and the scheduling task that acquires the distributed lock is executed, so that consistency of the target content and a processing result corresponding to the target content is ensured through a distributed lock mechanism, thereby improving data security.
FIG. 3 is a link structure diagram of a first content processing link determined based on an initial set of content attributes of target content, under an embodiment. As shown in fig. 3, the first content processing link includes 9 link processing nodes including nodes 1 to 9, and a plurality of link processing paths formed by the 9 link processing nodes, where each link processing node corresponds to one scheduling subtask in the first scheduling task, that is, corresponds to one algorithm or functional capability unit. It can be understood that, if the update of the content attribute of the target content is detected during the execution of the first scheduling task according to the first content processing link or after the execution of the first scheduling task is completed, and the link processing nodes 5 and 8 are not included in the second content processing link determined according to the target content attribute set after the update of the content attribute, during the execution of the first scheduling task, no matter whether the scheduling subtasks corresponding to the link processing nodes 5 and 8 are executed or not, during the execution of the second scheduling task according to the second content processing link, there is no need to execute the scheduling subtasks corresponding to the two link processing nodes, and therefore, after the update of the content attribute, the optimal path can be automatically found to obtain the optimal execution path, so that the invocation of modules can be reduced, thereby, while saving resources of the scheduling server, the content processing speed can be improved.
Further, in the process of executing the first scheduling task according to the first content processing link, assuming that the content attribute of the target content is updated when the scheduling subtask corresponding to the link processing node 7 is executed, the second content processing link determined according to the target content attribute set after the content attribute update coincides with the first content processing link shown in fig. 3, and the link processing node 2 in the second content processing link depends on the updated content attribute, and the link processing node 5 depends on the node output result of the link processing node 2, thereby determining that the scheduling subtask corresponding to the link processing node 2 needs to be re-executed. If the node output result obtained by re-executing the scheduling subtask corresponding to the link processing node 2 is consistent with the node output result obtained for the link processing node 2 in the process of executing the first scheduling task, it indicates that the node output result corresponding to the link processing node 5 can be reused, and the scheduling subtask corresponding to the link processing node 5 does not need to be repeatedly executed, so that resources can be saved and content processing efficiency can be improved.
In one embodiment, the content processing link adjustment method is applied to a scheduling server; the scheduling server comprises a task scheduling unit and a task execution unit; when the first content processing link and the second content processing link have conflicted link processing nodes, a second scheduling task and a corresponding second content processing link are sent to a task execution unit through a task scheduling unit; and stopping executing the first scheduling task through the task execution unit, executing the second scheduling task according to the second content processing link, and feeding back the task execution result to the task scheduling unit through an asynchronous callback mode when the second scheduling task is executed to obtain a corresponding task execution result.
Specifically, the scheduling server includes a task scheduling unit and a task executing unit. The scheduling server acquires a first scheduling task corresponding to the target content and a first content processing link corresponding to the first scheduling task through the task scheduling unit, and sends the acquired first scheduling task and the corresponding first content processing link to the task execution unit. And the scheduling server executes the corresponding first scheduling task according to the received first content processing link through the task execution unit, and feeds back the obtained task execution result to the task scheduling unit through an asynchronous callback mode when the first scheduling task is executed, so that the task execution result is sent to the corresponding computer equipment through the task scheduling unit.
Further, the scheduling server dynamically detects the updated content attribute of the target content through the task scheduling unit, if the updated content attribute of the target content is detected during the execution of the first scheduling task or after the execution of the first scheduling task is finished, a target content attribute set of the target content is obtained according to the updated content attribute, a second scheduling task corresponding to the target content is generated, a second content processing link corresponding to the second scheduling task is determined according to the target content attribute set, and when a link processing node that conflicts between the first content processing link and the second content processing link is judged, the second scheduling task and the corresponding second content processing link are sent to the task executing unit. And the scheduling server stops executing the first scheduling task through the task execution unit, executes the second scheduling task according to the second content processing link, and feeds back the obtained task execution result to the task scheduling unit in an asynchronous callback mode when the second scheduling task is finished and the corresponding task execution result is obtained.
It can be understood that the scheduling server executes, by the task execution unit, a corresponding second scheduling task according to the second scheduling task execution manner provided in one or more embodiments of the present application, according to the second content processing link.
In one embodiment, the target content to be processed is generally multiple, and each target content may correspond to one or more scheduling tasks, so that the scheduling server abstracts the processing of the target content into the scheduling tasks, sends the content processing link determined for each scheduling task and the corresponding scheduling task to the task execution unit through the task scheduling unit, processes the corresponding scheduling task according to the received content processing link through the scheduling cluster, and feeds back the obtained task processing result to the task scheduling unit through an asynchronous callback mode. Therefore, the asynchronous mechanism is adopted in the scheduling server, and the coupling dependence among all the internal functional units is reduced.
In one embodiment, the task scheduling unit comprises a scheduling gateway unit, a task gateway unit, a priority policy unit, a topology policy unit and a task management unit. The task execution unit comprises a scheduling cluster unit, a distributed flow control unit and the like. The scheduling server generates a scheduling task corresponding to the target content through the task gateway unit, caches the generated scheduling task to the task management unit, and informs the scheduling gateway unit of extracting the scheduling task from the task management unit through an event. The scheduling server calls the priority policy unit through the scheduling gateway unit to determine the task priority corresponding to the scheduling task, and calls the topology policy unit to determine the content processing link corresponding to the scheduling task. The dispatching server executes corresponding dispatching tasks according to the content processing link through the dispatching cluster unit and feeds back task processing results to the dispatching gateway unit through an asynchronous callback mode. And the scheduling server controls the sending time of the scheduling task through the distributed flow control unit.
FIG. 4 is an architecture diagram of a scheduling system in one embodiment. The scheduling server summarized in one or more embodiments of the present application corresponds to the scheduling system shown in fig. 4. As shown in fig. 4, the scheduling system includes a warehousing unit scheduling gateway unit, a task gateway unit, a priority policy unit, a topology policy unit, a task management unit, a scheduling cluster unit, a distributed flow control unit, a pipeline unit, a replay unit, a pressure measurement unit, and the like. When a user releases contents from each platform, the contents enter the scheduling system through the warehousing unit, the scheduling system creates scheduling tasks through the task gateway unit, caches the scheduling tasks to the task management unit, and informs the scheduling gateway unit to extract the scheduling tasks through events to execute scheduling. The scheduling system extracts the scheduling task through the scheduling gateway unit, then calls the priority policy unit to obtain the task priority of the scheduling task, and caches the scheduling task into the waiting queue according to the task priority, when the scheduling task can be executed, namely when the task execution condition is met, calls the topology policy unit through the scheduling gateway unit to obtain a content processing link which is required to be executed by the scheduling task, and sends the scheduling task and the content processing link to the scheduling cluster unit so as to start the execution flow of the scheduling content. The dispatching system executes corresponding dispatching tasks according to the content processing link through the dispatching cluster unit, feeds back task execution results to the dispatching gateway unit through an asynchronous callback mode after the dispatching tasks are executed, and finally delivers the content out of the warehouse through the dispatching gateway unit.
Further, in the process that the scheduling system executes the scheduling task according to the content processing link, when the content associated with the scheduling task is updated externally, the scheduling system receives an event notification through the task gateway unit, and the updated content attribute carried in the event notification is recorded in the HBase (open source storage component) storage corresponding to the task management unit for storing the content to be processed. The scheduling system detects whether the updated content attribute and the scheduling task currently being executed have conflict dependency through the scheduling gateway unit, if the updated content attribute and the scheduling task currently being executed have conflict dependency (for example, a certain link processing node in a content processing link depends on the content attribute of a content title, and the content attribute is modified by external updating, so that read-write conflict is generated), the scheduling system automatically terminates the scheduling task currently being executed, and re-optimizes a new content processing link corresponding to the content according to a target content attribute set obtained by updating the content attribute, so as to process the content after the content attribute is updated according to the optimized content processing link.
Furthermore, the scheduling system is connected with the priority management platform through the priority strategy unit, and is connected with the strategy management platform through the topology strategy unit. The task management unit also corresponds to a CDB (Cloud Database) and a CKV (Cloud Key Value, Key Value pair Cloud storage service) for storing data generated by the scheduling system in the process of processing content. The dispatching system controls the flow or the quantity of dispatching tasks required to be executed by the dispatching cluster unit through the distributed flow control. The scheduling system collects logs or running water of the scheduling cluster unit through a running water pipeline and stores the logs or running water to an ES cluster (elastic search, a distributed full-text search server). When the node version of the link processing node in the content processing link is updated, the scheduling system can compare the task execution results of the scheduling subtasks corresponding to the link processing nodes before and after the node version update request through the playback unit. The dispatching system carries out pressure measurement through the pressure measurement unit and is in butt joint with the replay management platform.
As shown in fig. 4, the scheduling cluster unit further corresponds to a capable shelf, which is divided into a preprocessing layer, a pre-filter layer, an understanding processing layer, and a post-filter layer according to a processing sequence, where each functional layer includes scheduling subtasks corresponding to a plurality of link processing nodes, for example, the preprocessing layer includes inner chain storage, structuring, transcoding and frame extraction, the pre-filter layer includes old filtering, security hit, and policy filtering, the understanding processing layer includes intelligent map selection, classification tag, video painting, and the like, and the post-filter layer includes de-emphasis filtering, low-quality filtering, and the like. And the scheduling system executes scheduling subtasks corresponding to the processing nodes of each link in the content processing link by the scheduling cluster unit through the capacity shelf.
As shown in fig. 5, in an embodiment, a content processing link adjustment method is provided, which specifically includes the following steps:
step 502, a first scheduling task corresponding to target content to be processed is obtained.
Step 504, determining task priority of the first scheduling task according to the initial content attribute set of the target content.
Step 506, buffering the first scheduled task to a waiting queue according to the task priority.
And step 508, when the task execution condition is met, extracting the first scheduling task from the waiting queue, and determining a first content processing link corresponding to the first scheduling task according to the initial content attribute set.
Step 510, a first scheduled task is executed according to a first content processing link.
And step 512, when the updated content attribute of the target content is detected, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content.
And 514, determining the task execution time of the second scheduling task according to the updated content attribute.
And step 516, acquiring a distributed lock corresponding to the target content when the task execution time is reached.
Step 518, if the distributed lock is obtained, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
And step 520, when the first content processing link and the second content processing link have conflicted link processing nodes, stopping the execution of the first scheduling task, and traversing the link processing nodes in the second content processing link according to the time sequence.
Step 522, obtaining node input parameters corresponding to the currently traversed link processing node; the node input parameters comprise content attributes on which the currently traversed link processing node depends, a node output result currently corresponding to a previous link processing node of the currently traversed link processing node, and a node version identifier currently corresponding to the currently traversed link processing node.
And step 524, obtaining the output result identifier corresponding to the currently traversed link processing node according to the node input parameter.
Step 526, when the output result of the corresponding node is not queried according to the currently obtained output result identifier, executing the scheduling subtask corresponding to the currently traversed link processing node.
In the foregoing embodiment, in the process of executing the first scheduling task according to the first content processing link, or after the first scheduling task is executed, when it is detected that the content attribute of the target content is updated, the second content processing link corresponding to the second scheduling corresponding to the target content is obtained through optimization according to the target content attribute set after the content attribute is updated, and according to the node input parameter currently corresponding to each link processing node in the second content processing link, the corresponding output result identifier is obtained, and according to the output result identifier, it is determined whether the scheduling subtask corresponding to the corresponding link processing node needs to be re-executed, and the processing of the target content after the content attribute is updated is realized by executing the scheduling subtask corresponding to the link processing node that needs to be re-executed. Therefore, the data processing pressure caused by redundant execution of the scheduling subtasks can be reduced, the processing capacity and performance can be improved, and meanwhile, the content processing link can be automatically calculated and optimized by establishing the relation among the content, the content attribute, the node input parameter, the node output result and the topology dependence, so that the processing efficiency of the content in the link processing is realized. Further, through the preemption of the distributed lock, the data consistency of the content can be included, so that the safety of the content can be ensured.
In one embodiment, the content processing links referred to in one or more embodiments herein may be understood as a topology map or a topology structure, the link processing paths in the content processing links may be understood as topology paths in the topology map, and the link processing nodes in the content processing links may be understood as topology nodes in the topology map. Therefore, when the content attribute of the target content is updated, the second content processing link is obtained through optimization according to the updated content attribute, and the second content processing link can also be understood as optimizing the topological graph on which the target content depends and the topological path in the topological graph, so that the technical effects of saving processing resources, improving processing speed, reducing processing conflicts and accelerating content scheduling are achieved.
It should be understood that although the steps in the flowcharts of fig. 2 and 5 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 and 5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternatively with other steps or at least a portion of the other steps or stages.
In one embodiment, as shown in fig. 6, there is provided a content processing link adjusting apparatus 600, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, and specifically includes: an obtaining module 601, a task executing module 602, and a link adjusting module 603, where:
an obtaining module 601, configured to obtain a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
a task execution module 602, configured to execute a first scheduling task according to the first content processing link;
the obtaining module 601 is further configured to, when an updated content attribute of the target content is detected, obtain a target content attribute set of the target content according to the updated content attribute, and generate a second scheduling task corresponding to the target content;
the obtaining module 601 is further configured to determine a second content processing link corresponding to the second scheduling task according to the target content attribute set;
a link adjusting module 603, configured to stop execution of the first scheduling task when a conflicting link processing node exists between the first content processing link and the second content processing link, traverse the link processing node in the second content processing link according to a time sequence, and obtain an output result identifier corresponding to the currently traversed link processing node;
the task execution module 602 is further configured to execute a scheduling sub-task corresponding to the currently traversed link processing node when the corresponding node output result is not queried according to the currently obtained output result identifier.
In one embodiment, the link adjustment module 603 is further configured to traverse the link processing nodes in the second content processing link according to a time sequence; acquiring node input parameters corresponding to the currently traversed link processing nodes; the node input parameters comprise content attributes on which the currently traversed link processing node depends; and obtaining the output result identification currently corresponding to the currently traversed link processing node according to the node input parameters.
In one embodiment, the node input parameters further include a node output result currently corresponding to a previous link processing node of the currently traversed link processing node.
In one embodiment, the node input parameters further include a node version identification currently corresponding to the currently traversed link processing node.
In an embodiment, the obtaining module 601 is further configured to obtain a first scheduling task corresponding to target content to be processed; determining the task priority of a first scheduling task according to the initial content attribute set of the target content; caching the first scheduling task to a waiting queue according to the task priority; and when the task execution condition is met, extracting the first scheduling task from the waiting queue, and determining a first content processing link corresponding to the first scheduling task according to the initial content attribute set.
In an embodiment, the obtaining module 601 is further configured to determine a task execution timing of the second scheduling task according to the updated content attribute; when the task execution time is reached, acquiring a distributed lock corresponding to the target content; and if the distributed lock is acquired, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
In one embodiment, the content processing link adjusting means is applied to a scheduling server; the scheduling server comprises a task scheduling unit and a task execution unit; when the first content processing link and the second content processing link have conflicted link processing nodes, a second scheduling task and a corresponding second content processing link are sent to a task execution unit through a task scheduling unit; and stopping executing the first scheduling task through the task execution unit, executing the second scheduling task according to the second content processing link, and feeding back the task execution result to the task scheduling unit through an asynchronous callback mode when the second scheduling task is executed to obtain a corresponding task execution result.
For the specific definition of the content processing link adjustment device, reference may be made to the above definition of the content processing link adjustment device method, which is not described herein again. The modules in the content processing link adjusting apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a dispatch server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is for storing target content and corresponding sets of content attributes, and content processing links preconfigured for the sets of content attributes. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a content processing link adjustment apparatus method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (16)

1. A method for content processing link adjustment, the method comprising:
acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
executing the first scheduling task according to the first content processing link;
when the updated content attribute of the target content is detected, if the updated content attribute has conflict dependency with the first scheduling task, obtaining a target content attribute set of the target content according to the updated content attribute, and generating a second scheduling task corresponding to the target content; wherein when a link processing node of the first content processing link relies on the updated content attribute, the updated content attribute has conflicting dependence on the first scheduling task;
determining a second content processing link corresponding to the second scheduling task according to the target content attribute set;
when the first content processing link and the second content processing link have conflicted link processing nodes, stopping the execution of the first scheduling task, traversing the link processing nodes in the second content processing link according to a time sequence, and obtaining an output result identifier corresponding to the currently traversed link processing node;
and when the corresponding node output result is not inquired according to the currently obtained output result identifier, executing a scheduling subtask corresponding to the currently traversed link processing node.
2. The method of claim 1, wherein traversing the link processing nodes in the second content processing link according to the time sequence to obtain the output result identifier corresponding to the currently traversed link processing node comprises:
traversing link processing nodes in the second content processing link according to a time sequence;
acquiring node input parameters corresponding to the currently traversed link processing nodes; the node input parameters comprise content attributes on which the currently traversed link processing node depends;
and obtaining the output result identification currently corresponding to the currently traversed link processing node according to the node input parameters.
3. The method of claim 2, wherein the node input parameters further include a node output result currently corresponding to a previous link processing node of the currently traversed link processing nodes.
4. The method of claim 3, wherein the node input parameters further include a node version identification currently corresponding to the currently traversed link processing node.
5. The method of claim 1, wherein obtaining the first scheduling task corresponding to the target content and the first content processing link corresponding to the first scheduling task comprises:
acquiring a first scheduling task corresponding to target content to be processed;
determining the task priority of the first scheduling task according to the initial content attribute set of the target content;
caching the first scheduling task to a waiting queue according to the task priority;
and when a task execution condition is met, extracting the first scheduling task from the waiting queue, and determining a first content processing link corresponding to the first scheduling task according to the initial content attribute set.
6. The method of claim 1, wherein determining the second content processing link corresponding to the second scheduling task according to the target content attribute set comprises:
determining task execution time of the second scheduling task according to the updated content attribute;
when the task execution time is reached, acquiring a distributed lock corresponding to the target content;
and if the distributed lock is acquired, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
7. The method according to any one of claims 1 to 6, wherein the method is applied to a scheduling server; the scheduling server comprises a task scheduling unit and a task execution unit; when the first content processing link and the second content processing link have conflicted link processing nodes, the task scheduling unit sends the second scheduling task and the corresponding second content processing link to the task execution unit; and stopping executing the first scheduling task through the task execution unit, executing the second scheduling task according to the second content processing link, and feeding back the task execution result to the task scheduling unit through an asynchronous callback mode when the second scheduling task is executed to obtain a corresponding task execution result.
8. An apparatus for content processing link adjustment, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a first scheduling task corresponding to target content and a first content processing link corresponding to the first scheduling task;
a task execution module, configured to execute the first scheduling task according to the first content processing link;
the obtaining module is further configured to, when an updated content attribute of the target content is detected, if a conflict dependency exists between the updated content attribute and the first scheduling task, obtain a target content attribute set of the target content according to the updated content attribute, and generate a second scheduling task corresponding to the target content; wherein when a link processing node of the first content processing link relies on the updated content attribute, the updated content attribute has conflicting dependence on the first scheduling task;
the obtaining module is further configured to determine a second content processing link corresponding to the second scheduling task according to the target content attribute set;
a link adjustment module, configured to stop execution of the first scheduling task when a conflicting link processing node exists between the first content processing link and the second content processing link, traverse the link processing node in the second content processing link according to a time sequence, and obtain an output result identifier corresponding to the currently traversed link processing node;
and the task execution module is also used for executing the scheduling subtask corresponding to the currently traversed link processing node when the corresponding node output result is not inquired according to the currently obtained output result identifier.
9. The apparatus of claim 8, wherein the link adjustment module is further configured to traverse link processing nodes in the second content processing link in a time-sequential manner; acquiring node input parameters corresponding to the currently traversed link processing nodes; the node input parameters comprise content attributes on which the currently traversed link processing node depends; and obtaining the output result identification currently corresponding to the currently traversed link processing node according to the node input parameters.
10. The apparatus of claim 9, wherein the node input parameters further comprise a node output result currently corresponding to a previous link processing node of the currently traversed link processing node.
11. The apparatus of claim 10, wherein the node input parameters further comprise a node version identification currently corresponding to the currently traversed link processing node.
12. The apparatus according to claim 8, wherein the obtaining module is further configured to obtain a first scheduling task corresponding to target content to be processed; determining the task priority of the first scheduling task according to the initial content attribute set of the target content; caching the first scheduling task to a waiting queue according to the task priority; and when a task execution condition is met, extracting the first scheduling task from the waiting queue, and determining a first content processing link corresponding to the first scheduling task according to the initial content attribute set.
13. The apparatus according to claim 8, wherein the obtaining module is further configured to determine a task execution timing of the second scheduling task according to the updated content attribute; when the task execution time is reached, acquiring a distributed lock corresponding to the target content; and if the distributed lock is acquired, determining a second content processing link corresponding to the second scheduling task according to the target content attribute set.
14. The apparatus according to any one of claims 8 to 13, wherein the content processing link adjusting means is applied to a scheduling server; the scheduling server comprises a task scheduling unit and a task execution unit; when the first content processing link and the second content processing link have conflicted link processing nodes, the task scheduling unit sends the second scheduling task and the corresponding second content processing link to the task execution unit; and stopping executing the first scheduling task through the task execution unit, executing the second scheduling task according to the second content processing link, and feeding back the task execution result to the task scheduling unit through an asynchronous callback mode when the second scheduling task is executed to obtain a corresponding task execution result.
15. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
16. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
CN202011091563.8A 2020-10-13 2020-10-13 Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium Active CN112235396B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011091563.8A CN112235396B (en) 2020-10-13 2020-10-13 Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011091563.8A CN112235396B (en) 2020-10-13 2020-10-13 Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN112235396A CN112235396A (en) 2021-01-15
CN112235396B true CN112235396B (en) 2022-03-01

Family

ID=74112521

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011091563.8A Active CN112235396B (en) 2020-10-13 2020-10-13 Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112235396B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242641B (en) * 2021-04-23 2023-12-19 奇安信科技集团股份有限公司 Method and device for previewing strategy issuing result and computer equipment
CN113918126B (en) * 2021-09-14 2022-06-10 北京柏睿数据技术股份有限公司 AI modeling flow arrangement method and system based on graph algorithm
CN115174286B (en) * 2022-05-12 2023-11-14 芜湖美的厨卫电器制造有限公司 Method and device for preventing link from being repeatedly executed, computer storage medium and equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109087054A (en) * 2018-06-01 2018-12-25 平安科技(深圳)有限公司 Synergetic office work data flow processing method, device, computer equipment and storage medium
CN110597858A (en) * 2019-08-30 2019-12-20 深圳壹账通智能科技有限公司 Task data processing method and device, computer equipment and storage medium
WO2020151177A1 (en) * 2019-01-25 2020-07-30 平安科技(深圳)有限公司 Method for scheduling application task based on big data behavior, server and storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9444885B2 (en) * 2014-03-18 2016-09-13 Vmware, Inc. Workflow processing in a distributed computing environment
CN107295027A (en) * 2016-03-30 2017-10-24 中国移动通信有限公司研究院 A kind of content injection method, device and controller
US10719795B2 (en) * 2017-10-27 2020-07-21 International Business Machines Corporation Cognitive learning workflow execution
CN110018860B (en) * 2019-04-04 2022-11-08 深圳市永兴元科技股份有限公司 Workflow management method, device, equipment and computer storage medium
CN111224843B (en) * 2019-12-31 2022-01-07 北京百度网讯科技有限公司 Resource link monitoring method, device, equipment and storage medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109087054A (en) * 2018-06-01 2018-12-25 平安科技(深圳)有限公司 Synergetic office work data flow processing method, device, computer equipment and storage medium
WO2020151177A1 (en) * 2019-01-25 2020-07-30 平安科技(深圳)有限公司 Method for scheduling application task based on big data behavior, server and storage medium
CN110597858A (en) * 2019-08-30 2019-12-20 深圳壹账通智能科技有限公司 Task data processing method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN112235396A (en) 2021-01-15

Similar Documents

Publication Publication Date Title
CN112235396B (en) Content processing link adjustment method, content processing link adjustment device, computer equipment and storage medium
CN109033123B (en) Big data-based query method and device, computer equipment and storage medium
CN113766035B (en) Service acceptance and consensus method and device
US9563426B1 (en) Partitioned key-value store with atomic memory operations
CN112948450B (en) Method and device for Flink streaming processing engine for real-time recommendation and computer equipment
CN106469087A (en) Metadata output intent, client and meta data server
WO2019128984A1 (en) Container security policy handling method and related device
WO2021208844A1 (en) Virtualized container management method and system, and storage medium
US20200342008A1 (en) System for lightweight objects
CN115185679A (en) Task processing method and device for artificial intelligence algorithm, server and storage medium
CN115455058A (en) Cache data processing method and device, computer equipment and storage medium
CN110784498B (en) Personalized data disaster tolerance method and device
US20160203032A1 (en) Series data parallel analysis infrastructure and parallel distributed processing method therefor
CN112199200B (en) Resource scheduling method and device, computer equipment and storage medium
CN107181773A (en) Data storage and data managing method, the equipment of distributed memory system
US20170286440A1 (en) Method, business processing server and data processing server for storing and searching transaction history data
CN111737021A (en) Parallel task processing method and device, electronic equipment and storage medium
CN114969165B (en) Data query request processing method, device, equipment and storage medium
US9009731B2 (en) Conversion of lightweight object to a heavyweight object
KR101694301B1 (en) Method for processing files in storage system and data server thereof
US11762860B1 (en) Dynamic concurrency level management for database queries
US11379268B1 (en) Affinity-based routing and execution for workflow service
CN111796767B (en) Distributed file system and data management method
US20180165018A1 (en) Partial storage of large files in distinct storage systems
CN113961586A (en) Control method and device for SQL (structured query language) statements

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant