CN114785790B - Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium - Google Patents

Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium Download PDF

Info

Publication number
CN114785790B
CN114785790B CN202210476425.4A CN202210476425A CN114785790B CN 114785790 B CN114785790 B CN 114785790B CN 202210476425 A CN202210476425 A CN 202210476425A CN 114785790 B CN114785790 B CN 114785790B
Authority
CN
China
Prior art keywords
scheduling
cross
task
application
local
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210476425.4A
Other languages
Chinese (zh)
Other versions
CN114785790A (en
Inventor
宋炜
陈爽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision System Technology Co Ltd
Original Assignee
Hangzhou Hikvision System Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision System Technology Co Ltd filed Critical Hangzhou Hikvision System Technology Co Ltd
Priority to CN202210476425.4A priority Critical patent/CN114785790B/en
Publication of CN114785790A publication Critical patent/CN114785790A/en
Application granted granted Critical
Publication of CN114785790B publication Critical patent/CN114785790B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H04L67/1014—Server selection for load balancing based on the content of a request
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1031—Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The application discloses a cross-domain analysis system, a cross-domain resource scheduling method, a cross-domain resource scheduling device and a storage medium, relates to the technical field of intelligent analysis of video images, and can realize cross-domain scheduling of video image analysis resources. The cross-domain analysis system includes: a first office point and a second office point; the first office point includes: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first scheduling device is used for receiving the cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises the following steps: the task to be analyzed is used for applying for executing the task to be analyzed at other local points except the first local point; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the second scheduling device; the second scheduling device is used for forwarding the cross-domain resource scheduling application to the second data analysis device; and the second data analysis device is used for analyzing the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result.

Description

Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium
Technical Field
The application relates to the technical field of intelligent analysis of video images, in particular to a cross-domain analysis system, a cross-domain resource scheduling method, a device and a storage medium.
Background
At present, in the technical field of intelligent analysis of video images, the main working mode of analysis tasks is to schedule resources in a single intelligent analysis system of video images, wherein the scheduled intelligent analysis computing resources of the video images can be deployed by a single node, and the intelligent analysis system also supports multi-node distributed cluster deployment.
It can be seen that the scheduling of resources by the conventional intelligent video image analysis system is often only completed in the local point, and the following problems are caused by such a working mode: when the resources in the office of the office point are insufficient, if a new analysis task appears, the new analysis task cannot be responded in time and can only be queued or discarded, so that the problems of untimely task processing and low processing efficiency can be caused. In addition, if a plurality of intelligent video image analysis systems exist at the same time, because the intelligent video image analysis systems are isolated from each other, the situation that each office point is busy and idle is uneven exists, so that the utilization rate of the whole resources of the intelligent video image analysis systems is low.
Disclosure of Invention
The application provides a cross-domain analysis system, a cross-domain resource scheduling method, a cross-domain resource scheduling device and a storage medium, which can realize cross-domain (namely cross-local point) scheduling of video image analysis resources.
In a first aspect, the present application provides a cross-domain resource scheduling method, applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the method comprises the following steps: the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
It can be understood that the cross-domain resource scheduling method provided by the application is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises a plurality of local points (one local point can be regarded as a video image intelligent analysis platform), and each local point comprises a scheduling device. One local point can receive a cross-domain resource scheduling application (comprising tasks to be analyzed) sent by a scheduling device of the local point through a scheduling device of the local point, and analyze the tasks to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the cross-domain resource scheduling application can perform cross-domain (information interaction of cross-local points is called cross-domain here) scheduling of analysis resources, so that the integration of computing power resources is realized, and the utilization rate of the global computing power resources is improved.
In one possible implementation manner, the cross-domain analysis system further includes: the central scheduling device is respectively connected with the first scheduling device and the second scheduling device, and the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device and comprises: the second dispatching device receives a first cross-domain resource dispatching application through the central dispatching device, wherein the first cross-domain resource dispatching application is sent to the central dispatching device by the first dispatching device, so that the central dispatching device determines a second local point from other local points according to the first cross-domain resource dispatching application and sends the first cross-domain resource dispatching application to the second dispatching device; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed.
It can be understood that the cross-domain resource scheduling method provided by the application is applied to a cross-domain analysis system, and the cross-domain analysis system further comprises: a central scheduling device; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device. One local point can send a cross-domain resource scheduling application (including tasks to be analyzed) to the central scheduling device through the scheduling device of the local point, so that the central scheduling device determines the local point which can be used for executing the tasks to be analyzed from other local points according to the cross-domain resource scheduling application, and then the central scheduling device can send the cross-domain resource scheduling application to the scheduling device of the local point, and the tasks to be analyzed are analyzed in the local point to obtain an analysis result. Therefore, by utilizing the method provided by the application, on one hand, when a certain local point can not bear a task to be analyzed, the task to be analyzed is timely sent to other idle local points for analysis by initiating a cross-domain resource scheduling application, so that the task processing efficiency is improved, and the utilization rate of the global computing power resource is improved; on the other hand, the intra-office scheduling device can synchronize the resource information in the local office point to the central scheduling device, and the central scheduling device makes a decision (namely determines the applied party of the cross-domain resource scheduling application), so that the configuration of the intra-office scheduling device is lighter.
In another possible implementation manner, under the condition that the amount of resources required by the first task to be analyzed is increased, the second scheduling device receives a first capacity expansion application through the central scheduling device, wherein the first capacity expansion application is sent to the central scheduling device by the first scheduling device, so that the first capacity expansion application is sent to the second scheduling device by the central scheduling device; the first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point.
It can be understood that in the cross-domain resource scheduling method provided by the application, under the condition that the amount of resources required by the first task to be analyzed is increased, the increased amount of resources of the first task to be analyzed can be applied to the local point for executing the first task to be analyzed again, so that the processing process of the first task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In another possible implementation manner, the method further includes: the second scheduling device sends an algorithm query application to the second data analysis device, wherein the algorithm query application comprises: the identification of the algorithm required by the first task to be analyzed is carried out, so that the second data analysis device inquires whether the second data analysis device comprises the algorithm required by the first task to be analyzed according to the identification of the algorithm required by the first task to be analyzed; the second scheduling device sends an algorithm downloading application to the second data analysis device under the condition that the second data analysis device does not comprise the algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: and the downloading address of the algorithm required by the first task to be analyzed is used for enabling the second data analysis device to download the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
It can be understood that, based on the method provided by the application, before the second data analysis device analyzes the task to be analyzed, the second scheduling device sends an algorithm query request to the second data analysis device to query whether the second data analysis device has the algorithm required by the task to be analyzed, and if the second data analysis device does not have the algorithm required by the task to be analyzed, the second scheduling device sends the identification of the algorithm required by the task to be analyzed to the second data analysis device, so that the second data analysis device downloads the algorithm required by the task to be analyzed according to the identification of the algorithm required by the task to be analyzed. Therefore, before the task to be analyzed is analyzed, the second data analysis device is ensured to have a related algorithm, and further the task to be analyzed can be ensured to be analyzed normally.
In another possible implementation manner, the method further includes: the second scheduling device receives the control instruction sent by the first scheduling device and controls the second data analysis device to analyze the state of the first task to be analyzed according to the control instruction.
It can be understood that, based on the method provided by the application, the processing state of the task to be analyzed can be obtained in real time by the second data analysis device through the first scheduling device, and the state is controlled. In this way, when the upper layer application modifies the task to be analyzed, corresponding modification operation can be timely made through the first scheduling device.
In another possible implementation manner, the method further includes: the second scheduling device receives algorithm change information sent by the first scheduling device; the algorithm change information comprises the identification of the changed target algorithm in the first data analysis device and the downloading address of the target algorithm; the second scheduling device sends the identification of the target algorithm to the second data analysis device so that the second data analysis device inquires whether the target algorithm exists in the second data analysis device according to the identification of the target algorithm; in the case where the target algorithm does not exist in the second data analysis means, the second scheduling means transmits a download address of the target algorithm to the second data analysis means so that the second data analysis means downloads the target algorithm according to the download address of the target algorithm.
It will be appreciated that cross-domain resource scheduling may only be performed if the algorithm information within each local point in the cross-domain analysis system is consistent. For example, if the resource to be analyzed is not required by the task to be analyzed, the task to be analyzed cannot be processed. Therefore, based on the method provided by the application, algorithm synchronization can be performed regularly through the scheduling device, so that the applied party of the resource can process the task to be analyzed in time in cross-domain resource scheduling.
In another possible implementation manner, the method further includes: the second scheduling device receives the computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining power information of the first office point; the second scheduling device changes the remaining power information of the first local point recorded in the second scheduling device into the current remaining power information of the first local point according to the power resource change information.
It can be understood that, in the method provided by the present application, the applied party of the cross-domain resource scheduling application needs to be determined according to the local resource allowance of each local point in the cross-domain analysis system, so that the local resource allowance of each local point recorded in the scheduling device needs to be updated periodically (for example, the computing power resource synchronization is performed periodically), so that the applied party of the cross-domain resource scheduling application can be ensured to have the capability of processing the task to be analyzed.
In another possible implementation, in case that the first office point and the second office point cannot communicate directly; the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device, and the method comprises the following steps: the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device through the boundary security server.
It can be understood that, based on the method provided by the present application, in the case that the first office point and the second office point cannot directly communicate (for example, the internal local area network of a certain organization cannot directly connect with the video private network), information is transmitted through the boundary security server, so that the method provided by the present application can be suitable for richer usage scenarios.
In a second aspect, the present application provides a cross-domain resource scheduling method, applied to a cross-domain analysis system, where the cross-domain analysis system includes: the first office point, the second office point and the third office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: the second data analysis device and the second scheduling device, the third office point includes: a third scheduling device and a third data analysis device; the method comprises the following steps: the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point; the second scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device so that the third data analysis device executes a second task to be analyzed; and the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
It can be understood that the cross-domain resource scheduling method provided by the application is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises a plurality of local points (one local point can be regarded as a video image intelligent analysis platform), and each local point comprises a scheduling device. One local point can send a cross-domain resource scheduling application (including tasks to be analyzed) to the scheduling devices of other local points through the scheduling device of the local point, so that the data analysis device of the other local points analyzes the tasks to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, by using the method provided by the application, when a certain local point cannot bear the task to be analyzed, the task to be analyzed is timely sent to other idle local points for analysis by initiating a cross-domain resource scheduling application, so that the task to be analyzed is timely processed under the condition that the normal service of the local point is not interrupted. In this way, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle of different local points can be solved, and the utilization rate of the computing power resources in the whole domain is improved.
In one possible implementation manner, the second scheduling device sends a second capacity expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed increases; the second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
It can be understood that in the cross-domain resource scheduling method provided by the application, under the condition that the resource amount required by the second task to be analyzed is increased, the increased resource amount of the second task to be analyzed can be applied to the local point for executing the second task to be analyzed again, so that the processing process of the second task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the determining, according to the second cross-domain resource scheduling application, a third local point from the other local points includes: the second scheduling device sends a second cross-domain resource scheduling application to the terminal equipment, so that the terminal equipment receives the second cross-domain resource scheduling application and receives a third local point indicated by the user for the second cross-domain resource scheduling application.
It can be understood that, based on the method provided by the application, the cross-domain resource scheduling application can be sent to the terminal device through the scheduling device, and further, the terminal device can display the cross-domain resource scheduling application to the user, so that the user can indicate the applied party for the cross-domain resource scheduling application according to the content of the cross-domain resource scheduling application. Therefore, decision weights can be provided for users through the terminal equipment, so that the method provided by the application can be suitable for richer use scenes.
In a third aspect, the present application provides a cross-domain resource scheduling method, applied to a cross-domain analysis system, where the cross-domain analysis system includes: the second office point, the third office point and the central dispatching device; wherein the second local point comprises: the second data analysis device and the second scheduling device, the third office point includes: a third scheduling device and a third data analysis device; the central dispatching device is respectively connected with the second dispatching device and the third dispatching device; the method further comprises the following steps: the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point; the second scheduling device sends a second cross-domain resource scheduling application to the central scheduling device, so that the central scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application, and sends the cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes a second task to be analyzed; wherein, the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed; under the condition that the amount of resources required by the second task to be analyzed is increased, the second scheduling device sends a capacity expansion application to the central scheduling device, so that the central scheduling device sends the capacity expansion application to the third scheduling device; the capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
In a fourth aspect, the present application provides a cross-domain resource scheduling method, applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point, a second office point, and a fourth office point. Wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: the second data analysis device and the second scheduling device, the fourth office point includes: a fourth scheduling means; the method further comprises the following steps: the second dispatching device receives a task analysis request sent by the fourth dispatching device, wherein the task analysis request is used for applying the second dispatching device for executing the task to be analyzed in the second local point in the fourth local point, and the task analysis request comprises: address information of the fourth local point and a local resource margin of the fourth local point.
It can be understood that the cross-domain resource scheduling method provided by the application is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises a plurality of local points (one local point can be regarded as a video image intelligent analysis platform), and each local point comprises a scheduling device. One local point can initiate a task analysis request to the scheduling device of the other local point through the scheduling device of the local point so as to request to execute tasks to be analyzed of the other local point. Therefore, by utilizing the method provided by the application, on one hand, when the idle resources of one local point are more, the task to be analyzed can be actively requested to other local points, so that the task to be analyzed can be timely processed. In this way, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle of different local points can be solved, and the utilization rate of the computing power resources in the whole domain is improved.
In a fifth aspect, the present application provides a cross-domain resource scheduling method, applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the method comprises the following steps: the second data analysis device receives a first cross-domain resource scheduling application sent by the second scheduling device; the first cross-domain resource scheduling application includes: a first task to be analyzed; the first cross-domain resource scheduling application is used for applying to execute a first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device; and the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
It can be understood that the cross-domain resource scheduling method provided by the application is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises a plurality of local points (one local point can be regarded as a video image intelligent analysis platform), and each local point comprises a scheduling device. The data analysis device of one local point can receive a cross-domain resource scheduling application (comprising a task to be analyzed) initiated by the data analysis device of the other local point through the scheduling device of the local point, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the cross-domain resource scheduling application can perform cross-domain (information interaction of cross-local points is called cross-domain here) scheduling of analysis resources, so that the integration of computing power resources is realized, and the utilization rate of the global computing power resources is improved.
In one possible implementation manner, the first task to be analyzed is a task of analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: an acquisition address of data to be analyzed; the method further comprises the following steps: the second data analysis device acquires the data to be analyzed according to the acquisition address of the data to be analyzed; the first cross-domain resource scheduling application further includes: the download address of the algorithm required by the first task to be analyzed; the method further comprises the following steps: the second data analysis device directly downloads the algorithm required by the first task to be analyzed from the first data analysis device according to the downloading address of the algorithm required by the first task to be analyzed; in another possible implementation manner, the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result, where the analysis result includes: and the second data analysis device analyzes the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed contained in the algorithm downloading application, and an analysis result is obtained.
It can be appreciated that, based on the method provided in the present application, the data to be analyzed (for example, video data or picture data) can be obtained directly according to the obtaining address of the data to be analyzed; according to the identification of the algorithm required by the data to be analyzed, the algorithm required by the data to be analyzed is obtained, namely the second data analysis device can directly obtain the service data without forwarding by a dispatching device, and the dispatching device only needs to be responsible for forwarding the task control data.
By adopting the mode of separating and forwarding the task control data and the service data, the configuration of the scheduling device is lighter, and meanwhile, the data forwarding efficiency can be improved.
In another possible implementation manner, the method further includes: the second data analysis device receives an algorithm query application sent by the second scheduling device; the algorithm query application comprises the following steps: identification of an algorithm required by a first task to be analyzed; the second data analysis device inquires whether the second data analysis device comprises the algorithm required by the first task to be analyzed according to the identification of the algorithm required by the first task to be analyzed; when the second data analysis device does not comprise the algorithm required by the first task to be analyzed, the second data analysis device receives an algorithm downloading application sent by the second scheduling device; the algorithm downloading application comprises the following steps: the download address of the algorithm required by the first task to be analyzed; and the second data analysis device downloads the algorithm required by the first task to be analyzed according to the download address of the algorithm required by the first task to be analyzed.
In a sixth aspect, the present application provides a scheduling apparatus applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis device and the scheduling device; the scheduling device comprises: the receiving unit is used for receiving a first cross-domain resource scheduling application sent by the first scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; and the sending unit is used for sending the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
In one possible implementation, the cross-domain analysis system further comprises: the central dispatching device is respectively connected with the first dispatching device and the dispatching device, and the receiving unit is specifically used for receiving a first cross-domain resource dispatching application through the central dispatching device, wherein the first cross-domain resource dispatching application is sent to the central dispatching device by the first dispatching device, so that the central dispatching device determines a second local point from other local points according to the first cross-domain resource dispatching application and sends the first cross-domain resource dispatching application to the dispatching device; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed.
In another possible implementation manner, the receiving unit is further configured to receive, by the central scheduling device, a first expansion application when the amount of resources required by the first task to be analyzed increases, where the first expansion application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device sends the first expansion application to the second scheduling device; the first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point.
In another possible implementation manner, the cross-domain analysis system further includes: a third local point, the third local point comprising: the receiving unit is also used for receiving a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point; the scheduling apparatus further includes: the determining unit is used for determining a third local point from other local points according to the second cross-domain resource scheduling application; the sending unit is further used for sending a second cross-domain resource scheduling application to the third scheduling device so that the third data analysis device executes a second task to be analyzed; and the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
In another possible implementation manner, the sending unit is further configured to send a second expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed increases; the second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
In another possible implementation manner, the central scheduling device is further connected to a third scheduling device, and the sending unit is further configured to send a second cross-domain resource scheduling application to the central scheduling device, so that the central scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application, and sends the cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes a second task to be analyzed; wherein, the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed; the sending unit is further used for sending a second capacity expansion application to the central scheduling device under the condition that the amount of resources required by the second task to be analyzed is increased, so that the central scheduling device sends the second capacity expansion application to the third scheduling device; the second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
In another possible implementation manner, the cross-domain analysis system further includes: a fourth local point, the fourth local point comprising: a fourth scheduling means; a receiving unit; the task analysis request is used for applying to the scheduling device to execute the task to be analyzed in the second local point in the fourth local point, and the task analysis request comprises: address information of the fourth local point and a local resource margin of the fourth local point.
In another possible implementation manner, the sending unit is further configured to send an algorithm query application to the second data analysis device, where the algorithm query application includes: the identification of the algorithm required by the first task to be analyzed is carried out, so that the second data analysis device inquires whether the second data analysis device comprises the algorithm required by the first task to be analyzed according to the identification of the algorithm required by the first task to be analyzed; the sending unit is also used for sending an algorithm downloading application to the second data analysis device under the condition that the algorithm required by the first task to be analyzed is not included in the second data analysis device; the algorithm downloading application comprises the following steps: and the downloading address of the algorithm required by the first task to be analyzed is used for enabling the second data analysis device to download the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
In another possible implementation manner, the receiving unit is further configured to receive a control instruction sent by the first scheduling device, and control the second data analysis device to analyze the state of the first task to be analyzed according to the control instruction.
In another possible implementation manner, the receiving unit is further configured to receive algorithm change information sent by the first scheduling device; the algorithm change information comprises the identification of the changed target algorithm in the first data analysis device and the downloading address of the target algorithm; the sending unit is further used for sending the identification of the target algorithm to the second data analysis device so that the second data analysis device inquires whether the target algorithm exists in the second data analysis device according to the identification of the target algorithm; and the sending unit is further used for sending the downloading address of the target algorithm to the second data analysis device in the condition that the target algorithm does not exist in the second data analysis device, so that the second data analysis device downloads the target algorithm according to the downloading address of the target algorithm.
In another possible implementation manner, the receiving unit is further configured to receive computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining power information of the first office point; the scheduling apparatus further includes: and an information changing unit for changing the remaining power information of the first local point recorded in the scheduling device to the current remaining power information of the first local point according to the power resource changing information.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the sending unit is further configured to send a second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third local point indicated by the user for the second cross-domain resource scheduling application.
In another possible implementation manner, the receiving unit is further configured to receive, through the boundary security server, a first cross-domain resource scheduling application sent by the first scheduling device, where direct communication between the first local station and the second local station is not possible.
In a seventh aspect, the present application provides a data analysis device, applied to a cross-domain analysis system, where the cross-domain analysis system includes: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: the data analysis device and the second scheduling device; the data analysis device includes: the receiving unit is used for receiving a first cross-domain resource scheduling application sent by the second scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device; and the processing unit is used for analyzing the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
In one possible implementation manner, the first task to be analyzed is a task of analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: an acquisition address of data to be analyzed; the data analysis device further includes: the acquisition unit is used for acquiring the data to be analyzed according to the acquisition address of the data to be analyzed; the first cross-domain resource scheduling application further includes: the download address of the algorithm required by the first task to be analyzed; the acquisition unit is further used for directly downloading the algorithm required by the first task to be analyzed from the first data analysis device according to the downloading address of the algorithm required by the first task to be analyzed; the processing unit is specifically used for analyzing the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed contained in the algorithm downloading application, and obtaining an analysis result.
In another possible implementation manner, the receiving unit is further configured to receive an algorithm query application sent by the second scheduling device; the algorithm query application comprises the following steps: identification of an algorithm required by a first task to be analyzed; the data analysis device further includes: the query unit is used for querying whether the data analysis device comprises the algorithm required by the first task to be analyzed according to the identification of the algorithm required by the first task to be analyzed; the receiving unit is further used for receiving an algorithm downloading application sent by the second scheduling device when the data analysis device does not comprise the algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: the download address of the algorithm required by the first task to be analyzed; the acquisition unit is further used for downloading the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
In an eighth aspect, the present application provides a local point, comprising: a scheduling device and a data analysis device; the scheduling device comprises a first memory and a first processor; the first memory is coupled to the first processor; the first memory is for storing computer program code, the computer program code comprising computer instructions; wherein the first processor, when executing the computer instructions, causes the scheduling apparatus to perform the method as provided by the first, second, third or fourth aspect and any one of its possible designs; the data analysis device comprises a second memory and a second processor; the second memory is coupled to the second processor; the second memory is for storing computer program code, the computer program code comprising computer instructions; wherein the second processor, when executing the computer instructions, causes the data analysis apparatus to perform the method as provided by the fifth aspect and any one of its possible designs.
In a ninth aspect, the present application provides a cross-domain analysis system, comprising: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first scheduling device is used for receiving a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises the following steps: tasks to be analyzed; the cross-domain resource scheduling application is used for applying to execute tasks to be analyzed at other local points except the first local point; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the second scheduling device; the second scheduling device is used for forwarding the cross-domain resource scheduling application to the second data analysis device; and the second data analysis device is used for analyzing the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result.
In one possible implementation manner, the cross-domain resource scheduling application further includes: the size of the resources required by the task to be analyzed; the first scheduling device is further used for determining a second local point from other local points according to the cross-domain resource scheduling application.
The local resource allowance of the second local point is larger than or equal to the size of the resource required by the task to be analyzed.
In another possible implementation manner, the system further includes: a central scheduling device; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device; the cross-domain resource scheduling application further includes: the size of the resources required by the task to be analyzed; the first scheduling device is also used for forwarding the cross-domain resource scheduling application to the central scheduling device; the central scheduling device is used for determining a second local point from other local points according to the cross-domain resource scheduling application, and the local resource allowance of the second local point is larger than or equal to the size of resources required by the task to be analyzed; and the central scheduling device is also used for sending a cross-domain resource scheduling application to the second scheduling device.
In another possible implementation manner, the first data analysis device is further configured to: when the size of the resources required by the task to be analyzed is detected to be larger than or equal to a first threshold value, a cross-domain resource scheduling application is sent to a first scheduling device; or when detecting that the computing power resource in the first local point is in an offline state, sending a cross-domain resource scheduling application to the first scheduling device.
In another possible implementation manner, the task to be analyzed is a task of analyzing the data to be analyzed; the cross-domain resource scheduling application further includes: an acquisition address of data to be analyzed; the second data analysis device is also used for acquiring the address according to the data to be analyzed and acquiring the data to be analyzed; and/or, the cross-domain resource scheduling application further comprises: the download address of the algorithm required by the task to be analyzed; the second data analysis device is also used for downloading the algorithm required by the task to be analyzed according to the downloading address of the algorithm required by the task to be analyzed; the second data analysis device is specifically used for analyzing the task to be analyzed according to the data to be analyzed and the algorithm required by the task to be analyzed to obtain an analysis result.
In another possible implementation manner, the second scheduling device is further configured to send an algorithm query application to the second data analysis device; the algorithm query application is used for enabling the second data analysis device to query whether the second data analysis device has an algorithm required by a task to be analyzed; the second data analysis device is also used for carrying out algorithm query according to the algorithm query application and sending the query result to the second scheduling device; the second scheduling device is also used for sending an algorithm downloading application to the second data analysis device when the query result is that the algorithm required by the task to be analyzed is not contained in the second data analysis device; the algorithm downloading application comprises the downloading address of the algorithm required by the task to be analyzed; the second data analysis device is specifically configured to directly download the algorithm required by the task to be analyzed from the first data analysis device according to the download address of the algorithm required by the task to be analyzed included in the algorithm download application.
In another possible implementation manner, the cross-domain resource scheduling application further includes: analyzing the storage address of the result; the second data analysis device is further used for storing the analysis result in the storage space indicated by the analysis result storage address.
In another possible implementation manner, the first scheduling device is further configured to send a control instruction to the second scheduling device; the second scheduling device is also used for controlling the processing state of the task to be analyzed by the second data analysis device according to the control instruction.
In another possible implementation manner, the first data analysis device is further configured to send algorithm change information to the first scheduling device; the algorithm change information comprises a download address of a changed target algorithm in the first data analysis device and an identification of the target algorithm; the first scheduling device is also used for forwarding algorithm change information to the second scheduling device; the second scheduling device is also used for sending the identification of the target algorithm and the download address of the target algorithm to the second data analysis device; and the second data analysis device is also used for inquiring whether the target algorithm exists in the second data analysis device according to the identification of the target algorithm and downloading the target algorithm according to the downloading address of the target algorithm under the condition that the target algorithm does not exist in the second data analysis device.
In another possible implementation manner, the first data analysis device is further configured to send computing power resource change information to the first scheduling device; the computing power resource variation information includes: current remaining power information of the first office point; the first scheduling device is also used for forwarding the computing power resource change information to the second scheduling device; the second scheduling device is further used for changing the remaining computing power information of the first local point recorded in the second scheduling device into the current remaining computing power information of the first local point according to the computing power resource change information.
In a tenth aspect, the present application provides a computer-readable storage medium comprising computer instructions. Wherein the computer instructions, when run on a computer, cause the computer to perform the method as provided by the first, second, third, fourth or fifth aspects and any one of their possible designs.
In an eleventh aspect, the present application provides a computer program product comprising computer instructions. When the computer instructions are run on a computer, the computer is caused to perform the method as provided by the first, second, third, fourth or fifth aspect and any one of its possible designs.
For a detailed description of the fifth to eleventh aspects and various implementations thereof in this application, reference may be made to the detailed description of the first, second, third, fourth aspects and various implementations thereof. The advantageous effects of the fifth aspect to the twelfth aspect and the various implementation manners thereof may be referred to for the advantageous effect analysis of the first aspect, the second aspect, the third aspect, the fourth aspect and the various implementation manners thereof, and are not described herein.
These and other aspects of the present application will be more readily apparent from the following description.
Drawings
Fig. 1 is a schematic structural diagram of a local point according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram III of a cross-domain analysis system according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application;
fig. 6 is a registration flowchart of a data analysis device according to an embodiment of the present application;
fig. 7 is a registration flowchart of a scheduling device according to an embodiment of the present application;
FIG. 8 is a flowchart I of an algorithm synchronization provided in an embodiment of the present application;
FIG. 9 is a second flowchart of an algorithm synchronization according to an embodiment of the present application;
FIG. 10 is a flowchart I of a computing power resource synchronization method according to an embodiment of the present application;
FIG. 11 is a second flowchart of computing resource synchronization according to an embodiment of the present disclosure;
fig. 12 is a flowchart of a cross-domain resource scheduling method provided in an embodiment of the present application;
fig. 13 is a schematic diagram one of a display interface of a terminal device according to an embodiment of the present application;
fig. 14 is a schematic diagram two of a display interface of a terminal device according to an embodiment of the present application;
fig. 15 is a schematic diagram III of a display interface of a terminal device according to an embodiment of the present application;
fig. 16 is a schematic diagram one of a data forwarding mode provided in an embodiment of the present application;
fig. 17 is a schematic diagram two of a data forwarding mode provided in the embodiment of the present application;
fig. 18 is a flowchart two of a cross-domain resource scheduling method provided in an embodiment of the present application;
fig. 19 is a flowchart III of a cross-domain resource scheduling method provided in an embodiment of the present application;
fig. 20 is a flowchart fourth of a cross-domain resource scheduling method provided in an embodiment of the present application;
Fig. 21 is a flowchart fifth of a cross-domain resource scheduling method provided in an embodiment of the present application;
fig. 22 is a schematic structural diagram of a scheduling apparatus according to an embodiment of the present application;
fig. 23 is a schematic structural diagram of a data analysis device according to an embodiment of the present application;
fig. 24 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone.
The terms "first" and "second" and the like in the description and in the drawings are used for distinguishing between different objects or for distinguishing between different processes of the same object and not for describing a particular sequential order of objects.
Furthermore, references to the terms "comprising" and "having" and any variations thereof in the description of the present application are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or apparatus.
It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
In the description of the present application, unless otherwise indicated, the meaning of "a plurality" means two or more.
At present, in the technical field of intelligent analysis of video images, a main working mode of an analysis task is to schedule resources in a single intelligent analysis system of video images, and the following problems can occur in the working mode: when the resources in the office of the office point are insufficient, if a new analysis task appears, the new analysis task cannot be responded in time and can only be queued or discarded, so that the problems of untimely task processing and low processing efficiency can be caused. In addition, if a plurality of intelligent video image analysis systems exist at the same time, because the intelligent video image analysis systems are isolated from each other, the situation that each office point is busy and idle is uneven exists, so that the utilization rate of the whole resources of the intelligent video image analysis systems is low.
In view of the problem, the embodiment of the application provides a cross-domain analysis system, which comprises a plurality of local points (one local point can be regarded as a video image intelligent analysis platform), wherein each local point comprises a scheduling device, and one local point can initiate a cross-domain resource scheduling application (comprising tasks to be analyzed) to another local point through the scheduling device; and the corresponding other local point can receive the cross-domain resource scheduling application through the scheduling device, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the system provided by the embodiment of the application does not need to carry out large-scale transformation on each local point, only needs to introduce the scheduling device on the basis of the original data analysis device, so that information can be circulated among all the local points, cross-domain (information interaction of the cross-local points is called as cross-domain here) scheduling of analysis resources can be realized, calculation resource integration is realized, and the utilization rate of the global calculation resource is improved.
In addition, the cross-domain analysis system provided by the embodiment of the application can timely send the task to be analyzed to other idle local points for analysis when a certain local point cannot bear the task to be analyzed, so that the task to be analyzed can be timely processed under the condition that normal service of the local point is not interrupted. In this way, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle of different local points can be solved, and the utilization rate of the computing power resources in the whole domain is improved.
For example, if there are 40 chips in the local area of the west lake in Hangzhou, 84 chips in the local area of the coast river, 56 chips in the villa area, the resources in each area are not uniform. If emergency occurs in the western lake region, video monitoring points which need to be ensured are more, resources are inevitably insufficient, and if temporary purchasing equipment in the western lake region is time-consuming, resource waste can be caused after event processing is completed. Therefore, by using the technical scheme provided by the embodiment of the application, the local point of the west lake region can initiate a cross-domain resource scheduling application to the local point of the coastal river region and/or the local point of the villa region, and temporarily borrow the resources of the local point of the coastal river region and/or the local point of the villa region.
The implementation of the examples of the present application will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present application provides a local point (i.e. a video image intelligent analysis platform), including: data analysis means and scheduling means. The data analysis device may register in the local scheduling device.
And the data analysis device is used for receiving an analysis task (such as a video image intelligent analysis task) and distributing corresponding computing power resources and algorithms for the analysis task. And the data analysis device is also used for initiating a cross-domain resource scheduling application to the scheduling device of the local point when the local point cannot bear the analysis task. The data analysis device may be, for example, a video image intelligent analysis system.
The dispatching device is used for receiving the cross-domain resource dispatching application sent by the data analysis device of the local point, determining the applied party (other local points except the local point) of the cross-domain resource dispatching application, and sending the cross-domain resource dispatching application to the dispatching device of the applied party.
And the scheduling device is also used for receiving cross-domain resource scheduling applications (forwarded to the local office point by the scheduling devices of other local offices or forwarded to the local office point by the central scheduling device) initiated by the data analysis devices of other local offices except the local office point, and forwarding the cross-domain resource scheduling applications to the data analysis device of the local office point.
In some embodiments, the scheduling means may be one network node physically; alternatively, the scheduling means may be one component (or functional module) logically. The scheduling means may be an AI gateway, for example.
In some embodiments, the scheduling means and the data analysis means may be two independent devices; alternatively, the scheduling means and the first data analysis means may be integrated.
The embodiment of the application also provides a cross-domain analysis system based on the local points shown in fig. 1, wherein the cross-domain analysis system comprises a plurality of local points. The data analysis device of each local point is registered to the dispatching device of the local point, and the dispatching devices of each local point are mutually registered; alternatively, each local scheduling device registers in the central scheduling device to construct a logically uniform large-capacity resource POOL (POOL). Therefore, the resource scheduling can be realized among the office points which finish the registration, and the purpose of resource sharing is achieved.
It may be understood that, in the embodiments of the present application, the cross-domain resource scheduling refers to: and (5) cross-office point resource scheduling, namely, scheduling resources among different office points.
Optionally, the deployment modes of the cross-domain analysis system provided in the embodiment of the present application include: a fully symmetric deployment mode and a central deployment mode.
Referring to fig. 2, a schematic structural diagram of a cross-domain analysis system according to an embodiment of the present application is shown. As shown in fig. 2, the cross-domain analysis system is in a fully symmetric deployment mode, including: a first office point 100 and a second office point 200. Wherein the first office point 100 includes: a first data analysis means 110 and a first scheduling means 120; the second office point 200 includes: a second data analysis means 210 and a second scheduling means 220.
It will be appreciated that the fully symmetric deployment mode of the cross-domain analysis system refers to the deployment of one scheduler within each office, each scheduler functioning consistently. For example, the first scheduler 120 is consistent with the function of the second scheduler 220.
When the cross-domain analysis system is in the full-symmetry deployment mode, the dispatching device of each local point is used for receiving the cross-domain resource dispatching application sent by the data analysis device of the local point, determining the applied party (other local points except the local point) of the cross-domain resource dispatching application, and sending the cross-domain resource dispatching application to the dispatching device of the applied party. Meanwhile, the dispatching device of each local point is also used for receiving the cross-domain resource dispatching application sent by the dispatching devices of other local points except the local point and forwarding the cross-domain resource dispatching application to the data analysis device of the local point.
For example, the first scheduling device 120 is configured to receive the cross-domain resource scheduling application sent by the first data analysis device 110, determine that the applied party of the cross-domain resource scheduling application is the second local point 200, and send the cross-domain resource scheduling application to the second scheduling device 220.
For another example, the second scheduling device 220 is configured to receive the cross-domain resource scheduling application sent by the first scheduling device 120, and forward the cross-domain resource scheduling application to the second data analysis device 210.
In addition, a connection is established between the dispatching devices of each local point, so that information such as algorithm identification, calculation force state, tasks, control instructions and the like can be transmitted in two directions in real time. For example, in fig. 2, between the first scheduler 120 and the second scheduler 220, the interface may be provided via an IF3a interface (one or more IF3a interfaces may be provided, for example IF3a 1 、IF3a 2 、IF3a 3 Etc., not shown in fig. 2) connected in a wired or wireless manner to enable information to be transferred bi-directionally.
Meanwhile, the data analysis device of each local point is connected with the dispatching device of the local point, so that information such as algorithm identification, calculation force state, tasks and control instructions can be transmitted in two directions in real time. For example, in fig. 2, between the first data analysis device 110 and the first scheduling device 120, an IF1a interface may be provided (one or more IF1a interfaces may be provided, for example, IF1a 1 、IF1a 2 、IF1a 3 Etc., not shown in fig. 2), connected in a wired or wireless manner, to realize information bidirectional transfer; between the second data analysis means 210 and the second scheduling means 220, one or more interfaces may be provided via an IF2a interface (the IF2a interface may be provided, for example IF2a 1 、IF2a 2 、IF2a 3 Etc., not shown in fig. 2), and is connected in a wired or wireless manner, thereby realizing information bidirectional transfer.
In some embodiments, in the case where direct communication between the first office point 100 and the second office point 200 is not possible (for example, direct connection between an internal local area network and a video private network of a certain department is not possible), as shown in fig. 3, a boundary security server 400 may be introduced, so that the first scheduling apparatus 120 and the second scheduling apparatus 220 are connected to the boundary security server 400, respectively.
Wherein the boundary security server 400 is configured to perform information transmission between the first scheduler 120 and the second scheduler 220.
Alternatively, the boundary security server 400 may be a file transfer protocol (file transfer protocol, FTP) server. The FTP server is a computer that provides file storage and access services over the internet, and provides services according to the FTP protocol.
Specifically, the first scheduling device 120 may upload the cross-domain scheduling information that needs to be sent to the second scheduling device 220 to the storage space of the boundary security server 400, and then the second scheduling device 220 may view the cross-domain scheduling information in the storage space of the boundary security server 400.
Referring to fig. 4, a schematic structural diagram of another cross-domain analysis system according to an embodiment of the present application is shown. As shown in fig. 4, the cross-domain analysis system is a central deployment mode, including: a first office point 100, a second office point 200 and a central scheduler 300. Wherein the first office point 100 includes: a first data analysis means 110 and a first scheduling means 120; the second office point 200 includes: a second data analysis means 210 and a second scheduling means 220.
It is to be understood that the central deployment mode of the cross-domain analysis system means that one scheduling device is deployed in each office point, the functions of each scheduling device are consistent, and one central scheduling device is deployed outside each office point and is respectively connected with the scheduling device of each office point, and the functions of the central scheduling devices are inconsistent with the functions of the scheduling devices deployed in each office point. For example, the first scheduler 120 is consistent with the function of the second scheduler 220. The central scheduler 300 is logically disposed outside the first office point 100 and the second office point 200, and is connected to the first scheduler 120 and the second scheduler 220, respectively, and the central scheduler 300 is inconsistent with the functions of the first scheduler 120 and the second scheduler 220.
When the cross-domain analysis system is in a central deployment mode, the dispatching device of each local point is used for receiving the cross-domain resource dispatching application sent by the data analysis device of the local point and forwarding the cross-domain resource dispatching application to the central dispatching device. The central dispatching device is used for receiving the cross-domain resource dispatching application sent by the dispatching device of each local point, determining the applied party of the cross-domain resource dispatching application, and sending the cross-domain resource dispatching application to the dispatching device of the applied party. Meanwhile, the dispatching device of each local point is also used for receiving the cross-domain resource dispatching application sent by the central dispatching device and forwarding the cross-domain resource dispatching application to the data analysis device of the local point.
For example, the first scheduling device 120 is configured to receive the cross-domain resource scheduling application sent by the first data analysis device 110, and forward the cross-domain resource scheduling application to the central scheduling device 300. The central scheduling device 300 is configured to receive the cross-domain resource scheduling application sent by the first scheduling device 120, determine that the applied party of the cross-domain resource scheduling application is the second local point 200, and send the cross-domain resource scheduling application to the second scheduling device 220.
For another example, the second scheduler 220 is configured to receive the cross-domain resource scheduling application sent by the central scheduler 300, and forward the cross-domain resource scheduling application to the second data analysis device 210.
In some embodiments, the central dispatching device establishes connection with the dispatching devices of the local points, and can perform information such as algorithm identification, calculation force state, task, control instruction and the like in real timeIs provided). For example, in fig. 4, between the central scheduler 300 and the first scheduler 120, an IF3b interface may be used (one or more IF3b interfaces, such as IF3b, may be used 1 、IF3b 2 、IF3b 3 Etc., not shown in fig. 4), connected in a wired or wireless manner; between the central scheduler 300 and the second scheduler 220, an IF4b interface may be provided (the IF4b interface may be provided with one or more, for example IF4b interfaces 1 、IF4b 2 、IF4b 3 Etc., not shown in fig. 4), connected in a wired or wireless manner.
Meanwhile, the data analysis device of each local point is connected with the dispatching device of the local point, so that information such as algorithm identification, calculation force state, tasks and control instructions can be transmitted in two directions in real time. For example, in fig. 4, between the first data analysis device 110 and the first scheduling device 120, an IF1b interface may be provided (one or more IF1b interfaces may be provided, for example, IF1b 1 、IF1b 2 、IF1b 3 Etc., not shown in fig. 4), connected in a wired or wireless manner; between the second data analysis means 210 and the second scheduling means 220, one or more interfaces may be provided via an IF2b interface (an IF2b interface may be provided, for example IF2 b) 1 、IF2b 2 、IF2b 3 Etc., not shown in fig. 4), connected in a wired or wireless manner.
In some embodiments, no direct communication is possible between the first local site 100 and the central scheduler 300; and/or, in the case where the central scheduler 300 and the second office 200 cannot communicate directly, as shown in fig. 5, a boundary security server 400 may be introduced, so that the first scheduler 120 and the central scheduler 300 are connected to the boundary security server 400, respectively; and/or the central scheduler 300 and the second scheduler 220 are connected to the boundary security server 400, respectively.
Wherein the boundary security server 400 is used for information transmission between the first scheduler 120 and the central scheduler 300; and/or the boundary security server 400 is used for information transmission between the central scheduler 300 and the second scheduler 220.
Specifically, the first scheduling device 120 may upload the cross-domain scheduling information that needs to be sent to the second scheduling device 220 to the storage space of the boundary security server 400, so that the central scheduling device 300 may view the cross-domain scheduling information in the storage space of the boundary security server 400; and/or, the central scheduler 300 may upload the cross-domain scheduling information to be transmitted to the second scheduler 220 into the storage space of the boundary security server 400, and thus the second scheduler 220 may view the cross-domain scheduling information in the storage space of the boundary security server 400.
In some embodiments, before cross-domain resource scheduling is performed based on the cross-domain analysis system provided in the embodiments of the present application, the following operation flow needs to be performed: registration flow, algorithm synchronization flow and information synchronization flow of computing resources.
1. Registration process
The registration flow includes: registration of the data analysis device, registration of the scheduling device, and registration of the boundary security server.
(1) Registration of data analysis device
The registration of the data analysis devices means that each data analysis device needs to register at the local scheduling device. For example, a first data analysis device needs to register with a first scheduler and a second data analysis device needs to register with a second scheduler.
Optionally, when the data analysis device registers with the scheduling device, the content to be filled in includes at least one of the following: internet protocol (internet protocol, IP) address, port number, etc.
It will be appreciated that after registration is completed, connection communication is established between the data analysis device and the scheduling device in the local site, so that information such as algorithm, computing resource, task, state and the like can be transferred.
For example, taking the first data analysis device registering in the first scheduling device as shown in fig. 6, the registration flow includes:
The Sa1, the first scheduling device receives a registration application initiated by the first data analysis device.
Wherein the registration application includes: the IP address and port number of the first data analysis device.
And Sa2, the first scheduling device judges whether the content of the registration application accords with the standard.
In some embodiments, in case the content of the registration application does not meet the criteria, the following step Sa3 is performed; in the case where the content of the registration application meets the criterion, the following step Sa4 is performed.
Optionally, the first scheduling device may adopt a manual auditing or automatic auditing mode to determine whether the content of the registration application meets the standard.
Sa3, the first scheduler refutes the registration application.
In some embodiments, after the first scheduler refutes the registration application, the first data analysis device may modify the registration application and reinitiate until the audit passes; alternatively, the first data analysis means may cancel the registration application.
Sa4, the first scheduling device applies for the registration.
Thus, the first data analysis device successfully registers with the first scheduling device.
(2) Registration of scheduling device
The system comprises a cross-domain analysis system in different deployment modes, and a scheduling device is registered in different modes.
Illustratively, when the cross-domain analysis system is in a fully symmetric deployment mode as shown in fig. 2, the registration of the scheduler refers to: the scheduling device of each local office needs to register with the scheduling devices of other local offices except the local office. For example, a first scheduler needs to register with a second scheduler, and the second scheduler needs to register with the first scheduler.
Illustratively, when the cross-domain analysis system is in a central deployment mode as shown in fig. 4, the registration of the scheduler refers to: each local scheduler needs to register with the central scheduler. For example, a first scheduler needs to register with a central scheduler and a second scheduler needs to register with the central scheduler.
Optionally, when the scheduling device of each office point registers, the content to be filled in includes at least one of the following: the name of the local point, internet protocol (internet protocol, IP) address, port number, etc.
It will be appreciated that after registration is completed, connection communication is established between the scheduling devices of each office point (or between each office point and the central scheduling device), and information such as algorithm identification, calculation force state, task, control instruction and the like can be transferred.
For example, taking the first scheduling device registering in the second scheduling device as shown in fig. 7, the registration procedure includes:
and Sb1, the second dispatching device receives a registration application initiated by the first dispatching device.
Wherein the registration application includes: the name of the first office point, the IP address of the first scheduler, the port number and the Kafka address.
And Sb2, the second scheduling device judges whether the content of the registration application meets the standard or not.
In some embodiments, in case the content of the registration application does not meet the criteria, the following step Sa3 is performed; in the case where the content of the registration application meets the criterion, the following step Sa4 is performed.
Optionally, the second scheduling device may adopt a manual auditing or automatic auditing mode to determine whether the content of the registration application meets the standard.
Sb3, the second scheduling device refutes the registration application.
In some embodiments, after the second scheduler refutes the registration request, the first scheduler may modify the registration request and reinitiate until the audit is passed; alternatively, the first scheduler may cancel the registration application.
Sb4, the second scheduling device applies for the registration.
Thus, the first scheduler successfully registers with the second scheduler.
In some embodiments, the procedure of the first scheduling device registering in the second scheduling device is the same as the procedure of the first scheduling device registering in the second scheduling device, and will not be described herein.
In another embodiment, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the first scheduling device and the second scheduling device need to register in the central scheduling device, and the registration flow is the same as the registration flow shown in fig. 7, and only the execution subject in fig. 7 needs to be replaced by the second scheduling device to be replaced by the central scheduling device, which is not described herein again.
(3) Registration of boundary security servers
In the case where direct communication is not possible between two office points, a boundary security server is introduced, at which point the boundary security server acts as an intermediary to enable indirect communication between the two office points. Therefore, the boundary security server needs to register in the scheduling apparatuses of the two local sites, respectively. Thus, through registration, the boundary security server can realize information transmission with the dispatching devices of the two local points.
For example, in the case where direct communication is not possible between the first local point and the second local point, a boundary security server is introduced, and the boundary security server needs to be registered in the first scheduling apparatus and the second scheduling apparatus, respectively.
Specifically, the first scheduling device adds registration information of the boundary security server, so that the boundary security server registers in the first scheduling device. The second scheduling device adds registration information of the boundary security server so that the boundary security server registers in the second scheduling device. The registration information includes: the IP address, port, identification information, authentication information (e.g., account number, password, etc.), and address information (e.g., kafka address, mapped Kafka address, memory address, and mapped memory address) of the boundary security server, etc.
Meanwhile, the first scheduling device or the second scheduling device creates a shared folder in the storage space of the boundary safety server and is used for storing cross-domain scheduling information. Meanwhile, the first scheduling device or the second scheduling device can also generate a first subfolder and a second subfolder in the shared folder, wherein the first subfolder is used for storing algorithm information, and the second subfolder is used for storing task information.
For example, in the case where direct communication between the central dispatcher and the second local site is not possible, a boundary security server is introduced, and the boundary security server needs to be registered in the central dispatcher and the second dispatcher, respectively. The registration flow of the boundary security server in the central dispatching device is the same as that of the boundary security server in the first dispatching device in the example.
2. Algorithm synchronization flow
Algorithm synchronization means that the algorithm information of each local point needs to be kept consistent. That is, each local scheduling device needs to synchronize (or broadcast) the algorithm change information of the target algorithm changed in the local data analysis device to the other scheduling devices of each local in real time, so that the other scheduling devices of each local perform algorithm synchronization according to the algorithm change information.
It can be appreciated that, because the algorithm package itself is relatively large, if the algorithm package is downloaded during use, the timeliness of the task is affected, and the task analysis progress is delayed. For example, if the applied party of the cross-domain resource scheduling application does not have the algorithm required by the task to be analyzed, the task to be analyzed cannot be processed. Therefore, before cross-domain resource scheduling is performed, an algorithm synchronization flow is needed to ensure that algorithm information of each local point is kept consistent.
The algorithm synchronization flow of the cross-domain analysis system in different deployment modes is also different.
For example, when the cross-domain analysis system is in a fully symmetric deployment mode as shown in fig. 2, as shown in fig. 8, the algorithm synchronization flow may include:
sc1, the first scheduling device receives the algorithm change information sent by the first data analysis device.
The algorithm change information comprises identification of a target algorithm changed in the first data analysis device and a download address of the target algorithm. The target algorithm may be, for example, an own algorithm (an algorithm not downloaded from another local point) newly added in the first data analysis device.
For example, when an own algorithm (an algorithm not downloaded from another local point) is newly added to the first data analysis device, the first data analysis device transmits the algorithm change information to the first scheduling device.
Sc2, the second scheduling device receives algorithm change information sent by the first scheduling device.
Sc3, the second scheduling device sends the download address of the target algorithm to the second data analysis device.
In some embodiments, prior to step Sc3, the method further comprises: the second scheduling device sends an algorithm query instruction to the second data analysis device, the algorithm query instruction comprising: and the algorithm query instruction is used for enabling the second data analysis device to query whether the target algorithm is included in the second data analysis device according to the identification of the target algorithm.
In the case where the target algorithm is not included in the second data analysis means, step Sc3 is performed.
And Sc4, the second data analysis device downloads the target algorithm according to the downloading address of the target algorithm.
In some embodiments, in the case where direct communication between the first office point and the second office point is not possible, the algorithm synchronization procedure in the case where the cross-domain analysis system is in the fully symmetric deployment mode may be implemented as:
the first scheduling device uploads the algorithm change information and the algorithm package of the target algorithm changed in the first data analysis device to a first subfolder (used for storing the algorithm information) in the shared folder of the boundary security server; meanwhile, the first scheduling device adds a summary file in the first subfolder, wherein the summary file is used for quickly looking up information of a target algorithm, and the summary file comprises: the algorithm package name, algorithm manufacturer, algorithm version, algorithm identification, chip type, file size, etc. of the target algorithm.
The second scheduling device may check the summary file of the first subfolder in the shared folder of the boundary security server at regular time, and when a new target algorithm is found, send an algorithm query instruction to the second data analysis device, so that the second data analysis device queries whether the target algorithm is included.
In the case where the target algorithm is not included in the second data analysis means, the second scheduling means transmits the download address of the target algorithm to the second data analysis means. The download address of the target algorithm is the address of a first subfolder in the shared folder of the boundary security server.
The second data analysis device downloads the target algorithm in the boundary security gateway according to the download address of the target algorithm.
In some embodiments, after the second data analysis device completes downloading the target algorithm, the border security gateway may delete the algorithm information of the target algorithm stored in the first subfolder of the shared folder, thereby reducing space occupation and improving storage space utilization.
Illustratively, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, as shown in fig. 9, the algorithm synchronization flow includes:
sd1, the first scheduling device receives algorithm change information sent by the first data analysis device.
Sd2, the central dispatching device receives algorithm change information sent by the first dispatching device.
Sd3, the second dispatching device receives the algorithm change information sent by the central dispatching device.
Sd4, the second scheduling device sends the download address of the target algorithm to the second data analysis device.
In some embodiments, prior to step Sd4, the method further comprises: the second scheduling device sends an algorithm query instruction to the second data analysis device, the algorithm query instruction comprising: and the algorithm query instruction is used for enabling the second data analysis device to query whether the target algorithm is included in the second data analysis device according to the identification of the target algorithm.
In case no target algorithm is included in the second data analysis means, step Sd4 is performed.
And Sd5, the second data analysis device downloads the target algorithm according to the identification of the target algorithm.
In some embodiments, in the case where direct communication between the central dispatcher and the second local site is not possible, the algorithm synchronization procedure in the deployment mode with the cross-domain analysis system as the center may be implemented as:
the first scheduling device receives the algorithm change information sent by the first data analysis device and the algorithm packet of the target algorithm changed in the first data analysis device; the central dispatching device receives the algorithm change information and the algorithm packet of the target algorithm sent by the first dispatching device.
The central dispatching device uploads the algorithm change information and the algorithm package of the target algorithm to a first subfolder (used for storing the algorithm information) in the shared folder of the boundary security server; meanwhile, the central scheduling device adds the abstract file in the first subfolder.
The second scheduling device may check the summary file of the first subfolder of the shared folder of the boundary security server at regular time, and when a new target algorithm is found, send an algorithm query instruction to the second data analysis device, so that the second data analysis device queries whether the target algorithm is included.
In the case where the target algorithm is not included in the second data analysis means, the second scheduling means transmits a download address of the target algorithm to the second data analysis means so that the second data analysis means downloads the target algorithm in the border security gateway according to the download address of the target algorithm. The download address of the target algorithm is the address of a first subfolder in the shared folder of the boundary security server.
In some embodiments, after the second data analysis device completes downloading the target algorithm, the border security gateway may delete the algorithm information of the target algorithm stored in the first subfolder of the shared folder, thereby reducing space occupation and improving storage space utilization.
3. Information synchronization process of computing power resource
The information synchronization flow of the computing power resources refers to that the scheduling device of each local point needs to synchronize (or broadcast) the basic information of the computing power resources of the local point to the scheduling devices of other local points in real time.
Wherein, the basic information of the computing power resource comprises: chip type, total calculation scale, calculation resource allowance, etc.
It can be understood that, since the scheduling device can determine which local point can be the applied party of the cross-domain resource scheduling application under the condition of knowing the computing power resource of each local point in the cross-domain analysis system, the scheduling device needs to perform the information synchronization flow of the computing power resource in real time.
The information synchronization flow of the computing power resources of the cross-domain analysis system in different deployment modes is also different.
For example, when the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, as shown in fig. 10, the information synchronization flow of the computing power resource may include:
se1, the first scheduling device receives the computing power resource change information sent by the first data analysis device.
Wherein the computing power resource variation information includes: the current remaining power information of the first office point.
For example, when the first data analysis device detects that the computing power resource is changed, the first data analysis device transmits computing power resource change information to the first scheduling device.
Se2, the second scheduling device receives the computing power resource change information sent by the first scheduling device.
Se3, the second scheduling device changes the remaining power information of the first local point recorded in the second scheduling device into the current remaining power information of the first local point according to the power resource change information.
In some embodiments, in the event that direct communication between the first local point and the second local point is not possible, the first scheduling device of the first local point may upload the computing power resource change information (the computing power resource change information includes current remaining computing power information of the first local point) to the shared folder of the boundary security server; correspondingly, the second scheduling device of the second office point can check the shared folder of the boundary security server regularly, and when the newly added computing resource change information is detected, the second scheduling device changes the remaining computing power information of the first office point recorded in the second scheduling device into the current remaining computing power information of the first office point according to the computing power resource change information.
For example, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, as shown in fig. 11, the information synchronization flow of the computing power resource may include:
and Sf1, the first scheduling device receives the computing power resource change information sent by the first data analysis device.
And Sf2, the central scheduling device receives the computing power resource change information sent by the first scheduling device.
And Sf3, the second scheduling device receives the computing power resource change information sent by the central scheduling device.
And Sf4, the second scheduling device changes the residual calculation power information of the first local point recorded in the second scheduling device into the current residual calculation power information of the first local point according to the calculation power resource change information.
In some embodiments, in the case that the central dispatching device and the second office point cannot directly communicate, the central dispatching device uploads the computing power resource change information (the computing power resource change information comprises the current residual computing power information of the first office point) sent by the first dispatching device to a shared folder of the boundary security server after receiving the computing power resource change information; correspondingly, the second scheduling device of the second office point can check the shared folder of the boundary security server regularly, and when the newly added computing resource change information is detected, the second scheduling device changes the remaining computing power information of the first office point recorded in the second scheduling device into the current remaining computing power information of the first office point according to the computing power resource change information.
The cross-domain resource scheduling method provided by the embodiment of the application is described in detail below.
Specifically, the cross-domain resource scheduling method provided by the embodiment of the application is applied to a cross-domain analysis system, and the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: and the second data analysis device and the second scheduling device.
Referring to fig. 12, a flowchart of a cross-domain resource scheduling method according to an embodiment of the present application is provided. As shown in fig. 12, the method may include S101-S104.
S101, a first scheduling device receives a first cross-domain resource scheduling application sent by a first data analysis device.
The first cross-domain resource scheduling application comprises the following steps: and a first task to be analyzed. And the first cross-domain resource scheduling application is used for applying to execute the first task to be analyzed at other local points (such as a second local point) except the first local point. Illustratively, the first task to be analyzed may include: gait recognition tasks (recognizing gait information in the target video/picture), human body recognition tasks (recognizing human body in the target video/picture), face recognition tasks (analyzing face information in the target video/picture), vehicle recognition tasks (recognizing vehicle in the target video/picture), and the like.
In addition, the first cross-domain resource scheduling application may further include at least one of: the method comprises the steps of obtaining address (for example, if the data to be analyzed is video data, obtaining address is video streaming IP address, if the data to be analyzed is picture data, obtaining address is URL address of picture) of first task to be analyzed, algorithm information (comprising identification of algorithm required by the task to be analyzed, downloading address of algorithm required by the task to be analyzed, and the like) required by the first task to be analyzed, analysis result storage address, application reason, and the like.
In some embodiments, there are two scheduling schemes when cross-domain resource scheduling is performed: 1. and borrowing the resolving power to the applied party of the resource by taking the video path number as a unit, namely, the resource amount applied by the first cross-domain resource scheduling application is the video path number needing to be analyzed. For example, if the first task to be analyzed needs to analyze 10 paths of videos, the amount of resources applied by the first cross-domain resource scheduling is the resources required for analyzing 10 paths of videos. 2. And borrowing the GPU cards from the resource applied party by taking the chip as a unit, namely, applying for the first cross-domain resource scheduling, wherein the amount of the applied resources is the number and the type of the needed GPU cards. For example, if the first task to be analyzed requires three GPU cards, the amount of resources applied for the first cross-domain resource scheduling is three GPU cards.
The difference between the two scheduling schemes is shown in table 1:
TABLE 1
It can be understood that, in the embodiment of the present application, the scheduling scheme in the cross-domain resource scheduling is not particularly limited, and a user may select according to actual use situations.
For example, if the first task to be analyzed is to analyze a vehicle in the target video. The first cross-domain resource scheduling application is: first task to be analyzed: analyzing vehicles in the target video; first task applicant side information to be analyzed: the identification of the first local point and the address information of the first local point; the size of the resources required by the first task to be analyzed: analyzing the maximum limit of the required computing power resources of the target video, for example, the first task to be analyzed needs to analyze GPU computing power resources of 10 paths of target videos altogether; data storage address to be analyzed: video streaming IP address of the target video; algorithm information required by a first task to be analyzed: a video structuring algorithm; the analysis result storage address is: the analyzed vehicle attributes, the vehicle feature model and the storage address of the vehicle picture; the reason for the application is as follows: the intra-office resource margin of the first office point does not meet the resource size required for the task to be analyzed.
In some embodiments, each local point is connected to a fixed video monitoring point, for example, a first local point is connected to a first video monitoring point, and the first local point is used for analyzing a video data analysis task corresponding to the first video monitoring point; and the second local point is connected with the second video monitoring point, and is used for analyzing a video data analysis task corresponding to the second video monitoring point.
The first cross-domain resource scheduling application may be used to apply for analyzing a video data analysis task corresponding to the second video monitoring point in the second local point; or, the first cross-domain resource scheduling application can also be used for applying for analyzing video data analysis tasks corresponding to third-party video monitoring points (other video monitoring points except the first video monitoring point and the second video monitoring point) in the second local point.
If the first local point is a west lake scheduling platform of a west lake region and the second local point is a coastal scheduling platform of a coastal region, the first cross-domain resource scheduling application is initiated by the west lake scheduling platform and is used for applying for analyzing video monitoring points of the coastal region to the coastal scheduling platform; or the first cross-domain resource scheduling application is initiated by a western lake scheduling platform and is used for applying the video monitoring point position of the analysis arch villa area (third party) to the coastal river scheduling platform.
As a possible implementation manner, when the first data analysis device detects that the size of the resource required by the task to be analyzed is greater than or equal to a first threshold, a first cross-domain resource scheduling application is generated, and the first cross-domain resource scheduling application is sent to the first scheduling device.
Illustratively, if the local resource margin of the first local point is GPU computing power resource capable of analyzing 5 paths of videos; and when the first threshold value is 5, the first data analysis device generates a first cross-domain resource scheduling application according to the first task to be analyzed when detecting that the size of the resource required by the first task to be analyzed is GPU computing power resource for analyzing the 8 paths of videos, and sends the first cross-domain resource scheduling application to the first scheduling device.
As another possible implementation manner, when the first data analysis device detects that the computing power resource in the first local point is in an offline state, a first cross-domain resource scheduling application is generated, and the first cross-domain resource scheduling application is sent to the first scheduling device.
For example, if the execution node 1 in the first local node is in an offline state, when the first data analysis device detects that the execution node 1 is included in the execution node required by the first task to be analyzed, the first data analysis device generates a first cross-domain resource scheduling application according to the first task to be analyzed, and sends the first cross-domain resource scheduling application to the first scheduling device.
In some embodiments, the first scheduling device records the resource allowance situation of all local points in the cross-domain analysis system. For example, the first scheduling device records the resource margin of the second local station. The first scheduling device may determine a second local point from the other local points according to the first cross-domain resource scheduling application, and forward the first cross-domain resource scheduling application to the second scheduling device.
The local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed.
It can be understood that the first local point is the application party of the resource, and the second local point is the applied party of the resource. When the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed, the second local point can be used as an applied party of the resource, the first cross-domain resource scheduling application is forwarded to the second scheduling device, and the resource amount applied by the first cross-domain resource scheduling application is the size of the resource required by the task to be analyzed.
In other embodiments, when the intra-local resource margins of all local points in the cross-domain analysis system do not meet (i.e. are smaller than) the size of the resources required by the first task to be analyzed, the first scheduling device determines, according to the size of the resources required by the first task to be analyzed and the preset application rule, a second local point (the second local point is a plurality of at the moment).
As a possible implementation manner, the preset application rule may be a proportional approximation policy, and the first scheduling device is specifically configured to determine, according to the proportional approximation policy, the second local points (at this time, the second local points are a plurality of) according to the size of the resources required by the first task to be analyzed and the resource allowance situation of all the local points in the cross-domain analysis system, and determine, according to the resource allowance of each second local point, the size of the resource amount applied in the first cross-domain resource scheduling application sent to the local point.
The proportion approximation strategy is used for determining the applied side of the cross-domain resource scheduling application and the size of the resource quantity applied to each applied side according to the proportion of the resource allowance of each local point in the cross-domain analysis system to the total amount of the resource allowance of all local points in the cross-domain analysis system.
Illustratively, the cross-domain analysis system is assumed to include: local point 1, local point 2, local point 3, and local point 4. If the local point 1 is the application party of the resource, and the size of the resource required by the task to be analyzed is GPU computing power resource capable of analyzing 10 paths of videos. The resource allowance of the local point 2 is GPU (graphics processing Unit) power resources capable of analyzing 2 paths of videos, the resource allowance of the local point 3 is GPU power resources capable of analyzing 4 paths of videos, the resource allowance of the local point 4 is GPU power resources capable of analyzing 6 paths of videos, and then the second local point is determined to be respectively: local point 2, local point 3 and local point 4; the cross-domain resource scheduling application sent to the local point 2 is applied for the resource amount: 10× (2/(2+4+6)); the cross-domain resource scheduling application sent to the local point 3, the applied resource amount is: 10× (4/(2+4+6)); the cross-domain resource scheduling application sent to the local point 4, the applied resource amount is: 10× (6/(2+4+6)).
As another possible implementation manner, the preset application rule may be a sequential allocation policy, and the first scheduling device is specifically configured to determine, according to the size of resources required by the first task to be analyzed and the resource allowance situation of all local points in the cross-domain analysis system, the second local points (at this time, the second local points are plural) according to the sequential allocation policy, and determine, according to the resource allowance of each second local point, the size of the resource amount applied by the first cross-domain resource scheduling application sent to the local point.
The sequence allocation strategy is to sequence the resource allowance of each local point in the cross-domain analysis system according to the sequence from big to small, firstly, send a cross-domain resource scheduling application to the first local point (namely, the local point with the largest resource allowance), then send the cross-domain resource scheduling application to the second local point, and then send the cross-domain resource scheduling application in sequence until the resources required by the target task are allocated.
Illustratively, the cross-domain analysis system is assumed to include: local point 1, local point 2, local point 3, and local point 4. If the local point 1 is the application party of the resource, and the resource required by the target task is the GPU computing resource capable of analyzing 10 paths of videos. The resource allowance of the local point 2 is GPU (graphics processing Unit) power resources capable of analyzing 2 paths of videos, the resource allowance of the local point 3 is GPU power resources capable of analyzing 4 paths of videos, the resource allowance of the local point 4 is GPU power resources capable of analyzing 6 paths of videos, and then the second local points are determined to be respectively: local point 4 and local point 3; firstly, a cross-domain resource scheduling application is sent to a local point 4, the applied resource amount is GPU computing power resources for analyzing 6 paths of videos, then the cross-domain resource scheduling application is sent to the local point 3, and the applied resource amount is GPU computing power resources for analyzing 4 paths of videos.
It can be understood that, in the cross-domain resource scheduling method provided by the embodiment of the present application, the scheduling device may determine that the applied party of the cross-domain resource scheduling application is a local point or a plurality of local points according to the size of the resources required by the task to be analyzed and the local resource margins of all the local points in the system. Meanwhile, under the condition that the applied party of the cross-domain resource scheduling application is a plurality of local points, the method also supports to determine a plurality of applied parties according to a plurality of preset application rules (comprising a proportion approximation strategy or a sequence allocation strategy). Therefore, the scheduling device can flexibly process the cross-domain resource scheduling application, so that the resource scheduling is more reasonable, and the resource utilization rate is improved.
As yet another possible implementation manner, the first scheduling device may feed back the intra-office resource margins of all the office points in the cross-domain analysis system to the user, and the user manually selects one or more second office points (i.e., the applied party of the cross-domain resource scheduling application).
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. In this way, the first scheduling device may send the first cross-domain resource scheduling application to the terminal device, and the terminal device displays the first cross-domain resource scheduling application to the user. Further, the user may indicate the applied party of the cross-domain resource scheduling application.
For example, as shown in (a) in fig. 13, the terminal device may display a cross-domain resource scheduling application interface on the display screen, so that the user may learn the content of the first cross-domain resource scheduling application; further, as shown in (b) of fig. 13, the user may indicate, on the resource-indicated-to-be-applied interface, a to-be-applied party for the first cross-domain resource scheduling application, and illustratively, the to-be-applied party for the first cross-domain resource scheduling application may be: a second local point.
Correspondingly, the terminal device can also send the applied side of the first cross-domain resource scheduling application indicated by the user to the first scheduling device, so that the first scheduling device sends the first cross-domain resource scheduling application to the scheduling device of the applied side of the resource indicated by the user.
It can be understood that, in the embodiment of the present application, a cross-domain resource scheduling application may be sent to a terminal device by using a scheduling apparatus, and further, the terminal device may display the cross-domain resource scheduling application to a user, so that the user may indicate, for the cross-domain resource scheduling application, that a resource is applied for according to the content of the cross-domain resource scheduling application. Therefore, decision weights can be provided for users through the terminal equipment, so that the method provided by the application can be suitable for richer use scenes.
In some embodiments, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the first scheduling device sends the first cross-domain resource scheduling application to the central scheduling device, so that the central scheduling device determines a second local point from other local points according to the first cross-domain resource scheduling application, and sends the first cross-domain resource scheduling application to the second scheduling device.
The central dispatching device records the resource allowance condition of all local points in the cross-domain analysis system.
Optionally, the manner in which the central scheduling device determines the second local point is the same as the manner in which the first scheduling device determines the second local point.
In some embodiments, the first scheduling device may directly send the first capacity expansion application to the second scheduling device when the amount of resources required by the first task to be analyzed increases; or the first scheduling device may send the first capacity expansion application to the second scheduling device through the central scheduling device.
The first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point.
S102, the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device.
In some embodiments, the second scheduler receives the first cross-domain resource scheduling application through the central scheduler when the cross-domain analysis system is in a central deployment mode as shown in fig. 4.
In some embodiments, the second scheduling device determines whether the amount of resources that can be shared externally in the second local station meets the amount of resources applied by the first cross-domain resource scheduling application sent by the first scheduling device.
And when the resource quantity which can be shared outside in the second local point meets the resource quantity applied by the first cross-domain resource scheduling application sent by the first scheduling device, the second scheduling device checks the first cross-domain resource scheduling application sent by the first scheduling device.
It is understood that the amount of resources that can be shared externally in the second local site may be the same or different from the amount of resources remaining in the second local site. For example, if the intra-office resource margin of the second office is 20, the second office may set the amount of resources that can be shared externally to 10, and use the remaining amount of resources for processing tasks inside the second office.
Optionally, the second scheduling device may audit the first cross-domain resource scheduling application in an automatic audit manner; or, a manual auditing mode is adopted to audit the first cross-domain resource scheduling application sent by the first scheduling device.
In some embodiments, the second scheduling device may receive the verification code sent by the first scheduling device, and determine, according to the verification code, whether the identity of the first local point meets the specification (to avoid malicious attack), and audit the first cross-domain resource scheduling application when the first local point is a local point meeting the specification.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. The terminal equipment comprises a display screen, and can display a first cross-domain resource scheduling application.
In this way, the above-mentioned manual auditing method, which is implemented by auditing the first cross-domain resource scheduling application sent by the first scheduling device, may be: the first scheduling device sends a first cross-domain resource scheduling application to the terminal equipment, and the terminal equipment displays the first cross-domain resource scheduling application on an audit interface of the cross-domain resource scheduling application, so that a user audits the first cross-domain resource scheduling application.
For example, as shown in fig. 14, the audit interface of the cross-domain resource scheduling application may include: the method comprises the steps of applying cross-domain resource scheduling to be audited, applying cross-domain resource scheduling passing audit and applying cross-domain resource scheduling not passing audit. The user can click on any window of the cross-domain resource scheduling application to check the detailed content of the cross-domain resource scheduling application.
The second scheduling device, for example, performs an audit on the first cross-domain resource scheduling application sent by the first scheduling device, including: auditing the application party information of the first task to be analyzed, auditing the algorithm information required by the first task to be analyzed, auditing the application reason and the like.
And under the condition that the first cross-domain resource scheduling application audit sent by the first scheduling device is not passed, the second scheduling device refuses the first cross-domain resource scheduling application sent by the first scheduling device.
And under the condition that the first cross-domain resource scheduling application audit sent by the first scheduling device passes, the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device.
In some embodiments, after the first cross-domain resource scheduling application sent by the first scheduling device passes the audit, the second scheduling device may further modify the amount of resources applied by the first cross-domain resource scheduling application sent by the first scheduling device, for example, in the case that the amount of resources applied by the first cross-domain resource scheduling application sent by the first scheduling device is GPU power resources for analyzing 5 paths of video, the amount of resources that the second scheduling device may grant is GPU power resources for analyzing 2 paths of video, that is, the amount of resources applied by the first scheduling device is modified to GPU power resources for analyzing 2 paths of video.
It can be understood that after the second scheduling device receives the cross-domain resource scheduling application, the second scheduling device further examines the cross-domain resource scheduling application from the perspective of the applied party of the resource, and determines whether the application meets the requirement of the second local point. On the one hand, the second scheduling device needs to confirm whether the resources in the local point meet the size of the resources required by the task to be analyzed; on the other hand, the second scheduling device needs to confirm whether the cross-domain resource scheduling application accords with the relevant rule of the local point or not. And processing the task to be analyzed in the cross-domain resource scheduling application under the condition that the auditing is passed. Therefore, by further auditing, the problems in the cross-domain resource scheduling application can be discovered in advance before the task to be analyzed is analyzed, the task analysis efficiency is improved, and the cross-domain analysis system can orderly perform cross-domain resource scheduling.
In some embodiments, in the case where the first office point and the second office point cannot directly communicate, the step S102 may be implemented as: the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device through the boundary security server.
Specifically, the first scheduling device may apply the first cross-domain resource scheduling to generate a lightweight data exchange format (java script object notation, JSON) file, upload the JSON file to a second subfolder (for storing task information) of the shared folder of the boundary security server, and name the JSON file as a name of the first task to be analyzed.
Correspondingly, the second scheduling device can regularly check a second subfolder of the shared folder of the boundary security server, when detecting the new first task to be analyzed, the second scheduling device generates a new task (same as the first task to be analyzed) in the second local point according to the content of the first task to be analyzed stored in the second subfolder, and executes the new task in the second local point.
S103, the second data analysis device receives the first cross-domain resource scheduling application sent by the second scheduling device.
In some embodiments, the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device in case the first cross-domain resource scheduling application passes the audit, and the second data analysis device further receives the first cross-domain resource scheduling application.
S104, the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
The first task to be analyzed is a task for analyzing the data to be analyzed.
In some embodiments, the first cross-domain resource scheduling application further comprises: an acquisition address of data to be analyzed; the method further comprises: the second data analysis device acquires the data to be analyzed according to the acquisition address of the data to be analyzed.
In some embodiments, in the case that direct communication between the first local point and the second local point is not possible, the first scheduling device further needs to store the data to be analyzed and/or the acquisition address of the data to be analyzed in the second subfolder of the shared folder of the boundary security server.
For example, if the data to be analyzed is real-time video data, the first scheduling device stores the video point location information and the streaming address of the real-time video data in the second subfolder of the shared folder of the boundary security server in the format of JSON file. Correspondingly, the second scheduling device can acquire the video point location information and the streaming address of the real-time video data in the second subfolder of the shared folder of the boundary safety server, and further the second data analysis device acquires the real-time video data according to the video point location information and the streaming address of the real-time video data.
In still another example, if the data to be analyzed is video data, the first scheduling device stores the video data in the second subfolder of the shared folder of the border security server in the format of JSON file, and the data analysis device may download the video data in the second subfolder of the shared folder of the border security server.
If the first scheduling device detects that no corresponding video data exists locally, the first scheduling device writes task abnormality information of 'video data does not exist' in a second subfolder of a shared folder of the boundary safety server, and then the first scheduling device downloads video from locally, stores the video as offline video data in the second subfolder of the shared folder of the boundary safety server, deletes the previous task, and generates a new task.
In still another example, if the data to be analyzed is offline video data, the first scheduling device stores the offline video data in the second subfolder of the shared folder of the boundary security server in the format of JSON file, and the data analysis device may download the offline video data in the second subfolder of the shared folder of the boundary security server.
In some embodiments, after the second data analysis device obtains the data to be analyzed, the boundary security server may delete the data file to be analyzed in the second subfolder of the shared folder, thereby reducing space occupation and improving storage space utilization.
In some embodiments, if the second data analysis device cannot obtain the data to be analyzed, that is, if there is a video streaming abnormality, the streaming abnormality result is sent to the second scheduling device, and the second scheduling device forwards the streaming abnormality result to the first scheduling device, so that the first scheduling device knows the streaming abnormality result. The flow taking abnormal result comprises the following steps: the error code returned when the video data is acquired according to the acquisition address of the data to be analyzed for the first time, and the code rate of the video data. Further, the first scheduling device analyzes whether the flow taking bottleneck of the data acquisition server reaches an upper limit according to the flow taking abnormal result (for example, the flow taking bottleneck of a single data acquisition server can be 600M), and if the flow taking bottleneck reaches the upper limit, the data acquisition server is prompted to be added; if the upper limit is not reached, judging that other streaming abnormality exists, and at the moment, if the data to be analyzed is real-time video data, retransmitting the task to be analyzed; and if the data to be analyzed is video data or offline video data, re-acquiring the stream address and then re-issuing the task to be analyzed.
In some embodiments, the first cross-domain resource scheduling application further comprises: the identification of the algorithm required by the first task to be analyzed and the download address of the algorithm required by the first task to be analyzed; the method further comprises: the second data analysis device inquires whether the algorithm exists in the second data analysis device according to the identification of the algorithm required by the first task to be analyzed, and directly downloads the algorithm required by the first task to be analyzed from the first data analysis device according to the download address of the algorithm required by the first task to be analyzed under the condition that the algorithm does not exist in the second data analysis device.
For example, the second scheduling device sends an algorithm query application to the second data analysis device; the algorithm query application comprises an identification of an algorithm required by a first task to be analyzed; the algorithm query application is used for enabling the second data analysis device to query whether an algorithm required by the task to be analyzed exists; the second data analysis device queries the algorithm according to the algorithm query application, and sends the query result to the second scheduling device; the second scheduling device sends an algorithm downloading application to the second data analysis device when the query result is that the algorithm required by the task to be analyzed does not exist in the second data analysis device; the algorithm downloading application comprises the downloading address of the algorithm required by the task to be analyzed; and the second data analysis device directly downloads the algorithm required by the task to be analyzed from the first data analysis device according to the downloading address of the algorithm required by the task to be analyzed.
In some embodiments, the first scheduler stores an algorithm package of algorithms required for the task to be analyzed in a second subfolder of the shared folder of the boundary security server in the event that direct communication between the first local point and the second local point is not possible. Further, in the case where the algorithm does not exist in the second data analysis device, the second data analysis device may download an algorithm package of an algorithm required for the task to be analyzed in a second subfolder of the shared folder of the boundary security server.
In some embodiments, if the second local node does not have the usage rights of the algorithm required by the first task to be analyzed, the second local node cannot execute the first task to be analyzed. Therefore, in the case that the second local point does not have the usage right of the algorithm required by the first task to be analyzed, the second local point needs to acquire the usage right of the algorithm required by the first task to be analyzed.
For example, the second office point may apply for a short-term trial right, and import an authorization file to obtain the usage right of the algorithm required by the first task to be analyzed.
Thus, the step S104 may be specifically implemented as: and the second data analysis device analyzes the first task to be analyzed according to the data to be analyzed and the algorithm required by the first task to be analyzed to obtain an analysis result.
In some embodiments, the first cross-domain resource scheduling application further comprises: analyzing the storage address of the result; the method further includes, after the analysis result is obtained in the step S104,: the second data analysis means stores the analysis result in a storage space indicated by the storage address of the analysis result (when storing the analysis result, the second data analysis means directly stores the analysis result according to the result storage address without using the scheduling means).
In some embodiments, the second data analysis device stores the analysis result in a memory space indicated by a memory address of the analysis result through the Kafka system. Wherein Kafka is a high throughput distributed publish-subscribe messaging system that can process large amounts of data in real time to meet various demand scenarios. The Kafka system consists essentially of two modes of messaging: a point-to-point delivery mode and a publish-subscribe delivery mode. In the embodiment of the present application, the message transfer mode is not particularly limited.
In some embodiments, in the event that direct communication is not possible between the first local point and the second local point, the second data analysis device may store the analysis result in a storage space indicated by a storage address of the analysis result through a video image transmission server (e.g., a view library gateway).
For example, the second data analysis means may transmit the analysis result and the analysis result storage address to the video image transmission server, and the video image transmission server stores the analysis result in the storage space indicated by the analysis result storage address.
It can be understood that, in the cross-domain resource scheduling method provided by the embodiment of the present application, the second data analysis device may directly obtain the data to be analyzed (for example, video data or picture data) according to the obtaining address of the data to be analyzed; under the condition that the algorithm required by the data to be analyzed does not exist in the second data analysis device, directly acquiring the algorithm required by the data to be analyzed according to the download address of the algorithm required by the data to be analyzed; and storing the analysis result in the corresponding position according to the analysis result storage address. That is, the second data analysis device can directly acquire service data or directly store analysis result data, and the scheduling device is not required to forward. And the scheduling device only needs to be responsible for forwarding the task control data. Thus, the arrangement of the scheduling device is lighter, and the data forwarding efficiency can be improved.
In some embodiments, in performing the step S104, the method further includes: the second scheduling device receives the control instruction sent by the first scheduling device and forwards the control instruction to the second data analysis device, so that the second data analysis device controls the state of analyzing the first task to be analyzed according to the control instruction.
For example, the user at the first office point may send a control instruction through the first scheduling device, forward the control instruction through the second scheduling device, send the control instruction to the analysis task in the second data analysis device, and perform the following control operation on the processing state of the analysis task: start, modify, delete, stop, etc.
In another exemplary embodiment, in the process of executing the second task to be analyzed by the second data analysis device, if the first local point knows that the local resource allowance of the second local point is smaller than the size of the resource required by the first task to be analyzed, the first local point may also replace the local point, and send a local point replacement message to the second scheduling device through the first scheduling device.
In another example, if the first local point knows that the first task to be analyzed is in the waiting state or the abnormal state in the second local point for a long time, the first scheduling device may send an abnormality troubleshooting instruction to the second scheduling device, so that the second local point troubleshoots the cause of the abnormality; alternatively, the first scheduler may send a priority execution instruction to the second scheduler, so that the second local point stops the task with low priority and preferentially executes the first task to be analyzed.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: and the terminal equipment can interact with the user and receive the indication of the user. The terminal equipment comprises a display screen which can display the execution state of the first cross-domain resource scheduling application, so that a user can know the execution state of the first cross-domain resource scheduling application.
For example, as shown in fig. 15, a task execution status interface may be displayed on a display screen of the terminal device, where the task execution status interface may include: the name of each task to be analyzed, the task execution state, the resources required by the task and the like.
In some embodiments, in the case that the first local point and the second local point cannot directly communicate, the second scheduling device may transmit the state of the first task to be analyzed by the second data analyzing device to the first scheduling device through the boundary security server; the first dispatching device can transmit the control instruction to the second dispatching device through the boundary safety server, and the second dispatching device forwards the control instruction to the second data analysis device, so that the second data analysis device controls the state of the first task to be analyzed according to the control instruction.
The second scheduling device generates a JSON file by analyzing the identifier of the first task to be analyzed and the state of the first task to be analyzed, stores the JSON file in a second subfolder of the shared folder of the boundary security server, and names the JSON file as the name and the execution state of the first task to be analyzed.
Correspondingly, the first scheduling device can check the progress of the first task to be analyzed in the second subfolder of the shared folder of the boundary safety server.
In some embodiments, after the first scheduling device obtains the progress of the first task to be analyzed, the boundary security server may delete the task progress file in the second subfolder of the shared folder, thereby reducing space occupation and improving storage space utilization.
In another example, the first dispatcher generates a JSON file from the control instruction of the first task to be analyzed, stores the JSON file in a second subfolder of the shared folder of the boundary security server, and names the JSON folder as a name of the first task to be analyzed and a timestamp of the control instruction.
Correspondingly, the second scheduling device can regularly check the second subfolders of the shared folder of the boundary safety server, and when a new control instruction is detected, the control instruction is issued to the second data analysis device, so that the second data analysis device controls the state of analyzing the first task to be analyzed according to the control instruction.
In some embodiments, after the second scheduling device obtains the control instruction, the boundary security server may delete the control instruction file in the second subfolder of the shared folder, thereby reducing space occupation and improving storage space utilization.
It can be understood that the first scheduling device can acquire the processing state of the task to be analyzed by the second data analysis device in real time and control the state. In this way, when the upper layer application modifies the task to be analyzed, the first scheduling device can make corresponding modification operation in time.
It can be understood that, according to the cross-domain resource scheduling method provided by the embodiment of the application, based on the cross-domain analysis system, each local point can initiate a cross-domain resource scheduling application (including a task to be analyzed) to another local point through a scheduling device; and the corresponding other local point can receive the cross-domain resource scheduling application through the scheduling device, and analyze the task to be analyzed according to the cross-domain resource scheduling application to obtain an analysis result. Therefore, the analysis resource cross-domain (information interaction of cross-local points is called cross-domain here) scheduling can be realized by introducing the scheduling device on the basis of the original data analysis device without large-scale transformation of each local point, so that the integration of computing power resources is realized, and the utilization rate of the global computing power resources is improved.
In addition, by the cross-domain resource scheduling method, when a certain local point cannot bear a task to be analyzed, the task to be analyzed can be timely sent to other idle local points for analysis, so that the task to be analyzed can be timely processed under the condition that normal service of the local point is not interrupted. In this way, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle of different local points can be solved, and the utilization rate of the computing power resources in the whole domain is improved.
It can be seen that, in the cross-domain resource scheduling method provided in the embodiment of the present application, the task control data (for example, the cross-domain resource scheduling application, the control instruction, the algorithm change information or the computing power resource change information, etc.) and the service data (for example, the data to be analyzed, the algorithm required by the task to be analyzed, the analysis result, etc.) are separated, where the forwarding mode is a separate mode, where the forwarding of the task control data is performed by the scheduling device, and the service data is directly acquired or stored by the data processing module of the data analysis device. Thus, the performance requirement of the scheduling device is lower, so that the configuration of the scheduling device is lighter.
In addition, the embodiment of the application also provides a mode of integrating task control data and business data into a whole. For ease of understanding, two data forwarding modes will be described below with reference to fig. 16 and 17.
By way of example, fig. 16 provides a schematic diagram of a manner of unified forwarding of task control data and traffic data. As shown in fig. 16, after receiving a cross-domain resource scheduling application sent by a first data analysis device, a first scheduling device obtains data to be analyzed according to an address of data to be analyzed in the cross-domain resource scheduling application, and forwards the cross-domain resource scheduling application (task control data) and the data to be analyzed (service data) to a second scheduling device in a unified way; after the second scheduling device passes the auditing, forwarding the cross-domain resource scheduling application (task control data) and the data to be analyzed (business data) to a second data analysis device; and the second data analysis device is used for storing the analysis result into the storage system after the analysis result is obtained, and then the storage system is used for storing the analysis result into a result storage place.
It can be understood that the above-mentioned mode of forwarding task control data and service data in one has higher performance requirements for the scheduling device, and the scheduling device needs to have a larger storage space and more processing resources, and may affect other modules inside the local point due to a larger amount of forwarded data.
By way of example, fig. 17 provides a schematic diagram of a manner of separating forwarding from task control data and traffic data. As shown in fig. 17, after receiving the cross-domain resource scheduling application sent by the first data analysis device, the first scheduling device selects the second office as the applied party of the resource, and then sends the cross-domain resource scheduling application to the second scheduling device, and after the second scheduling device passes the audit, the second scheduling device sends the cross-domain resource scheduling application to the second data analysis device; after receiving the cross-domain resource scheduling application, the second data analysis device directly acquires data to be analyzed from the data storage place according to the data storage address to be analyzed, directly acquires an algorithm required by the task to be analyzed according to the address of the algorithm required by the task to be analyzed (under the condition that the algorithm required by the task to be analyzed is not available in the second data analysis device), further analyzes the target task according to the algorithm required by the task to be analyzed, obtains an analysis result, and directly stores the analysis result to the result storage place.
It can be understood that the above-mentioned mode of separating and forwarding task control data and service data has lower performance requirements for the scheduling device, so that the scheduling device is only responsible for the circulation of task control data, and is not responsible for the circulation of service data (in the field of image and video processing, the bytes of service data are larger). Thus, the arrangement of the dispatching device is lighter, and the burden of each office point is not increased. In some embodiments of the present application, a manner of forwarding task control data and service data separately is adopted.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a third local point, the third local point comprising: a third scheduling means and a third data analysis means. When the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, the method further includes steps S201 to S205 as shown in fig. 18.
S201, the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device.
The second cross-domain resource scheduling application comprises a second task to be analyzed. The second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point;
As a possible implementation manner, the second data analysis device generates a second cross-domain resource scheduling application when detecting that the size of the resource required by the second task to be analyzed is greater than or equal to the first threshold value, and sends the second cross-domain resource scheduling application to the third scheduling device.
As another possible implementation manner, the second data analysis device generates a second cross-domain resource scheduling application when detecting that the computing power resource in the second local site is in an offline state, and sends the second cross-domain resource scheduling application to the third scheduling device.
S202, the second scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application.
And the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a terminal device that can receive an indication of a user. Thus, the step S202 may be implemented as follows:
and g1, the second scheduling device sends a second cross-domain resource scheduling application to the terminal equipment.
And g2, the terminal equipment receives a second cross-domain resource scheduling application and receives a third local point indicated by the user for the second cross-domain resource scheduling application.
And g3, the terminal equipment sends the identification of the third local point to the second scheduling device.
S203, the second scheduling device forwards the second cross-domain resource scheduling application to the third scheduling device.
S204, the third scheduling device forwards the second cross-domain resource scheduling application to the third data analysis device.
S205, the third data analysis device analyzes the second task to be analyzed according to the second cross-domain resource scheduling application to obtain an analysis result.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a third local point, the third local point comprising: a third scheduling means and a third data analysis means. When the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the above method further includes steps S301-S306 as shown in fig. 19.
S301, the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device.
The second cross-domain resource scheduling application comprises a second task to be analyzed. The second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point;
as a possible implementation manner, the second data analysis device generates a second cross-domain resource scheduling application when detecting that the size of the resource required by the second task to be analyzed is greater than or equal to the first threshold value, and sends the second cross-domain resource scheduling application to the third scheduling device.
As another possible implementation manner, the second data analysis device generates a second cross-domain resource scheduling application when detecting that the computing power resource in the second local site is in an offline state, and sends the second cross-domain resource scheduling application to the third scheduling device.
S302, the second scheduling device sends a second cross-domain resource scheduling application to the central scheduling device.
S303, the central scheduling device determines a third local point from other local points according to the second cross-domain resource scheduling application.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a terminal device that can receive an indication of a user. Thus, the step S303 may be further implemented as:
and step h1, the central scheduling device sends a second cross-domain resource scheduling application to the terminal equipment.
And step h2, the terminal equipment receives a second cross-domain resource scheduling application and receives a third local point indicated by the user for the second cross-domain resource scheduling application.
And step h3, the terminal equipment sends the identification of the third local point to the central dispatching device.
And the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
S304, the central scheduling device sends a second cross-domain resource scheduling application to the third scheduling device.
And S305, the third scheduling device forwards the second cross-domain resource scheduling application to the third data analysis device.
S306, the third data analysis device analyzes the second task to be analyzed according to the second cross-domain resource scheduling application to obtain an analysis result.
It can be understood that the second local point may also be used as an applicant party of a cross-domain resource scheduling application, and the second scheduling device or the central scheduling device determines a third local point according to the second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third local point, and the third local point executes the second task to be analyzed. That is, each local point provided by the embodiment of the application can be used as an applicant or an applied party of cross-domain resource scheduling, so that the cross-domain scheduling of resources is truly realized, and the utilization rate of the resources in the whole domain is effectively improved.
In some embodiments, in the process of the third data analysis device analyzing the second task to be analyzed, if the second data analysis device receives an indication that the amount of resources required by the second task to be analyzed increases, the second data analysis device may generate the second expansion application according to the increased amount of resources of the second task to be analyzed. Specifically, as shown in fig. 20, the method may be implemented as the following steps:
S401, the second data analysis device sends a second capacity expansion application to the second scheduling device.
The second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
Optionally, the second capacity expansion application includes an increased amount of resources for the second task to be analyzed.
S402, the second scheduling device sends a second capacity expansion application to the third scheduling device.
As a possible implementation manner, when the cross-domain analysis system is in the fully symmetric deployment mode as shown in fig. 2, the second scheduling device directly sends the second capacity expansion application to the third scheduling device.
As another possible implementation manner, when the cross-domain analysis system is in the central deployment mode as shown in fig. 4, the second scheduling device sends a second capacity expansion application to the central scheduling device, and the central scheduling device forwards the second capacity expansion application to the third scheduling device.
S403, the third scheduling device examines the second capacity expansion application.
In some embodiments, the third scheduling means auditing the second capacity expansion application may include: the third scheduling device judges whether the intra-office resource allowance of the third office point meets the resource amount applied by the second capacity expansion application (namely, the resource amount increased by the second task to be analyzed); if yes, the third scheduling device determines that the second capacity expansion application passes the auditing; if the second capacity expansion request does not meet the first capacity expansion request, the third scheduling device determines that the second capacity expansion request does not pass the verification, and rejects the second capacity expansion request.
And S404, under the condition that the second capacity expansion application passes the auditing, the third scheduling device sends the second capacity expansion application to the third data analysis device.
Correspondingly, after receiving the second capacity expansion application, the third data analysis device allocates the resource amount applied by the second capacity expansion application (namely, the resource amount increased by the second capacity expansion application) to the second task to be analyzed.
It can be understood that in the cross-domain resource scheduling method provided by the application, under the condition that the resource amount required by the second task to be analyzed is increased, the increased resource amount of the second task to be analyzed can be applied to the local point for executing the second task to be analyzed again, so that the processing process of the second task to be analyzed is not interrupted, and the task analysis efficiency is improved.
In some embodiments, the cross-domain analysis system provided in the embodiments of the present application further includes: a fourth local point, the fourth local point comprising: fourth scheduling means. As shown in fig. 21, the above method further includes steps S501 to S503.
S501, the fourth scheduling device sends a task analysis request to the second scheduling device.
The task analysis request is used for applying to the second scheduling device to execute the task to be analyzed in the second local point in the fourth local point.
Optionally, the task analysis request includes: address information of the fourth local point, local resource margin of the fourth local point, and authentication information.
S502, the second scheduling device examines the task analysis request.
In some embodiments, the second scheduler auditing the task analysis request includes: the second scheduling device verifies the legitimacy of the fourth local point (prevents malicious network attack) according to the verification information, and determines that the task analysis request passes the audit under the condition that the fourth local point is legal; and/or the second scheduling device judges whether the local resource allowance of the fourth local point meets the size of the resources required by the task to be analyzed in the second local point, and determines that the task analysis request passes the audit under the condition of meeting the size.
In some embodiments, the second scheduler rejects the task analysis request if the task analysis request fails the audit.
S503, under the condition that the task analysis request passes the audit, the second scheduling device sends a third cross-domain resource scheduling application to the fourth scheduling device.
The third cross-domain resource scheduling application comprises a third task to be analyzed; the third cross-domain resource scheduling application is used for applying to execute the third task to be analyzed in the fourth local point.
It can be understood that in the cross-domain resource scheduling method provided by the embodiment of the present application, a local node may initiate a task analysis request to a scheduling device of another local node through a scheduling device of the local node, so as to request to execute tasks to be analyzed of the other local node. Therefore, by utilizing the method provided by the application, on one hand, when the idle resources of one local point are more, the task to be analyzed can be actively requested to other local points, so that the task to be analyzed can be timely processed. In this way, on one hand, the task analysis efficiency can be improved; on the other hand, the problem of uneven busy and idle of different local points can be solved, and the utilization rate of the computing power resources in the whole domain is improved.
The embodiment of the application provides a scheduling device, which is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: and a second data analysis device and the scheduling device.
As shown in fig. 22, the scheduling apparatus 600 includes: a receiving unit 601 and a transmitting unit 602. In some embodiments, the scheduling apparatus 600 further comprises: a determination unit 603 and an information change unit 604.
A receiving unit 601, configured to receive a first cross-domain resource scheduling application sent by a first scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device.
And the sending unit 602 is configured to send the first cross-domain resource scheduling application to the second data analysis device, so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
In one possible implementation manner, the cross-domain analysis system further includes: a central dispatching device which is respectively connected with the first dispatching device and the dispatching device 600; the receiving unit 601 is specifically configured to receive, by using a central scheduling device, a first cross-domain resource scheduling application, where the first cross-domain resource scheduling application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device determines, according to the first cross-domain resource scheduling application, a second local point from other local points, and sends the first cross-domain resource scheduling application to the scheduling device; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed.
In another possible implementation manner, the receiving unit 601 is further configured to receive, by using the central scheduling device, a first expansion application when the amount of resources required by the first task to be analyzed increases, where the first expansion application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device sends the first expansion application to the scheduling device 600; the first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point.
In another possible implementation manner, the cross-domain analysis system further includes: a third local point, the third local point comprising: the third scheduling device and the third data analysis device, the receiving unit 601 is further configured to receive a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute a second task to be analyzed at other local points except the second local point; a determining unit 603, configured to determine a third local point from the other local points according to the second cross-domain resource scheduling application; the sending unit 602 is further configured to send a second cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes a second task to be analyzed; and the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
In another possible implementation manner, the sending unit 602 is further configured to send a second expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed increases; the second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point.
In another possible implementation manner, the cross-domain analysis system further includes: a fourth local point, the fourth local point comprising: a fourth scheduling means; a receiving unit 601; the task analysis request is used for applying to the scheduling device to execute the task to be analyzed in the second local point in the fourth local point, and the task analysis request comprises: address information of the fourth local point and a local resource margin of the fourth local point.
In another possible implementation manner, the sending unit 602 is further configured to send an algorithm query application to the second data analysis device, where the algorithm query application includes: the identification of the algorithm required by the first task to be analyzed is carried out, so that the second data analysis device inquires whether the second data analysis device comprises the algorithm required by the first task to be analyzed according to the identification of the algorithm required by the first task to be analyzed; the sending unit 602 is further configured to send an algorithm download application to the second data analysis device, where the second data analysis device does not include an algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: and the downloading address of the algorithm required by the first task to be analyzed is used for enabling the second data analysis device to download the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
In another possible implementation manner, the receiving unit 601 is further configured to receive a control instruction sent by the first scheduling device, and control the state of the first task to be analyzed according to the control instruction by the second data analysis device.
In another possible implementation manner, the receiving unit 601 is further configured to receive algorithm change information sent by the first scheduling device; the algorithm change information comprises the identification of the changed target algorithm in the first data analysis device and the downloading address of the target algorithm; the sending unit 602 is further configured to send an identifier of the target algorithm to the second data analysis device, so that the second data analysis device queries whether the target algorithm exists in the second data analysis device according to the identifier of the target algorithm; the sending unit 602 is further configured to send, in a case where the target algorithm does not exist in the second data analysis device, a download address of the target algorithm to the second data analysis device, so that the second data analysis device downloads the target algorithm according to the download address of the target algorithm.
In another possible implementation manner, the receiving unit 601 is further configured to receive computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining power information of the first office point; an information changing unit 604 for changing the remaining power information of the first local point recorded in the scheduling device to the current remaining power information of the first local point according to the power resource change information.
In another possible implementation manner, the cross-domain analysis system further includes: a terminal device; the sending unit 601 is further configured to send a second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third local point indicated by the user for the second cross-domain resource scheduling application.
In another possible implementation manner, the receiving unit 601 is further configured to receive, through the boundary security server, a first cross-domain resource scheduling application sent by the first scheduling device, where direct communication between the first local station and the second local station is not possible.
Of course, the scheduling apparatus 600 provided in the embodiment of the present application includes, but is not limited to, the above modules.
The embodiment of the application provides a data analysis device, which is applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: the data analysis device and the second scheduling device.
As shown in fig. 23, the data analysis device 700 includes: a receiving unit 701 and a processing unit 702. In some embodiments, the data analysis device 700 further comprises: an acquisition unit 703 and a query unit 704.
A receiving unit 701, configured to receive a first cross-domain resource scheduling application sent by a second scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device.
The processing unit 702 is configured to analyze a first task to be analyzed according to the first cross-domain resource scheduling application, and obtain an analysis result.
In one possible implementation manner, the first task to be analyzed is a task of analyzing data to be analyzed; the first cross-domain resource scheduling application further includes: an acquisition address of data to be analyzed; an acquiring unit 703, configured to acquire data to be analyzed according to an acquisition address of the data to be analyzed; the first cross-domain resource scheduling application further includes: the download address of the algorithm required by the first task to be analyzed; the obtaining unit 703 is further configured to directly download the algorithm required by the first task to be analyzed from the first data analysis device according to the download address of the algorithm required by the first task to be analyzed; the processing unit 702 is specifically configured to analyze the first task to be analyzed according to the data to be analyzed and an algorithm required by the first task to be analyzed contained in the algorithm download application, and obtain an analysis result.
In another possible implementation manner, the receiving unit 701 is further configured to receive an algorithm query application sent by the second scheduling device; the algorithm query application comprises the following steps: identification of an algorithm required by a first task to be analyzed; a query unit 704, configured to query whether the data analysis device includes an algorithm required by the first task to be analyzed according to the identifier of the algorithm required by the first task to be analyzed; the receiving unit 701 is further configured to receive an algorithm download application sent by the second scheduling device when the data analysis device does not include an algorithm required by the first task to be analyzed; the algorithm downloading application comprises the following steps: the download address of the algorithm required by the first task to be analyzed; the acquisition unit is further used for downloading the algorithm required by the first task to be analyzed according to the downloading address of the algorithm required by the first task to be analyzed.
Of course, the data analysis device 700 provided in the embodiment of the present application includes, but is not limited to, the above-described modules.
In the case of implementing the functions of the integrated modules in the form of hardware, another embodiment of the present application provides a schematic structural diagram of an electronic device. Alternatively, the electronic device may be a scheduling device; alternatively, the electronic device may be a data analysis apparatus.
As shown in fig. 24, the electronic device 800 includes: a processor 802, a communication interface 803, and a bus 804. Optionally, the electronic device 800 may also include a memory 801.
The processor 802 may be any processor that implements or performs the various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein. The processor 802 may be a central processor, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 802 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of DSP and microprocessor, etc.
A communication interface 803 for connecting with other devices through a communication network. The communication network may be an ethernet, a radio access network, a wireless local area network (wireless local area networks, WLAN), etc.
The memory 801 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
As a possible implementation, the memory 801 may exist separately from the processor 802, and the memory 801 may be connected to the processor 802 through the bus 804 for storing instructions or program code. The processor 802, when calling and executing instructions or program code stored in the memory 801, is capable of implementing the cross-domain resource scheduling method provided in the embodiments of the present application.
In another possible implementation, the memory 801 may also be integrated with the processor 802.
Bus 804 may be an extended industry standard architecture (extended industry standard architecture, EISA) bus or the like. The bus 804 may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 24, but not only one bus or one type of bus.
From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the electronic device is divided into different functional modules to perform all or part of the functions described above.
Another embodiment of the present application also provides a computer-readable storage medium having computer instructions stored therein. When the computer instructions are executed on the computer, the computer is caused to perform the steps in the cross-domain resource scheduling method flow shown in the method embodiment.
Another embodiment of the present application also provides a computer program product comprising computer instructions. When the computer instructions are executed on the computer, the computer is caused to perform the steps in the cross-domain resource scheduling method flow shown in the method embodiment.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer-executable instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, a website, computer, server, or data center via a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices including one or more servers, data centers, etc. that can be integrated with the media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
The foregoing is merely a specific embodiment of the present application. Variations and alternatives will occur to those skilled in the art from the detailed description provided herein and are intended to be included within the scope of the present application.

Claims (21)

1. The cross-domain resource scheduling method is characterized by being applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first data analysis device is registered to the first scheduling device, the second data analysis device is registered to the second scheduling device, and the first scheduling device and the second scheduling device are mutually registered; the method comprises the following steps:
the second scheduling device receives a first cross-domain resource scheduling application sent by the first scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; the first scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
And the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
2. The method of claim 1, wherein the cross-domain analysis system further comprises: a third local point, the third local point comprising: a third scheduling device and a third data analysis device, the method further comprising:
the second scheduling device receives a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute the second task to be analyzed at other local points except the second local point;
the second scheduling device determines the third local point from the other local points according to the second cross-domain resource scheduling application, and sends the second cross-domain resource scheduling application to the third scheduling device so that the third data analysis device executes the second task to be analyzed; and the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed.
3. The method according to claim 2, wherein the method further comprises:
under the condition that the amount of resources required by the second task to be analyzed is increased, the second scheduling device sends a second capacity expansion application to the third scheduling device; and the second capacity expansion application is used for applying the increased resource quantity of the second task to be analyzed to the third local point.
4. The method of claim 1, wherein the cross-domain analysis system further comprises: a fourth local point, the fourth local point comprising: a fourth scheduling apparatus, the method further comprising:
the second scheduling device receives a task analysis request sent by the fourth scheduling device, where the task analysis request is used to apply to the second scheduling device to execute a task to be analyzed in the second local point in the fourth local point, and the task analysis request includes: address information of the fourth local point and local resource allowance of the fourth local point.
5. The method according to any one of claims 1 to 4, further comprising:
the second scheduling device receives the computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining computing power information of the first office point;
The second scheduling device changes the remaining calculation power information of the first local point recorded in the second scheduling device into the current remaining calculation power information of the first local point according to the calculation power resource change information.
6. A method according to claim 2 or 3, wherein the cross-domain analysis system further comprises: a terminal device;
the determining, according to the second cross-domain resource scheduling application, the third local point from the other local points includes:
the second scheduling device sends the second cross-domain resource scheduling application to the terminal equipment, so that the terminal equipment receives the second cross-domain resource scheduling application and receives a third local point indicated by a user for the second cross-domain resource scheduling application.
7. The cross-domain resource scheduling method is characterized by being applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: the system comprises a first local point, a second local point and a central dispatching device; the first local point includes: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device; the first data analysis device is registered with the first scheduling device, the second data analysis device is registered with the second scheduling device, and both the first scheduling device and the second scheduling device are registered with the central scheduling device; the method comprises the following steps:
The second dispatching device receives a first cross-domain resource dispatching application through the central dispatching device, wherein the first cross-domain resource dispatching application is sent to the central dispatching device by the first dispatching device, so that the central dispatching device determines the second local point from other local points according to the first cross-domain resource dispatching application and sends the first cross-domain resource dispatching application to the second dispatching device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; the central scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
and the second scheduling device sends the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
8. The method of claim 7, wherein the method further comprises:
under the condition that the amount of resources required by the first task to be analyzed is increased, the second scheduling device receives a first capacity expansion application through the central scheduling device, wherein the first capacity expansion application is sent to the central scheduling device by the first scheduling device, so that the first capacity expansion application is sent to the second scheduling device by the central scheduling device; the first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point.
9. The method of claim 7, wherein the cross-domain analysis system further comprises: a fourth local point, the fourth local point comprising: a fourth scheduling apparatus, the method further comprising:
the second scheduling device receives a task analysis request sent by the fourth scheduling device, where the task analysis request is used to apply to the second scheduling device to execute a task to be analyzed in the second local point in the fourth local point, and the task analysis request includes: address information of the fourth local point and local resource allowance of the fourth local point.
10. The method according to any one of claims 7 to 9, further comprising:
the second scheduling device receives the computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining computing power information of the first office point;
the second scheduling device changes the remaining calculation power information of the first local point recorded in the second scheduling device into the current remaining calculation power information of the first local point according to the calculation power resource change information.
11. The cross-domain resource scheduling method is characterized by being applied to a cross-domain analysis system, wherein the cross-domain analysis system comprises: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first data analysis device is registered to the first scheduling device, the second data analysis device is registered to the second scheduling device, and the first scheduling device and the second scheduling device are mutually registered; the method comprises the following steps:
the second data analysis device receives a first cross-domain resource scheduling application sent by the second scheduling device; the first cross-domain resource scheduling application includes: a first task to be analyzed; the first cross-domain resource scheduling application is used for applying to execute the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device; the first scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
And the second data analysis device analyzes the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
12. A scheduling apparatus, for use in a cross-domain analysis system, the cross-domain analysis system comprising: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first data analysis device is registered to the first scheduling device, the second data analysis device is registered to the second scheduling device, and the first scheduling device and the second scheduling device are mutually registered; the scheduling device comprises:
a receiving unit, configured to receive a first cross-domain resource scheduling application sent by the first scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; the first scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
And the sending unit is used for sending the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
13. The scheduling apparatus of claim 12, wherein the scheduling apparatus,
the cross-domain analysis system further comprises: a third local point, the third local point comprising: the receiving unit is further used for receiving a second cross-domain resource scheduling application sent by the second data analysis device; the second cross-domain resource scheduling application comprises a second task to be analyzed; the second cross-domain resource scheduling application is used for applying to execute the second task to be analyzed at other local points except the second local point; the scheduling apparatus further includes: the determining unit is used for determining the third local point from the other local points according to the second cross-domain resource scheduling application; the sending unit is further configured to send the second cross-domain resource scheduling application to the third scheduling device, so that the third data analysis device executes the second task to be analyzed; wherein, the local resource allowance of the third local point is larger than or equal to the size of the resource required by the second task to be analyzed;
The sending unit is further configured to send a second capacity expansion application to the third scheduling device when the amount of resources required by the second task to be analyzed increases; the second capacity expansion application is used for applying the increased resource amount of the second task to be analyzed to the third local point;
the cross-domain analysis system further comprises: a fourth local point, the fourth local point comprising: a fourth scheduling means; the receiving unit; the task analysis request is further used for receiving a task analysis request sent by the fourth scheduling device, the task analysis request is used for applying to the scheduling device to execute a task to be analyzed in the second local point in the fourth local point, and the task analysis request comprises: address information of the fourth local point and local resource allowance of the fourth local point;
the receiving unit is further configured to receive computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining computing power information of the first office point; the scheduling apparatus further includes: an information changing unit configured to change remaining power information of the first local point recorded in the scheduling device to current remaining power information of the first local point according to the power resource change information;
The cross-domain analysis system further comprises: a terminal device; the sending unit is further configured to send the second cross-domain resource scheduling application to the terminal device, so that the terminal device receives the second cross-domain resource scheduling application, and receives a third local point indicated by a user for the second cross-domain resource scheduling application.
14. A scheduling apparatus, for use in a cross-domain analysis system, the cross-domain analysis system comprising: the system comprises a first local point, a second local point and a central dispatching device; the first local point includes: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device; the first data analysis device is registered with the first scheduling device, the second data analysis device is registered with the second scheduling device, and both the first scheduling device and the second scheduling device are registered with the central scheduling device; the scheduling device comprises:
the receiving unit is configured to receive, by using the central scheduling device, a first cross-domain resource scheduling application, where the first cross-domain resource scheduling application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device determines, according to the first cross-domain resource scheduling application, the second local point from other local points, and sends, to the second scheduling device, the first cross-domain resource scheduling application; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is an application initiated by the first data analysis device; the central scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
And the sending unit is used for sending the first cross-domain resource scheduling application to the second data analysis device so that the second data analysis device analyzes the first task to be analyzed to obtain an analysis result.
15. The apparatus of claim 14, wherein the device comprises a plurality of sensors,
the receiving unit is specifically configured to receive, by using the central scheduling device, a first capacity expansion application when the amount of resources required by the first task to be analyzed increases, where the first capacity expansion application is sent by the first scheduling device to the central scheduling device, so that the central scheduling device sends the first capacity expansion application to the second scheduling device; the first capacity expansion application is used for applying the increased resource quantity of the first task to be analyzed to the second local point;
the cross-domain analysis system further comprises: a fourth local point, the fourth local point comprising: the receiving unit is further configured to receive a task analysis request sent by the fourth scheduling device, where the task analysis request is used to apply to the second scheduling device for executing a task to be analyzed in the second local point in the fourth local point, and the task analysis request includes: address information of the fourth local point and local resource allowance of the fourth local point;
The receiving unit is further configured to receive computing power resource change information sent by the first scheduling device; the computing power resource variation information includes: current remaining computing power information of the first office point; the scheduling apparatus further includes: and an information changing unit configured to change remaining power information of the first local point recorded in the scheduling device to current remaining power information of the first local point according to the power resource change information.
16. A data analysis device, characterized by being applied to a cross-domain analysis system, the cross-domain analysis system comprising: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first data analysis device is registered to the first scheduling device, the second data analysis device is registered to the second scheduling device, and the first scheduling device and the second scheduling device are mutually registered; the data analysis device includes:
a receiving unit, configured to receive a first cross-domain resource scheduling application sent by the second scheduling device; the first cross-domain resource scheduling application includes: the first task to be analyzed is used for applying for executing the first task to be analyzed at other local points except the first local point; the first cross-domain resource scheduling application is initiated by the first data analysis device and forwarded to the second scheduling device through the first scheduling device; the first scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the first task to be analyzed;
And the processing unit is used for analyzing the first task to be analyzed according to the first cross-domain resource scheduling application to obtain an analysis result.
17. A local point, comprising: a scheduling device and a data analysis device;
the scheduling device comprises a first memory and a first processor; the first memory is coupled to the first processor; the first memory is for storing computer program code, the computer program code comprising computer instructions;
wherein the first processor, when executing the computer instructions, causes the scheduling apparatus to perform the cross-domain resource scheduling method of any one of claims 1 to 6, or any one of claims 7 to 10;
the data analysis device comprises a second memory and a second processor; the second memory is coupled to the second processor; the second memory is for storing computer program code, the computer program code comprising computer instructions;
wherein the second processor, when executing the computer instructions, causes the data analysis apparatus to perform the cross-domain resource scheduling method of claim 11.
18. A cross-domain analysis system, comprising: a first office point and a second office point; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the first data analysis device is registered to the first scheduling device, the second data analysis device is registered to the second scheduling device, and the first scheduling device and the second scheduling device are mutually registered;
The first scheduling device is used for receiving a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises the following steps: tasks to be analyzed; the cross-domain resource scheduling application is used for applying to execute the task to be analyzed at other local points except the first local point; the first scheduling device records the resource allowance conditions of all local points in the cross-domain analysis system; the local resource allowance of the second local point is larger than or equal to the size of the resource required by the task to be analyzed;
the first scheduling device is further configured to forward the cross-domain resource scheduling application to the second scheduling device.
19. The system of claim 18, wherein the cross-domain resource scheduling application further comprises: the size of the resources required by the task to be analyzed;
the first scheduling device is further configured to determine, according to the cross-domain resource scheduling application, the second local point from the other local points.
20. A cross-domain analysis system, comprising: the system comprises a first local point, a second local point and a central dispatching device; wherein the first local point comprises: the first data analysis device and the first scheduling device, the second local point includes: a second data analysis means and a second scheduling means; the central dispatching device is respectively connected with the first dispatching device and the second dispatching device; the first data analysis device is registered with the first scheduling device, the second data analysis device is registered with the second scheduling device, and both the first scheduling device and the second scheduling device are registered with the central scheduling device;
The first scheduling device is used for receiving a cross-domain resource scheduling application of the first data analysis device; the cross-domain resource scheduling application comprises the following steps: the task to be analyzed and the size of the resources required by the task to be analyzed; the cross-domain resource scheduling application is used for applying to execute the task to be analyzed at other local points except the first local point;
the first scheduling device is further configured to forward the cross-domain resource scheduling application to the central scheduling device;
the central scheduling device is used for determining the second local point from the other local points according to the cross-domain resource scheduling application, and the local resource allowance of the second local point is larger than or equal to the size of the resources required by the task to be analyzed;
the central scheduling device is further configured to send the cross-domain resource scheduling application to the second scheduling device.
21. A computer-readable storage medium, the computer-readable storage medium comprising computer instructions;
wherein the computer instructions, when run on a computer, cause the computer to perform the method of any one of claims 1 to 11.
CN202210476425.4A 2022-04-29 2022-04-29 Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium Active CN114785790B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210476425.4A CN114785790B (en) 2022-04-29 2022-04-29 Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210476425.4A CN114785790B (en) 2022-04-29 2022-04-29 Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium

Publications (2)

Publication Number Publication Date
CN114785790A CN114785790A (en) 2022-07-22
CN114785790B true CN114785790B (en) 2024-03-08

Family

ID=82436006

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210476425.4A Active CN114785790B (en) 2022-04-29 2022-04-29 Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium

Country Status (1)

Country Link
CN (1) CN114785790B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207814A (en) * 2012-12-27 2013-07-17 北京仿真中心 Decentralized cross cluster resource management and task scheduling system and scheduling method
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN106293950A (en) * 2016-08-23 2017-01-04 成都卡莱博尔信息技术股份有限公司 A kind of resource optimization management method towards group system
CN107066332A (en) * 2017-01-25 2017-08-18 广东神马搜索科技有限公司 Distributed system and its dispatching method and dispatching device
CN112134802A (en) * 2020-09-23 2020-12-25 杭州雾联科技有限公司 Edge computing power resource scheduling method and system based on terminal triggering

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11669367B2 (en) * 2020-04-01 2023-06-06 Bank Of America Corporation System and methods for generation and analysis of real-time resource requests

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207814A (en) * 2012-12-27 2013-07-17 北京仿真中心 Decentralized cross cluster resource management and task scheduling system and scheduling method
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN106293950A (en) * 2016-08-23 2017-01-04 成都卡莱博尔信息技术股份有限公司 A kind of resource optimization management method towards group system
CN107066332A (en) * 2017-01-25 2017-08-18 广东神马搜索科技有限公司 Distributed system and its dispatching method and dispatching device
CN112134802A (en) * 2020-09-23 2020-12-25 杭州雾联科技有限公司 Edge computing power resource scheduling method and system based on terminal triggering

Also Published As

Publication number Publication date
CN114785790A (en) 2022-07-22

Similar Documents

Publication Publication Date Title
CN110647391B (en) Edge computing method and system for satellite-ground cooperative network
EP3461087B1 (en) Network-slice resource management method and apparatus
US11706088B2 (en) Analyzing and configuring workload distribution in slice-based networks to optimize network performance
JP2022532007A (en) How to select mobile edge computing nodes, devices and systems, and computer programs
US20150317583A1 (en) Business Scheduling Method and Apparatus and Convergence Device
CN108901046A (en) Cotasking unloading algorithm and system design scheme towards mobile edge calculations
KR101893963B1 (en) System and method for a software defined protocol network node
CN107306232B (en) Network device, controller, queue management method and flow management chip
US20020118700A1 (en) Flow allocation in a ring topology
JP2017531958A (en) Method and system for implementing a content distribution network based on a software defined network
US7369505B2 (en) Method and system to efficiently manage network connection to connect a client and a resource
WO2016107598A1 (en) Service acceleration method and apparatus
US9043468B2 (en) Method and arrangement for network resource management
EP1699417B1 (en) Interprocessor communication network providing dynamic dedication of ports
CN1643946B (en) Controlling service stream
Masoumi et al. Dynamic online VNF placement with different protection schemes in a MEC environment
CN111356182A (en) Resource scheduling and processing method and device
CN114785790B (en) Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium
CN114827286B (en) Cross-domain analysis system, cross-domain resource scheduling method, device and storage medium
CN107113186A (en) The method and public service entity of data transfer in unified machine to machine system
CN111770179B (en) High-performance high-availability cloud networking gateway implementation method, medium and terminal
CN114221948A (en) Cloud network system and task processing method
Fichera et al. Experimental evaluation of dynamic resource orchestration in multi-layer (packet over flexi-grid optical) networks
CN112732409B (en) Method and device for enabling zero-time-consumption network flow load balancing under VNF architecture
CN114641014A (en) User plane entity, configuration method, system and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant