CN116827942B - Balancing method based on distributed AI video analysis task - Google Patents
Balancing method based on distributed AI video analysis task Download PDFInfo
- Publication number
- CN116827942B CN116827942B CN202211497141.XA CN202211497141A CN116827942B CN 116827942 B CN116827942 B CN 116827942B CN 202211497141 A CN202211497141 A CN 202211497141A CN 116827942 B CN116827942 B CN 116827942B
- Authority
- CN
- China
- Prior art keywords
- video
- video analysis
- analysis application
- tasks
- leader
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 301
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000003491 array Methods 0.000 claims description 3
- 230000010076 replication Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1029—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers using data related to the state of servers by a load balancer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1031—Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1034—Reaction to server failures by a load balancer
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a balancing method based on a distributed AI video analysis task, which relates to the technical field of AI video analysis methods, and comprises the following steps of S1, establishing a Raft system; s2, the AI video analysis application receives video tasks and numbers the video tasks in sequence; s3, detecting AI video analysis application and a server by using a log structure; s4, a cluster deploys a video task and solves the problem that an AI video analysis application is removed or down; and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems. The invention relates to a distributed AI video analysis task-based balancing method, which is characterized in that the number of video tasks distributed by each AI video analysis application is the same, the distribution is balanced, the analysis completion time of each AI video analysis application on the video tasks is the same, thus the condition of missing report of analysis results is not easy to occur, and the working efficiency is improved.
Description
Technical Field
The invention relates to the technical field of AI video analysis methods, in particular to a distributed AI video analysis task-based balancing method.
Background
The AI video analysis processing technology is to find the moving object in the image, track and analyze the moving object, find abnormal behavior in time, and trigger an alarm system or perform intervention. AI video analysis processing technology is currently the most interesting image processing technology in the security industry. In general, the basic process of intelligent analysis of video is to read each frame of image from a given video, pre-process the input image, such as filtering, gray level conversion, etc., then determine whether there is a moving object in the input image, then determine whether the moving object is a monitoring object, and finally perform analysis such as monitoring, tracking or behavior understanding on the object according to the requirement.
The current AI video analysis system in the market basically adopts a mode that video streams are fixedly distributed to AI video analysis applications, and the AI video analysis applications perform frame extraction analysis on the video streams; with the increase of the number of video stream access paths, the AI video analysis applications can correspondingly perform multi-point deployment, and a plurality of AI video analysis applications analyze a plurality of tasks simultaneously so as to complete the task amount within a specified time.
However, the existing video stream fixed allocation task mode cannot flexibly allocate the computing power of a server where the AI video analysis application is deployed; the uneven distribution can cause excessive video flow distributed by one or more AI video analysis applications, other AI video analysis applications can be idle after the distributed tasks are processed, and the distributed more AI video analysis applications are not timely processed for analysis tasks, so that the condition of missing report of analysis results occurs, and the working efficiency of the analysis tasks is affected.
Disclosure of Invention
The invention mainly aims to provide a distributed AI video analysis task-based balancing method, which can effectively solve the problem that the conventional video stream fixed allocation task mode in the background art cannot flexibly allocate the computing power of a server where an AI video analysis application is deployed; the uneven distribution can cause excessive video flow distributed by one or more AI video analysis applications, other AI video analysis applications can be idle after the distributed tasks are processed, and the distributed more AI video analysis applications are not timely processed for analysis tasks, so that the condition of missing report of analysis results occurs, and the working efficiency of the analysis tasks is affected.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the invention discloses a distributed AI video analysis task-based balancing method, which comprises the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server;
s2, a task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in a Raft, each AI video analysis application stores a log array, each log array stores a log command, an arbitrary number, a log index, a value of a next index and an additional log remote procedure call, and the AI video analysis applications sequentially number the received video tasks and elect a leader in the AI video analysis application;
s3, sequencing the leader in the AI video analysis application according to the number of video tasks of the task collection application, numbering newly-added video tasks received by the task collection application according to the number of the AI video analysis applications, sequentially issuing video tasks named as the same number to other different AI video analysis applications, recording process data of the issued video tasks into a log structure by the AI video analysis application, detecting the number of video tasks of each AI video analysis application and the state of a server by the leader in the AI video analysis application, and then performing cluster deployment by the leader in the AI video analysis application according to the actual number of video tasks of the AI video analysis application;
s4, when the clusters are deployed, a plurality of AI video analysis application clusters analyze video tasks with the same number, when one AI video analysis application in the clusters is removed or down, a leader in the AI video analysis application can combine a server state to perform balanced distribution of video tasks of the removed or down AI video analysis application again, the value of a next index in a log structure is set to be the next position of the latest position of a log array, when the remote procedure call of an additional log in the log structure fails, the number of the video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the number of the next index is reduced, the video tasks are retried until the position consistent with that of the leader in the AI video analysis application is found, and after the latest position of the next index is found, the subsequent video tasks are directly copied to the failed AI video analysis application;
and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems.
Preferably, in the steps S1 and S2, the Raft system selects a leader in the AI video analysis application through a heartbeat mechanism and a timeout mechanism, and manages the replication of the log structure through the leader in the AI video analysis application, and when the leader in the AI video analysis application exists, the leader in the AI video analysis application synchronizes the log structure to other AI video analysis applications.
Preferably, the leader in the AI video analysis application is the leader, has the largest task number, and stores the value of nextsdex, and stores the video index value to be sent to other AI video analysis applications.
Preferably, the value of nextIndex in the leader in the AI video analysis application stores the video index value of the next log entry to be sent to each AI video analysis application, and the leader in the AI video analysis application directly sends the task to the more idle AI video analysis application.
Preferably, the number of video tasks in the plurality of AI video analysis applications is not necessarily the same, but is not more than the number of video tasks in the leader in the AI video analysis application.
Preferably, in the case that the leader in the AI video analysis application is suspended by mistake, the election of the leader in the new AI video analysis application needs to be reelected through a heartbeat mechanism and a timeout mechanism.
Preferably, in the Raft system, a comparison check is made during election to ensure that the leader in the new AI video analysis application already has all log arrays submitted, avoiding additional log array transmissions.
Preferably, when one or more AI video analysis applications in step S4 are removed, the leader in the AI video analysis application assigns video tasks to other intact AI video analysis applications according to the index value, and the other intact AI video analysis applications renumber and analyze the video tasks.
Preferably, the video tasks that have failed to be processed in step S5 are reported to other systems, provided that the leader in the AI video analysis application and all AI video analysis applications cannot continue to process the increased number of video tasks.
Compared with the prior art, the invention has the following beneficial effects:
1. in the invention, in step S2, the task collection application respectively drops different video tasks to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the plurality of AI video analysis applications sequentially number the received video tasks and elect the leader in the AI video analysis application, the plurality of AI video analysis applications are responsible for maintaining the consistency of the logs in the cluster, the consistency of the logs facilitates the regulation and control of the subsequent AI video analysis applications, and the number and analysis state of the video tasks in the AI video analysis applications can be rapidly detected, thereby detecting faults or completing new allocation, and further improving the working efficiency.
2. In the invention, in step S3, the leader in the AI video analysis application is sequenced according to the number of video tasks of the task collection application, the newly added video tasks received by the task collection application are numbered according to the number of the AI video analysis application in a circulating way, and the video tasks named as the same number are sequentially issued to other different AI video analysis applications, the AI video analysis application records data into a log structure, so that the number of the video tasks distributed by different AI video analysis applications can be conveniently detected later, the leader of the AI video analysis application can obtain new video tasks from the task collection application and distribute the new video tasks to the AI video analysis application with fewer tasks after the state of the server is detected, and the calculation power of the server where the AI video analysis application is located can be flexibly allocated in the mode, thereby improving the analysis speed.
3. In the invention, in step S4, when a certain AI video analysis application fails or is down, the leader in the AI video analysis application can perform balanced allocation of video tasks of the failed or down AI video analysis application again in combination with the server state, when the remote procedure call of the additional log in the log structure fails, the situation that the number of video task analysis in other AI video analysis applications is not consistent with that of the leader in the AI video analysis application is indicated, whether the AI video analysis application fails or is down can be determined by reducing the value of the next index, where the number of the video tasks allocated to the failure is smaller, so that new video tasks can be allocated to the AI video analysis application directly through the task collection application, the problem that the number of the video tasks allocated to the AI video analysis application is smaller is solved, if the allocation scheme is optimized, after the value of the next index is always repeated, the position consistent with the leader in the AI video analysis application still cannot be found, the AI video analysis application is regarded as a machine, the video tasks allocated to the AI video analysis application can be allocated to the same in sequence, and the same number of the AI video tasks can be allocated to the AI video analysis application, and the same as the task analysis can be easily performed, and the task allocation time is not completed.
4. In the invention, in step S5, when the number of video tasks in the task collection application exceeds the cluster processing capability, a large number of applications are stored in the task collection application, the complete AI video analysis application cannot process more video tasks, the leader in the AI video analysis application cannot continue to copy the log array to other AI video analysis applications, so that the position of the leader is lost, and reelection is needed.
Drawings
FIG. 1 is a flow chart of a balancing method based on a distributed AI video analysis task of the invention;
FIG. 2 is a flow chart of the task of the Raft system in a distributed AI video analysis task based balancing method of the present invention;
FIG. 3 is a schematic diagram of an distribution process of an AI video analysis application in a distributed AI video analysis task-based balancing method of the invention;
FIG. 4 is a schematic diagram of a fault recovery process during distribution in a distributed AI video analysis task-based balancing method according to the present invention;
fig. 5 is a schematic diagram of recovery of AI video analysis application downtime during distribution in a distributed AI video analysis task-based balancing method of the present invention.
Detailed Description
The invention is further described in connection with the following detailed description, in order to make the technical means, the creation characteristics, the achievement of the purpose and the effect of the invention easy to understand.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "front", "rear", "both ends", "one end", "the other end", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific direction, be configured and operated in the specific direction, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "provided," "connected," and the like are to be construed broadly, and may be fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Referring to fig. 1-3, embodiment 1 includes the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server, wherein the Raft system elects a leader in the AI video analysis applications through a heartbeat mechanism and a timeout mechanism;
s2, the task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in a Raft, each AI video analysis application stores a log array, each log array stores a log command, an optional number, a log index, a numerical value of a next index and an additional log remote procedure call, the numerical value of the next index is an index value sent to the next entry of each following person, the plurality of AI video analysis applications sequentially numbers the received video tasks and selects the leader in the AI video analysis application, the leader in the AI video analysis application manages replication of the log structure through the leader in the AI video analysis application, when the leader in the AI video analysis application exists, the leader in the AI video analysis application synchronizes the log structure to other AI video analysis applications, the leader in the AI analysis application has the most number, the numerical value of the next index is the index, the numerical value of the next index in the AI analysis application is more convenient for each following person, the numerical value of the next index in the AI analysis application is more than the AI analysis application, the numerical value in the AI analysis application is more convenient for the subsequent video analysis application is not required to be regulated and the numerical value in the AI analysis application is more convenient for the video analysis application in the following the number of the video tasks, and the numerical value in the AI analysis application is more convenient for the AI analysis application is more than the numerical value of the leader in the AI analysis application, the number and the analysis state of video tasks in the AI video analysis application can be rapidly detected, so that faults are detected or new allocation is completed, and the working efficiency is improved;
s3, the leader in the AI video analysis application sequences according to the number of video tasks of the task collection application, the newly-added video tasks received by the task collection application are circularly numbered according to the number of the AI video analysis applications, video tasks named as the same number are sequentially issued to other different AI video analysis applications, the AI video analysis application records process data of the issued video tasks into a log structure, the leader in the AI video analysis application detects the number of video tasks of each AI video analysis application and the state of a server, the subsequent detection of video tasks distributed by different AI video analysis applications is facilitated, the leader of the AI video analysis application can acquire new video tasks from the task collection application and distribute the new video tasks to the AI video analysis application with fewer video tasks after the state of the server is detected, the leader in the AI video analysis application can flexibly allocate the calculation power of the server where the AI video analysis application is located in the mode, the analysis speed is improved, and then the leader in the AI video analysis application is deployed in clusters according to the actual number of the AI video tasks of the AI video analysis application, under the condition that the leader in the AI video analysis application is suspended by error, the leader in the AI video analysis application needs to be newly-out of the new video analysis application, and the leader in the system is prevented from being submitted by the system to the system for the comparison with the new leader in the time-out timer after the state is detected;
s4, when the cluster is deployed, the cluster analysis of the AI video analysis application analyzes video tasks with the same number, when a certain AI video analysis application in the cluster fails or is down, a leader in the AI video analysis application can carry out balanced distribution of the video tasks of the failed or down AI video analysis application again in combination with a server state, the numerical value of the next index in a log structure is set to be the next position of the latest position of the log array, when the call of an additional log remote process in the log structure fails, the number of video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the numerical value of the next index is reduced until the position which is consistent with that of the leader in the AI video analysis application is found, the number of the next index is reduced, the AI video analysis application can be determined to be the failed or down, the failed by reducing the numerical value of the next index, the number of the failed video analysis application is less, the number of the new video tasks can be directly distributed to the AI video analysis application by the task collection application, accordingly, the number of the new video tasks can be distributed to the AI video analysis application can be directly distributed to the position of the AI video analysis application after the video task is not be distributed, the number of the same, the number of the video tasks can be well distributed to the position of the video analysis application is not be found, the number of the video analysis application can be well distributed to be accomplished, the number of the AI video analysis application can be well, and the number of the task can be well distributed to the failure is found, and the number to the task can be directly distributed to the task is well, the number to the task can be directly, when one or more AI video analysis applications are removed, the leader in the AI video analysis application also distributes video tasks to other intact AI video analysis applications according to the video index values, and the other intact AI video analysis applications renumber and analyze the video tasks;
s5, when the number of video tasks in the task collection application exceeds the cluster processing capability, a large number of applications are stored in the task collection application, the leader in the AI video analysis application and all the AI video analysis applications cannot continuously process the added video tasks, the leader in the AI video analysis application cannot continuously copy logs to other AI video analysis applications, so that the position of the leader is lost, reelection is needed, and because the number of video tasks of a plurality of AI video analysis applications is the same, the leader cannot be reelected by means of a heartbeat mechanism and a timeout mechanism and can fall into a cycle, so that the situation that the video tasks cannot be processed again can be known, a control server sends a system alarm, and the video tasks which cannot be processed are reported to other systems, and the applicability of a Raft system is improved.
According to example 2, as shown in FIG. 4
In the Raft system, after a fault occurs in a certain AI video analysis application, after the fault occurs in the AI video analysis application, the leader of the AI video analysis application initiates an additional log request to the AI video analysis application, the leader of the AI video analysis application stores a value of next index, and an index value of the next log entry required to be sent to each AI video analysis application is stored, so that the leader of the AI video analysis application can be directly sent to the log required to be added by the dropped AI video analysis application, when the remote procedure call of the additional log fails, the condition that the AI video analysis application is not consistent with the leader of the AI video analysis application on the latest log, namely, a certain number of video tasks is deleted, the numerical value of the next index is reduced until the leader of the AI video analysis application is found, after the latest position of the next index is found, the whole log is directly copied to a fault handling position, namely, more video tasks are given to the AI video analysis application, in FIG. 4, the AI analysis application 1 is a video analysis application, the AI analysis application is a video application 1, the AI analysis application is assigned with the index of the AI analysis application 2, and the AI analysis application is assigned with the fault 2 in the video analysis application 2, and the AI analysis application is assigned with the fault 2, and the AI analysis application is assigned with the priority 2, and the AI analysis application is assigned with the video analysis 2, and the fault analysis application is assigned with the AI analysis application 2.
According to example 3, as shown in FIG. 5
In the Raft system, after a certain AI video analysis application is down, after the video analysis application is on line again, a leader of the AI video analysis application uses an additional log to remotely call a procedure, but because the AI video analysis application is down, when the value of the next index is minimum, a corresponding log is still not found, so that the failure can be caused, which indicates that the AI video analysis application is down, in fig. 5, the AI video analysis application 1 is a leader of the AI video analysis application, the AI video analysis application 3 is down, and the leader of the AI video analysis application cannot be connected with the AI video analysis application 3, so that video tasks in the original AI video analysis application 3 are sequentially distributed to the AI video analysis applications 1, 2, 4 and 5, thereby achieving the distribution rationality and the working efficiency.
The working principle of the invention is as follows: firstly, a Raft system is established, then, different video tasks are respectively downloaded to a plurality of AI video analysis applications by using a task collection application, each AI video analysis application starts to establish a log structure after receiving the video tasks, the plurality of AI video analysis applications sequentially number the received video tasks and elect the leader in the AI video analysis application, the plurality of AI video analysis applications are responsible for maintaining the consistency of log arrays in a cluster, the AI video analysis applications record data into the log structure, the video tasks distributed by the different AI video analysis applications are conveniently detected later, the leader of the AI video analysis application can obtain new video tasks from the task collection application after detecting the state of a server and distribute the new video tasks to the AI video analysis applications with less video tasks, the analysis speed is improved, when a certain AI video analysis application fails or is down, the leader in the AI video analysis application can carry out balanced allocation again in combination with the server state, whether the AI video analysis application is faulty or down can be determined by reducing the numerical value of the next index, so that a new video task can be directly allocated to the AI video analysis application through the task collection application, the number of video tasks allocated by each AI video analysis application is the same, the allocation is balanced, the analysis completion time of each AI video analysis application on the video task is the same, the condition that analysis results are not reported easily occurs, the working efficiency is further improved, when the number of video tasks in the task collection application exceeds the cluster processing capability, the leader cannot be selected again because the number of video tasks of a plurality of AI video analysis applications is the same, the situation that the video tasks cannot be processed again can be known, the control server sends out a system alarm, and the video tasks which cannot be processed are reported to other systems, the applicability of the Raft system is improved.
The foregoing has shown and described the basic principles and main features of the present invention and the advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (7)
1. A balancing method based on a distributed AI video analysis task is characterized by comprising the following steps: the method comprises the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server;
s2, a task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in a Raft, each AI video analysis application stores a log array, each log array stores a log command, an option number, a log index, a value of a next index and an additional log remote procedure call, the plurality of AI video analysis applications sequentially number the received video tasks and elect a leader in the AI video analysis application, the Raft system elects a leader in the AI video analysis application through a heartbeat mechanism and a timeout mechanism, and manages replication of the log structure through the leader in the AI video analysis application, and when the leader in the AI video analysis application exists, the leader in the AI video analysis application synchronizes the log structure to other AI video analysis applications;
s3, sequencing the leader in the AI video analysis application according to the number of video tasks of the task collection application, numbering newly-added video tasks received by the task collection application according to the number of the AI video analysis applications, sequentially issuing video tasks named as the same number to other different AI video analysis applications, recording process data of the issued video tasks into a log structure by the AI video analysis application, detecting the number of video tasks of each AI video analysis application and the state of a server by the leader in the AI video analysis application, and then performing cluster deployment by the leader in the AI video analysis application according to the actual number of video tasks of the AI video analysis application;
s4, when the cluster is deployed, the video tasks with the same number are analyzed by the AI video analysis application cluster, when a certain AI video analysis application in the cluster fails or is down, the leader in the AI video analysis application can be combined with the server state to carry out balanced distribution of the video tasks of the failed or down AI video analysis application again, the value of the next index in the log structure is set to be the next position of the latest position of the log array, when the remote procedure call of the additional log in the log structure fails, the number of the video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the number of the AI video analysis application is determined to be the failure or down by reducing the value of the next index, the subsequent video tasks are directly copied to the AI video analysis application with the failure after the latest position of the value of the next index is found, when the value of the next index is the latest position of the next index is the latest, the number of the next index is still not found, the number of the video tasks which are consistent with the leader in the AI video analysis application is indicated to be the failure video analysis application, the AI video tasks are not orderly distributed to the AI video analysis application by the AI video analysis application, and the AI video analysis application is not well distributed to the AI video analysis application;
and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems.
2. The method for balancing a distributed AI video analysis task of claim 1, wherein: the leader in the AI video analysis application is the leader and has the largest task number, and the leader in the AI video analysis application stores the value of nextIndex and stores the video index value which needs to be sent to other AI video analysis applications.
3. The method for balancing a distributed AI video analysis task of claim 2, wherein: the value of the nextIndex in the leader in the AI video analysis application stores the video index value of the next log entry to be sent to each AI video analysis application, and the leader in the AI video analysis application directly sends the task to the more idle AI video analysis application.
4. The method for balancing a distributed AI video analysis task of claim 3, wherein: the number of video tasks in the plurality of AI video analysis applications is not necessarily the same, but is not more than the number of video tasks in the leader in the AI video analysis application.
5. The method for balancing a distributed AI video analysis task of claim 4, wherein: under the condition that the leader in the AI video analysis application is hung up by mistake, the leader in the new AI video analysis application needs to be reelected through a heartbeat mechanism and a timeout mechanism.
6. The method for balancing a distributed AI video analysis task of claim 5, wherein: in the Raft system, the election period is checked to ensure that the leader in the new AI video analysis application already has all the journal arrays submitted, avoiding additional journal array transmission.
7. The method of balancing a distributed AI video analysis task of claim 6, wherein: the video tasks which cannot be processed in the step S5 are reported to other systems, provided that the leader in the AI video analysis application and all AI video analysis applications cannot continue to process the increased number of video tasks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497141.XA CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497141.XA CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116827942A CN116827942A (en) | 2023-09-29 |
| CN116827942B true CN116827942B (en) | 2024-04-12 |
Family
ID=88128080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211497141.XA Active CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116827942B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110430395A (en) * | 2019-07-19 | 2019-11-08 | 苏州维众数据技术有限公司 | Video data AI processing system and processing method |
| CN112183222A (en) * | 2020-09-04 | 2021-01-05 | 方正株式(武汉)科技开发有限公司 | Equipment linkage system and method based on distributed artificial intelligence video analysis |
| CN114821957A (en) * | 2022-05-13 | 2022-07-29 | 湖南工商大学 | AI video analysis system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9760970B2 (en) * | 2015-03-18 | 2017-09-12 | Hitachi, Ltd. | Video analysis and post processing of multiple video streams |
-
2022
- 2022-11-23 CN CN202211497141.XA patent/CN116827942B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110430395A (en) * | 2019-07-19 | 2019-11-08 | 苏州维众数据技术有限公司 | Video data AI processing system and processing method |
| CN112183222A (en) * | 2020-09-04 | 2021-01-05 | 方正株式(武汉)科技开发有限公司 | Equipment linkage system and method based on distributed artificial intelligence video analysis |
| CN114821957A (en) * | 2022-05-13 | 2022-07-29 | 湖南工商大学 | AI video analysis system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116827942A (en) | 2023-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN202282837U (en) | Video quality diagnosis system | |
| CN107862393A (en) | A kind of IT operation management system | |
| CN107612756A (en) | A kind of operation management system with intelligent trouble analyzing and processing function | |
| CN107453929B (en) | Cluster system self-construction method and device and cluster system | |
| CN110807064A (en) | Data recovery device in RAC distributed database cluster system | |
| CN113282635A (en) | Micro-service system fault root cause positioning method and device | |
| CN106484565A (en) | Method of data synchronization between multiple data centers and relevant device | |
| CN109901951A (en) | A kind of storage system and method for ceph company-data | |
| CN104780220A (en) | Intelligent monitoring system and method for large distributed system oriented to security futures industry | |
| CN107918570A (en) | A kind of method that dual-active system shares arbitration disk | |
| CN102594596A (en) | Method and device for recognizing available partitions, and clustering network system | |
| CN111064672A (en) | Cloud platform communication system, election method and resource scheduling management method | |
| CN112346925A (en) | Process-level dual-computer hot standby redundancy system and method | |
| CN108932182A (en) | A kind of messaging bus performance test methods and system | |
| CN116827942B (en) | Balancing method based on distributed AI video analysis task | |
| CN116433589A (en) | Train checking operation system and TFDS intelligent identification method | |
| CN116755920A (en) | Fault positioning method, device, apparatus, storage medium and electronic equipment | |
| CN111475386A (en) | Fault early warning method and related device | |
| CN111130899A (en) | Service recovery method and system for distributed system | |
| CN107395768A (en) | Two-shipper adaptive load balancing method for file security transmission | |
| CN112379640B (en) | Distributed intelligent sensing controller | |
| CN117194154A (en) | APM full-link monitoring system and method based on micro-service | |
| CN117851099A (en) | Fault processing method of k8s cluster, computing equipment and cluster | |
| CN116346582A (en) | Method, device, equipment and storage medium for realizing redundancy of main network and standby network | |
| CN113467955A (en) | Intelligent distribution method and system for data acquisition link of distribution network master station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |