CN116827942A - Balancing method based on distributed AI video analysis task - Google Patents
Balancing method based on distributed AI video analysis task Download PDFInfo
- Publication number
- CN116827942A CN116827942A CN202211497141.XA CN202211497141A CN116827942A CN 116827942 A CN116827942 A CN 116827942A CN 202211497141 A CN202211497141 A CN 202211497141A CN 116827942 A CN116827942 A CN 116827942A
- Authority
- CN
- China
- Prior art keywords
- video
- video analysis
- tasks
- leader
- analysis application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 300
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1029—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers using data related to the state of servers by a load balancer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1031—Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1034—Reaction to server failures by a load balancer
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a balancing method based on a distributed AI video analysis task, which relates to the technical field of AI video analysis methods, and comprises the following steps of S1, establishing a Raft system; s2, the AI video analysis application receives video tasks and numbers the video tasks in sequence; s3, detecting AI video analysis application and a server by using a log structure; s4, a cluster deploys a video task and solves the problem that an AI video analysis application is removed or down; and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems. The invention relates to a distributed AI video analysis task-based balancing method, which is characterized in that the number of video tasks distributed by each AI video analysis application is the same, the distribution is balanced, the analysis completion time of each AI video analysis application on the video tasks is the same, thus the condition of missing report of analysis results is not easy to occur, and the working efficiency is improved.
Description
Technical Field
The invention relates to the technical field of AI video analysis methods, in particular to a distributed AI video analysis task-based balancing method.
Background
The AI video analysis processing technology is to find the moving object in the image, track and analyze the moving object, find abnormal behavior in time, and trigger an alarm system or perform intervention. AI video analysis processing technology is currently the most interesting image processing technology in the security industry. In general, the basic process of intelligent analysis of video is to read each frame of image from a given video, pre-process the input image, such as filtering, gray level conversion, etc., then determine whether there is a moving object in the input image, then determine whether the moving object is a monitoring object, and finally perform analysis such as monitoring, tracking or behavior understanding on the object according to the requirement.
The current AI video analysis system in the market basically adopts a mode that video streams are fixedly distributed to AI video analysis applications, and the AI video analysis applications perform frame extraction analysis on the video streams; with the increase of the number of video stream access paths, the AI video analysis applications can correspondingly perform multi-point deployment, and a plurality of AI video analysis applications analyze a plurality of tasks simultaneously so as to complete the task amount within a specified time.
However, the existing video stream fixed allocation task mode cannot flexibly allocate the computing power of a server where the AI video analysis application is deployed; the uneven distribution can cause excessive video flow distributed by one or more AI video analysis applications, other AI video analysis applications can be idle after the distributed tasks are processed, and the distributed more AI video analysis applications are not timely processed for analysis tasks, so that the condition of missing report of analysis results occurs, and the working efficiency of the analysis tasks is affected.
Disclosure of Invention
The invention mainly aims to provide a distributed AI video analysis task-based balancing method, which can effectively solve the problem that the conventional video stream fixed allocation task mode in the background art cannot flexibly allocate the computing power of a server where an AI video analysis application is deployed; the uneven distribution can cause excessive video flow distributed by one or more AI video analysis applications, other AI video analysis applications can be idle after the distributed tasks are processed, and the distributed more AI video analysis applications are not timely processed for analysis tasks, so that the condition of missing report of analysis results occurs, and the working efficiency of the analysis tasks is affected.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the invention discloses a distributed AI video analysis task-based balancing method, which comprises the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server;
s2, a task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in a Raft, each AI video analysis application stores a log array, each log array stores a log command, an arbitrary number, a log index, a value of a next index and an additional log remote procedure call, and the AI video analysis applications sequentially number the received video tasks and elect a leader in the AI video analysis application;
s3, sequencing the leader in the AI video analysis application according to the number of video tasks of the task collection application, numbering newly-added video tasks received by the task collection application according to the number of the AI video analysis applications, sequentially issuing video tasks named as the same number to other different AI video analysis applications, recording process data of the issued video tasks into a log structure by the AI video analysis application, detecting the number of video tasks of each AI video analysis application and the state of a server by the leader in the AI video analysis application, and then performing cluster deployment by the leader in the AI video analysis application according to the actual number of video tasks of the AI video analysis application;
s4, when the clusters are deployed, a plurality of AI video analysis application clusters analyze video tasks with the same number, when one AI video analysis application in the clusters is removed or down, a leader in the AI video analysis application can combine a server state to perform balanced distribution of video tasks of the removed or down AI video analysis application again, the value of a next index in a log structure is set to be the next position of the latest position of a log array, when the remote procedure call of an additional log in the log structure fails, the number of the video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the number of the next index is reduced, the video tasks are retried until the position consistent with that of the leader in the AI video analysis application is found, and after the latest position of the next index is found, the subsequent video tasks are directly copied to the failed AI video analysis application;
and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems.
Preferably, in the steps S1 and S2, the Raft system selects a leader in the AI video analysis application through a heartbeat mechanism and a timeout mechanism, and manages the replication of the log structure through the leader in the AI video analysis application, and when the leader in the AI video analysis application exists, the leader in the AI video analysis application synchronizes the log structure to other AI video analysis applications.
Preferably, the leader in the AI video analysis application is the leader, has the largest task number, and stores a nextInder array, and stores video index values to be sent to other AI video analysis applications.
Preferably, the next index array in the leader in the AI video analysis application stores an index value of a next log entry to be sent to each AI video analysis application, and the leader in the AI video analysis application directly sends a task to the AI video analysis application that is more idle.
Preferably, the number of video tasks in the plurality of AI video analysis applications is not necessarily the same, but is not more than the number of video tasks in the leader in the AI video analysis application.
Preferably, in the case that the leader in the AI video analysis application is suspended by mistake, the election of the leader in the new AI video analysis application needs to be reelected through a heartbeat mechanism and a timeout mechanism.
Preferably, in the Raft system, a comparison check is made during election to ensure that the leader in the new AI video analysis application already has all logs submitted, avoiding additional log transfer.
Preferably, when one or more AI video analysis applications in step S4 are removed, the leader in the AI video analysis application assigns video tasks to other intact AI video analysis applications according to the index value, and the other intact AI video analysis applications renumber and analyze the video tasks.
Preferably, the video tasks that have failed to be processed in step S5 are reported to other systems, provided that the leader in the AI video analysis application and all AI video analysis applications cannot continue to process the increased number of video tasks.
Compared with the prior art, the invention has the following beneficial effects:
1. in the invention, in step S2, the task collection application respectively drops different video tasks to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the plurality of AI video analysis applications sequentially number the received video tasks and elect the leader in the AI video analysis application, the plurality of AI video analysis applications are responsible for maintaining the consistency of the logs in the cluster, the consistency of the logs facilitates the regulation and control of the subsequent AI video analysis applications, and the number and analysis state of the video tasks in the AI video analysis applications can be rapidly detected, thereby detecting faults or completing new allocation, and further improving the working efficiency.
2. In the invention, in step S3, the leader in the AI video analysis application is sequenced according to the number of video tasks of the task collection application, the newly added video tasks received by the task collection application are numbered according to the number of the AI video analysis application in a circulating way, and the video tasks named as the same number are sequentially issued to other different AI video analysis applications, the AI video analysis application records data into a log structure, so that the number of the video tasks distributed by different AI video analysis applications can be conveniently detected later, the leader of the AI video analysis application can obtain new video tasks from the task collection application and distribute the new video tasks to the AI video analysis application with fewer tasks after the state of the server is detected, and the calculation power of the server where the AI video analysis application is located can be flexibly allocated in the mode, thereby improving the analysis speed.
3. In the invention, in step S4, when a certain AI video analysis application fails or is down, the leader in the AI video analysis application can perform balanced allocation of video tasks of the AI video analysis application removed or down again in combination with the server state, when the remote procedure call of the additional log in the log structure fails, the number of video task analysis in other AI video analysis applications is not consistent with that of the leader in the AI video analysis application, whether the AI video analysis application fails or is down can be determined by reducing the number of the next index, where the number of the video tasks allocated to the failure is smaller, so that new video tasks can be allocated to the AI video analysis application directly through the task collection application, the problem that the number of the video tasks allocated to the AI video analysis application is smaller is solved, if the allocation scheme is optimized, after the number of the next index is always repeated, the position consistent with that of the leader in the AI video analysis application still cannot be found, the AI video analysis application is regarded as a machine, the number of the video tasks allocated to the AI video analysis application can be allocated to the same in sequence, and the same number of the video tasks allocated to the AI video analysis application can be easily completed, and the same time is not be allocated to the AI video analysis application, and the same as the task analysis can be completed, and the task analysis can be easily completed.
4. In the invention, in step S5, when the number of video tasks in the task collection application exceeds the cluster processing capability, a large number of applications are stored in the task collection application, the complete AI video analysis application cannot process more video tasks, the leader in the AI video analysis application cannot continue to copy logs to other AI video analysis applications, so that the status of the leader is lost, and the leader needs to be reelected.
Drawings
FIG. 1 is a flow chart of a balancing method based on a distributed AI video analysis task of the invention;
FIG. 2 is a flow chart of the task of the Raft system in a distributed AI video analysis task based balancing method of the present invention;
FIG. 3 is a schematic diagram of an distribution process of an AI video analysis application in a distributed AI video analysis task-based balancing method of the invention;
FIG. 4 is a schematic diagram of a fault recovery process during distribution in a distributed AI video analysis task-based balancing method according to the present invention;
fig. 5 is a schematic diagram of recovery of AI video analysis application downtime during distribution in a distributed AI video analysis task-based balancing method of the present invention.
Detailed Description
The invention is further described in connection with the following detailed description, in order to make the technical means, the creation characteristics, the achievement of the purpose and the effect of the invention easy to understand.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "front", "rear", "both ends", "one end", "the other end", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific direction, be configured and operated in the specific direction, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "provided," "connected," and the like are to be construed broadly, and may be fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
Referring to fig. 1-3, embodiment 1 includes the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server, wherein the Raft system elects a leader in the AI video analysis applications through a heartbeat mechanism and a timeout mechanism;
s2, the task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in the Raft, each AI video analysis application stores a log array, each log array stores a log command, an optional number, a log index, a value of a next index and an additional log remote procedure call, the value of the next index means that for each follower, the index value of the next entry is sent to the follower, the AI video analysis applications sequentially number the received video tasks and elect a leader in the AI video analysis application, and manage replication of the log structure through the leader in the AI video analysis application, when the leader in the AI video analysis application exists, synchronizing log structures from the leader in the AI video analysis application to other AI video analysis applications, wherein the leader in the AI video analysis application is a leader, has the largest number of video tasks, stores a next index array, stores video index values to be sent to other AI video analysis applications, stores index values to be sent to the next log entry of each AI video analysis application, directly sends video tasks to idle AI video analysis applications, and the number of video tasks in the AI video analysis applications is not necessarily the same, but is not more than the number of video tasks in the leader in the AI video analysis application, the consistency of the log facilitates the regulation of subsequent AI video analysis applications, the number and the analysis state of video tasks in the AI video analysis application can be rapidly detected, so that faults are detected or new allocation is completed, and the working efficiency is improved;
s3, the leader in the AI video analysis application sequences according to the number of video tasks of the task collection application, the newly-added video tasks received by the task collection application are circularly numbered according to the number of the AI video analysis applications, video tasks named as the same number are sequentially issued to other different AI video analysis applications, the AI video analysis application records process data of the issued video tasks into a log structure, the leader in the AI video analysis application detects the number of video tasks of each AI video analysis application and the state of a server, the subsequent detection of video tasks distributed by different AI video analysis applications is facilitated, the leader of the AI video analysis application can acquire new video tasks from the task collection application and distribute the new video tasks to the AI video analysis application with fewer video tasks after the state of the server is detected, the leader in the AI video analysis application can flexibly allocate the calculation power of the server where the AI video analysis application is located in the mode, the analysis speed is improved, and then the leader in the AI video analysis application is deployed in clusters according to the actual number of the AI video tasks of the AI video analysis application, under the condition that the leader in the AI video analysis application is suspended by error, the leader in the AI video analysis application needs to be newly-out of the new video analysis application, and the leader in the system is prevented from being submitted by the system to the system for the comparison with the new leader in the time-out timer after the state is detected;
s4, when the clusters are deployed, a plurality of AI video analysis application clusters analyze video tasks with the same number, when one AI video analysis application in the clusters fails or is down, a leader in the AI video analysis application can combine the server state to perform balanced distribution to remove or down the video tasks of the AI video analysis application again, the value of the next index in the log structure is set to be the next position of the latest position of the log array, when the remote procedure call of the additional log in the log structure fails, the number of the video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the number of the next index is reduced, the retry is carried out until the position consistent with that of the leader in the AI video analysis application is found, the number of the next index is reduced to determine whether the AI video analysis application fails or is down, the number of the video tasks allocated is less, so that a new video task can be directly distributed to the AI video analysis application through the task collection application, the problem that the video task distributed to the AI video analysis application is less is solved, the distribution scheme is optimized, after the latest position of the next index is found, the subsequent video task is directly copied to the failed AI video analysis application, if the position which is consistent with the leader in the AI video analysis application can not be found after the numerical value of the next index is repeatedly reduced all the time, the AI video analysis application is regarded as downtime, the video task distributed to the AI video analysis application is sequentially distributed to other perfect AI video analysis applications, the distribution balance is realized, the analysis completion time of each AI video analysis application to the video task is the same, the condition that the analysis result is not easy to report is not easy to occur, when one or more AI video analysis applications are removed, the leader in the AI video analysis application also distributes video tasks to other intact AI video analysis applications according to the index value, and the other intact AI video analysis applications renumber and analyze the video tasks;
s5, when the number of video tasks in the task collection application exceeds the cluster processing capability, a large number of applications are stored in the task collection application, the leader in the AI video analysis application and all the AI video analysis applications cannot continuously process the added video tasks, the leader in the AI video analysis application cannot continuously copy logs to other AI video analysis applications, so that the position of the leader is lost, reelection is needed, and because the number of video tasks of a plurality of AI video analysis applications is the same, the leader cannot be reelected by means of a heartbeat mechanism and a timeout mechanism and can fall into a cycle, so that the situation that the video tasks cannot be processed again can be known, a control server sends a system alarm, and the video tasks which cannot be processed are reported to other systems, and the applicability of a Raft system is improved.
According to example 2, as shown in FIG. 4
In the Raft system, after a fault occurs in a certain AI video analysis application, after the fault occurs in the certain AI video analysis application, the leader of the AI video analysis application initiates an additional log request to the AI video analysis application, the leader of the AI video analysis application stores a next index array, and an index value of the next log entry required to be sent to each AI video analysis application is stored, so that the leader of the AI video analysis application can directly send to the video analysis application requiring an additional log of a fall-off, when the remote procedure call of the additional log fails, the condition that the AI video analysis application is not consistent with the leader of the AI video analysis application on the latest log is indicated, namely, a certain number of video tasks is deleted, the number of the next index is reduced until the leader of the AI video analysis application is found, after the latest position of the next index is found, the whole log is directly copied to a fault handling position, namely, more video tasks are given to the AI analysis application, in fig. 4, the AI analysis application 1 is the video analysis application, when the fault handling index of the AI video analysis application 1 is not consistent with the leader of the AI video analysis application 2, the AI video analysis application 2 is assigned to the video analysis application 2, and the AI analysis application 2 is assigned to the fault handling stage of the AI analysis application, and the AI analysis application is assigned to the video application 2, and the AI analysis application is assigned to the fault handling stage 2, and the AI analysis application is assigned to the video application 2, and the fault handling stage is assigned to the video analysis application 2.
According to example 3, as shown in FIG. 5
In the Raft system, after a certain AI video analysis application is down, after the video analysis application is on line again, a leader of the AI video analysis application uses an additional log to remotely call a procedure, but because the AI video analysis application is down, when the next index value is minimum, a corresponding log is still not found, so that the failure can be caused, the AI video analysis application is down, in fig. 5, the AI video analysis application 1 is a leader of the AI video analysis application, the AI video analysis application 3 is down, and the leader of the AI video analysis application cannot be connected with the AI video analysis application 3, so that video tasks in the original AI video analysis application 3 are sequentially distributed to the AI video analysis applications 1, 2, 4 and 5, so that the distribution rationality and the working efficiency are achieved.
The working principle of the invention is as follows: firstly, a Raft system is established, then, different video tasks are respectively downloaded to a plurality of AI video analysis applications by using a task collection application, each AI video analysis application starts to establish a log structure after receiving the video tasks, the plurality of AI video analysis applications sequentially number the received video tasks and elect the leader in the AI video analysis application, the plurality of AI video analysis applications are responsible for maintaining the log consistency in a cluster, the AI video analysis applications record data into the log structure, the video tasks distributed by the different AI video analysis applications are conveniently detected later, the leader of the AI video analysis application can obtain new video tasks from the task collection application after detecting the state of a server and distribute the new video tasks to the AI video analysis applications with less video tasks, the analysis speed is improved, when a certain AI video analysis application fails or is down, the leader in the AI video analysis application can carry out balanced allocation again in combination with the server state, whether the AI video analysis application is faulty or down can be determined by reducing the numerical value of the next index, so that a new video task can be directly allocated to the AI video analysis application through the task collection application, the number of video tasks allocated by each AI video analysis application is the same, the allocation is balanced, the analysis completion time of each AI video analysis application on the video task is the same, the condition that analysis results are not reported easily occurs, the working efficiency is further improved, when the number of video tasks in the task collection application exceeds the cluster processing capability, the leader cannot be selected again because the number of video tasks of a plurality of AI video analysis applications is the same, the situation that the video tasks cannot be processed again can be known, the control server sends out a system alarm, and the video tasks which cannot be processed are reported to other systems, the applicability of the Raft system is improved.
The foregoing has shown and described the basic principles and main features of the present invention and the advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (9)
1. A balancing method based on a distributed AI video analysis task is characterized by comprising the following steps: the method comprises the following steps:
s1, establishing a Raft system comprising a task collection application, a plurality of AI video analysis applications and a server;
s2, a task collection application stores a plurality of video tasks, different video tasks are respectively downloaded to a plurality of AI video analysis applications, each AI video analysis application starts to establish a log structure after receiving the video tasks, the log structure is the most basic data structure in a Raft, each AI video analysis application stores a log array, each log array stores a log command, an arbitrary number, a log index, a value of a next index and an additional log remote procedure call, and the AI video analysis applications sequentially number the received video tasks and elect a leader in the AI video analysis application;
s3, sequencing the leader in the AI video analysis application according to the number of video tasks of the task collection application, numbering newly-added video tasks received by the task collection application according to the number of the AI video analysis applications, sequentially issuing video tasks named as the same number to other different AI video analysis applications, recording process data of the issued video tasks into a log structure by the AI video analysis application, detecting the number of video tasks of each AI video analysis application and the state of a server by the leader in the AI video analysis application, and then performing cluster deployment by the leader in the AI video analysis application according to the actual number of video tasks of the AI video analysis application;
s4, when the clusters are deployed, a plurality of AI video analysis application clusters analyze video tasks with the same number, when one AI video analysis application in the clusters is removed or down, a leader in the AI video analysis application can combine a server state to perform balanced distribution of video tasks of the removed or down AI video analysis application again, the value of a next index in a log structure is set to be the next position of the latest position of a log array, when the remote procedure call of an additional log in the log structure fails, the number of the video tasks in other AI video analysis applications is indicated to be not consistent with that of the leader in the AI video analysis application, the number of the next index is reduced, the video tasks are retried until the position consistent with that of the leader in the AI video analysis application is found, and after the latest position of the next index is found, the subsequent video tasks are directly copied to the failed AI video analysis application;
and S5, when the number of video tasks in the task collection application exceeds the cluster processing capacity, the server sends out a system alarm and reports the video tasks which cannot be processed to other systems.
2. The method for balancing a distributed AI video analysis task of claim 1, wherein: in the steps S1 and S2, the Raft system selects a leader in the AI video analysis application through a heartbeat mechanism and a timeout mechanism, manages copying of the log structure through the leader in the AI video analysis application, and synchronizes the log structure to other AI video analysis applications by the leader in the AI video analysis application when the leader in the AI video analysis application exists.
3. The method for balancing a distributed AI video analysis task of claim 2, wherein: the leader in the AI video analysis application is the leader and has the largest task number, and the leader in the AI video analysis application stores a nextInder array and stores video index values which need to be sent to other AI video analysis applications.
4. The method for balancing a distributed AI video analysis task of claim 3, wherein: the next index array in the leader in the AI video analysis application stores the index value of the next log entry to be sent to each AI video analysis application, and the leader in the AI video analysis application directly sends the task to the AI video analysis application which is idle.
5. The method for balancing a distributed AI video analysis task of claim 4, wherein: the number of video tasks in the plurality of AI video analysis applications is not necessarily the same, but is not more than the number of video tasks in the leader in the AI video analysis application.
6. The method for balancing a distributed AI video analysis task of claim 5, wherein: under the condition that the leader in the AI video analysis application is hung up by mistake, the leader in the new AI video analysis application needs to be reelected through a heartbeat mechanism and a timeout mechanism.
7. The method of balancing a distributed AI video analysis task of claim 6, wherein: in the Raft system, the election period is checked against to ensure that the leader in the new AI video analysis application already has all submitted logs, avoiding additional log transmission.
8. The method for balancing a distributed AI video analysis task of claim 7, wherein: when one or more AI video analysis applications in step S4 are removed, the leader in the AI video analysis application will assign the video task to other intact AI video analysis applications according to the index value, and the other intact AI video analysis applications renumber and analyze the video task.
9. The method for balancing a distributed AI video analysis task of claim 8, wherein: the video tasks which cannot be processed in the step S5 are reported to other systems, provided that the leader in the AI video analysis application and all AI video analysis applications cannot continue to process the increased number of video tasks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497141.XA CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211497141.XA CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116827942A true CN116827942A (en) | 2023-09-29 |
| CN116827942B CN116827942B (en) | 2024-04-12 |
Family
ID=88128080
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211497141.XA Active CN116827942B (en) | 2022-11-23 | 2022-11-23 | Balancing method based on distributed AI video analysis task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116827942B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160275642A1 (en) * | 2015-03-18 | 2016-09-22 | Hitachi, Ltd. | Video analysis and post processing of multiple video streams |
| CN110430395A (en) * | 2019-07-19 | 2019-11-08 | 苏州维众数据技术有限公司 | Video data AI processing system and processing method |
| CN112183222A (en) * | 2020-09-04 | 2021-01-05 | 方正株式(武汉)科技开发有限公司 | Equipment linkage system and method based on distributed artificial intelligence video analysis |
| CN114821957A (en) * | 2022-05-13 | 2022-07-29 | 湖南工商大学 | AI video analysis system and method |
-
2022
- 2022-11-23 CN CN202211497141.XA patent/CN116827942B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160275642A1 (en) * | 2015-03-18 | 2016-09-22 | Hitachi, Ltd. | Video analysis and post processing of multiple video streams |
| CN110430395A (en) * | 2019-07-19 | 2019-11-08 | 苏州维众数据技术有限公司 | Video data AI processing system and processing method |
| CN112183222A (en) * | 2020-09-04 | 2021-01-05 | 方正株式(武汉)科技开发有限公司 | Equipment linkage system and method based on distributed artificial intelligence video analysis |
| CN114821957A (en) * | 2022-05-13 | 2022-07-29 | 湖南工商大学 | AI video analysis system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116827942B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106126346B (en) | A kind of large-scale distributed data collection system and method | |
| CN110807064B (en) | Data recovery device in RAC distributed database cluster system | |
| US8615578B2 (en) | Using a standby data storage system to detect the health of a cluster of data storage servers | |
| CN108234170A (en) | The monitoring method and device of a kind of server cluster | |
| CN107453929B (en) | Cluster system self-construction method and device and cluster system | |
| CN109474465A (en) | A kind of method and system of the high availability that can dynamically circulate based on server cluster | |
| JPH04229741A (en) | Method and apparatus for isolating fault and analyzing fault in link coupling system | |
| CN106484565A (en) | Method of data synchronization between multiple data centers and relevant device | |
| CN109901951A (en) | A kind of storage system and method for ceph company-data | |
| CN109842505A (en) | A kind of cloud clustering fault processing method and processing device | |
| CN111064672A (en) | Cloud platform communication system, election method and resource scheduling management method | |
| CN110677282A (en) | Hot backup method of distributed system and distributed system | |
| DE112020000145T5 (en) | Redundant network IP intelligent switching method and system based on redundant computer | |
| CN112346925A (en) | Process-level dual-computer hot standby redundancy system and method | |
| CN111130899A (en) | Service recovery method and system for distributed system | |
| CN116827942B (en) | Balancing method based on distributed AI video analysis task | |
| EP1103146B1 (en) | Fault control and restoration in a data communication system | |
| CN116346582A (en) | Method, device, equipment and storage medium for realizing redundancy of main network and standby network | |
| CN106250048A (en) | The method and device of management storage array | |
| CN110399254A (en) | A kind of server CMC dual-locomotive heat activating method, system, terminal and storage medium | |
| CN110830281B (en) | Hot standby method and system based on mesh network structure | |
| CN113467955A (en) | Intelligent distribution method and system for data acquisition link of distribution network master station | |
| CN113946474A (en) | Efficient disaster tolerance protection method and disaster tolerance processing system for storage system | |
| CN108897645B (en) | Database cluster disaster tolerance method and system based on standby heartbeat disk | |
| CN112564968A (en) | Fault processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |