WO2023213073A1 - 移动视频ai平台及视频流处理方法 - Google Patents

移动视频ai平台及视频流处理方法 Download PDF

Info

Publication number
WO2023213073A1
WO2023213073A1 PCT/CN2022/136890 CN2022136890W WO2023213073A1 WO 2023213073 A1 WO2023213073 A1 WO 2023213073A1 CN 2022136890 W CN2022136890 W CN 2022136890W WO 2023213073 A1 WO2023213073 A1 WO 2023213073A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
task
platform
video stream
management server
Prior art date
Application number
PCT/CN2022/136890
Other languages
English (en)
French (fr)
Inventor
张湛梅
张晓川
陈智扬
陈雅娟
Original Assignee
中国移动通信集团广东有限公司
中国移动通信集团有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国移动通信集团广东有限公司, 中国移动通信集团有限公司 filed Critical 中国移动通信集团广东有限公司
Publication of WO2023213073A1 publication Critical patent/WO2023213073A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
    • H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/6437—Real-time Transport Protocol [RTP]

Definitions

  • This application relates to the field of video stream processing technology, and in particular to a mobile video artificial intelligence (Artificial Intelligence, AI) platform and a video stream processing method.
  • AI Artificial Intelligence
  • the video AI analysis application support platform is based on the video AI analysis needs of some smart buildings, bright kitchens, smart construction sites, etc.
  • many platforms follow the principle of minimum system and take rapid implementation and launch as the starting point. Therefore, they adopt a simplified design in the system architecture and retain the most basic business processes and functions.
  • the simplified design brings simplification and rapid implementation of the system, and has good performance in meeting the current video stream access requirements of less than 500 channels.
  • the simplified design also brings about performance and process defects, including the difficulty of the platform supporting the access of large-scale video streams and the waste of resources in task scheduling.
  • the system is connected to less than 500 video streams, the number of concurrent video AI tasks can reach 1,500 or even higher.
  • the system's video encoding and decoding modules will become unstable; when the video stream When access reaches 10,000 channels, the above problems will become more serious.
  • the simplified design of the system architecture of the video AI analysis application support platform in related technologies makes it difficult for the architecture to support the massive video stream access requirements, and there are problems such as low system resource utilization, insufficient system robustness, and poor business robustness. Insufficiency and other problems.
  • Most of the current systems are mainly composed of main functional modules such as client (World Wide Web, Web, Application, App, etc.), central management server and AI video analysis server.
  • client World Wide Web, Web, Application, App, etc.
  • central management server and AI video analysis server.
  • AI video analysis server are all deployed on a single node, which is the bottleneck of the system, resulting in low system load and difficulty in supporting subsequent large-scale commercial integration.
  • This application provides a mobile video AI platform and video stream processing method to solve the problem in related technologies that the video AI analysis application support platform system has low load capacity and is difficult to support subsequent large-scale commercial integration, by using a distributed cluster. Capacity expansion can enhance the platform’s distributed architecture management capabilities and improve the platform’s expansion capabilities.
  • the mobile video AI platform includes an AI task management server, a central management server, middleware and an anti-agent node.
  • the mobile video AI platform also includes an AI capability pool.
  • the AI capability pool Concurrency management for video AI analysis;
  • the central management server is used to synchronize the running status and load status of each functional module in real time to achieve dynamic scheduling, and then perform circuit breaker or queue processing on newly accessed tasks based on the nature of the task;
  • the AI task management server, the central management server, the middleware and the anti-agent node are all expanded in a distributed cluster manner.
  • the AI task management server includes a 1:N task configuration sub-module, an inspection task management sub-module, a status monitoring sub-module and a node information management sub-module;
  • the AI capability pool includes a capability opening platform, an AI platform and an AI capability server.
  • the middleware includes a database, a message queue module, an image data interaction module and an alarm picture storage module;
  • the database, the message queue module, the image data interaction module and the alarm picture storage module are deployed in a distributed structure.
  • the AI task management server and the database (Redis) module of the middleware are deployed in a hot standby manner;
  • the Master node in the AI task management server is the management node of the AI task management server and is used for scheduling AI video analysis tasks.
  • the video stream processing method is applied to any of the above mobile video AI platforms.
  • the video stream processing method includes:
  • the video stream processing method provided according to this application includes:
  • the module status is managed on the master node in the AI task manager
  • the video stream processing method provided according to this application includes:
  • the video stream processing method provided according to this application includes:
  • the target thread is also used to determine whether the connection with the video cloud server is disconnected at a preset time interval, and if disconnected, automatically reconnect to the video cloud server.
  • the video stream processing method provided according to this application includes:
  • the one-to-many task configuration of video frames is performed on the decoding frame extraction task of the video code stream, where the one-to-many task configuration includes issuing multiple AI analysis tasks for the video code stream.
  • This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the program, it implements any of the above video stream processing. method.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program When executed by a processor, it implements any one of the above video stream processing methods.
  • the present application also provides a computer program product, which includes a computer program.
  • the computer program When the computer program is executed by a processor, it implements any one of the above video stream processing methods.
  • the mobile video AI platform and video stream processing method provided by this application are implemented by changing the AI task management server, central management server, middleware and anti-agent nodes in the mobile video AI platform into distributed clusters, using a distributed cluster approach. Capacity expansion, enhanced distributed architecture management capabilities, and platform expansion capabilities; and a new AI capability pool to enable dynamic access to AI capabilities and concurrent management of video AI.
  • Figure 1 is a schematic system architecture diagram of the mobile video AI platform provided by this application.
  • FIG. 2 is a schematic structural diagram of the AI task management server in the mobile video AI platform provided by this application;
  • FIG. 3 is a schematic structural diagram of the AI capability pool in the mobile video AI platform provided by this application.
  • Figure 4 is a schematic structural diagram of the middleware of the mobile video AI platform provided by this application.
  • FIG. 5 is one of the flow diagrams of the video stream processing method provided by this application.
  • FIG. 6 is the second schematic flow chart of the video stream processing method provided by this application.
  • FIG. 7 is the third schematic flow chart of the video stream processing method provided by this application.
  • Figure 8 is the fourth schematic flow chart of the video stream processing method provided by this application.
  • Figure 9 is a schematic structural diagram of an electronic device provided by this application.
  • Figure 1 is a mobile video AI platform proposed by this application.
  • the mobile video AI platform includes an AI task management server, a central management server, middleware and an anti-agent node.
  • the mobile video AI platform also includes AI capability pool, which is used for concurrent management of video AI analysis;
  • the central management server is used to synchronize the running status and load status of each functional module in real time to achieve dynamic scheduling, and then perform circuit breaker or queue processing on newly accessed tasks based on the nature of the task;
  • the AI task management server, the central management server, the middleware and the anti-agent node are all expanded in a distributed cluster manner.
  • the business application system is responsible for all management, issuance of query and preview requirements, and result display.
  • the central management server is responsible for the management of data and instructions for the entire system, including the distribution of business application system requirements, pushing data to the business application system, registration of business-related information (equipment information, AI capabilities and AI tasks, etc.), and task distribution. Send and receive alarms, as well as store all data in the database and query it.
  • the central management server is usually associated with a database, and only the central management server can access the database.
  • the AI task management server cluster is mainly responsible for the preparation and connection of AI tasks, including the Real Time Streaming Protocol (RTSP) code stream required for AI tasks, decoding frames or screenshots of RTSP code streams, and AI tasks.
  • RTSP Real Time Streaming Protocol
  • the AI capability server cluster is responsible for providing various AI capabilities. Each AI capability is deployed in an independent container (docker) and provides external capability responses through services.
  • the middleware is mainly responsible for the data flow within the platform. It includes four middlewares including database, message queue module, image interaction module, and alarm image storage module.
  • the anti-proxy node cluster is mainly implemented based on the high-performance Hyper Text Transfer Protocol (HTTP) and reverse proxy web server (Nginx), and is responsible for real-time preview RTSP code stream forwarding of video AI analysis.
  • HTTP Hyper Text Transfer Protocol
  • Nginx reverse proxy web server
  • the anti-proxy node cluster only has 2 nodes.
  • RTSP stream data is transmitted between the mobile video AI platform and the video cloud server through a dedicated line. Within the platform, the transfer process involving image data and interaction also requires communication through dedicated lines.
  • the current AI video analysis server of the system involves the encoding and decoding of RTSP code streams, which consumes a large amount of hardware resources.
  • the system adopts a distributed design for this module.
  • the AI capability requirements can basically be met by a high-performance AI server, and the research and development cycle is relatively tight, so the AI capability pool does not adopt a distributed design.
  • the distributed architecture management capabilities of the system are enhanced, specifically including the use of distributed deployment for middleware such as databases and Redis modules, central management servers, and AI task management services to improve the system's high concurrency capabilities. Expand the system capacity to accommodate subsequent large-scale access.
  • the mobile video AI platform changed the AI task management server, central management server, middleware and anti-agent nodes in the mobile video AI platform into distributed clusters, and adopts the distributed cluster method to expand the capacity and enhance the distribution Architectural management capabilities improve the platform’s expansion capabilities; and a new AI capability pool is added to enable dynamic access to AI capabilities and concurrent management of video AI.
  • the AI task management server includes a 1:N task configuration sub-module, an inspection task management sub-module, a status monitoring sub-module and nodes.
  • FIG 3 is a schematic structural diagram of the AI capability pool of this application.
  • the AI capability pool includes a capability opening platform, an AI platform and an AI capability server.
  • the middleware includes a database, an image data interaction module and an alarm picture storage module;
  • the database, the image data interaction module and the alarm picture storage module are deployed in a distributed structure.
  • the transformation of the system architecture of the mobile video AI platform also includes: the middleware includes the database, message queue module, image data interaction module and alarm picture storage module, which solves the problem of insufficient business robustness; the database, image data interaction module, alarm picture Distributed design of middleware such as storage modules.
  • Middleware is responsible for the data flow within the platform. It includes four middlewares including database, message queue module, image interaction module, and alarm image storage module.
  • database is built based on the relational database management system (MySQL)
  • message queue module is implemented using MQ technology
  • image interaction module uses Redis distributed
  • alarm image storage module uses the open source lightweight distributed file system (fastDFS).
  • the middleware adopts a distributed design and synchronizes status and load information with the central management service when the task status changes to avoid system overload and solve the problem of low system carrying capacity.
  • AI task management server and the Redis module of the middleware are deployed in a hot standby manner
  • the Master node in the AI task management server is the management node of the AI task management server and is used for scheduling AI video analysis tasks.
  • the current system lacks effective management of bottleneck nodes.
  • the current platform adopts a lightweight architecture and cannot support large-scale commercial use. Encoding and decoding and Redis will become bottlenecks. When the bottleneck reaches the performance threshold, tasks are still issued, which can easily cause system congestion and affect the normal development of other upstream and downstream functional components. For the above Key nodes lack effective management mechanisms.
  • the Redis module of the AI task management server and middleware is an important part of the system and is also the bottleneck.
  • the two modules are deployed in a hot standby manner to improve system stability. .
  • the Master node is the management node of the AI task management server and is mainly responsible for the scheduling of AI video analysis tasks.
  • the functions of the Master node include managing task information, managing cluster address information, managing RTSP forwarding addresses, and formulating Nginx forwarding rules.
  • the AI task management server also includes Node nodes, which are responsible for processing specific AI video analysis tasks, including parsing task instructions issued by the central management server, pulling RTSP streams from the video cloud based on device information, and decoding and extracting the RTSP streams.
  • Frame (or screenshot), store the decoded picture into redis, request the redis address to the AI capability pool as a parameter and return the analysis results, push the alarm information to the MQ queue, mark the analysis results on the picture according to the real-time analysis preview requirements, and Encoding streaming and closing tasks.
  • the Redis module of the AI task management server and middleware is deployed in a hot standby manner and adopts a high-availability architecture to support large-scale commercial use.
  • the AI video analysis task is scheduled through the Master node in the AI task management server, adding a management mechanism and improving the robustness of the system.
  • the video stream processing method is applied to any of the above mobile video AI platforms.
  • the video stream processing method includes:
  • Step 10 Monitor the status of the video stream pulling process so that when the video stream pulling is abnormal or fails, the video stream can be pulled again;
  • Step 20 Assign the same video stream to a single video encoding and decoding process, and synchronize the encoded and decoded video data to different video AI capabilities.
  • the simplified design of the original video AI platform made the main processes of the system relatively simple, so some extended functions were not implemented. This application enhances system functions and data flow logic. On the basis of improving the system function tree, it also achieves significant optimization in efficiency and resources.
  • the main performances include:
  • the task configuration strategy is changed so that when delivering multiple video AI capabilities to the same video stream, only one video encoding and decoding process is implemented, and the data is synchronized to different devices according to different strategies.
  • Video AI capabilities greatly reduce resource utilization.
  • the video stream processing method corresponding to the mobile video AI platform of this application also includes:
  • Step 30 Manage the module status of the master node in the AI task manager through keep alive technology
  • Step 40 Implement status monitoring and reporting for all modules and middleware in the mobile video AI platform, and handle abnormal situations.
  • keep alive technology is used on the master node in the AI task manager to prevent it from malfunctioning and improve the multi-activity of the system.
  • the video stream processing method corresponding to the mobile video AI platform of this application also includes:
  • Step 50 Configure the circuit breaker mechanism and the queuing mechanism for the real-time stream mode and the single-frame screenshot mode respectively;
  • Step 60 When the response time of the downstream service is lower than the preset time or the downstream service fails to respond, the call to the downstream service is cut off and the continuous cutoff state is maintained for the preset time.
  • the video stream processing method corresponding to the mobile video AI platform of this application also includes:
  • Step 70 When receiving the streaming task, create a new target thread, where the target thread is used to obtain time-sensitive anti-piracy video code streams from the video cloud server;
  • Step 80 Decode the anti-piracy video stream to obtain a picture, and store the picture in a database;
  • Step 90 Send the AI task instructions corresponding to the pictures to the AI capability pool, and obtain the AI analysis results fed back by the AI capability pool;
  • the target thread is also used to determine whether the connection with the video cloud server is disconnected at a preset time interval, and if disconnected, automatically reconnect to the video cloud server.
  • the master node sends the task to the new node (new node) node, and the new node creates a new thread 1 to pull the stream, that is Pull the anti-piracy RTSP code stream with only 10s timeliness from the video cloud through the device information.
  • Thread 1 stores the decoded image into redis, and sends the redis address as a parameter along with the AI task instructions to the AI capability pool, and receives the returned AI analysis results (including alarm information), in order to avoid the problem of interruption caused by pulling RTSP code stream for more than 10 seconds, a new interruption and reconnection mechanism is added.
  • the video stream processing method corresponding to the mobile video AI platform of this application also includes:
  • the one-to-many task configuration of video frames is performed on the decoding frame extraction task of the video code stream, where the one-to-many task configuration includes issuing multiple AI analysis tasks for the video code stream.
  • the system AI video analysis server is mainly responsible for the preparation and connection of AI tasks, including the pulling of RTSP code streams required for AI tasks, decoding frames or screenshots of RTSP code streams, data interaction of AI tasks, and the return of AI task results and alarms.
  • AI video analysis real-time preview of RTSP code stream encoding and streaming.
  • Each task orchestration functional component occupies an exclusive thread, but the configuration strategies for different tasks are different. When some nodes reach performance bottlenecks, other nodes may be idle, resulting in a waste of resources in the streaming media access part. Therefore, the current system still has the problem of a single task configuration strategy, resulting in a waste of streaming media access resources.
  • the system thread selects the method of decoding frame extraction based on the frame extraction interval parameter passed in by the AI task.
  • the screenshot operation is performed when the frame is greater than 250, and the decoding frame extraction operation is performed when the frame is less than or equal to 250.
  • the frame interval parameters may be set differently for different tasks. Similarly, it results in multiple extractions of the same video frame. However, through the video frame one-to-multi-task configuration function, repeated extraction of video streams can be reduced.
  • the business application system can issue a single AI analysis task for the specified RTSP code stream/video code stream (here called 1:1), or it can issue multiple AI analysis tasks for the specified RTSP code stream/video code stream. Tasks (called 1:N here) can also issue scheduled tasks and other inspection tasks as needed;
  • the AI task management service receives the AI analysis task issued by the central management service, whether it is a 1:1 task, a 1:N task, or an inspection task, it will obtain the device ID and other information from the video cloud. And after the task is started, the RTSP code stream is pulled from the video cloud. If it is a 1:N task here, N tasks will only pull one RTSP code stream;
  • the AI task management service creates a new task node, selects the method of decoding frame extraction, decodes the video code stream pulled from the video cloud, and stores different numbers of images to the Redis node according to the number of AI analysis tasks for the code stream. data, and finally summarize the task information, and distribute the tasks to the AI capability pool through the capability open platform and AI platform;
  • the AI capability pool performs AI analysis based on the tasks assigned by the AI task management service and the method of decoding frames, reads image data from the Redis node, performs AI analysis, and returns the results and alarm information to the AI task management service and Redis storage node respectively.
  • the alarm information returned by the AI capability pool can be used to encode the image, and the stored RTSP forwarding address is transmitted to the ngix service node.
  • the web end will use this RTSP The forwarding address goes to Nginx to pull the RTSP code stream;
  • the AI task management service When the AI task management service decodes the RTSP code stream, it will maintain the heartbeat of the decoding thread of the Node node. When the RTSP code stream fails to be pulled, it will report the interruption information and start the interruption and reconnection function;
  • the central management service When each task is issued or executed, the central management service will synchronize the operating status information and load status of each functional module to determine whether the entire high-performance AI platform has reached the load red line;
  • Figure 9 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (processor) 910, a communication interface (Communications Interface) 920, a memory (memory) 930 and a communication bus 940.
  • the processor 910, the communication interface 920, and the memory 930 complete communication with each other through the communication bus 940.
  • the processor 910 can call logical instructions in the memory 930 to execute a video stream processing method.
  • the method includes: performing status monitoring on the video code stream pulling process, so that when the video code stream pulling process is abnormal or fails, the video code stream processing method is performed.
  • the stream is re-pulled; the same video stream is assigned to a single video encoding and decoding process, and the encoded and decoded video data is synchronized to different video AI capabilities.
  • the above-mentioned logical instructions in the memory 930 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of the present application is essentially or contributes to the relevant technology or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes several The instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
  • the present application also provides a computer program product.
  • the computer program product includes a computer program.
  • the computer program can be stored on a non-transitory computer-readable storage medium.
  • the computer can Execute the video stream processing method provided by each of the above methods.
  • the method includes: monitoring the status of the video stream pulling process, so that when the video stream pulling is abnormal or fails, the video stream is pulled again;
  • the same video stream is assigned to a single video encoding and decoding process, and the encoded and decoded video data is synchronized to different video AI capabilities.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is implemented when executed by the processor to perform the video stream processing method provided by each of the above methods.
  • the method includes : Monitor the status of the video stream pulling process to re-pulse the video stream when there is an exception or failure in the video stream pulling process; allocate the same video stream to a single video codec process, and The coded video data is synchronized to different video AI capabilities.
  • the device embodiments described above are only illustrative.
  • the units described as separate components may or may not be physically separated.
  • the components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Persons of ordinary skill in the art can understand and implement the method without any creative effort.
  • each embodiment can be implemented by software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the computer software products can be stored in computer-readable storage media, such as ROM/RAM, disks. , optical disk, etc., including a number of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请提供一种移动视频AI平台及视频流处理方法，所述移动视频AI平台包括AI任务管理服务器、中心管理服务器、中间件和反代理节点，所述移动视频AI平台还包括AI能力池，所述AI能力池用于视频AI分析的并发管理；中心管理服务器，用于实时同步各功能模块的运行状态以及负载情况实现动态调度，再结合任务性质，对新接入的任务执行熔断或者排队处理；所述AI任务管理服务器、所述中心管理服务器、所述中间件和所述反代理节点均采用分布式集群的方式进行扩容。

Description

移动视频AI平台及视频流处理方法
相关申请的交叉引用
本申请主张在2022年5月5日在中国提交的中国专利申请No.202210483462.8的优先权，其全部内容通过引用包含于此。
技术领域
本申请涉及视频流处理技术领域，尤其涉及一种移动视频人工智能(Artificial Intelligence，AI)平台及视频流处理方法。
背景技术
视频AI分析应用支撑平台基于一些智慧楼宇、明厨亮灶、智慧工地等视频AI分析需求。一般很多平台都遵循最小系统原则，以快速实施及上线为出发点，因此在系统架构上采取简易化设计，保留最基本业务流程和功能。简易化设计带来了系统的精简和快速实现，在目前支撑的500路以下的视频码流接入需求上有良好的表现。
简易化设计也带来了性能和流程上的缺陷，包括平台难以支撑大规模视频码流的接入，在任务调度上也存在资源浪费等不足。当系统接入500路以下的视频码流时，需要并发的视频AI任务数量可达到1500甚至更高，这种情况下系统的视频编解码等模块就会变得不稳定；而当视频码流接入达到10000路，上述问题将变得更加严重。相关技术中的视频AI分析应用支撑平台在系统架构上的简易化设计，使得架构上较难支撑海量的视频码流接入需求，且存在系统资源利用率低、系统健壮性不足、业务健壮性不足等多种问题。
目前系统大多都是主要由客户端(万维网(World Wide Web，Web)、应用程序(Application，App)等)、中心管理服务器及AI视频分析服务器等主要功能模块组成。其中中心服务管理、AI视频分析服务器等均为单节点部署，为系统瓶颈，造成系统承载较低，难以支撑后续大规模商用集成。
发明内容
本申请提供一种移动视频AI平台及视频流处理方法，用以解决相关技术中视频AI分析应用支撑平台系统承载较低，难以支撑后续大规模商用集成的缺陷，通过采用分布式集群的方式进行扩容，实现增强平台分布式架构管理能力，提高平台的扩容能力。
本申请提供一种移动视频AI平台，所述移动视频AI平台包括AI任务管理服务器、中心管理服务器、中间件和反代理节点，所述移动视频AI平台还包括AI能力池，所述AI能力池用于视频AI分析的并发管理；
中心管理服务器，用于实时同步各功能模块的运行状态以及负载情况实现动态调度，再结合任务性质，对新接入的任务执行熔断或者排队处理；
所述AI任务管理服务器、所述中心管理服务器、所述中间件和所述反代理节点均采用分布式集群的方式进行扩容。
根据本申请提供的一种移动视频AI平台，所述AI任务管理服务器包括1:N任务配置子模块、巡检任务管理子模块、状态监控子模块和节点信息管理子模块；
所述AI能力池包括能力开放平台、AI平台和AI能力服务器。
根据本申请提供的一种移动视频AI平台，所述中间件包括数据库、消息队列模块、图像数据交互模块和告警图片存储模块；
所述数据库、所述消息队列模块、所述图像数据交互模块和所述告警图片存储模块采用分布式结构进行部署。
根据本申请提供的一种移动视频AI平台，所述AI任务管理服务器和所述中间件的数据库(Redis)模块采用热备的方式部署；
所述AI任务管理服务器中的Master节点为所述AI任务管理服务器的管理节点，用于AI视频分析任务的调度。
根据本申请提供的视频流处理方法，所述视频流处理方法应用于上述任一项的移动视频AI平台，所述视频流处理方法包括：
对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；
将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
根据本申请提供的视频流处理方法，包括：
通过保持(keep alive)技术，对所述AI任务管理器中的主(master)节点进行模块状态的管理；
对所述移动视频AI平台中的所有的模块和中间件实现状态监控和上报，并对异常情况进行处理。
根据本申请提供的视频流处理方法，包括：
分别对实时码流方式和单帧截图方式进行配置熔断机制及排队机制；
当下游服务的响应时长低于预设时长或所述下游服务响应失败，进行切断对下游服务的调用，并保持持续切断状态预设时长。
根据本申请提供的视频流处理方法，包括：
在接收到拉流任务时，新建目标线程，其中，所述目标线程用于从视频云服务器中获取具有时效的防盗播视频码流；
将所述防盗播视频码流进行解码后得到图片，并将所述图片存入数据库；
将所述图片对应的AI任务指令下发至所述AI能力池，得到所述AI能力池反馈的AI分析结果；
其中，所述目标线程还用于相隔预设时间，判断与所述视频云服务器连接是否断开，若断开，则自动重连所述视频云服务器。
根据本申请提供的视频流处理方法，包括：
对视频码流的解码抽帧任务执行视频帧的一对多任务配置，其中，所述一对多任务配置包括为所述视频码流下发多个AI分析任务。
本申请还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述任一种所述视频流处理方法。
本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如上述任一种所述视频流处理方法。
本申请还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上述任一种所述视频流处理方法。
本申请提供的移动视频AI平台及视频流处理方法，通过移动视频AI平台中的AI任务管理服务器、中心管理服务器、中间件和反代理节点均改成分布式集群，采用分布式集群的方式进行扩容，增强分布式架构管理能力，提高平台的扩容能力；以及新增AI能力池，实现动态接入AI能力，实现视频AI的并发管理。
附图说明
为了更清楚地说明本申请或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请提供的移动视频AI平台的系统架构示意图；
图2是本申请提供的移动视频AI平台中AI任务管理服务器的结构示意图；
图3是本申请提供的移动视频AI平台中AI能力池的结构示意图；
图4是本申请提供的移动视频AI平台的中间件结构示意图；
图5是本申请提供的视频流处理方法的流程示意图之一；
图6是本申请提供的视频流处理方法的流程示意图之二；
图7是本申请提供的视频流处理方法的流程示意图之三；
图8是本申请提供的视频流处理方法的流程示意图之四；
图9是本申请提供的电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
请参照图1，图1为本申请提出的一种移动视频AI平台，所述移动视频AI平台包括AI任务管理服务器、中心管理服务器、中间件和反代理节点，所述移动视频AI平台还包括AI能力池，所述AI能力池用于视频AI分析的并发管理；
中心管理服务器，用于实时同步各功能模块的运行状态以及负载情况实现动态调度，再结合任务性质，对新接入的任务执行熔断或者排队处理；
所述AI任务管理服务器、所述中心管理服务器、所述中间件和所述反代理节点均采用分布式集群的方式进行扩容。
其中，业务应用系统负责全部管理、查询及预览需求的下发和结果展示。中心管理服务器负责整个系统的数据和指令的管理，包括业务应用系统的需求的下发、向业务应用系统推送数据、业务相关信息的注册(设备信息、AI能力及AI任务等)、任务的下发、告警的接收，以及将所有数据存入数据库并进行查询。中心管理服务器通常和数据库关联，且只有中心管理服务器可以访问数据库。AI任务管理服务器集群主要负责AI任务的准备和衔接工作，包括AI任务所需的实时流传输协议(Real Time Streaming Protocol，RTSP)码流拉取、RTSP码流的解码抽帧或者截图、AI任务及巡检任务的配置、AI任务的数据交互、AI任务的结果和告警返回以及AI视频分析实时预览RTSP码流的编码和推流。因为AI视频分析服务器涉及到RTSP码流的编解码，对硬件资源消耗较大，系统对该模块采用分布式设计。此外，AI任务管理服务和AI能力池的图像数据交互是通过中间件的Redis模块进行的。AI能力服务器集群负责提供各类AI能力，每一个AI能力部署在独立的容器(docker) 里，并通过服务的方式对外提供能力响应。中间件主要负责平台内部的数据流转，这里包括数据库、消息队列模块、图像交互模块、告警图像存储模块等4个中间件。反代理节点集群主要是基于高性能的超文本传输协议(Hyper Text Transfer Protocol，HTTP)和反向代理web服务器(Nginx)实现，负责视频AI分析的实时预览RTSP码流转发。考虑到视频AI平台的受众为特点运维人员，因此这里设计实时预览最高支持100路并发。因此，反代理节点集群只有2个节点。此外，移动视频AI平台和视频云服务器之间通过专线传输RTSP码流数据。在平台内部，涉及到图像数据的传输及交互的流转过程也都需通过专线进行通信。
目前系统大多都是主要由业务应用系统(web、app等)、中心管理服务器、AI视频分析服务器及AI能力池等主要功能模块组成。其中中心服务管理、redis模块等为单节点部署，为系统瓶颈，造成系统承载较低，难以支撑后续大规模商用集成。
系统当前AI视频分析服务器涉及到RTSP码流的编解码，对硬件资源消耗较大，系统对该模块采用分布式设计。而AI能力需求由一台高性能AI服务器基本可以满足，且研发周期较为紧迫，故AI能力池未采用分布式设计。考虑到相关技术中结构的不足，对系统增强分布式架构管理能力，具体包括，对数据库和Redis模块等中间件、中心管理服务器、AI任务管理服务等采用分布式部署，提高系统高并发能力，对系统进行扩容以适应后续大规模接入。
本申请提供的移动视频AI平台，通过移动视频AI平台中的AI任务管理服务器、中心管理服务器、中间件和反代理节点均改成分布式集群，采用分布式集群的方式进行扩容，增强分布式架构管理能力，提高平台的扩容能力；以及新增AI能力池，实现动态接入AI能力，实现视频AI的并发管理。
进一步地，请参照图2，图2为本申请中的任务管理服务器的结构示意图，所述AI任务管理服务器包括1:N任务配置子模块、巡检任务管理子模块、状态监控子模块和节点信息管理子模块；
请参照图3，图3为本申请的AI能力池的结构示意图，所述AI能力池 包括能力开放平台、AI平台和AI能力服务器。
进一步地，请参照图4，图4为本申请的中间件的结构示意图，所述中间件包括数据库、图像数据交互模块和告警图片存储模块；
所述数据库、所述图像数据交互模块和所述告警图片存储模块采用分布式结构进行部署。
移动视频AI平台的系统架构的改造还包括：中间件包括数据库、消息队列模块、图像数据交互模块和告警图片存储模块，解决了业务健壮性不足的问题；对数据库、图像数据交互模块、告警图片存储模块等中间件的分布式设计。
中间件用于负责平台内部的数据流转，这里包括数据库、消息队列模块、图像交互模块、告警图像存储模块等4个中间件。其中，数据库基于关系型数据库管理系统(MySQL)搭建，消息队列模块利用MQ技术实现，图像交互模块使用Redis分布式，告警图像存储模块使用开源的轻量级分布式文件系统(fastDFS)，每一个中间件都采用分布式设计，并在任务状态发生变化时和中心管理服务同步状态和负荷信息，以避免系统超载，解决了系统承载力少的问题。
进一步地，所述AI任务管理服务器和所述中间件的Redis模块采用热备的方式部署；
所述AI任务管理服务器中的Master节点为所述AI任务管理服务器的管理节点，用于AI视频分析任务的调度。
当前系统缺少对瓶颈节点的有效管理。目前平台采用轻量级架构，不能支撑大规模商用，编解码和Redis将成为瓶颈，当瓶颈达到性能阈值后任务依旧被下发容易造成系统阻塞，影响其他上下游功能组件的正常开展，对于上述关键节点缺乏有效的管理机制。
AI任务管理服务器和中间件的Redis模块是系统重要组成部分，也是瓶颈所在，为解决原有系统容量小、缺少瓶颈节点有效管理等问题，将两模块采用热备的方式部署，提高系统稳定性。对AI任务管理服务器和redis集群 实现相互监听，并由后台实现系统层面的多活机制。
Master节点为AI任务管理服务器的管理节点，主要负责AI视频分析任务的调度。Master节点的功能包括管理任务信息、管理集群地址信息、管理RTSP转发地址、制定Nginx转发规则。AI任务管理服务器还包括Node节点，Node节点负责处理具体的AI视频分析任务，包括解析中心管理服务器下发的任务指令、并根据设备信息从视频云上拉取RTSP流、对RTSP流进行解码抽帧(或截图)、将解码后图片存入redis、将redis地址以参数形式请求到AI能力池并返回分析结果、推送告警信息到MQ队列、根据实时分析预览需求将分析结果标注到图片上并编码推流、以及对任务的关闭操作。
本实施例中通过AI任务管理服务器和中间件的Redis模块采用热备的方式部署，采用高可用架构，从而能支撑大规模商用。并且通过AI任务管理服务器中的Master节点对AI视频分析任务的调度，增加了管理机制，提升了系统的健壮性。
请参照图5，本申请还提出一种视频流处理方法，所述视频流处理方法应用于上述任一项所述的移动视频AI平台，所述视频流处理方法包括：
步骤10，对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；
步骤20，将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
相关技术中的系统还存在流媒体的断流问题，目前大多AI平台是从视频云平台拉取RTSP码流，但是由于视频云平台具备防盗播机制，当收到请求后一段时间没有被拉流的话，原有拉流通道将会关闭，造成AI平台拉流失败，影响后续任务的开展。
原视频AI平台的简易化设计，使得系统的主要流程都较为简洁，因此在一些扩展功能上都未能实现。本申请在系统功能及数据流转逻辑上进行增强，在完善系统功能树的基础上，也在效率和资源上实现大幅优化，主要表现包括：
加强对数据源的管理。增加对视频码流拉取过程进行状态监听，当视频码流拉取出现异常或失败时，对视频码流进行重新拉取。实现和视频码流断流重连机制。
对于对不同视频AI任务分别拉流的问题，改变任务配置策略，使得对同一路视频码流下发多种视频AI能力时，只实现一个视频编解码流程，并将数据按照不同策略同步给不同的视频AI能力，极大减少对资源的利用率。
在一种实施例中，请参照图6，基于本申请移动视频AI平台对应的视频流处理方法，还包括：
步骤30，通过keep alive技术，对所述AI任务管理器中的master节点进行模块状态的管理；
步骤40，对所述移动视频AI平台中的所有的模块和中间件实现状态监控和上报，并对异常情况进行处理。
通过keep alive技术防止主要模块的管理节点出现故障，提高系统的多活性，并对系统中所有的模块和中间件实现状态监控和上报，并对异常情况进行处理，有效防止系统超载。
本实施例中，对AI任务管理器中的master节点采用keep alive技术防止其出现故障，提高系统的多活性。
在一种实施例中，请参照图7，基于本申请移动视频AI平台对应的视频流处理方法，还包括：
步骤50，分别对实时码流方式和单帧截图方式进行配置熔断机制及排队机制；
步骤60，当下游服务的响应时长低于预设时长或所述下游服务响应失败，进行切断对下游服务的调用，并保持持续切断状态预设时长。
相关技术中的系统还存在缺乏熔断机制的问题，当前的系统功能模块A在达到性能阈值后往往会对正在运行的其他依赖服务A的程序产生影响。甚至，一旦服务A的阻塞过程耗时过长，问题会更严重，大量请求会阻塞，产生级联影响，导致整个系统卡慢，易造成系统宕机，影响业务开展。
在一种实施例中，请参照图8，基于本申请移动视频AI平台对应的视频流处理方法，还包括：
步骤70，在接收到拉流任务时，新建目标线程，其中，所述目标线程用于从视频云服务器中获取具有时效的防盗播视频码流；
步骤80，将所述防盗播视频码流进行解码后得到图片，并将所述图片存入数据库；
步骤90，将所述图片对应的AI任务指令下发至所述AI能力池，得到所述AI能力池反馈的AI分析结果；
其中，所述目标线程还用于相隔预设时间，判断与所述视频云服务器连接是否断开，若断开，则自动重连所述视频云服务器。
相关技术中的系统还存在流媒体的断流问题，目前大多AI平台是从视频云平台拉取RTSP码流，但是由于视频云平台具备防盗播机制，当收到请求后一段时间没有被拉流的话，原有拉流通道将会关闭，造成AI平台拉流失败，影响后续任务的开展。
为了便于对断流重连机制的理解，下述进行举例说明，在拉流的过程中，master节点将任务下发至新增节点(new node)节点，new node新建线程1进行拉流，即通过设备信息从视频云拉取只有10s时效的防盗播的RTSP码流，线程1将解码后图片存入redis，并将redis地址作为参数连同AI任务指令下发到AI能力池，并接收返回的AI分析结果(含告警信息)，为避免超过10s拉取RTSP码流导致断流的问题，新增断流重连机制，在一个线程中，每隔一定时间，判断如果连接断开，就自动重新连接一次视频云服务器。防盗播生效的时候，我们不立即去做重连接，而是延迟1s-2s再去做重连接。这是因为，我们需要给视频云服务重启一些时间，不然重连接可能不会成功，或者数据不能够同步。
在一种实施例中，基于本申请移动视频AI平台对应的视频流处理方法，还包括：
对视频码流的解码抽帧任务执行视频帧的一对多任务配置，其中，所述 一对多任务配置包括为所述视频码流下发多个AI分析任务。
系统AI视频分析服务器主要负责AI任务的准备和衔接工作，包括AI任务所需的RTSP码流拉取、RTSP码流的解码抽帧或者截图、AI任务的数据交互、AI任务的结果和告警返回以及AI视频分析实时预览RTSP码流的编码和推流。每个任务编排功能组件，独占一个线程，但是不同任务配置策略不一，在部分节点到达性能瓶颈时另外节点可能处于空闲状态，造成了流媒体接入部分的资源浪费。因此当前的系统还存在任务配置策略单一，导致流媒体接入资源浪费的问题。
相关技术中系统线程根据AI任务传入的抽帧间隔参数选择解码抽帧的方式，大于250帧执行截图操作，小于等于250帧执行解码抽帧操作，但是不同任务可能设置的抽帧间隔参数不一样，导致了同一视频帧的多次抽取，而通过视频帧一对多任务配置功能,可以降低对视频流的重复拉取。
基于本申请提出的移动视频AI平台实现下述的视频流分析流程，包括：
①首先，移动视频AI平台的所有任务指令都由业务应用系统发起，并中心管理服务创建新的AI能力分析任务，传达给AI任务管理服务。这里可以在业务应用系统下发对指定RTSP码流/视频码流进行单一AI分析任务(这里称之为1:1)，也可以下发对指定RTSP码流/视频码流进行多个AI分析任务(这里称之为1:N),也可以根据需要下发定时任务等巡检任务；
②AI任务管理服务接受到中心管理服务下发的AI分析任务后，不管时1:1的任务，还是1:N的任务，又或者是巡检任务，都将从视频云获取设备ID等信息，并在任务启动后从视频云拉取RTSP码流。这里如果是1:N的任务，N个任务只会拉取一路RTSP码流；
③AI任务管理服务创建新的任务节点、选择解码抽帧的方式，将从视频云拉取的视频码流进行解码，并根据该路码流的AI分析任务数量，向Redis节点存储不同数量的图像数据，最后汇总任务信息，通过能力开放平台和AI平台将任务下发到AI能力池；
④AI能力池根据AI任务管理服务下达的任务以及解码抽帧的方式进行 AI分析，从Redis节点中读取图像数据，并进行AI分析，将结果、告警信息分别返回AI任务管理服务及Redis存储节点；
⑤如果中心管理服务下发的任务包含视频AI分析实时预览需求，则可利用AI能力池返回的告警信息对图像编码，并将存储的RTSP转发地址传输到ngix服务节点，web端会根据这个RTSP转发地址去Nginx上拉取RTSP码流；
⑥AI任务管理服务在对RTSP码流进行解码的时候，会保持和Node节点的解码线程的心跳，当发生RTSP码流拉取失败的时候，上报断流信息，并启动断流重连功能；
⑦中心管理服务在每次任务下发或执行时，都会同步每个功能模块的运行状态信息及负荷情况，作为判断整个高性能AI平台是否达到负荷红线；
⑧当出现某个功能模块的负荷过高，由中心管理服务根据不同的任务性质，对新接入的任务实施熔断或者排队处理。
图9示例了一种电子设备的实体结构示意图，如图9所示，该电子设备可以包括：处理器(processor)910、通信接口(Communications Interface)920、存储器(memory)930和通信总线940，其中，处理器910，通信接口920，存储器930通过通信总线940完成相互间的通信。处理器910可以调用存储器930中的逻辑指令，以执行视频流处理方法，该方法包括：对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
此外，上述的存储器930中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方 法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
另一方面，本申请还提供一种计算机程序产品，所述计算机程序产品包括计算机程序，计算机程序可存储在非暂态计算机可读存储介质上，所述计算机程序被处理器执行时，计算机能够执行上述各方法所提供的视频流处理方法，该方法包括：对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
又一方面，本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行上述各方法提供的视频流处理方法，该方法包括：对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施 例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (12)

  1. 一种移动视频AI平台，所述移动视频AI平台包括AI任务管理服务器、中心管理服务器、中间件和反代理节点，所述移动视频AI平台还包括AI能力池，所述AI能力池用于视频AI分析的并发管理；
    中心管理服务器，用于实时同步各功能模块的运行状态以及负载情况实现动态调度，再结合任务性质，对新接入的任务执行熔断或者排队处理；
    所述AI任务管理服务器、所述中心管理服务器、所述中间件和所述反代理节点均采用分布式集群的方式进行扩容。
  2. 根据权利要求1所述的移动视频AI平台，其中，所述AI任务管理服务器包括1:N任务配置子模块、巡检任务管理子模块、状态监控子模块和节点信息管理子模块；
    所述AI能力池包括能力开放平台、AI平台和AI能力服务器。
  3. 根据权利要求1所述的移动视频AI平台，其中，所述中间件包括数据库、消息队列模块、图像数据交互模块和告警图片存储模块；
    所述数据库、所述消息队列模块、所述图像数据交互模块和所述告警图片存储模块采用分布式结构进行部署。
  4. 根据权利要求1所述的移动视频AI平台，其中，所述AI任务管理服务器和所述中间件的Redis模块采用热备的方式部署；
    所述AI任务管理服务器中的Master节点为所述AI任务管理服务器的管理节点，用于AI视频分析任务的调度。
  5. 一种视频流处理方法，所述视频流处理方法应用于1至4任一项的移动视频AI平台，所述视频流处理方法包括：
    对视频码流拉取过程进行状态监听，以在视频码流拉取出现异常或失败时，对视频码流进行重新拉取；
    将同一路视频码流分配至单个视频编解码流程，并将编解码后的视频数据同步至不同的视频AI能力。
  6. 根据权利要求5所述的视频流处理方法，其中，包括：
    通过keep alive技术，对所述AI任务管理器中的master节点进行模块状态的管理；
    对所述移动视频AI平台中的所有的模块和中间件实现状态监控和上报，并对异常情况进行处理。
  7. 根据权利要求5所述的视频流处理方法，其中，包括：
    分别对实时码流方式和单帧截图方式进行配置熔断机制及排队机制；
    当下游服务的响应时长低于预设时长或所述下游服务响应失败，进行切断对下游服务的调用，并保持持续切断状态预设时长。
  8. 根据权利要求5所述的视频流处理方法，其中，包括：
    在接收到拉流任务时，新建目标线程，其中，所述目标线程用于从视频云服务器中获取具有时效的防盗播视频码流；
    将所述防盗播视频码流进行解码后得到图片，并将所述图片存入数据库；
    将所述图片对应的AI任务指令下发至AI能力池，得到所述AI能力池反馈的AI分析结果；
    其中，所述目标线程还用于相隔预设时间，判断与所述视频云服务器连接是否断开，若断开，则自动重连所述视频云服务器。
  9. 根据权利要求5所述的视频流处理方法，其中，包括：
    对视频码流的解码抽帧任务执行视频帧的一对多任务配置，其中，所述一对多任务配置包括为所述视频码流下发多个AI分析任务。
  10. 一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其中，所述处理器执行所述程序时实现如权利要求5至9任一项所述视频流处理方法。
  11. 一种非暂态计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求5至9任一项所述视频流处理方法。
  12. 一种计算机程序产品，包括计算机程序，其中，所述计算机程序被 处理器执行时实现如权利要求5至9任一项所述视频流处理方法。
PCT/CN2022/136890 2022-05-05 2022-12-06 移动视频ai平台及视频流处理方法 WO2023213073A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210483462.8A CN117061505A (zh) 2022-05-05 2022-05-05 移动视频ai平台及视频流处理方法
CN202210483462.8 2022-05-05

Publications (1)

Publication Number Publication Date
WO2023213073A1 true WO2023213073A1 (zh) 2023-11-09

Family

ID=88646190

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/136890 WO2023213073A1 (zh) 2022-05-05 2022-12-06 移动视频ai平台及视频流处理方法

Country Status (2)

Country Link
CN (1) CN117061505A (zh)
WO (1) WO2023213073A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626430A (zh) * 2019-04-18 2020-09-04 中科寒武纪科技股份有限公司 一种数据处理方法及相关产品
US20210377310A1 (en) * 2020-05-29 2021-12-02 Shoptaki Inc. Method, system, apparatus and program for secure distributed data management using collaborative artificial intelligence
CN113869125A (zh) * 2021-08-30 2021-12-31 浪潮软件科技有限公司 一种视频流的分析处理方法、装置、设备及可读存储介质
CN114255432A (zh) * 2021-12-24 2022-03-29 中电信数智科技有限公司 视频流处理方法、装置、电子设备、存储介质及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626430A (zh) * 2019-04-18 2020-09-04 中科寒武纪科技股份有限公司 一种数据处理方法及相关产品
US20210377310A1 (en) * 2020-05-29 2021-12-02 Shoptaki Inc. Method, system, apparatus and program for secure distributed data management using collaborative artificial intelligence
CN113869125A (zh) * 2021-08-30 2021-12-31 浪潮软件科技有限公司 一种视频流的分析处理方法、装置、设备及可读存储介质
CN114255432A (zh) * 2021-12-24 2022-03-29 中电信数智科技有限公司 视频流处理方法、装置、电子设备、存储介质及系统

Also Published As

Publication number Publication date
CN117061505A (zh) 2023-11-14

Similar Documents

Publication Publication Date Title
CN109327509B (zh) 一种主/从架构的低耦合的分布式流式计算系统
CN110769278B (zh) 一种分布式视频转码方法及系统
WO2016197864A1 (zh) 连接状态检测方法、装置和智能电视系统
US20230300203A1 (en) Data processing method and system, related device, storage medium, and product
CN109936593B (zh) 一种消息分发的方法和系统
CN109672856A (zh) 资源同步方法和装置
CN105357273A (zh) 异步通信模式下socket通信与进程管理通用平台及方法
CN112804289B (zh) 一种资源同步方法、装置、设备及存储介质
CN113630438B (zh) 流处理任务调度方法和分布式流处理系统
WO2016202201A1 (zh) 一种资源分配方法、装置和智能电视系统
WO2017185615A1 (zh) 一种业务处理设备的业务状态确定方法及调度设备
CN111432025A (zh) 一种面向云边协同的分布式服务目录管理方法及系统
CN108737566B (zh) 一种分布式的实时消息过滤系统
CN112954392A (zh) 一种自动实现视频格式转换处理的平台
CN116319790A (zh) 全真场景的渲染方法、装置、设备和存储介质
KR20220077899A (ko) 오디오 및 비디오 처리 방법, 장치, 전자 기기, 저장 매체 및 프로그램
CN109388501B (zh) 基于人脸识别请求的通信匹配方法、装置、设备及介质
WO2023213073A1 (zh) 移动视频ai平台及视频流处理方法
CN103139607B (zh) 一种实时网络监控系统及其方法
CN113676750A (zh) 一种流媒体服务器集群管理系统
CN112543374A (zh) 一种转码控制方法、装置及电子设备
CN104702663B (zh) 云处理装置和方法
CN110519397B (zh) 一种基于nginx的sip终端接入的负载均衡系统及方法
CN110365631B (zh) 一种数据处理方法和视联网系统
CN106850283B (zh) 一种基于事件驱动的云ac告警处理系统及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22940767

Country of ref document: EP

Kind code of ref document: A1