WO2021196559A1 - 一种视频处理系统 - Google Patents

一种视频处理系统 Download PDF

Info

Publication number
WO2021196559A1
WO2021196559A1 PCT/CN2020/120304 CN2020120304W WO2021196559A1 WO 2021196559 A1 WO2021196559 A1 WO 2021196559A1 CN 2020120304 W CN2020120304 W CN 2020120304W WO 2021196559 A1 WO2021196559 A1 WO 2021196559A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
terminal
server
video frame
executable program
Prior art date
Application number
PCT/CN2020/120304
Other languages
English (en)
French (fr)
Inventor
吕胜伟
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2021196559A1 publication Critical patent/WO2021196559A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion

Definitions

  • This application relates to the technical field of video processing, and in particular to a video processing system.
  • the terminal is usually replaced with a device with higher computing power.
  • each corresponding server is deployed in each scene. After the terminal collects the video, it uploads the video to its corresponding server, and the server completes the video processing operation.
  • each scene needs to deploy a set of video processing systems.
  • the video processing systems of each scene are independent of each other, and the computing resources cannot be coordinated and allocated among the scenes, resulting in waste of resources .
  • This application provides a video processing system.
  • the terminal sends the storage capacity, processing power and executable program required for video processing to the server, and the server allocates the corresponding storage space, and executes the executable program with the above processing power.
  • the video of the terminal does not need to update the hardware of the terminal, and the computing power can be automatically expanded by adjusting the processing power through the server.
  • the embodiment of the application provides a video processing system, including a terminal and a server;
  • the terminal is configured to send the storage capacity, processing power, and executable program for processing the video required for processing the video to the server; and send the video frame to be processed to the server;
  • the server is configured to allocate the storage space of the storage capacity for the terminal, and store the processing computing power and the executable program in the storage space; and upon receiving the pending processing from the terminal
  • the processor loads the executable program from the storage space, inputs the to-be-processed video frame into the executable program, and executes the executable program with the processing power to treat the to-be-processed program. Process video frames for processing.
  • the terminal is also used to send new storage capacity and new processing power to the server when the storage capacity and processing power required for video processing change;
  • the server is further configured to adjust the storage space according to the new storage capacity, and store the new processing power and the executable program in the adjusted storage space.
  • the terminal is further configured to send a new version of the executable program to the server when the executable program version is upgraded;
  • the server is also used to update the executable program stored in the storage space to the new version of the executable program.
  • the terminal is also used to send video frames that need to be stored to the server;
  • the server is further configured to store the video frame in the storage space, and send the first storage address storing the video frame to the terminal.
  • the terminal is further configured to send a video decoding request to the server, where the video decoding request includes the first storage address and encoding type;
  • the server is configured to read the video frame stored at the first storage address from the storage space; and use the decoder corresponding to the encoding type to decode the read video frame, and decode the decoded video frame.
  • the video frame of is stored in the storage space, and the second storage address storing the decoded video frame is sent to the terminal.
  • the terminal is further configured to send a video analysis request to the server, where the video analysis request includes the second storage address;
  • the server is further configured to read the video frame stored at the second storage address from the storage space, load the executable program from the storage space through a graphics processor GPU, and load all the read video frames from the storage space.
  • the video frame is input to the executable program, and the executable program is executed with the processing power to analyze the video frame; the analysis result is stored in the storage space, and the first analysis result is stored 3. Send the storage address to the terminal.
  • the terminal is further configured to send a video encoding request to the server, where the video encoding request includes the second storage address and encoding type;
  • the server is further configured to read the video frame stored at the second storage address from the storage space; use the encoder corresponding to the encoding type to encode the read video frame; and encode The subsequent video frame is stored in the storage space, and the fourth storage address storing the encoded video frame is sent to the terminal.
  • the terminal is further configured to receive the second storage address returned by the server, and insert the second storage address into the end of the queue to be analyzed; and, according to the The video frame rate of the video collected by the terminal and the preset unit analysis duration of the video analyzed by the server are calculated to calculate the first minimum interval frame number; in the collected video data, the minimum dwell time of the target within the preset duration is counted; according to the minimum Calculate the second minimum interval frame number based on the dwell time and the video frame rate; according to the first minimum interval frame number, the second minimum interval frame number, and the number of second storage addresses included in the queue to be analyzed , Selecting the second storage address corresponding to the video frame to be analyzed from the queue to be analyzed.
  • the embodiment of the present application decouples the executable program for processing video from the hardware on the terminal side, and relies on the processor on the server side to execute the executable program for video processing, thereby achieving the purpose of elastic expansion of processing computing power.
  • the terminal does not need to be equipped with hardware with high processing power, and there is no need to be equipped with a storage device with large storage capacity.
  • the terminal sends processing power, storage space and executable programs for video processing to the server, and the server is the terminal Allocate storage space and execute executable programs with the processing power to process the video of the terminal.
  • this application does not need to update the terminal hardware.
  • the computing power can be automatically expanded, and the hardware resources of the server can be used efficiently, and it can be implemented in different video processing Reasonable deployment of computing resources between scenarios reduces the waste of computing resources and facilitates the expansion of new application scenarios.
  • Fig. 1 shows a schematic diagram of a video processing system provided by an embodiment of the present application
  • Fig. 2 shows a flowchart of a video processing method provided by an embodiment of the present application.
  • the network architecture of the system is shown in FIG. 1.
  • the video processing system includes a server and at least one terminal, and each terminal is connected to the server through a network.
  • the terminal may be a device capable of collecting video, such as a camera, a camera, and so on.
  • the server may be a cloud GPU (Graphics Processing Unit, graphics processor) service platform, and the server may also be a server cluster composed of multiple server devices. Only three terminals A, B, and C are shown schematically in Figure 1.
  • a video processing system provided by an embodiment of the present application includes a terminal and a server, where:
  • the terminal is used to send the storage capacity, processing power, and executable program for processing the video required for processing the video to the server; and send the video frame to be processed to the server;
  • the server is configured to allocate the storage space of the storage capacity for the terminal, and store the processing power and the executable program in the storage space; and upon receiving the to-be-processed video frame from the terminal When the processor loads the executable program from the storage space, inputs the to-be-processed video frame into the executable program, and executes the executable program with the processing power to control the to-be-processed video Frame processing.
  • the terminal sends the storage capacity, processing power and executable program required for video processing to the server, and the server allocates the corresponding storage space for the terminal, and sends the corresponding processing power and power of the terminal to the server.
  • the execution program is stored in this storage space.
  • the server receives the to-be-processed video frame sent by the terminal, the server loads the executable program from the storage space through the processor, inputs the to-be-processed video frame into the executable program, and executes the executable program with the aforementioned processing power. Process video frames for processing. In this way, the terminal does not need to be equipped with hardware with high processing power.
  • the processing power required by the terminal for video processing is upgraded, there is no need to update the hardware on the terminal side to increase the processing power.
  • the terminal only needs to send the processing power required after the upgrade to the server, and the processing power can be expanded, and the hardware resources of the server can be efficiently used with low cost.
  • the processing power can be expanded through the server through the method provided in the embodiments of this application, so as to realize the reasonable allocation of computing resources between different video processing scenarios and reduce the waste of computing resources. , To facilitate the expansion of new application scenarios.
  • the present invention provides a video processing method based on the above-mentioned video processing system, which specifically includes the following steps:
  • Step 101 The terminal sends the storage capacity required for processing the video, the processing power and the executable program for processing the video to the server.
  • the executable program required for processing the video can be stored locally on the terminal, and the storage capacity and processing power required for processing the video can be set on the terminal.
  • storage capacity is the size of storage capacity.
  • the terminal connects to the server through the network, obtains its own device identification, and obtains the locally stored storage capacity, processing power and executable program, and sends the device identification and the obtained storage capacity, processing power and executable program to the server.
  • the executable program can be set according to the actual needs of the video processing scene.
  • the corresponding relationship between the executable program identifier and the processing that the executable program can perform on the video can be recorded, and then the When the video needs to be processed, the executable program identifier is determined according to the corresponding relationship, and the executable program corresponding to the executable program identifier is sent to the server.
  • the terminal may be a camera used to capture faces, and the executable program may be a program used to perform face recognition.
  • the storage capacity required by the terminal for face recognition processing is 50G
  • the required processing power is 2TOPS (Tera Operations Per Second, the processor performs one trillion operations per second)
  • the terminal sends a storage capacity of 50G and processes
  • the computing power 2T and the executable program for face recognition are given to the server.
  • the terminal connects to the server through the network, and uploads the storage capacity, processing power, and executable program required to process the video to the server, so that the executable program used to process the video and the hardware on the terminal side are different.
  • the demand for computing power in application scenarios increases, there is no need to replace the hardware on the terminal side.
  • the simple configuration on the server side achieves the purpose of automatic expansion of processing power, with low cost and flexible and convenient expansion of processing power.
  • Step 102 The server receives the storage capacity, the processing power and the executable program uploaded by the terminal, allocates the storage space of the storage capacity to the terminal, and stores the processing power and the executable program in the storage space.
  • the server receives the storage capacity, processing power and executable program uploaded by the terminal, the server also receives the device identification uploaded by the terminal, allocates storage space of the storage capacity for the terminal, and stores the received processing power and executable program in the distribution In the storage space of the terminal, a mapping relationship between the device identifier of the terminal and the storage space is established.
  • the terminal can send the device identification and the new storage capacity to the server.
  • the server receives the device identification and new storage capacity sent by the terminal, and obtains processing power, executable programs, and historical data of video processing from the storage space previously allocated for the terminal according to the device identification, that is, obtains the previously allocated storage space All data stored.
  • the server allocates the storage space of the new storage capacity for the terminal, stores the acquired processing power, executable programs, and historical data of processed videos in the newly allocated storage space, and establishes the terminal’s device identification and the new storage space.
  • the mapping relationship between the allocated storage space is the mapping relationship between the allocated storage space.
  • the server After the server receives the device identification and the new storage capacity sent by the terminal, it can also query the storage capacity of the storage space previously allocated to the terminal according to the device identification, and compare the queried storage capacity with the size of the new storage capacity. If the capacity is greater than the new storage capacity, the difference between the queried storage capacity and the new storage capacity is calculated, and the storage space with the capacity of the difference is released from the storage space previously allocated for the terminal, and the remaining storage space The capacity is the new storage capacity mentioned above.
  • the difference between the new storage capacity and the queried storage capacity is calculated, the storage space with the capacity as the difference is allocated, and the storage space with the capacity as the difference is established and
  • the mapping relationship between the device identifiers of the terminal, and the sum of the storage space whose capacity is the difference and the storage capacity of the storage space previously allocated for the terminal is the aforementioned new storage capacity.
  • the terminal side does not need to be equipped with a storage device with a large storage capacity. Low, convenient and flexible to adjust storage capacity.
  • the terminal can send the device identification and the new processing power to the server.
  • the server receives the device identification and the new processing power sent by the terminal, and updates the processing power stored in the storage space corresponding to the device identification to the new processing power, so that the terminal can be subsequently processed according to the new processing power.
  • the pending video frames are processed.
  • the terminal can send the device identification and the new version of the executable program to the server.
  • the server receives the device identification and the new version of the executable program sent by the terminal, and updates the executable program stored in the storage space corresponding to the device identification to the new version of the executable program.
  • the terminal sends a new version of the executable program to the server to realize the program upgrade.
  • any terminal can access the server through the above operations, and the server allocates the storage space corresponding to the terminal, and stores the processing power and executable programs required by the terminal to process the video in the storage space corresponding to the terminal.
  • the storage space corresponding to the terminal is configured in the server, and the processing power and executable programs required by the terminal to process the video are stored in the storage space. Then, when the terminal needs to process the video, pass The following steps 103 and 104 are used for video processing.
  • Step 103 The terminal sends the to-be-processed video frame to the server.
  • Step 104 The server receives the to-be-processed video frame from the terminal, loads the executable program from the storage space corresponding to the terminal through the processor, inputs the to-be-processed video frame into the executable program, and executes the executable program to be processed with the aforementioned processing power Video frames are processed.
  • the server can provide at least four basic services: cloud storage service, decoding service, encoding service, and algorithm processing service.
  • the cloud storage service is used to store the video frames of the terminal.
  • the decoding service is used to decode and restore video frames encoded by encoding protocols such as H264 or H265 to the original video frames.
  • Encoding service is used to encode video frames using encoding protocols such as H264 or H265.
  • the algorithm processing service is used to analyze and process the video frame using the executable program required to process the video sent by the terminal, such as the recognition process of the video frame through the executable program for face recognition.
  • the above terminal may also be used to send the video frame that needs to be stored to the server; the above server may also be used to store the video frame in the storage space, and store the first storage address of the video frame Sent to the terminal.
  • the terminal can send the device identification and the video frame to be stored to the server.
  • the server receives the device identification sent by the terminal and the video frame that needs to be stored, stores the video frame in the storage space corresponding to the device identification of the terminal, and sends the first storage address storing the video frame to the terminal.
  • the server allocates the storage space corresponding to the device identification of the terminal in step 102
  • the storage space corresponding to the device identification is mapped to the terminal locally, so that a mapping directory corresponding to the storage space is generated locally in the terminal.
  • the terminal needs to store the video, it directly copies the video frame to be stored to the local mapping directory.
  • the server obtains the video frames copied by the terminal to the local mapping directory.
  • the terminal may also determine the storage address for storing the video frame in the mapping directory as the first storage address, and save the first storage address.
  • the terminal when the video frame that needs to be stored is a video frame to be decoded, the terminal stores the video frame that needs to be stored in the storage space corresponding to the terminal in the server through any of the above methods, and obtains the storage of the video. After the first storage address of the frame, the first storage address can be inserted into the end of the queue to be decoded. Wherein, the queue to be decoded is used to store the storage address of the video frame to be decoded.
  • the server sends failure indication information to the terminal, and the failure indication information includes an error code for indicating storage failure.
  • the terminal can send the video frame that needs to be stored to the server again, and record the number of uploads corresponding to the video frame. If the number of uploads of the video frame reaches the preset upper limit number of times, the video frame is discarded, and the video frame is no longer uploaded to the server for storage.
  • the foregoing terminal may also be used to send a video decoding request to the server, where the video decoding request includes the first storage address and encoding type; the foregoing server may also be used to read the first storage space from the storage space.
  • the second storage address of the subsequent video frame is sent to the terminal.
  • the terminal sends a video decoding request to the server, and the video decoding request includes the first storage address and the encoding type. Specifically, the terminal obtains one or more first storage addresses from the head of the queue to be decoded, and sends a video decoding request to the server.
  • the video decoding request carries the obtained one or more first storage addresses and carries each first storage address.
  • the encoding type corresponding to the video frame stored in a storage address. Among them, the encoding type is used to indicate the encoding protocol corresponding to which encoding type the video frame needs to be decoded is encoded, and the encoding type can be H264 or H265, etc.
  • the server receives the video decoding request sent by the terminal; according to the first storage address in the video decoding request, obtains the video frame to be decoded from the storage space corresponding to the terminal; obtains the video frame corresponding to the encoding type included in the video decoding request The video frame is decoded; the decoded video frame is stored in the storage space corresponding to the terminal, and the second storage address storing the decoded video frame is sent to the terminal.
  • the terminal when the video frame to be decoded is the video frame to be analyzed, the terminal may also be used to receive the second storage address returned by the server, and insert the second storage address into the end of the queue to be analyzed.
  • the terminal After the server sends the second storage address storing the decoded video frame to the terminal, the terminal receives the second storage address returned by the server, and can insert the second storage address at the end of the queue to be analyzed.
  • the queue to be analyzed is used to store the storage address of the video frame to be analyzed.
  • the server sends failure indication information to the terminal, and the failure indication information includes an error code for indicating decoding failure.
  • the terminal may send a video decoding request corresponding to the video frame that needs to be decoded to the server again, and record the number of decoding requests corresponding to the video frame. If the number of decoding requests corresponding to the video frame reaches the preset upper limit number of times, the video frame is discarded, and the video frame is no longer requested to be decoded.
  • the above terminal may also be used to send a video analysis request to the server, where the video analysis request includes the second storage address; the above server may also be used to read the second storage address from the storage space Load the executable program from the storage space through the graphics processor GPU, input the read video frame into the executable program, and execute the executable program with the processing power
  • the program analyzes the video frame; stores the analysis result in the storage space, and sends a third storage address storing the analysis result to the terminal.
  • the terminal obtains the second storage address from the head of the queue to be analyzed, and sends a video analysis request to the server.
  • the video analysis request includes the device identification of the terminal and the obtained second storage address.
  • the server receives the video analysis request sent by the terminal, reads the video frame stored at the second storage address from the storage space corresponding to the device identifier, and obtains the video frame from the storage space corresponding to the device identifier through GPU (Graphics Processing Unit) Load the executable program, input the read video frame into the executable program, use the processing power stored in the storage space corresponding to the device identifier, execute the executable program to analyze the read video frames, and store the analysis results In the storage space corresponding to the device identifier, the third storage address storing the analysis result is sent to the terminal.
  • the executable program is an executable program used to analyze video frames to implement algorithm processing services.
  • the terminal After receiving the third storage address returned by the server, the terminal can download the corresponding analysis result from the server according to the third storage address.
  • the terminal can calculate the first minimum interval frame number by the following formula (1) according to the video frame rate of the video collected by the terminal and the preset unit analysis duration of the video analyzed by the server; in the collected video data, the statistical target is in the preset duration According to the minimum stay time and the video frame rate of the video collected by the terminal, the second minimum interval frame number is calculated by the following formula (2); according to the first minimum interval frame number, the second minimum interval frame number and to be analyzed For the number of second storage addresses included in the queue, the second storage address corresponding to the video frame to be analyzed is selected from the queue to be analyzed according to the following formula (3).
  • a is the first minimum interval frame number
  • b is the second minimum interval frame number
  • f is the video frame rate
  • ⁇ t is the analysis duration of the preset unit
  • T 1 is the minimum stay time
  • c is the number of interval frames
  • d is the strategy determination coefficient
  • n is the number of video frames in the queue to be analyzed
  • INT(a) is rounded to a
  • INT(b) is rounded to b.
  • the above-mentioned first minimum interval frame number means that at least every first minimum interval frame number is analyzed at least once to avoid data congestion.
  • the second minimum interval frame number means that at least once every second minimum interval frame number is analyzed, the target will not be missed.
  • the aforementioned preset unit analysis time length is the time required for the server to analyze one video frame.
  • the minimum stay time is the total time the target appears in the video collected by the terminal within the preset time period, and the initial value of the minimum stay time can be 30ms or 40ms, etc.
  • the preset duration can be 1 hour or 2 hours, etc., such as the total duration of the target face appearing in a 1-hour video in a face recognition scene.
  • the strategy determination coefficient d determines whether the strategy determination coefficient d is greater than the preset threshold, and if so, discard it from the queue to be analyzed 1/2 of the second storage address.
  • one second storage address can be eliminated every second storage address, so that one frame can be eliminated by analyzing one frame, which reduces the number of video frames that need to be analyzed, saves server resources, and improves the real-time performance of video analysis and processing.
  • the strategy determination coefficient d is less than or equal to the preset threshold value, a second storage address is acquired in the queue to be analyzed every c second storage addresses of the number of frames at intervals, and the video frame corresponding to the second storage address is analyzed. This reduces the number of video frames that need to be analyzed, saves server resources, and improves the real-time performance of video analysis and processing.
  • the aforementioned preset threshold may be 2 or 3, etc.
  • the above terminal may also be used to send a video encoding request to the server, where the video encoding request includes the second storage address and encoding type; the above server may also be used to read the first video from the storage space 2.
  • the fourth storage address of the subsequent video frame is sent to the terminal.
  • the terminal obtains the second storage address from the head of the queue to be analyzed, and sends a video coding request to the server.
  • the video coding request includes the coding type and the obtained second storage address.
  • the server receives the video encoding request sent by the terminal, reads the video frame stored at the second storage address from the storage space corresponding to the terminal, and uses the encoder corresponding to the encoding type included in the video encoding request to encode the read video frame Processing, storing the encoded video frame in the storage space corresponding to the terminal, and sending the fourth storage address storing the encoded video frame to the terminal.
  • the terminal receives the fourth storage address returned by the server, and can download the encoded video frame from the server according to the fourth storage address.
  • the server sends failure indication information to the terminal, and the failure indication information includes an error code for indicating encoding failure.
  • the terminal may send a video encoding request corresponding to the video frame to be encoded to the server again, and record the number of encoding requests. If the number of encoding requests reaches the preset upper limit number of times, the video frame is discarded, and the video frame is no longer requested to be encoded.
  • the terminal can also select the second storage address that needs to be encoded according to the formulas (1), (2), and (3) described in the above algorithm processing service. The specific selection process is not described in detail here.
  • the terminal may also send a deletion request to the server, and the deletion request may include the device identification and the first storage address.
  • the server deletes the video frame stored at the first storage address from the storage space corresponding to the device identifier.
  • the deletion request may also include the device identification and the second storage address. Accordingly, after receiving the deletion request, the server deletes the decoded video frame stored at the second storage address from the storage space corresponding to the device identification.
  • the deletion request may also include the device identification and the third storage address. Accordingly, after receiving the deletion request, the server deletes the analysis result stored at the third storage address from the storage space corresponding to the device identification.
  • the deletion request may also include the device identification and the fourth storage address. Accordingly, after receiving the deletion request, the server deletes the encoded video frame stored at the fourth storage address from the storage space corresponding to the device identification.
  • the processing power, executable programs, and various data generated by processing videos stored in the storage space corresponding to the terminal can be deleted in the server, and the terminal corresponding to the data can be released.
  • Storage space Since the storage space and processing power required to process the video of the terminal are allocated by the server according to the needs of the terminal and can be configured at will, the expansion of the processing power is more flexible and convenient.
  • the embodiment of the present application decouples the executable program for processing video from the hardware on the terminal side, and relies on the processor on the server side to execute the executable program for video processing, achieving the purpose of elastic expansion of processing computing power.
  • the terminal does not need to be equipped with hardware with high processing power, and there is no need to be equipped with a storage device with large storage capacity.
  • the terminal sends processing power, storage space and executable programs for video processing to the server, and the server is the terminal Allocate storage space and execute executable programs with the processing power to process the video of the terminal.
  • this application does not need to update the hardware of the terminal.
  • the automatic expansion of the computing power can be achieved by configuring the server, and the hardware resources of the server are used efficiently, so that the hardware of the terminal is not updated. In the case of flexible expansion of processing power.
  • modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and/or processes or units are mutually exclusive, any combination can be used to compare all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or methods disclosed in this manner or All the processes or units of the equipment are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
  • the various component embodiments of the present application may be implemented by hardware, or by software modules running on one or more processors, or by a combination of them.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation apparatus according to the embodiments of the present application.
  • This application can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein.
  • Such a program for implementing the present application may be stored on a computer-readable medium, or may have the form of one or more signals.
  • Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请公开一种视频处理系统，该系统包括：终端，用于将处理视频所需的存储容量、处理算力和用于处理视频的可执行程序发送给服务器；将待处理视频帧发送给服务器；服务器，用于为终端分配上述存储容量的存储空间，并将处理算力和可执行程序存储到该存储空间中；在接收到来自终端的待处理视频帧时，通过处理器从存储空间中加载可执行程序，将待处理视频帧输入可执行程序，以处理算力执行可执行程序对待处理视频帧进行处理。本申请不需要更新终端的硬件，通过对服务器进行配置即可达到算力自动扩展，高效利用服务器的硬件资源。且能实现在不同的视频处理场景之间进行运算资源的合理调配，减少运算资源的浪费，便于扩展新的应用场景。

Description

一种视频处理系统
本申请要求于2020年4月3日提交中国专利局、申请号为202010260779.6发明名称为“一种视频处理系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，具体涉及一种视频处理系统。
背景技术
随着视频处理技术的发展，不同场景下视频处理的运算量都急剧增加，对视频处理系统的计算能力要求越来越高。例如，人脸识别以及智能交通等场景的运算量都非常大，而且实时性要求也非常高，对视频处理系统的计算能力要求很高。
目前，为了满足对视频处理的计算能力越来越高的需求，通常将终端替换为具有更高运算能力的设备。或者，各个场景中均部署各自对应的服务器，终端采集视频后，将视频上传其对应的服务器，由服务器完成视频处理运算。
但通过更换终端来提供更高的计算能力，成本很高。而由服务器完成其对应场景下的视频处理运算的方式，每个场景都需部署一套视频处理系统，各场景的视频处理系统相互独立，运算资源无法在各场景之间协调分配，造成资源浪费。
发明内容
本申请提供一种视频处理系统，终端将处理视频所需的存储容量、处理算力和可执行程序发送给服务器，由服务器分配相应的存储空间，并以上述处理算力执行可执行程序来处理该终端的视频，因此无需更新终端的硬件，只需通过服务器调整处理算力的大小即可达到算力自动扩展。
本申请实施例提供了一种视频处理系统，包括终端和服务器；
所述终端，用于将处理视频所需的存储容量、处理算力和用于处理视频的可执行程序发送给所述服务器；以及将待处理视频帧发送给所述服务器；
所述服务器，用于为所述终端分配所述存储容量的存储空间，并将所述处理算力和所述可执行程序存储到该存储空间中；以及在接收到来自所述终 端的待处理视频帧时，通过处理器从所述存储空间中加载所述可执行程序，将所述待处理视频帧输入所述可执行程序，以所述处理算力执行所述可执行程序对所述待处理视频帧进行处理。
在本申请的一些实施例中，所述终端，还用于在处理视频所需的存储容量和处理算力改变时，发送新的存储容量和新的处理算力给所述服务器；
所述服务器，还用于根据所述新的存储容量调整所述存储空间，将所述新的处理算力和所述可执行程序存储在调整后的所述存储空间中。
在本申请的一些实施例中，所述终端，还用于在所述可执行程序版本升级时，发送新版本的可执行程序给所述服务器；
所述服务器，还用于将所述存储空间中存储的可执行程序更新为所述新版本的可执行程序。
在本申请的一些实施例中，所述终端，还用于发送需要存储的视频帧给所述服务器；
所述服务器，还用于将所述视频帧存储到所述存储空间中，并将存储所述视频帧的第一存储地址发送给所述终端。
在本申请的一些实施例中，所述终端，还用于发送视频解码请求给所述服务器，所述视频解码请求包括所述第一存储地址和编码类型；
所述服务器，用于从所述存储空间中读取所述第一存储地址处存储的视频帧；并利用所述编码类型对应的解码器对读取的所述视频帧进行解码，将解码后的视频帧存储在所述存储空间中，并将存储所述解码后的视频帧的第二存储地址发送给所述终端。
在本申请的一些实施例中，所述终端，还用于发送视频分析请求给所述服务器，所述视频分析请求包括所述第二存储地址；
所述服务器，还用于从所述存储空间中读取所述第二存储地址处存储的视频帧，通过图形处理器GPU从所述存储空间中加载所述可执行程序，将读取的所述视频帧输入所述可执行程序，以所述处理算力执行所述可执行程序对所述视频帧进行分析；将分析结果存储在所述存储空间中，并将存储所述分析结果的第三存储地址发送给所述终端。
在本申请的一些实施例中，所述终端，还用于发送视频编码请求给所述 服务器，所述视频编码请求包括所述第二存储地址和编码类型；
所述服务器，还用于从所述存储空间中读取所述第二存储地址处存储的视频帧；利用所述编码类型对应的编码器对读取的所述视频帧进行编码处理；将编码后的视频帧存储在所述存储空间中，并将存储所述编码后的视频帧的第四存储地址发送给所述终端。
在本申请的一些实施例中，所述终端，还用于接收所述服务器返回的所述第二存储地址，并将所述第二存储地址插入待分析队列的队尾；以及，根据所述终端采集视频的视频帧率和所述服务器分析视频的预设单位分析时长，计算第一最少间隔帧数；在采集的视频数据中统计目标在预设时长内的最小停留时间；根据所述最小停留时间和所述视频帧率，计算第二最少间隔帧数；根据所述第一最少间隔帧数、所述第二最少间隔帧数和所述待分析队列中包括的第二存储地址的数目，从所述待分析队列中选取需要分析的视频帧对应的第二存储地址。
本申请实施例中提供的技术方案，至少具有如下技术效果或优点：
本申请实施例将用于处理视频的可执行程序和终端侧的硬件解耦，依靠服务器侧的处理器来执行可执行程序进行视频处理，实现了处理算力弹性扩展的目的。终端中不需要配备具有很高处理算力的硬件，也无需配备具有很大存储容量的存储器件，终端将处理算力、存储空间和用于处理视频的可执行程序发送给服务器，服务器为终端分配存储空间，并以该处理算力执行可执行程序对终端的视频进行处理。在终端处理视频所需的处理算力增长时，本申请不需要更新终端的硬件，通过对服务器进行配置即可达到算力自动扩展，高效利用服务器的硬件资源，且能够实现在不同的视频处理场景之间进行运算资源的合理调配，减少运算资源的浪费，便于扩展新的应用场景。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变的明显，或通过本申请的实践了解到。
附图说明
为了更清楚地说明本发明实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的 附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例所提供的一种视频处理系统的示意图；
图2示出了本申请实施例所提供的一种视频处理的方法流程图。
具体实施方式
为使本发明的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本发明进一步详细说明。显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本申请实施例提供一种视频处理系统，下面结合附图进行说明。
本申请实施例提供了一种视频处理系统，该系统的网络架构如图1所示，该视频处理系统包括服务器和至少一个终端，每个终端均通过网络与服务器连接。终端可以为能够采集视频的设备，如相机、摄像头等。服务器可以为云GPU(Graphics Processing Unit，图形处理器)服务平台，服务器也可以为多个服务器设备组成的服务器集群。图1中仅示意性地画出了3个终端A、B和C。
本申请实施例所提供的一种视频处理系统包括终端和服务器，其中：
终端，用于将处理视频所需的存储容量、处理算力和用于处理视频的可执行程序发送给所述服务器；以及将待处理视频帧发送给所述服务器；
服务器，用于为所述终端分配所述存储容量的存储空间，并将所述处理算力和所述可执行程序存储到该存储空间中；以及在接收到来自所述终端的待处理视频帧时，通过处理器从所述存储空间中加载所述可执行程序，将所述待处理视频帧输入所述可执行程序，以所述处理算力执行所述可执行程序对所述待处理视频帧进行处理。
在本申请实施例中，终端将其处理视频所需的存储容量、处理算力和可执行程序发送给服务器，由服务器为终端分配相应的存储空间，并将该终端对应的处理算力和可执行程序存储在该存储空间中。当服务器接收到该终端发送的待处理视频帧时，服务器通过处理器从该存储空间中加载可执行程序， 将待处理视频帧输入该可执行程序，以上述处理算力执行该可执行程序对待处理视频帧进行处理。如此终端中不需要配置具有很高处理算力的硬件，当终端处理视频所需的处理算力升级提高时，无需通过更新终端侧的硬件来提高处理算力。终端只需将升级后所需要的处理算力发送给服务器，即可实现处理算力的扩展，高效利用服务器的硬件资源，成本低。而且对于不同应用场景下的终端，都可以通过本申请实施例提供的方法，通过服务器来扩展处理算力，从而实现在不同的视频处理场景之间进行运算资源的合理调配，减少运算资源的浪费，便于扩展新的应用场景。
如图2所示，本发明提供了一种基于上述视频处理系统的视频处理方法，具体包括以下步骤：
步骤101：终端将处理视频所需的存储容量、处理算力和用于处理视频的可执行程序发送给服务器。
在新的视频处理场景中部署终端后，可以将用于处理视频所需的可执行程序存储在终端本地，并在终端上设置处理视频所需的存储容量和处理算力。其中，存储容量即为存储容量的大小。终端通过网络连接到服务器，获取自身的设备标识，以及获取本地存储的存储容量、处理算力和可执行程序，发送设备标识及获取的存储容量、处理算力和可执行程序给服务器。
其中，可执行程序可以根据视频处理场景的实际需要设置，当终端存储有多个可执行程序时，可以记录可执行程序标识与该可执行程序所能对视频进行的处理的对应关系，进而在需要对视频进行处理时，根据该对应关系确定可执行程序标识，将该可执行程序标识对应的可执行程序发送至服务器。
例如，在人脸识别的应用场景中，终端可以为用于抓拍人脸的相机，可执行程序可以为用于进行人脸识别的程序。假设终端进行人脸识别处理所需的存储容量为50G，所需的处理算力为2TOPS(Tera Operations Per Second，处理器每秒钟进行一万亿次操作)，则终端发送存储容量50G、处理算力2T和用于进行人脸识别的可执行程序给服务器。
在本申请实施例中，终端通过网络连接服务器，并将处理视频所需的存储容量、处理算力和可执行程序上传给服务器，从而使得用于处理视频的可执行程序与终端侧的硬件之间解耦，当应用场景对运算能力的需求增加时， 无需更换终端侧的硬件，只需调整服务器为该终端分配的处理算力的大小即可，实现通过终端与服务器之间的数据通信及服务器侧的简单配置达到处理算力自动扩展的目的，成本低，处理算力的扩展灵活方便。
步骤102：服务器接收终端上传的存储容量、处理算力和可执行程序，为该终端分配该存储容量的存储空间，并将处理算力和可执行程序存储到该存储空间中。
服务器接收终端上传的存储容量、处理算力和可执行程序，服务器还接收该终端上传的设备标识，为该终端分配该存储容量的存储空间，将接收的处理算力及可执行程序存储在分配的存储空间中，并建立该终端的设备标识与该存储空间之间的映射关系。
当终端处理视频所需的存储容量发生改变时，终端可以发送设备标识和新的存储容量给服务器。服务器接收终端发送的设备标识和新的存储容量，根据设备标识从之前为该终端分配的存储空间中获取处理算力、可执行程序及处理视频的历史数据等，即获取之前分配的存储空间中存储的所有数据。服务器为该终端分配该新的存储容量的存储空间，将上述获取的处理算力、可执行程序及处理视频的历史数据等存储在新分配的存储空间中，并建立该终端的设备标识与新分配的存储空间之间的映射关系。
服务器接收终端发送的设备标识和新的存储容量之后，还可以根据设备标识查询之前为该终端分配的存储空间的存储容量，比较查询的存储容量与该新的存储容量的大小，若查询的存储容量大于该新的存储容量，则计算查询的存储容量与该新的存储容量之间的差值，从之前为该终端分配的存储空间中释放容量为该差值的存储空间，剩余的存储空间的容量即为上述新的存储容量。若查询的存储容量小于该新的存储容量，则计算该新的存储容量与查询的存储容量之间的差值，分配容量为该差值的存储空间，建立容量为该差值的存储空间与该终端的设备标识之间的映射关系，容量为该差值的存储空间与之前为该终端分配的存储空间的存储容量之和即为上述新的存储容量。
通过上述方式，依靠终端与服务器之间的数据通信，简单地在服务器侧进行配置，就可以方便地调整终端处理视频所需的存储空间，终端侧无需配备具有很大存储容量的存储器件，成本低，调整存储容量的方式方便灵活。
当终端处理视频所需的处理算力发生改变时，终端可以发送设备标识和和新的处理算力给服务器。服务器接收终端发送的设备标识和新的处理算力，将该设备标识对应的存储空间中存储的处理算力更新为该新的处理算力，以便后续根据该新的处理算力对该终端的待处理视频帧进行处理。
本申请实施例通过终端与服务器之间的数据通信，简单地在服务器侧进行配置，就可以方便地调整终端处理视频所需的处理算力，终端侧无需配备具有很大处理算力的硬件，成本低，处理算力的扩展方式方便灵活。
当终端侧用于处理视频的可执行程序版本升级时，终端可以发送设备标识和新版本的可执行程序给服务器。服务器接收终端发送的设备标识和新版本的可执行程序，将该设备标识对应的存储空间中存储的可执行程序更新为新版本的可执行程序。通过终端与服务器之间的数据通信，终端将新版本的可执行程序发送给服务器，即可实现程序升级。
在本申请实施例中，任意终端均可通过上述操作接入服务器，并由服务器分配终端对应的存储空间，以及在终端对应的存储空间中存储终端处理视频所需的处理算力和可执行程序，打破了不同应用场景之间的界线，不同应用场景中的终端均可连接到相同的服务器中，能够满足各个应用场景下处理视频的业务需求。
通过上述步骤101和102的操作在服务器中配置好了终端对应的存储空间，以及在存储空间中存储了终端处理视频所需的处理算力和可执行程序，之后当终端需要处理视频时，通过如下步骤103和104的操作来进行视频处理。
步骤103：终端将待处理视频帧发送给服务器。
步骤104：服务器接收来自终端的待处理视频帧，通过处理器从该终端对应的存储空间中加载可执行程序，将待处理视频帧输入可执行程序，以上述处理算力执行可执行程序对待处理视频帧进行处理。
在本申请实施例中，服务器至少可以提供4种基本服务：云存储服务、解码服务、编码服务和算法处理服务。其中，云存储服务用于存储终端的视频帧。解码服务用于将经过H264或H265等编码协议编码的视频帧解码还原为原始视频帧。编码服务用于采用H264或H265等编码协议对视频帧进行编 码。算法处理服务用于采用终端发送的处理视频所需的可执行程序对视频帧进行分析处理，如通过用于人脸识别的可执行程序对视频帧进行识别处理等。
下面以上述4种基本服务为例具体说明处理视频帧的过程。
一、云存储服务
上述终端，还可以用于发送需要存储的视频帧给所述服务器；上述服务器，还可以用于将所述视频帧存储到所述存储空间中，并将存储所述视频帧的第一存储地址发送给所述终端。
终端可以发送设备标识和需要存储的视频帧给服务器。服务器接收终端发送的设备标识和需要存储的视频帧，将该视频帧存储到该终端的设备标识对应的存储空间中；将存储该视频帧的第一存储地址发送给终端。
或者，在步骤102中服务器分配终端的设备标识对应的存储空间后，将设备标识对应的存储空间映射到终端本地，如此在终端本地会产生存储空间对应的映射目录。当终端需要存储视频时，直接将待存储的视频帧拷贝到本地的映射目录中。之后服务器获取终端拷贝到本地的映射目录下的视频帧。终端还可以将映射目录中存储该视频帧的存储地址确定为第一存储地址，并保存该第一存储地址。
在一种实施方式中，当需要存储的视频帧为待解码的视频帧时，终端通过上述任一方式将需要存储的视频帧存储到服务器中该终端对应的存储空间中，并得到存储该视频帧的第一存储地址后，可以将第一存储地址插入待解码队列的队尾。其中，该待解码队列用于存放待解码视频帧的存储地址。
在本申请实施例中，也可能存在存储视频帧失败的情况，在此情况下，服务器发送失败指示信息给终端，该失败指示信息中包括用于指示存储失败的错误码。终端接收到该失败指示信息后，可以再次发送需要存储的视频帧给服务器，并记录该视频帧对应的上传次数。若该视频帧的上传次数达到预设上限次数，则放弃该视频帧，不再将该视频帧上传到服务器进行存储。
二、解码服务
上述终端，还可以用于发送视频解码请求给所述服务器，所述视频解码请求包括所述第一存储地址和编码类型；上述服务器，还可以用于从所述存储空间中读取所述第一存储地址处存储的视频帧；并利用所述编码类型对应 的解码器对读取的所述视频帧进行解码，将解码后的视频帧存储在所述存储空间中，并将存储所述解码后的视频帧的第二存储地址发送给所述终端。
终端发送视频解码请求给服务器，该视频解码请求包括第一存储地址和编码类型。具体地，终端从待解码队列的队首获取一个或多个第一存储地址，发送视频解码请求给服务器，在该视频解码请求中携带获取的一个或多个第一存储地址以及携带每个第一存储地址存储的视频帧对应的编码类型。其中，编码类型用于指示需要解码的视频帧是用哪种编码类型对应的编码协议进行编码的，编码类型可以为H264或H265等。
服务器接收终端发送的视频解码请求；根据视频解码请求中的第一存储地址，从终端对应的存储空间中，获取需要解码的视频帧；通过视频解码请求包括的编码类型对应的解码器对获取的视频帧进行解码操作；将解码后的视频帧存储到终端对应的存储空间中，将存储解码后的视频帧的第二存储地址发送给终端。
在一种实施方式中，当需要解码的视频帧为待分析的视频帧时，终端还可以用于接收服务器返回的第二存储地址，并将第二存储地址插入待分析队列的队尾。服务器将存储解码后的视频帧的第二存储地址发送给终端后，终端接收服务器返回的第二存储地址，可以将第二存储地址插入待分析队列的队尾。其中，该待分析队列用于存放待分析视频帧的存储地址。
在本申请实施例中，也可能存在解码失败的情况，在此情况下，服务器发送失败指示信息给终端，该失败指示信息中包括用于指示解码失败的错误码。终端接收到该失败指示信息后，可以再次发送需要解码的视频帧对应的视频解码请求给服务器，并记录该视频帧对应的解码请求次数。若该视频帧对应的解码请求次数达到预设上限次数，则放弃该视频帧，不再请求解码该视频帧。
三、算法处理服务
上述终端，还可以用于发送视频分析请求给所述服务器，所述视频分析请求包括所述第二存储地址；上述服务器，还可以用于从所述存储空间中读取所述第二存储地址处存储的视频帧，通过图形处理器GPU从所述存储空间中加载所述可执行程序，将读取的所述视频帧输入所述可执行程序，以所述 处理算力执行所述可执行程序对所述视频帧进行分析；将分析结果存储在所述存储空间中，并将存储所述分析结果的第三存储地址发送给所述终端。
终端从待分析队列的队首获取第二存储地址，发送视频分析请求给服务器，该视频分析请求包括终端的设备标识和获取的第二存储地址。服务器接收终端发送的视频分析请求，从该设备标识对应的存储空间中读取第二存储地址处存储的视频帧，通过GPU(Graphics Processing Unit，图形处理器)从该设备标识对应的存储空间中加载可执行程序，将读取的视频帧输入该可执行程序，以该设备标识对应的存储空间中存储的处理算力，执行该可执行程序对读取的视频帧进行分析，将分析结果存储在该设备标识对应的存储空间中，并将存储该分析结果的第三存储地址发送给终端。其中，可执行程序即为用于对视频帧进行分析的可执行程序，以实现算法处理服务。
终端接收到服务器返回的第三存储地址后，可以根据第三存储地址，从服务器下载对应的分析结果。
在本申请实施例中，对于上述算法处理服务，为了节省分析资源和提高分析的实时性，可以不必对每一帧进行分析，可以选择性的分析。具体地，终端可以根据终端采集视频的视频帧率和服务器分析视频的预设单位分析时长，通过如下公式(1)计算第一最少间隔帧数；在采集的视频数据中统计目标在预设时长内的最小停留时间；根据最小停留时间和终端采集视频的视频帧率，通过如下公式(2)计算第二最少间隔帧数；根据第一最少间隔帧数、第二最少间隔帧数和待分析队列中包括的第二存储地址的数目，根据如下公式(3)从待分析队列中选取需要分析的视频帧对应的第二存储地址。
Figure PCTCN2020120304-appb-000001
Figure PCTCN2020120304-appb-000002
Figure PCTCN2020120304-appb-000003
在上述公式(1)、(2)和(3)中，a为第一最少间隔帧数，b为第二最 少间隔帧数，f为视频帧率，△t为预设单位分析时长，T 1为最小停留时间，c为间隔帧数，d为策略判定系数，n为待分析队列中视频帧的数量，INT(a)为对a取整，INT(b)为对b取整。
在本申请实施例中，上述第一最少间隔帧数表示至少每隔第一最少间隔帧数分析一次才不会造成数据拥堵。第二最少间隔帧数表示最少每隔第二最少间隔帧数分析一次才不会造成目标漏抓。上述预设单位分析时长为服务器分析一帧视频帧所需的时长。最小停留时间为预设时长内目标在终端所采集的视频中出现的总时长，最小停留时间的初始值可以为30ms或40ms等。预设时长可以为1小时或2小时等，如在人脸识别场景中目标人脸在1小时的视频中出现的总时长。
通过上述公式(1)、(2)和(3)的方式计算出间隔帧数c和策略判定系数d后，判断策略判定系数d是否大于预设阈值，如果是，则从待分析队列中舍弃1/2的第二存储地址。具体地，可以每隔一个第二存储地址剔除一个第二存储地址，如此实现分析一帧剔除一帧，减少需要分析的视频帧的数目，节省服务器资源，提高视频分析处理的实时性。如果策略判定系数d小于等于预设阈值，则在待分析队列中每隔间隔帧数c个第二存储地址获取一个第二存储地址并对该第二存储地址对应的视频帧进行分析。如此减少需要分析的视频帧的数目，节省服务器资源，提高视频分析处理的实时性。上述预设阈值可以为2或3等。
四、编码服务
上述终端，还可以用于发送视频编码请求给所述服务器，所述视频编码请求包括所述第二存储地址和编码类型；上述服务器，还可以用于从所述存储空间中读取所述第二存储地址处存储的视频帧；利用所述编码类型对应的编码器对读取的所述视频帧进行编码处理；将编码后的视频帧存储在所述存储空间中，并将存储所述编码后的视频帧的第四存储地址发送给所述终端。
终端从待分析队列的队首获取第二存储地址，发送视频编码请求给服务器，该视频编码请求包括编码类型和获取的第二存储地址。服务器接收终端发送的该视频编码请求，从该终端对应的存储空间中读取第二存储地址处存储的视频帧，利用视频编码请求包括的编码类型对应的编码器对读取的视频 帧进行编码处理，将编码后的视频帧存储在该终端对应的存储空间中，并将存储编码后的视频帧的第四存储地址发送给终端。终端接收服务器返回的第四存储地址，可以根据第四存储地址从服务器下载编码后的视频帧。
在本申请实施例中，也可能存在编码失败的情况，在此情况下，服务器发送失败指示信息给终端，该失败指示信息中包括用于指示编码失败的错误码。终端接收到该失败指示信息后，可以再次发送需编码的视频帧对应的视频编码请求给服务器，并记录该编码请求次数。若该编码请求次数达到预设上限次数，则放弃该视频帧，不再请求编码该视频帧。
对于上述编码服务，为了节省服务器资源和提高编码的实时性，可以不必对待分析队列中每一帧进行编码，可以选择性的编码。具体地终端同样可以按照上述算法处理服务中所述的公式(1)、(2)和(3)的方式来选择需要进行编码的第二存储地址，具体选择过程在此不再详述。
在本申请实施例中，终端还可以发送删除请求给服务器，该删除请求中可以包括设备标识和第一存储地址。服务器接收到该删除请求后，从该设备标识对应的存储空间中删除第一存储地址处存储的视频帧。该删除请求中还可以包括设备标识和第二存储地址，相应地，服务器接收到删除请求后，从该设备标识对应的存储空间中删除第二存储地址处存储的解码后的视频帧。该删除请求中还可以包括设备标识和第三存储地址，相应地，服务器接收到删除请求后，从该设备标识对应的存储空间中删除第三存储地址处存储的分析结果。该删除请求中还可以包括设备标识和第四存储地址，相应地，服务器接收到删除请求后，从该设备标识对应的存储空间中删除第四存储地址处存储的编码后的视频帧。
在本申请实施例中，若某个终端不再使用，在服务器中可以删除该终端对应的存储空间中存储的处理算力、可执行程序及处理视频产生的各种数据，并释放该终端对应的存储空间。由于处理终端的视频所需的存储空间和处理算力都由服务器根据终端的需求进行分配，可以随意配置，因此处理算力的扩展更加灵活方便。
本申请实施例将用于处理视频的可执行程序和终端侧的硬件解耦，依靠服务器侧的处理器来执行可执行程序进行视频处理，实现了处理算力弹性扩 展的目的。终端中不需要配备具有很高处理算力的硬件，也无需配备具有很大存储容量的存储器件，终端将处理算力、存储空间和用于处理视频的可执行程序发送给服务器，服务器为终端分配存储空间，并以该处理算力执行可执行程序对终端的视频进行处理。在终端处理视频所需的处理算力增长时，本申请不需要更新终端的硬件，通过对服务器进行配置即可达到算力自动扩展，并且高效利用服务器的硬件资源，实现在不更新终端的硬件的情况下，弹性扩展处理算力。
需要说明的是：
在此提供的算法和显示不与任何特定计算机、虚拟装置或者其它设备有固有相关。各种通用装置也可以与基于在此的示教一起使用。根据上面的描述，构造这类装置所要求的结构是显而易见的。此外，本申请也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本申请的内容，并且上面对特定语言所做的描述是为了披露本申请的最佳实施方式。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本申请的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本申请并帮助理解各个发明方面中的一个或多个，在上面对本申请的示例性实施例的描述中，本申请的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本申请要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本申请的单独实施例。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者 单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本申请的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
本申请的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本申请实施例的虚拟机的创建装置中的一些或者全部部件的一些或者全部功能。本申请还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本申请的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。
应该注意的是上述实施例对本申请进行说明而不是对本申请进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本申请可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。
以上所述，仅为本申请较佳的具体实施方式，但本申请的保护范围并不 局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (8)

  1. 一种视频处理系统，其特征在于，包括终端和服务器；
    所述终端，用于将处理视频所需的存储容量、处理算力和用于处理视频的可执行程序发送给所述服务器；以及将待处理视频帧发送给所述服务器；
    所述服务器，用于为所述终端分配所述存储容量的存储空间，并将所述处理算力和所述可执行程序存储到该存储空间中；以及在接收到来自所述终端的待处理视频帧时，通过处理器从所述存储空间中加载所述可执行程序，将所述待处理视频帧输入所述可执行程序，以所述处理算力执行所述可执行程序对所述待处理视频帧进行处理。
  2. 根据权利要求1所述的系统，其特征在于，
    所述终端，还用于在处理视频所需的存储容量和处理算力改变时，发送新的存储容量和新的处理算力给所述服务器；
    所述服务器，还用于根据所述新的存储容量调整所述存储空间，将所述新的处理算力和所述可执行程序存储在调整后的所述存储空间中。
  3. 根据权利要求1所述的系统，其特征在于，
    所述终端，还用于在所述可执行程序版本升级时，发送新版本的可执行程序给所述服务器；
    所述服务器，还用于将所述存储空间中存储的可执行程序更新为所述新版本的可执行程序。
  4. 根据权利要求1所述的系统，其特征在于，
    所述终端，还用于发送需要存储的视频帧给所述服务器；
    所述服务器，还用于将所述视频帧存储到所述存储空间中，并将存储所述视频帧的第一存储地址发送给所述终端。
  5. 根据权利要求4所述的系统，其特征在于，
    所述终端，还用于发送视频解码请求给所述服务器，所述视频解码请求包括所述第一存储地址和编码类型；
    所述服务器，用于从所述存储空间中读取所述第一存储地址处存储的视频帧；并利用所述编码类型对应的解码器对读取的所述视频帧进行解码，将解码后的视频帧存储在所述存储空间中，并将存储所述解码后的视频帧的第 二存储地址发送给所述终端。
  6. 根据权利要求5所述的系统，其特征在于，
    所述终端，还用于发送视频分析请求给所述服务器，所述视频分析请求包括所述第二存储地址；
    所述服务器，还用于从所述存储空间中读取所述第二存储地址处存储的视频帧，通过图形处理器GPU从所述存储空间中加载所述可执行程序，将读取的所述视频帧输入所述可执行程序，以所述处理算力执行所述可执行程序对所述视频帧进行分析；将分析结果存储在所述存储空间中，并将存储所述分析结果的第三存储地址发送给所述终端。
  7. 根据权利要求5所述的系统，其特征在于，
    所述终端，还用于发送视频编码请求给所述服务器，所述视频编码请求包括所述第二存储地址和编码类型；
    所述服务器，还用于从所述存储空间中读取所述第二存储地址处存储的视频帧；利用所述编码类型对应的编码器对读取的所述视频帧进行编码处理；将编码后的视频帧存储在所述存储空间中，并将存储所述编码后的视频帧的第四存储地址发送给所述终端。
  8. 根据权利要求5所述的系统，其特征在于，
    所述终端，还用于接收所述服务器返回的所述第二存储地址，并将所述第二存储地址插入待分析队列的队尾；以及，根据所述终端采集视频的视频帧率和所述服务器分析视频的预设单位分析时长，计算第一最少间隔帧数；在采集的视频数据中统计目标在预设时长内的最小停留时间；根据所述最小停留时间和所述视频帧率，计算第二最少间隔帧数；根据所述第一最少间隔帧数、所述第二最少间隔帧数和所述待分析队列中包括的第二存储地址的数目，从所述待分析队列中选取需要分析的视频帧对应的第二存储地址。
PCT/CN2020/120304 2020-04-03 2020-10-12 一种视频处理系统 WO2021196559A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010260779.6A CN113497955B (zh) 2020-04-03 2020-04-03 一种视频处理系统
CN202010260779.6 2020-04-03

Publications (1)

Publication Number Publication Date
WO2021196559A1 true WO2021196559A1 (zh) 2021-10-07

Family

ID=77929275

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120304 WO2021196559A1 (zh) 2020-04-03 2020-10-12 一种视频处理系统

Country Status (2)

Country Link
CN (1) CN113497955B (zh)
WO (1) WO2021196559A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114339266A (zh) * 2021-12-14 2022-04-12 浪潮软件集团有限公司 一种基于国产cpu和操作系统的视频流队列处理方法
CN114390363A (zh) * 2021-12-22 2022-04-22 广州方硅信息技术有限公司 编码器的适配方法、装置、系统和存储介质
CN116610270A (zh) * 2023-07-21 2023-08-18 湖南马栏山视频先进技术研究院有限公司 一种视频处理的存算分离方法及视频存算分离系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101957911A (zh) * 2010-09-29 2011-01-26 汉王科技股份有限公司 一种人脸识别方法及系统
US20120110167A1 (en) * 2010-10-28 2012-05-03 Avvasi Device with video buffer modeling and methods for use therewith
CN105162874A (zh) * 2015-09-23 2015-12-16 上海斐讯数据通信技术有限公司 一种扩展移动终端运算能力的系统
CN107395413A (zh) * 2017-07-19 2017-11-24 湖南竞投智能信息技术有限公司 一种基于音视频的实时救援、处理及调度系统
CN110769175A (zh) * 2018-07-27 2020-02-07 华为技术有限公司 一种智能分析系统、方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3232667A1 (en) * 2016-04-12 2017-10-18 EVS Broadcast Equipment SA Modular software based video production server, method for operating the video production server and distributed video production system
CN109194984B (zh) * 2018-11-09 2021-02-12 中山大学 一种基于边缘计算的视频帧调度方法
CN110851255B (zh) * 2019-11-07 2022-04-15 之江实验室 一种基于终端设备和边缘服务器协同进行视频流处理的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101957911A (zh) * 2010-09-29 2011-01-26 汉王科技股份有限公司 一种人脸识别方法及系统
US20120110167A1 (en) * 2010-10-28 2012-05-03 Avvasi Device with video buffer modeling and methods for use therewith
CN105162874A (zh) * 2015-09-23 2015-12-16 上海斐讯数据通信技术有限公司 一种扩展移动终端运算能力的系统
CN107395413A (zh) * 2017-07-19 2017-11-24 湖南竞投智能信息技术有限公司 一种基于音视频的实时救援、处理及调度系统
CN110769175A (zh) * 2018-07-27 2020-02-07 华为技术有限公司 一种智能分析系统、方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114339266A (zh) * 2021-12-14 2022-04-12 浪潮软件集团有限公司 一种基于国产cpu和操作系统的视频流队列处理方法
CN114339266B (zh) * 2021-12-14 2023-09-01 浪潮软件集团有限公司 一种基于国产cpu和操作系统的视频流队列处理方法
CN114390363A (zh) * 2021-12-22 2022-04-22 广州方硅信息技术有限公司 编码器的适配方法、装置、系统和存储介质
CN116610270A (zh) * 2023-07-21 2023-08-18 湖南马栏山视频先进技术研究院有限公司 一种视频处理的存算分离方法及视频存算分离系统
CN116610270B (zh) * 2023-07-21 2023-10-03 湖南马栏山视频先进技术研究院有限公司 一种视频处理的存算分离方法及视频存算分离系统

Also Published As

Publication number Publication date
CN113497955B (zh) 2022-10-04
CN113497955A (zh) 2021-10-12

Similar Documents

Publication Publication Date Title
WO2021196559A1 (zh) 一种视频处理系统
WO2020093694A1 (zh) 生成视频分析模型的方法及视频分析系统
US11102641B2 (en) SIM card status determination method and SIM card status determination device
US9432672B2 (en) Image compression method and system with image compression time information
US9639459B2 (en) I/O latency and IOPs performance in thin provisioned volumes
CN110795244B (zh) 一种任务分配方法、装置、设备及介质
US20150309874A1 (en) A method and apparatus for code length adaptation for access to key-value based cloud storage systems
US20200159565A1 (en) Predicting transaction outcome based on artifacts in a transaction processing environment
CN113010260A (zh) 容器数量弹性伸缩方法以及容器数量弹性伸缩方法系统
US20100106683A1 (en) Computer system and replication method for the computer system
WO2019237774A1 (zh) 一种云存储系统中的视频数据存储方法及装置
WO2020057178A1 (zh) 智能分析设备资源调整方法及装置
US11810350B2 (en) Processing of surveillance video streams using image classification and object detection
US20160248834A1 (en) Algorithmic transcoding
WO2023142513A1 (zh) 一种分布式存储系统的存储优化方法及装置
CN113593541B (zh) 数据处理方法、装置、电子设备和计算机存储介质
CN116048740A (zh) 基于众核系统的任务调度方法、系统、电子设备及介质
CN111831408A (zh) 异步任务处理方法、装置、电子设备及介质
CN111090818A (zh) 资源管理方法、资源管理系统、服务器及计算机存储介质
US11733906B2 (en) Methods, apparatuses, computer programs and computer program products for data storage
WO2022028189A1 (zh) 投屏方法及装置、电子设备和计算机可读存储介质
US9575722B2 (en) Software interface for a specialized hardward device
CN115391051A (zh) 视频计算任务调度方法、设备及计算机可读介质
CN111510715B (zh) 视频处理方法、系统、计算机设备及存储介质
US11475022B2 (en) System and method for constructing a compound object in a distributed object storage system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20928177

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20928177

Country of ref document: EP

Kind code of ref document: A1