JP7191240B2 - Video stream decoding method, device, terminal equipment and program - Google Patents

Description

This application is a Chinese patent application with application number 201910636848.6 and titled "Video Stream Decoding Method, Apparatus, Terminal Equipment and Storage Medium" filed with the Chinese Patent Office on July 15, 2019. , the entire contents of which are hereby incorporated by reference into this application.

The present application relates to the field of computer technology, in particular to a video stream decoding method, apparatus, terminal equipment and storage medium.

With the rapid development of Internet and mobile Internet, the total amount of data generated by people shows a dramatic increase trend. Along with the increase in information data, the demand for information data processing capability is also increasing. Especially in the field of video, with the rapid development of the Internet and smart cities, the amount of video data to be stored, transmitted, and processed is increasing more and more, and video encoding and decoding, called video codecs, are often performed. .

With the enhancement of security management, many scenes now deploy high-density HD camera surveillance networks. However, with related technology, it is difficult to process such a large amount of video data efficiently. A large number of servers are required to process the multi-channel video streams captured by high-density HD cameras, as they occupy a lot of Central Processing Unit (CPU) resources.

In the actual development method of conventional video stream decoding, it is necessary to use a terminal device with excellent hardware performance, such as a high-performance CPU, which leads to high hardware cost and high energy consumption. tend to become Currently, there is still no efficient solution for decoding video streams.

According to various embodiments of the present application, a video stream decoding method, apparatus, terminal equipment and storage medium are provided.

A video stream decoding method performed by a terminal device,
obtaining multi-channel video streams input from surveillance equipment;
creating a plurality of threads in a thread pool corresponding to the graphics processing unit;
and transmitting the multi-channel video stream to the graphics processing device for video decoding processing based on the plurality of threads, thereby obtaining multi-channel decoded video data.

A video stream decoding device,
an acquisition unit for acquiring multi-channel video streams input from surveillance equipment;
a creation unit for creating a plurality of threads in a thread pool corresponding to the graphics processing device;
a decoding unit for obtaining multi-channel decoded video data by transmitting the multi-channel video stream to the graphics processing device for video decoding processing based on the plurality of threads.

A non-volatile storage medium having computer readable instructions stored therein which, when executed by one or more processors, instruct the one or more processors to perform a video stream decoding method. perform the steps.

A terminal device comprising a memory and a processor in which computer readable instructions are stored which, when executed by the processor, cause the processor to perform steps of a video stream decoding method.

The details of one or more embodiments of the application are set forth in the drawings and description below. Other features, objects, and advantages of the present application will become apparent from the specification, drawings, and claims.

In order to more clearly describe the technical solutions in the embodiments of the present application, the following briefly introduces the drawings that need to be used in the description of the embodiments. and those skilled in the art can obtain other drawings based on these drawings without creative efforts.

1 is a schematic diagram illustrating a scene of one implementation of a video stream decoding system according to embodiments of the present application; FIG. FIG. 3 is a schematic diagram showing a flow of an implementation of a video stream decoding method according to an embodiment of the present application; FIG. 5 is a schematic diagram showing a flow of another implementation of a video stream decoding method according to an embodiment of the present application; 1 is a schematic diagram showing the structure of one embodiment of a video stream decoding device according to an embodiment of the present application; FIG. FIG. 4 is a schematic diagram showing the structure of another implementation of a video stream decoding device according to an embodiment of the present application; It is a schematic diagram which shows the structure of the terminal device which concerns on embodiment of this application.

In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below in conjunction with the drawings and embodiments. It will be appreciated that the specific embodiments described herein are for interpretation of the application only and are not intended to limit the application. The present application is based on the embodiments of the present application, and all other embodiments obtained by persons skilled in the art without creative labor fall within the protection scope of the present application.

In the description that follows, specific embodiments of the present application will be described with reference to steps and operations that are performed by one or more computers, unless specified otherwise. Thus, these steps and operations are sometimes referred to as being computer-executed, and the computer-execution referred to herein is a computer processing unit that represents electronic signals of data in structured form. including operation by This manipulation transforms or retains the data in locations within the memory system of the computer, and the operation of the computer may be reconfigured or altered in other ways known to those skilled in the art. become. The data structure in which the data is held is its physical location in memory and has certain characteristics defined by its data format. However, while the principles of the present application have been described in the literal terms above, this is not intended to be limiting, and those skilled in the art will appreciate that many of the steps and operations described below can also be implemented in hardware. will understand.

A "module" or "unit" as used herein can be considered a software object that executes on the computing system. The different components, modules, engines and services described herein can be viewed as objects implemented on this computing system. Also, the devices and methods described herein are preferably implemented in software, but can of course also be implemented in hardware, all of which fall within the scope of protection of the present application.

Referring to FIG. 1, FIG. 1 is a schematic diagram showing a scene of a video stream decoding system according to an embodiment of the present application, which includes a terminal device 102 and at least one monitoring device 101. Alternatively, the monitoring equipment captures, monitors, encodes, and transmits the video stream to the terminal equipment 102 for decoding. Each monitoring device may correspond to one channel of video stream, and this monitoring device 101 may be a surveillance camera. The terminal equipment 102 is integrated with a video stream decoding device and may comprise a central processing unit 1021 , a graphics processing unit 1022 and a memory 1023 . In the embodiment of the present application, the terminal device 102 mainly uses the central processing unit 1021 to acquire the multi-channel video stream input from the monitoring device and create a plurality of threads in the thread pool corresponding to the graphics processing unit. Then, based on a plurality of threads, the multi-channel video stream is transmitted to the graphics processing device to perform video decoding processing, thereby obtaining multi-channel decoded video data.

In embodiments of the present application, this terminal device 102 may be a server, which may be an independent server, or a server network or server cluster of servers, such as the Servers described in the embodiments include, but are not limited to, computers, network hosts, single network servers, multiple network servers, or cloud servers composed of multiple servers. A cloud server consists of a number of computers or network servers based on Cloud Computing. In embodiments of the present application, communication between the server and the monitoring equipment may be accomplished by any means of communication, including 3rd Generation Partnership Project (3GPP), Long Term Evolution Evolution, LTE), Mobile communications based on Worldwide Interoperability for Microwave Access (WiMAX) or TCP/IP Protocol Suite (TCP/IP), User Datagram Protocol (User Datagram Protocol, UDP) based computer network communication, etc., but not limited to these.

The application environment shown in FIG. 1 is only one application scene of the present invention, and does not limit the application scene of the present invention. Those skilled in the art will appreciate that it may also include equipment or monitoring equipment. For example, although only one server and two monitoring devices are shown in FIG. 1, this video stream decoding system may include one or more other servers and/or one connected to a server network. or multiple monitoring devices, and is not limited herein.

Also, as shown in FIG. 1, the memory within the terminal equipment 102 in this video stream decoding system is for storing data such as decoded video data or surveillance video data for later examination when required. and the memory may include at least one of system memory, shared memory, and disk.

System memory is synonymous with ram, which is an abbreviation for random-access memory, which is the main area where the system temporarily stores program instructions and data, and each location in ram is called a memory address. Identified by number, when you exit the terminal equipment system, all data stored in ram will be lost. System memory consists of physical memory and virtual memory, and when physical memory runs out during runtime, the system creates virtual memory, which is typically 1.5 to 3 times the physical memory.

A disk is a memory that stores data using magnetic recording technology. Disks are the main storage medium for computers, and can store large amounts of binary data, which can be preserved even after a power failure. The disks used in early computers were soft disks, the disks commonly used today are hard disks, and the disks referred to in the embodiments of this application are soft disks or / and can be a hard disk.

Shared memory is a large amount of memory that is accessible from different central processing units (CPUs) within a terminal device. Since the CPU needs to access the disk quickly, it needs to cache the disk, and after any cached data is updated, other processors also need to access it, so shared memory is used. It must be updated immediately or different processors may use different data.

It should be noted that the schematic diagram of the scene of the video stream decoding system shown in FIG. It is for explanation and does not limit the technical solutions according to the embodiments of the present application. As those skilled in the art know, with the evolution of video stream decoding systems and the emergence of new business scenes , the technical solutions according to the embodiments of the present application are equally applicable to similar technical problems.

Hereinafter, it will be described in detail together with specific embodiments.

This embodiment will be described from the viewpoint of a video stream decoding device. Specifically, this video stream decoding device may be integrated into a terminal device having a central processing unit and a graphics processing unit. A video stream decoding method in an embodiment is performed by a central processing unit.

The present application provides a video stream decoding method, comprising the steps of obtaining a multi-channel video stream input from a surveillance device, and creating multiple threads in a thread pool corresponding to a graphics processing device. and transmitting the multi-channel video stream to a graphics processing device for video decoding processing based on multiple threads to obtain multi-channel decoded video data.

Referring to FIG. 2, FIG. 2 is a schematic diagram showing the flow of an embodiment of a video stream decoding method according to an embodiment of the present application, which video stream decoding method includes steps 201-203 as follows.

At 201, a multi-channel video stream is acquired from the surveillance equipment.

In embodiments of the present application, monitoring equipment may refer to monitoring equipment 101 as shown in FIG. It can also be a camera. The monitoring device may be one monitoring device or a plurality of monitoring devices, each monitoring device transmits a video stream of one channel, and the monitoring device transmits the captured video to the video. It can be transmitted directly to the terminal equipment as a video stream in the form of a file. In one example, the terminal device may establish a connection, such as via wire or wireless, with the surveillance equipment to receive the video stream captured by the surveillance equipment. The terminal equipment may obtain the video stream from the monitoring equipment via Rtsp protocol, GB/T28181 protocol, or third-party Software Development Kit (SDK).

Also, the video images captured by the surveillance equipment are stored in, for example, the Multimedia Container (MKV) format or the Stream Video format (Flash Video, FLV) to form a video stream for transmission to the terminal equipment. It can be understood that it can be packaged in various different formats and may be other video formats such as mp4 format, rmvb format, etc., and is not particularly limited herein.

The step of obtaining the video stream data input from the monitoring equipment includes obtaining the original video stream input from the monitoring equipment, analyzing the original video stream, and analyzing the video stream data input from the monitoring equipment. may include the step of obtaining In this case, the video stream data input from the surveillance equipment is stored in the form of packets, and similarly, as will be described later, the decoded video stream data may be stored in the form of decoded data. Second, the decoded video stream is stored in the form of image frames.

At 202, multiple threads are created in a thread pool corresponding to the graphics processing unit.

A graphics processing unit (GPU), also called a graphics core, is a microprocessor specialized for image computation tasks in personal computers, workstations, game consoles, and some mobile devices. It can convert and drive the display information required by the computer system, provide scanning signals to the display, and control the display to display correctly. In the embodiments of the present application, the terminal equipment is equipped with a GPU, and the floating point computing power of the GPU is much stronger than that of the CPU, while current monitoring equipment is generally low-end, and the processing power of the CPU is Therefore, in the embodiment of the present application, by transmitting the video stream to the GPU and having the GPU take charge of decoding the video stream, the load on the CPU can be greatly reduced and the work efficiency of the video codec can be improved. .

In the embodiments of the present application, the terminal device may be provided with multiple GPUs, and the number of GPUs is not particularly limited here. The video streams transmitted to the graphics processing unit can then be randomly assigned to these multiple GPUs. For example, if the video stream input from the monitoring device has 10 channels and the terminal device has 2 GPUs, 3 of the 10 channels are randomly assigned to one GPU and the remaining 7 channels to the other GPU. be able to. Of course, the video stream transmitted to the graphics processing device can be equally allocated to these multiple GPUs. , a video stream of 5 channels each can be assigned to the two GPUs.

In some other embodiments of the present application, if the terminal device includes multiple GPUs, the corresponding dynamic load balancing rules may be preconfigured, in which case the video stream is used for graphics processing. The transmitting to the device further causes the video streams input from the monitoring device to be allocated to multiple GPUs according to a preset dynamic load balancing rule when the terminal device receives the multi-channel video stream. may be broken. Dynamic load balancing rules can ensure balanced use of each of these multiple GPUs.

In this dynamic load balancing rule, the initial allocation may set the number of channels of video streams to allocate to each GPU, for example, the initial allocation may allocate 20 channels of video streams to each GPU. For different video streams, the time for the GPU to finish the codec is different, and the allocation of subsequent video streams can be dynamically allocated based on the processing status of each GPU according to the dynamic load balancing rule. For example, if the terminal device is equipped with three GPUs, GPU1, GPU2, and GPU3, and 20 channels of video streams are assigned to each GPU in the initial assignment, GPU1 is assigned 20 channels of video streams. may have ended first, at which time the 20-channel video streams could be reallocated to GPU1 according to this dynamic load balancing rule.

Hardware resource calls introduce the concept of mutual exclusion locks to ensure the integrity of shared data operations. Each hardware object corresponds to a mark called a "mutual exclusion lock", which ensures that only one thread can access this hardware object at any one time. Therefore, when a terminal device is provided with multiple GPUs as described above, a mutual exclusion lock corresponding to each GPU can be set in order to avoid interaction between GPUs when allocating video streams to GPUs. , this mutual exclusion lock ensures that a multi-channel video stream is assigned to only one GPU at the same time. Considering that there may be multiple threads in the thread pool corresponding to one GPU, set a corresponding mutex lock for each thread to avoid interaction between threads. This mutex lock ensures that only one thread calls the GPU at the same time.

If the terminal device is equipped with multiple GPUs, for each GPU, based on the information on the number of threads input by the user, multiple threads are set in the thread pool corresponding to that GPU, and the received multi-channel A video stream can be parsed, multiple threads can be used to call the GPU, and codec processing can be performed on the parsed multi-channel video stream. In this way, one thread can call the GPU to realize the codec processing of the analyzed one-channel video stream, and multiple threads can be set for one GPU, namely: Multiple threads can simultaneously call the GPU to process the parsed video stream, thus realizing parallel processing of multi-channel video streams by one GPU and effectively improving the work efficiency of the video codec. improve to

In an embodiment of the present application, the terminal device includes at least one GPU, each GPU corresponds to one thread pool, each thread pool can be provided with at least one thread, and each channel video stream can be simultaneously processed by the number of threads provided in one thread pool.

A thread can be understood as interface information for calling the GPU to implement the codec, and using this thread pool, the GPU can be called to implement the codec of the video stream. Thread pools can store threads, and the number of threads stored in one thread pool can be set according to user requirements. The number of threads included in the thread pool can be input by the user to the terminal device through a related interface, and at this time, the step of creating a plurality of threads in the thread pool corresponding to the graphics processing unit specifically includes: Specifically, it can include the steps of obtaining information on the number of threads input by a user, and creating a plurality of threads in a thread pool corresponding to the graphics processing device according to the information on the number of threads. A specific value for the number of threads may be determined according to the processing power of the GPU. If the number of threads in the pool is set to 10 and the number of threads set by the user exceeds 10, say 20, it may be disabled. Note that the number of threads included in the thread pool corresponding to each GPU may be the same or different, and is not limited here.

In some embodiments of the present application, the step of transmitting multi-channel video streams to a graphics processing device for video decoding processing on the basis of multiple threads to obtain multi-channel decoded video data includes: Specifically, a step of assigning one thread out of a plurality of threads to a video stream of each channel of a multi-channel video stream on a one-to-one basis; inputting the target channel video stream to the graphics processing device using the thread, wherein the target channel video stream is a one-channel video stream corresponding to the target thread among the multi-channel video streams; , may include In this case, the step of outputting the multi-channel decoded video data to the memory via the graphics processor includes the step of decoding the target video stream by the graphics processor to obtain the target decoded video data.

For example, if the terminal device includes two GPUs, GPU1 and GPU2, and five threads are created in the thread pool corresponding to GPU1 and ten threads are created in the thread pool corresponding to GPU2, the terminal device In the thread pool corresponding to the graphics processing unit, multiple threads can be created, ie, 15 threads.

If the terminal device includes a plurality of GPUs, when creating a plurality of threads in the thread pool corresponding to the graphics processing device, which GPU to create the plurality of threads depends on the actual situation. for example, dedicating decoding of video streams input from surveillance equipment to one or more GPUs and processing of other images to other one or more GPUs. can be in charge.

In another possible embodiment of the present application, the number of threads to create may be determined based on the number of video channels of the video stream input from the surveillance equipment, specifically the threads corresponding to the graphics processing unit. The step of creating a plurality of threads in the pool includes the step of obtaining the number of video channels of the video stream input from the monitoring device, and creating the same number of threads as the number of video channels in the thread pool corresponding to the graphics processing device. to obtain a plurality of threads. The processing capacity of the GPU in the terminal equipment is always limited, and the number of channels of the video stream input from the monitoring equipment is limited to the maximum number of threads that can be created in the thread pool corresponding to the GPU in the terminal equipment (when the terminal equipment has multiple When GPU is included, if this maximum number exceeds the maximum number of threads that can be created by multiple GPUs), the multiple threads are the number of threads that can be created in the thread pool corresponding to the GPU in this terminal device. It may be the maximum number, ie the terminal device may receive only the maximum number of channels of video streams.

At 203 , multi-channel decoded video data is obtained by transmitting the multi-channel video stream to the graphics processing device for video decoding processing based on the plurality of threads.

In some embodiments of the present application, the step of transmitting multi-channel video streams to a graphics processing device for video decoding processing on the basis of multiple threads to obtain multi-channel decoded video data includes: Specifically, a step of assigning one thread out of a plurality of threads to a video stream of each channel of a multi-channel video stream on a one-to-one basis; inputting the target video stream to the graphics processing device using the thread, wherein the target channel video stream is a one-channel video stream corresponding to the target thread among the multi-channel video streams; may contain. In this case, the step of outputting the multi-channel decoded video data to the memory via the graphics processor includes the step of decoding the target video stream by the graphics processor to obtain the target decoded video data.

If the terminal device has multiple GPUs, the target thread is used to call the graphics processing unit, where the graphics processing unit is the graphics processing unit corresponding to the target thread, that is, the target thread is the graphics processing unit corresponding to the target thread. A thread in the corresponding thread pool.

Taking multithreading as an example, when a thread calls the GPU to perform codec processing on a video stream, each thread corresponds to one channel of the video stream input from the monitoring device. When calling the GPU to decode the video stream, the thread analyzes the video stream to improve the decoding work efficiency, and the processed video stream is divided into smaller components. , for example, may be present in the form of packets, ie at least one packet may be included in a video stream of one channel. When the GPU decodes a multi-channel video stream, each channel's video stream is divided into at least one packet, and each thread buffers each channel's video stream in order to facilitate management of the packets corresponding to each channel's video stream. can be set. Specifically, a first buffer and a second buffer corresponding to each thread can be set on the GPU. A first buffer may be used to store video stream data to be decoded and a second buffer may be used to store decoded video stream data.

Video streams captured by surveillance equipment usually include long stretches of image frames with no objects of interest, such as unmanned scenes at night, so the video images of such scenes are not very meaningful. , while a large amount of decoding imposes a heavy load on the terminal equipment, the storage after decoding occupies a large memory space. Therefore, in the embodiments of the present application, it is possible to improve the decoding efficiency by performing frame skip determination on the image of each frame in the video stream.

Specifically, the step of obtaining the target decoded video data by using the target thread to call the graphics processing unit and decode the target channel video stream as described above includes: a step of determining whether or not the target image needs to be frame-skipped after the target image is decoded, and a step of rejecting the target image if necessary; may contain.

The step of determining whether the target image needs to be frame-skipped includes the step of detecting whether the target image contains a predetermined type of object; if not, determining that the target image needs to be frame-skipped. Predetermined types of objects may be set according to the needs of a specific scene, for example, in the case of surveillance equipment, the images in the captured video are the objects that the surveillance equipment needs to monitor, such as people or It may be determined according to the vehicle. For example, if the monitoring equipment is for monitoring the traffic of people, the predetermined type of object may be a person, and if the monitoring equipment is for monitoring vehicles, the predetermined type The object of may be a vehicle, but of course the above is only an example, and in a practical application, a given type of object may include one or more objects, such as people, vehicles, etc. In addition to objects, various moving objects such as animals and aircraft may also be included. In addition, the detection of objects (such as vehicles, people, etc.) in the target image is a common technical means in this field, and specific reference can be made to the prior art, and the description is omitted here. .

In the embodiment of the present application, a polling check can be performed on the video stream of each channel input from the monitoring equipment, and when it is found that the video stream of a certain channel has been decoded, the video stream of that channel is need to be played to determine if the decoding of the video stream is abnormal. If the decoding of the video stream is abnormal, it can be played back and re-decoded. Specifically, the video stream decoding method in the embodiments of the present application includes the steps of determining whether or not the video stream of the target channel needs to be reproduced when the decoding of the video stream of the target channel is finished; if not, a step of restarting the target thread, using the target thread again to call the graphics processing unit, and decoding the video stream of the target channel to obtain the target decoded video data; and recovering. Since thread resources in thread pools corresponding to GPUs in terminal equipment systems are limited, the number of threads that can be executed simultaneously is limited. Also, under default conditions, after a thread terminates, its corresponding GPU resource is not freed. As the thread pool for the GPU becomes exhausted and new threads cannot be created, the threads can be recovered in an appropriate manner so that the video stream decoding capability of the terminal device can be restored in a timely manner, thereby improving the decoding efficiency. can be improved.

Currently, there are various methods for encoding video streams, and each encoding method has a corresponding decoding method. When decoding a video stream, it is necessary to know how this video stream was coded, as well as elements such as resolution, color, and size of the video stream. Therefore, in the embodiment of the present application, these parameter information (decoding parameter information corresponding to encoding parameter information), i.e., decoding rules, are set before the thread calls the GPU to perform decoding processing. need to get. Parameter information may include decoding format, resolution, color, size, and the like. A codec's main function is to compress and decompress a video stream, and the decoding format can be used to indicate how the video stream is decompressed. These parameter information are arranged according to specific format requirements to obtain decoding rules. Decoding rules can be used to indicate requirements for decoding a video stream.

According to embodiments of the present application, if this pre-obtained decoding rule is followed, the related architecture of the video decoding library, such as the Compute Unified Device Architecture (CUDA) or the CUVID architecture, can be set, and , the GPU can realize decoding processing of video stream packets input from the monitoring equipment. In a specific implementation, a format change callback function, a decode callback function, and a display callback function are used when decoding video stream packets input from a monitoring device. ) are used. The format change callback function is called mainly when the format of the video stream is changed. A network glitch may change the format of the video stream, in which case a format change callback function can be called to change the format of the changed video stream back to the original format. The decode callback function can be used to decode packets, the decoded packets can be in the form of frames, and the display callback function can be used to display or store frames. can.

It should be noted that in the embodiment of the present application, if the GPU is capable of decoding the video stream input from the surveillance equipment, the GPU uses the CUVID video decoding library based on the CUDA architecture to A video stream may be decoded, and embodiments herein are not limited to the decoding architecture used by the GPU.

In one possible implementation, the format of the video stream input from the surveillance equipment is H.264. H.264 format, and after the GPU decodes the video stream input from the surveillance equipment, the format of the resulting decoded video data may be NV12 format. In an embodiment of the present application, the process of obtaining decoded video data by decoding the video stream input from the surveillance equipment by the GPU may further include a format conversion process. For example, the step of decoding the video stream input from the monitoring equipment by the GPU to obtain the decoded video data includes decoding the video stream input from the monitoring equipment by the GPU to obtain the first decoded video data. and performing a format conversion process on the first decoded video data to obtain second decoded video data. For example, if the format of the video stream input from the surveillance equipment is H.264. 264 format and the first decoded video data is NV12 format, format conversion processing is performed on the format of the first decoded video data, that is, format conversion of the first decoded video data in NV12 format is performed. to obtain the second decoded video data in RGB24 format. In one example, the GPU may use a CUDA architecture to format convert the first decoded video data, and embodiments herein are not limited to the format conversion method used by the GPU. If the video stream input from the surveillance equipment contains multiple channels, the format conversion process may be different when the GPU decodes different multi-channel video streams, for example, converting to different formats. In this way, it becomes possible to output video image frames in multiple formats, in different color spaces, to memories such as system memory, shared memory, and disk files.

In one possible implementation, the GPU performs format conversion processing on the second decoded video data, and then performs scaling processing on the second decoded video data to obtain decoded data of an appropriate size. You can get it. At this time, the step of obtaining decoded video data by decoding the video stream input from the monitoring equipment by the GPU performs scaling processing of the second decoded video data to obtain third decoded video data. Further steps may be included. Note that when the video stream input from the monitoring device includes multiple channels, the parameters corresponding to the scaling process when decoding the multi-channel video streams with different GPUs may not be the same. In this way, output of video image frames in multiple sizes to memory such as system memory, shared memory, and disk files can be achieved.

In one possible implementation, the video stream decoding device of the present embodiment further comprises a frame rate controller for receiving decoding frame rate information set by the user, and according to this decoding frame rate information, the graphics processing device The speed of the decoding process of the video stream by can be controlled, for example, 20 frames/second. When the video stream input from the monitoring equipment includes a multi-channel video stream, the decoding frame rate information corresponding to each channel video stream can be set individually. It will be appreciated that the frame rate may be 20 frames/second, the decoding frame rate of the second channel video stream may be 30 frames/second, etc., and is not limited here.

Specifically, the graphics processing device can obtain decoded video data by performing parallel decoding processing on the multi-channel video stream after receiving the multi-channel video stream input from the monitoring equipment. . Since GPUs typically do not support decoding video streams of all formats, embodiments herein may determine whether the GPU supports decoding the current video stream.

Specifically, in one possible implementation, the central processing unit obtains the performance parameters of the GPU and performs Based on the format, it may be determined whether the GPU supports hardware decoding of the currently received video stream. For example, if the format of the video stream input from the surveillance equipment is H.264. H.264 format, the video stream decoding device obtains the performance parameters of the GPU, and based on the performance parameters of the GPU, determines that the GPU supports hardware decoding of the video stream of that format. , the GPU can determine that it supports hardware decoding of the format, and then decode the video stream input from the monitoring equipment to obtain decoded video data.

In the embodiments of the present application, a central processing unit (CPU) in a terminal device is a very large scale integrated circuit that serves as a computing core (Core) and a control core (Control Unit) of a computer. Its function is mainly to interpret computer instructions and process data in computer software. The central processing unit mainly includes arithmetic logic unit (ALU, Arithmetic Logic Unit), cache memory (Cache), and a bus (Bus) for transmitting and receiving data, control and status between them. are collectively referred to as the three core components of a computer, including internal memory (Memory) and input/output (I/O) devices.

At present, as one possible method, video decoding can be performed by a central processing unit (CPU, Central Processing Unit) alone. is very difficult to do. Therefore, in the embodiment of the present application, it is preferable to obtain decoded video data by decoding the video stream input from the monitoring equipment by the graphics processing device.

In one possible implementation mode, when the video stream decoding device determines that the GPU does not support hardware decoding of the video stream input from the monitoring equipment, the CPU of the terminal equipment decodes the video input from the monitoring equipment. The stream may be decoded and the decoded video data after the decoding process may be transmitted to the GPU. In one example, the CPU can use FFMPGE (Fast Forward Mpeg) to decode the video stream input from the surveillance equipment, and FFMPGE can be used to record, convert, and convert digital audio and video. A set of open source computer programs that can be converted to streams. In this way, the video formats supported by the GPU are decoded preferentially by the GPU, and the video formats not supported by the GPU are decoded using the CPU. While the utilization rate of the GPU can be greatly improved and the decoding efficiency can be improved, the resources of the CPU can be saved and the processing efficiency of other tasks by the CPU can be improved.

In embodiments of the present application, the memory of the terminal device may include one or more memories of different types, specifically the memory of the terminal device may include, for example, system memory, shared memory, and It may contain multiple memories such as disk files. At this time, the video stream decoding method further includes obtaining a memory corresponding to each of the multi-channel decoded video data, and outputting the multi-channel decoded video data to the memory via the graphics processing unit. .

The multi-channel decoded video data in the embodiments of the present application may include first decoded video data and second decoded video data, wherein a memory corresponding to each of the multi-channel decoded video data is stored as: The obtaining step includes obtaining a first memory corresponding to the first decoded video data and obtaining a second memory corresponding to the second decoded video data, and graphics processing The step of outputting the multi-channel decoded video data to the memory via the device includes outputting the first decoded video data to the first memory and outputting the second decoded video data to the second memory via the graphics processing device. , wherein the first memory and the second memory are different memories within the terminal device. In this way, decoding of a multi-channel video stream by the GPU and divisional output to different memories become possible, and moreover, image frame data corresponding to the video stream are stored in a storage medium other than the GPU, such as system memory, shared memory, and more compatible with practical application scenes, such as those that need to be output to a disc or the like.

The first decoded video data may comprise decoded video data corresponding to at least one channel of video stream, the second decoded video data may also comprise decoded video data corresponding to at least one channel of video stream, In some embodiments of the present application, the multi-channel decoded video data comprises first decoded video data and second decoded video data, i.e. the multi-channel video stream comprises a first video stream and a second decoded video data. divided into two video streams, the multi-channel decoded video data is divided into first decoded video data and second decoded video data, for example, the multi-channel video stream has 10 channels; The first video stream includes a 5-channel video stream, the second video stream includes a 5-channel video stream, and the first decoded video data for the 5-channel video stream in the first video stream is , and the second decoded video data for the 5-channel video stream in the second video stream may be output to the second memory.

In some other embodiments of the present application, the first decoded video data and the second decoded video data may be part of the multi-channel decoded video data, e.g. There are 10-channel video streams, the first video stream comprises a 5-channel video stream, the second video stream comprises a 2-channel video stream, the multi-channel video stream includes the first video stream and there is another video stream in addition to the second video stream. In one specific embodiment, said multi-channel video stream further comprises a third video stream, i.e. said multi-channel decoded video data further comprises third decoded video data, wherein: Obtaining a memory corresponding to each of the multi-channel decoded video data further includes obtaining a third memory corresponding to a third decoded video data, wherein, via the graphics processing unit, decoding Outputting the video data to the memory includes outputting third decoded video data to the third memory via the graphics processing unit.

"Memory", one of the most important functional units of a computer, is a collection of many memories. In order for the CPU to accurately find the memory that stores certain information, it is necessary to assign mutually identifiable "ID numbers" to these units, and this "ID number" is the address code. In an embedded processor, multiple types of Memories are integrated, and the same type of Memory is usually called a Memory Block. In general, a processor designer allocates a set of consecutive natural numbers expressed in hexadecimal numbers equal to the number of memories to each Memory Block, and uses them as address codes of the Memory Block. A correspondence relationship between such a set of natural numbers and a memory block is a memory map, which is sometimes called an address map. In fact, the Address Map is more relevant in a literal sense. Memory Map is an operation that prepares a computer system for reset (power-on) and is an automatic process that assigns processor-owned address code resources to respective physical memory blocks in the system.

In embodiments of the present application, the graphics processing unit can be used to output decoded video data to memory through GPU COPY technology and memory mapping (Memory Map). Specifically, the memory mapping task in the prior art is left to the execution unit (Execution Unit, EU) of the GPU. The EU is the execution unit within the GPU processor, responsible for executing instructions, and in fact combines both the functions of a controller and an arithmetic unit.

In one specific embodiment, the step of outputting the multi-channel decoded video data to the memory via the graphics processing unit includes obtaining, by the graphics processing unit, a memory address code corresponding to the multi-channel decoded video data. determining a target memory corresponding to multi-channel decoded video data by performing memory mapping based on the address code of this memory by a graphics processing unit; and outputting the decoded video data of to a target memory.

In an embodiment of the present application, a multi-channel video stream input from a monitoring device is acquired, multiple threads are created in a thread pool corresponding to a graphics processing unit, and multi-channel video is generated based on the multiple threads. Multi-channel decoded video data is obtained by transmitting the stream to a graphics processing device to perform video decoding processing. In an embodiment of the present application, by creating multiple threads in a thread pool corresponding to the GPU, the GPU is used instead of the CPU to complete the task of parallel decoding of multi-channel video stream data, thereby enabling the device Greatly reduces overall energy consumption, improves video decoding efficiency and GPU device utilization, reduces CPU dependency, and is widely used in video stream decoding scenes where the CPU processing power of terminal equipment is not strong can be

As shown in FIG. 3, the video stream decoding method in the embodiment of the present application will be described below together with specific application scenes.

At 301, a 10-channel video stream is acquired from the surveillance equipment.

In this embodiment, the terminal device includes a CPU, a GPU, and a memory, the memory includes a shared memory and a system memory, and the number of monitoring devices connected to the terminal device may be ten, each monitoring device transmits a video stream corresponding to one channel of surveillance video.

At 302, the 10-channel video stream is transmitted to the GPU of the terminal device.

In this embodiment, assuming that the terminal device includes only one GPU, 20 threads can be set in the thread pool corresponding to the GPU. It can handle processing of 20 channels of video streams, indicating that the processing capability of the GPU is greater than the number of video stream channels. In this case, the 10-channel video stream can be directly transmitted to the GPU of this terminal device.

At 303, 10 threads are created in the thread pool corresponding to the GPU.

In this embodiment, the step of creating 10 threads in the thread pool corresponding to the GPU acquires information on the number of threads (10) input by the user, and based on the information on the number of threads (10) , to create 10 threads in the thread pool corresponding to the GPU. Further, the step of creating 10 threads in the thread pool corresponding to the GPU obtains the number of video channels (10 channels) of the video streams input from the 10 surveillance videos, and in the thread pool corresponding to the GPU , to create as many threads as the number of video channels to get 10 threads.

At 304, one thread of the ten threads is assigned on a one-to-one basis to each channel video stream of the ten channel video streams.

At 305, 10 threads are used to call the GPU to decode the 10-channel video streams in parallel, thereby obtaining 10-channel decoded video data.

The terminal device is provided with a frame rate controller, receives the decoding frame rate information set by the user, and according to this decoding frame rate information, the speed of the decoding processing of the video stream of each channel by the GPU is adjusted to, for example, 20 frames/second. etc. can be controlled. Of course, the decoding frame rate of these 10 channel video streams may be set individually, for example, the decoding frame rate of the video stream of the first channel is 20 frames per second, and the decoding frame rate of the video stream of the second channel is 20 frames per second. The rate may be 30 frames/second, or the like.

At 306, the five channels of decoded video data are output to the terminal's system memory and the five channels of decoded video data are output to the terminal's shared memory via the GPU.

In this embodiment, it is assumed that the memory of the terminal device includes a system memory and a shared memory, and these 10-channel video streams are output to the system memory and the shared memory respectively. The decoded video data is output to the system memory and the 5-channel decoded video data is output to the shared memory. Therefore, it is possible to directly output the 5-channel decoded video data to the system memory of the terminal device via the GPU and output the 5-channel decoded video data to the shared memory of the terminal device.

In this embodiment, a 10-channel video stream input from a monitoring device is acquired, the 10-channel video stream is transmitted to the GPU of the terminal device, and the GPU of the terminal device decodes the 10-channel video stream. , 10 channels of decoded video data are obtained, and the decoded video data are output to the system memory and shared memory of the terminal device via the GPU, respectively. Embodiments of the present application use the GPU instead of the CPU to complete the decoding work of the video stream data, which greatly reduces the energy consumption of the entire device and improves the video decoding efficiency and utilization of the GPU device. At the same time, it can directly output the decoded data from the GPU to the system memory, further reducing the dependence on the CPU, and better meet the actual application scene where the frame data needs to be output to a storage medium other than the GPU. , to meet more application scenes.

Although the individual steps of each flowchart in the foregoing embodiments are shown in order as indicated by the arrows, it is understood that these steps are not necessarily performed in order as indicated by the arrows. Will. Unless explicitly indicated herein, there is no strict order of performance of these steps, and these steps may be performed in other orders. Also, at least some of the steps in each flowchart may include multiple steps or multiple stages, and the execution order of these steps or stages is not necessarily executed at the same time. and the order of execution of these steps or stages is not necessarily sequential, but may alternate or alternate with other steps or at least portions of steps or stages in other steps. may be executed.

To better implement the video stream decoding method according to the embodiments of the present application, the embodiments of the present application further provide an apparatus based on the above video stream decoding method. The meaning of the noun is the same as in the video stream decoding method described above, and the description in the method embodiment can be referred to for specific implementation details.

In some embodiments of the present application, the decoding unit 403 specifically allocates one thread of the plurality of threads to each channel video stream of the multi-channel video stream on a one-to-one basis. each of the threads is a target thread, and the target thread is used to input the video stream of the target channel to the graphics processing device, and the video stream of the target channel corresponds to the target thread among the multi-channel video streams. The target channel video stream is a one-channel video stream, and the target channel video stream is decoded by the graphics processing device to obtain target decoded video data.

The acquisition unit 401 acquires multi-channel video streams input from surveillance equipment.

In some embodiments of the present application, the decoding unit 403 specifically restarts the target thread when the decoding of the target channel video stream is finished and the target channel video stream needs to be played. When activated, the target thread is used again to input the video stream of the target channel to the graphics processing device, and the video stream of the target channel is decoded by the graphics processing device to obtain target decoded video data.

The decoding unit 403 transmits the multi-channel video stream to the graphics processing device for video decoding processing based on multiple threads to obtain multi-channel decoded video data.

In some embodiments of the present application, the creation unit 402 specifically obtains an input number of threads, creates multiple threads in a thread pool corresponding to the graphics processing device based on the number of threads, and The number of threads equals the number of threads.

In some embodiments of the present application, the production unit 402 specifically obtains the video channel number of the multi-channel video stream via the central processing unit, and in the thread pool corresponding to the graphics processing unit, the video channel Create as many threads as the number.

In some embodiments of the present application, the decoding unit 403 specifically allocates one thread of the plurality of threads to each channel video stream of the multi-channel video stream on a one-to-one basis. Each of the threads is a target thread, and the target thread is used to input the target video stream to the graphics processing device, and the video stream of the target channel is one channel of the multi-channel video stream corresponding to the target thread. The video stream of the target channel is further decoded by the graphics processing device to obtain target decoded video data.

In some embodiments of the present application, the decoding unit 403 specifically sequentially takes the image of each frame in the video stream of the target channel as the target image, decodes the target image by the graphics processing device, and converts the target image into the target image. If the predetermined type of object is not included, it is determined that the target image should be frame-skipped.

In some embodiments of the present application, the decoding unit 403 specifically restarts the target thread when the decoding of the target channel video stream is finished and the target channel video stream needs to be played. When activated, the target thread is used again to input the target video stream to the graphics processing device, and the graphics processing device decodes the video stream of the target channel to obtain target decoded video data.

In some embodiments of the present application, the decoding unit 403 specifically further reclaims the target thread when the decoding of the target video stream is finished and there is no need to play the video stream of the target channel. do.

In some embodiments of the present application, the terminal device further comprises a memory, the device further comprises an output unit, the output unit obtains the memory corresponding to each of the multi-channel decoded video data, and the graphics processing device output the multi-channel decoded video data to the memory via.

In some embodiments of the present application, the multi-channel decoded video data includes first decoded video data and second decoded video data, and the output unit specifically decodes the first decoded video data. Obtaining a first memory corresponding to video data; obtaining a second memory corresponding to second decoded video data; and outputting the first decoded video data to the first memory via a graphics processing unit. and outputs the second decoded video data to the second memory.

In some embodiments herein, the multi-channel decoded video data includes first decoded video data and second decoded video data.

In some embodiments of the present application, the multi-channel decoded video data further includes third decoded video data, and the output unit specifically outputs the third decoded video data corresponding to the third decoded video data. 3 memory, and outputs third decoded video data to the third memory via the graphics processing unit.

In some embodiments herein, the memory includes at least one of system memory, shared memory, and disk.

In some embodiments of the present application, the output unit specifically obtains the address code of the memory corresponding to the multi-channel decoded video data by the graphics processing device, and the graphics processing device performs the A target memory corresponding to the multi-channel decoded video data is determined by performing memory mapping, and the multi-channel decoded video data is output to the target memory via the graphics processing unit.

In an embodiment of the present application, the acquisition unit 401 acquires the multi-channel video stream input from the surveillance equipment, the creation unit 402 creates multiple threads in a thread pool corresponding to the graphics processing device, and the decoding unit 403 transmits the multi-channel video stream to the graphics processing device for video decoding processing according to the plurality of threads, thereby obtaining multi-channel decoded video data. In an embodiment of the present application, by creating multiple threads in a thread pool corresponding to the GPU, the GPU is used instead of the CPU to complete the task of parallel decoding of multi-channel video stream data, thereby enabling the device Greatly reduces overall energy consumption, improves video decoding efficiency and GPU device utilization, reduces CPU dependency, and is widely used in video stream decoding scenes where the CPU processing power of terminal equipment is not strong can be

In the specific implementation, each of the above units may be implemented as an independent entity, or may be implemented as the same or several entities in any combination. Reference can be made to the foregoing method embodiments.

FIG. 5 is a schematic diagram showing the structure of another implementation of a video stream decoding device 500 in an embodiment of the present application, which video stream decoding device 500 comprises a management control module 501 and a decoding module 502 . Each module included in the video stream decoding device may be implemented in whole or in part by software, hardware, or a combination thereof.

The management control module 501 acquires the video stream input from the monitoring equipment and transmits the video stream to the graphics processing device.

The decoding module 502 obtains decoded video data by decoding the video stream through the graphics processing device, and outputs the decoded video data to the memory via the graphics processing device.

Management control module 501 and decoding module 502 may include further modules, for example, in FIG. 5 management control module 501 includes parameter management module, thread management module, state detection module, and reconnection control module may include The parameter management module manages decoding rules and parameters related to decoding (e.g. color space, output frame size, etc.), the thread management module manages threads in the thread pool corresponding to the GPU, and the state detection module manages the current and the reconnection control module detects whether the target video stream needs to be re-decoded and played back to avoid decoding anomalies. Similarly, the decoding module 502 may include modules such as a parameter manager, a video splicer, a decoded frame skip controller, a decoded frame rate controller, an output frame selector, and the like. The parameter management unit acquires the decoding rules and decoding-related parameters in the parameter management module in the management control module 501, and the video connection unit is connected to the monitoring equipment for multi-channel video streams input from the monitoring equipment. and the decoding frame skip control determines whether the current image frame needs to be frame-skipped during decoding of the video stream, for example, as described in the above embodiments, the image By detecting whether or not a predetermined type of object is included in the image, it is determined whether or not the image needs to be frame-skipped. The decoding frame rate control unit sets the decoding frame rate and controls the decoding frame rate to control the decoding rate of the video stream. The output frame selection unit selects the image frames of the output decoded data. For example, when the output frame selection unit sets the selected object to be "car", the decoded data includes "car". Select and output image frames.

Note that in this embodiment, only one management control module 501 may be provided, and a plurality of decryption modules 502 may be provided. The number of decoding modules 502 can correspond to the number of video channels of the input video stream. For example, if a 10-channel video stream is input, 10 decoding modules 502 can be set.

Embodiments of the present application further provide a terminal device, comprising a processor and a memory, the memory storing computer readable instructions, the computer readable instructions being executed by the processor to to perform the steps of the video stream decoding method provided herein.

Specifically, in one embodiment, the processor includes a central processing unit and a graphics processing unit, and the computer readable instructions include first computer readable instructions and second computer readable instructions. , the first computer readable instructions, when executed by the central processing unit, cause the central processing unit to acquire a multi-channel video stream input from the monitoring equipment; and a thread pool corresponding to the graphics processing unit. in creating a plurality of threads; and transmitting a multi-channel video stream to a graphics processing device based on the plurality of threads. Also, the second computer-readable instructions, when executed by the graphics processing device, cause the graphics processing device to perform parallel decoding processing on the multi-channel video stream to obtain multi-channel decoded video data. perform the steps.

FIG. 6 shows a schematic diagram showing the structure of the terminal device according to the embodiment of the present application. Specifically, the terminal device may include components such as one or more processing core processors 601 , one or more computer-readable storage media memory 602 , power supply 603 and input unit 604 . The structure of the terminal equipment shown in FIG. 6 is not intended to limit the terminal equipment and may include more or fewer components than those shown, or specific combinations of components, or different arrangements of components. A good thing will be understood by those skilled in the art.

Processor 601 is the control center for this terminal equipment, connecting parts of the overall terminal equipment using various interfaces and lines, executing software programs and/or modules stored in memory 602, executing software programs and/or modules stored in memory 602, Various functions and data processing of the terminal equipment are performed by calling up the stored data, and overall monitoring of the terminal equipment is performed. Alternatively, processor 601 may include one or more processing cores. Preferably, the processor 601 may be integrated with an application processor and a modem processor, the application processor mainly handling operating systems, user interfaces, application programs, etc., and the modem processor mainly handling wireless communications. deal. It will be appreciated that the modem processor need not be integral with processor 601 .

Memory 602 may be used to store software programs and modules, and processor 601 performs various functional applications and data processing by executing the software programs and modules stored in memory 602. The memory 602 may primarily include a program storage area and a data storage area, where the program storage area includes an operating system, applications required for at least one function (e.g., audio playback function, image playback function, etc.). A program may be stored, and the data storage area may store data created according to the usage status of the terminal device. Memory 602 may also include high speed random access memory and may also include non-volatile memory such as, for example, at least one disk memory device, flash memory device, or other volatile solid-state memory device. Accordingly, memory 602 may further include a memory controller that provides access to memory 602 by processor 601 .

The terminal equipment further includes a power supply 603 for powering each component, preferably the power supply 603 may be logically connected to the processor 601 via a power management system, whereby Through the power management system, it implements functions such as charging, discharging, and managing power consumption. Also, power supply 603 may further include optional components such as one or more DC or AC power supplies, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The terminal may further include an input unit 604 for receiving input digital or textual information and for generating keyboard, mouse, joystick, optical or trackball signal inputs associated with controlling user settings and functions. .

Also, although not shown, the terminal device may further include a display unit and the like. Specifically, in this embodiment, the processor 601 in the terminal device loads executable files corresponding to one or more application program processes into the memory 602 according to instructions such as Various functions are realized by executing the application programs stored in the memory 602 . The command is

An instruction to acquire a multi-channel video stream input from a monitoring device, an instruction to create a plurality of threads in a thread pool corresponding to the graphics processing device, and a multi-channel video stream based on the plurality of threads. and instructions for obtaining multi-channel decoded video data by transmitting it to a graphics processing unit to perform a video decoding process.

In one embodiment, there is provided a computer readable storage medium having computer readable instructions stored therein which, when executed by a processor, cause the processor to perform the steps of the video stream decoding method described above. let it run. The steps of the video stream decoding method referred to here may be the steps of the video stream decoding method in each of the above-described embodiments.

Implementing all or part of the flow in the methods of the above-described embodiments can be completed by instructing relevant hardware by computer-readable instructions, and the program is a non-volatile computer-readable Those skilled in the art will appreciate that this program, which may be stored in a storage medium, may include the flow of each method embodiment described above when executed. Any reference to memory, storage, database, or other medium used in each of the embodiments provided herein may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of example and not limitation, RAM may be Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink It is available in various forms such as DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be combined arbitrarily, and for the sake of brevity, not all possible combinations of the technical features of the above embodiments have been described. So long as the combination of features is not inconsistent, it should be considered within the scope described herein.

The above embodiments merely show some embodiments of the present application, and although the descriptions thereof are relatively specific and detailed, they should not be understood to limit the scope of the invention patent. In addition, it should be noted that those skilled in the art can make several modifications and improvements without departing from the spirit of the present application, and they are all within the protection scope of the present application. Therefore, the scope of protection of the present patent application should be determined based on the attached claims.

101 monitoring equipment 102 terminal equipment 1021 central processing unit 1022 graphics processing unit 1023 memory 400 video stream decoding device 401 acquisition unit 402 creation unit 403 decoding unit 500 video stream decoding device 501 management control module 502 decoding module 601 processor 602 memory 603 power supply 604 input unit

Claims (14)

A video stream decoding method performed by a terminal device,
obtaining multi-channel video streams input from surveillance equipment;
creating a plurality of threads in a thread pool corresponding to the graphics processing unit;
A step of obtaining multi-channel decoded video data by transmitting the multi-channel video stream to the graphics processing device for video decoding processing based on the plurality of threads,
assigning one thread out of the plurality of threads to each channel video stream of the multi-channel video stream;
each thread of the plurality of threads being a target thread, and using the target thread to input a video stream of a target channel to the graphics processing unit;
When the decoding of the target channel video stream is finished and the target channel video stream needs to be played, restart the target thread and use the target thread again to play the target channel video. inputting a stream to the graphics processing unit, and decoding the video stream of the target channel by the graphics processing unit to obtain target decoded video data ;
A video stream decoding method comprising: creating a plurality of threads in a thread pool corresponding to the graphics processing unit;
obtaining the number of threads entered;
creating the plurality of threads in a thread pool corresponding to the graphics processing device based on the number of threads, wherein the number of the plurality of threads is equal to the number of threads. 2. The video stream decoding method according to claim 1, wherein: creating a plurality of threads in a thread pool corresponding to the graphics processing unit;
obtaining a number of video channels of the multi-channel video stream;
2. The video stream decoding method according to claim 1, further comprising: creating a number of threads equal to the number of video channels in a thread pool corresponding to the graphics processing device. The assigning step includes assigning one thread of the plurality of threads to each channel video stream of the multi-channel video stream on a one-to-one basis;
4. The method of claim 3, wherein the target channel video stream is a one-channel video stream corresponding to the target thread among the multi-channel video streams. The step of obtaining target decoded video data by decoding the video stream of the target channel by the graphics processing device,
a step of sequentially setting an image of each frame in the video stream of the target channel as a target image, and decoding the target image by the graphics processing device;
5. The method of claim 4, comprising determining that the target image needs to be frame-skipped if the target image does not contain a predetermined type of object. 5. The method of claim 4, further comprising reclaiming the target thread when decoding of the target channel video stream is finished and there is no need to play the target channel video stream. video stream decoding method. The terminal device further comprises a memory,
The video stream decoding method comprises:
obtaining a memory corresponding to each of the multi-channel decoded video data;
2. The method of claim 1, further comprising outputting the multi-channel decoded video data to the memory via the graphics processing unit. the multi-channel decoded video data includes first decoded video data and second decoded video data;
The step of obtaining memory corresponding to each of the multi-channel decoded video data comprises:
obtaining a first memory corresponding to the first decoded video data;
obtaining a second memory corresponding to the second decoded video data;
The step of outputting the multi-channel decoded video data to the memory via the graphics processing unit comprises:
outputting said first decoded video data to said first memory and outputting said second decoded video data to said second memory via said graphics processing unit. 8. A video stream decoding method according to Item 7 . the multi-channel decoded video data further includes third decoded video data;
The step of obtaining memory corresponding to each of the multi-channel decoded video data comprises:
further comprising obtaining a third memory corresponding to the third decoded video data;
The step of outputting the multi-channel decoded video data to the memory via the graphics processing unit comprises:
9. The method of claim 8 , further comprising outputting the third decoded video data to the third memory via the graphics processing unit. 8. The method of claim 7 , wherein the memory comprises at least one of system memory, shared memory, and disk. The step of outputting the multi-channel decoded video data to the memory via the graphics processing unit comprises:
obtaining, by the graphics processing unit, a memory address code corresponding to multi-channel decoded video data;
determining, by the graphics processing unit, a target memory corresponding to the multi-channel decoded video data by performing memory mapping based on the address code;
and outputting multi - channel decoded video data to the target memory via the graphics processing unit. an acquisition unit for acquiring multi-channel video streams input from surveillance equipment;
a creation unit for creating a plurality of threads in a thread pool corresponding to the graphics processing unit;
a decoding unit for obtaining multi-channel decoded video data by transmitting the multi-channel video stream to the graphics processing device to perform video decoding processing based on the plurality of threads ,
Allocating one thread out of the plurality of threads to each channel video stream of the multi-channel video stream;
Each thread of the plurality of threads is set as a target thread, and a video stream of a target channel is input to the graphics processing device using the target thread;
When the decoding of the target channel video stream is finished and the target channel video stream needs to be played, restart the target thread and use the target thread again to play the target channel video. a decoding unit configured to input a stream to the graphics processing device and decode the video stream of the target channel by the graphics processing device to obtain target decoded video data ;
A video stream decoding device comprising: a memory in which computer readable instructions are stored;
a processor;
A terminal device, wherein the computer readable instructions, when executed by the processor, cause the processor to perform the method steps of any one of claims 1 to 11 . A program for causing a computer to perform the steps of the method according to any one of claims 1-11 .

Images