WO2022237165A1 - 视频监控数据的处理方法及装置 - Google Patents

视频监控数据的处理方法及装置 Download PDF

Info

Publication number
WO2022237165A1
WO2022237165A1 PCT/CN2021/138533 CN2021138533W WO2022237165A1 WO 2022237165 A1 WO2022237165 A1 WO 2022237165A1 CN 2021138533 W CN2021138533 W CN 2021138533W WO 2022237165 A1 WO2022237165 A1 WO 2022237165A1
Authority
WO
WIPO (PCT)
Prior art keywords
processed
image frames
processor
processing
image
Prior art date
Application number
PCT/CN2021/138533
Other languages
English (en)
French (fr)
Inventor
吴学文
韩承志
王小辉
葛建壮
陈安伟
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022237165A1 publication Critical patent/WO2022237165A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present application relates to the field of video surveillance, in particular to a method and device for processing video surveillance data.
  • the data processing device usually analyzes and processes the surveillance video collected by multiple cameras, so as to realize the intelligent monitoring of different services by the intelligent video surveillance system.
  • the intelligent video surveillance system can realize the intelligent monitoring of whether there are workers without hard hats on the construction site.
  • the data processing device when the above-mentioned data processing device analyzes and processes the surveillance videos collected by multiple cameras, for one surveillance video (the surveillance video collected by one camera can be called one surveillance video), the data processing device adopts serial processing method for its Carry out the following processing: video reading, video image preprocessing, target detection on video images (including target detection preprocessing, image reasoning, model postprocessing), target tracking in video images, logic processing of monitoring services, and alarm signals output etc.
  • the present application provides a method and device for processing video surveillance data.
  • the method can increase the number of channels that the video surveillance data processing device can analyze and process the surveillance video at the same time, thereby improving the data processing efficiency of the video surveillance system.
  • the present application provides a method for processing video surveillance data, the method being executed by a video surveillance data processing device including a first processor and a second processor.
  • the method may include: the first processor acquires a plurality of image frames to be processed from multiple surveillance video streams, and buffers the plurality of image frames to be processed in a first processing queue.
  • the second processor performs image reasoning on the plurality of image frames to be processed obtained from the first processing queue, so as to determine target information in the plurality of image frames to be processed.
  • the device for processing video surveillance data can decouple the arithmetic processing performed by the first processor and the second processor through the first processing queue, so that the first processor processes the number of channels of the surveillance video stream in parallel, It is no longer limited by the number of paths that the second processor can parallelly process the monitoring video streams, so that when the data processing capability and resources of the second processor are limited, the processing device for video surveillance data will not be limited to simultaneously process the paths of the monitoring video streams. number. That is to say, compared with the general processor and graphics processor of the processing device for video surveillance data, the number of channels of surveillance video streams that can be processed simultaneously when the surveillance video stream data is serially analyzed and processed, the method provided by the application improves the quality of video surveillance data. The processing device simultaneously processes the number of monitoring video streams, that is, the method provided by this application improves the data processing efficiency of the video monitoring system.
  • the above-mentioned second processor performs image reasoning on a plurality of image frames to be processed acquired from the first processing queue, including: the second processor executes image reasoning on the plurality of image frames to be processed through n processes running in parallel Image frames to be processed for image inference.
  • n is a positive integer
  • the value of n is determined according to the data processing capability of the second processor.
  • the above-mentioned first processor obtains a plurality of image frames to be processed from multiple monitoring video streams, including: the first processor processes m processes running in parallel, Acquire multiple image frames to be processed, m is a positive integer greater than 1.
  • the first processor and the second processor can respectively concurrently process multiple processes to process the data of multiple monitoring video streams, and the number of concurrent processes of the first processor and the second processor is irrelevant. That is to say, the method of the present application decouples the calculation and processing performed by the first processor and the second processor respectively, so that the first processor processes the number of monitoring video streams in parallel, and is no longer affected by the parallel processing of the monitoring video by the second processor.
  • the number of streams is limited, so that when the data processing capability and resources of the second processor are limited, the video monitoring data processing device will not limit the number of channels of monitoring video streams that can be processed simultaneously.
  • the aforementioned plurality of image frames to be processed are image frames corresponding to moving objects in the surveillance video stream to which the plurality of image frames to be processed belong.
  • the aforementioned plurality of image frames to be processed are image frames extracted from every t frames of images in the surveillance video stream to which the plurality of image frames to be processed belong, and t is a positive integer.
  • the first processor may determine, in the surveillance video stream to be analyzed, image frames to be processed that need to be subjected to graphics reasoning.
  • the first processor determines the image frame corresponding to the moving target in the monitoring video stream to be analyzed as the image frame to be processed, it can effectively avoid the image frame that does not need to be analyzed in the monitoring video stream (that is, there is no change in the monitoring video stream) image frames) to perform graphical reasoning, thereby saving the resources of the second processor for performing graphical reasoning on the image frames, so that the resources saved by the second processor can be used to process more channels of surveillance video streams
  • Data that is, the number of channels that the video surveillance data processing device can simultaneously process surveillance video streams, that is, the method provided by this application improves the data processing efficiency of the video surveillance system.
  • the above method further includes: the first processor determines whether there is an abnormal event in the surveillance video stream to which the multiple image frames to be processed belong according to target information in the multiple image frames to be processed. If there is an abnormal event in the surveillance video stream to which the multiple image frames to be processed belong, the first processor controls to output an alarm message.
  • the above-mentioned first processor determines whether there is an abnormal event in the surveillance video stream to which the multiple image frames to be processed belong according to the target information in the multiple image frames to be processed, specifically including: first The processor determines whether there is an abnormal event in the surveillance video stream to which the multiple image frames to be processed belong according to a preset algorithm and target information of the multiple image frames to be processed.
  • the preset algorithm is determined according to services monitored by the surveillance video stream to which the multiple image frames to be processed belong.
  • the first processor can determine the abnormal event in the surveillance video stream to which the image frame to be processed belongs based on the graphics reasoning result of the second processor, and alert the user, thereby realizing the video surveillance
  • the system monitors the monitored business.
  • the above target information includes the type and/or position of the target in the image to be processed.
  • the above method before buffering the plurality of image frames to be processed in the first processing queue, the above method further includes: the first processor performs an The image frames to be processed are subjected to target detection preprocessing, so that the plurality of image frames to be processed meet the requirements of the input image input to the reasoning model.
  • the reasoning model is preset in the first processor, and the reasoning model is used to perform graphical reasoning on the plurality of image frames to be processed, so as to determine target information in the image frames to be processed.
  • the target detection preprocessing includes at least one of the following methods: size scaling, normalization processing, black border filling, dimension expansion, and the like.
  • the above method also includes: the first processor performs target detection post-processing based on the graphics inference results of the plurality of image frames to be processed by the second processor to determine Obtain the target information of the plurality of image frames to be processed.
  • the method provided by this application advances the process of target detection preprocessing, That is, the first processor with high concurrent processing capability performs target detection pre-processing, and the target detection post-processing is moved backward, that is, the first processor with high concurrent processing capability performs target detection post-processing.
  • the method provided by the present application further saves the resources of the second processor, so that the saved resources of the second processor can be used to process data of more monitoring video streams, that is, the processing of video monitoring data is improved.
  • the device simultaneously processes the number of monitoring video streams, that is, the method provided by the present application improves the data processing efficiency of the video monitoring system.
  • the present application provides a device for processing video surveillance data.
  • the device for processing video surveillance data is configured to execute any one of the methods provided in the first aspect above.
  • the present application may divide the video monitoring data processing device into functional modules according to any one of the methods provided in the first aspect above.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the present application may divide the video surveillance data processing device into a first processing unit, a second processing unit, and the like according to functions.
  • the device includes a first processor, a second processor and a memory, the first processor and the second processor are configured to call the Computer instructions in the memory, so that the processing device of the video surveillance data executes the method executed by the first processor according to any one of claims 1-8 through the first processor, and executes the method described in the claim through the second processor The method of any one of claims 1-8 performed by the second processor.
  • the present application provides a computer-readable storage medium, the computer-readable storage medium includes program instructions, and when the program instructions are run on a computer or a processor, the computer or the processor executes any of the procedures in the first aspect. Either method provided by a possible implementation.
  • the present application provides a computer program product, which, when running on a video surveillance data processing device or device, enables any method provided in any possible implementation of the first aspect to be executed .
  • any device, device, computer storage medium or computer program product provided above can be applied to the corresponding method provided above, therefore, the beneficial effects it can achieve can refer to the corresponding method The beneficial effects in the above will not be repeated here.
  • FIG. 1 is a schematic diagram of a hardware structure of a data processing device provided in an embodiment of the present application
  • FIG. 2 is a schematic framework diagram of a video surveillance system provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a method for processing video surveillance data provided by an embodiment of the present application
  • Fig. 4 is another schematic diagram of the processing method of video surveillance data provided by the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a device for processing video surveillance data provided by an embodiment of the present application.
  • words such as “exemplary” or “for example” are used as examples, illustrations or illustrations. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as “exemplary” or “such as” is intended to present related concepts in a concrete manner.
  • first and second are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features.
  • a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features.
  • the data processing device serially analyzes and processes the data of each surveillance video stream, it can usually read the surveillance video stream through a central processing unit (CPU), and the read surveillance video The image frames in are preprocessed.
  • the data processing device may perform target detection on the preprocessed image frame through a graphics processing unit (graphics processing unit, GPU), so as to determine target information in the image frame.
  • the data processing device performs logic service post-processing on the detected target information through the CPU, so as to determine whether there is an abnormal event in the surveillance video stream.
  • the abnormal event corresponds to the business monitored by the video monitoring system.
  • the abnormal event in the surveillance video stream is that one or more image frames in the surveillance video stream include flames.
  • the abnormal event in the surveillance video stream is that one or more image frames in the surveillance video stream include There are characters not wearing hard hats. Wait, no more details.
  • the data processing device realizes the intelligent monitoring of the monitored service by the video monitoring system by determining whether there is an abnormal event in the monitoring video stream.
  • the data processing capabilities and resources of the GPU are usually extremely limited, so the number of channels that the GPU can simultaneously process surveillance video streams is extremely limited. Therefore, the serial analysis and processing methods of the CPU and GPU will cause the CPU to process channels of surveillance video streams in parallel. The number is limited by the number of monitoring video streams processed in parallel by the GPU, which leads to an extremely limited number of monitoring video streams that are serially analyzed and processed by the CPU and GPU, which in turn makes the data processing efficiency of the data monitoring system low.
  • the embodiments of the present application provide a method and device for processing video surveillance data.
  • the method is used to analyze and process multiple surveillance video streams in a video surveillance system, so as to realize intelligent surveillance of different services.
  • the embodiment of the present application hereinafter simply refers to the video surveillance data processing device as a data processing device.
  • the method provided by the embodiment of the present application can first determine the number of channels to be monitored in the multiple monitoring video streams through multiple parallel processes of the first processor of the data processing device.
  • the image frames are processed, and the determined image frames to be processed are buffered into the first processing queue.
  • the method uses multiple parallel processes of the second processor of the data processing device to sequentially perform image reasoning on the image frames to be processed in the first processing queue, so as to determine the target information in the image frames to be processed.
  • the method uses multiple parallel processes of the first processor of the data processing device to process the image frames to be processed after image reasoning, so as to determine whether there is an abnormal event in the surveillance video stream to which the image frames to be processed belong, so that Realize the intelligent monitoring of different services by the video surveillance system.
  • one process is used to process one surveillance video stream.
  • the number of processes run by the first processor and the number of processes run by the second processor may be the same or different. It can be seen that, through the method provided by the embodiment of the present application, the number of surveillance video streams processed by the first processor in parallel is no longer limited by the number of surveillance video streams processed by the second processor in parallel, so that the data processing capacity And the second processor with limited resources will not limit the number of channels that the data processing device can simultaneously process the surveillance video streams. That is to say, compared with the number of channels that can simultaneously process the monitoring video stream when the CPU and GPU of the data processing device serially analyze and process the monitoring video stream data, the method provided by the implementation of the present application improves the data processing device's ability to simultaneously process the monitoring video stream. The number of ways, that is, the method provided by the embodiment of the present application improves the data processing efficiency of the video surveillance system.
  • the above-mentioned data processing device may be any computing device with computing processing capabilities, such as a general-purpose computer, a notebook computer, a tablet computer, and other computing devices.
  • the data processing device may also be a server, such as a monitoring server in a video monitoring system, which is not limited in this embodiment of the present application.
  • FIG. 1 shows a schematic diagram of a hardware structure of a data processing device provided by an embodiment of the present application.
  • the data processing device 10 includes a processor 11 , a main memory (main memory) 12 , a storage medium 13 , a communication interface 14 and a bus 15 .
  • the processor 11 , the main memory 12 , the storage medium 13 and the communication interface 14 can be connected and communicated through the bus 15 .
  • Processor 11 is the control center of data processing device 10, can be general-purpose CPU, also can be other general-purpose processors, digital signal processor (digital signal processing, DSP), GPU, neural network processing unit (neural processing unit, NPU) , tensor processing unit (TPU), artificial intelligence (artificial intelligent) chip, application-specific integrated circuit (ASIC), field-programmable gate array (field-programmable gate array, FPGA) or others Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • DSP digital signal processor
  • GPU neural network processing unit
  • NPU neural network processing unit
  • TPU tensor processing unit
  • ASIC application-specific integrated circuit
  • FPGA field-programmable gate array
  • Programmable logic devices discrete gate or transistor logic devices, discrete hardware components, etc.
  • the processor 11 may include a first processor and a second processor.
  • the first processor and the second processor may be the same type of processor, or may be different types of processors, which is not specifically limited in this embodiment of the present application.
  • the first processor may be a CPU
  • the second processor may be a GPU
  • the main memory 12 is used to store program instructions, and the processor 11 can execute the program instructions in the main memory 12 to implement the video surveillance data processing method provided by the embodiment of the present application.
  • the main memory 12 may exist independently of the processor 11 .
  • the main memory 12 can be connected to the processor 11 through the bus 15, and is used for storing data, instructions or program codes.
  • the processor 11 invokes and executes the instructions or program codes stored in the main memory 12, the method for processing video surveillance data provided by the embodiment of the present application can be realized.
  • the main memory 12 may also be integrated with the processor 11 .
  • the storage medium 13 may be a volatile memory or a nonvolatile memory, or include both volatile and nonvolatile memory.
  • the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile memory can be random access memory (RAM), which acts as external cache memory.
  • RAM random access memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • Double data rate synchronous dynamic random access memory double data date SDRAM, DDR SDRAM
  • enhanced SDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous connection dynamic random access memory
  • direct rambus RAM direct rambus RAM
  • the communication interface 14 is used to connect the data processing device 10 with other equipment (such as surveillance cameras, etc.) through a communication network, and the communication network can be Ethernet, radio access network (radio access network, RAN), wireless local area network (wireless local area networks, WLAN), etc.
  • the communication interface 14 may include a receiving unit for receiving data, and a sending unit for sending data.
  • the bus 15 may be an industry standard architecture (industry standard architecture, ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, etc.
  • ISA industry standard architecture
  • PCI peripheral component interconnect
  • EISA extended industry standard architecture
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 1 , but it does not mean that there is only one bus or one type of bus.
  • FIG. 1 does not constitute a limitation on the data processing device 10, except for the components shown in FIG. 1, the data processing device 10 may include more or less components than those shown in FIG. components, or combinations of certain components, or different arrangements of components.
  • FIG. 2 shows a schematic framework diagram of a video surveillance system 20 provided by an embodiment of the present application.
  • the above-mentioned data processing device 10 is included in the video monitoring system 20, and at least one video capture device is included, such as the video capture device 21 shown in Figure 2, the video capture device 22, ..., and the video capture device 2k , where k is a positive integer.
  • the data processing device 10 and the at least one video acquisition device may be connected and communicated in a wired or wireless manner.
  • the video collection device is used for collecting surveillance video streams, and sending the collected surveillance video streams to the data processing device 10 through wired or wireless communication.
  • the data processing device 10 can analyze and process the surveillance video stream collected by the video collection device, so as to realize the intelligent monitoring of the services monitored by the video surveillance system 20 .
  • FIG. 3 shows a schematic flowchart of a method for processing video surveillance data provided by an embodiment of the present application.
  • the method can be applied to the video surveillance system shown in FIG. 2 and executed by the data processing device 10 shown in FIG. 1 .
  • the method may include the following steps:
  • the first processor reads a surveillance video stream.
  • the data processing device may first receive m monitoring video streams received in advance or in real time from multiple (for example, m, m is a positive integer) video acquisition devices through wired or wireless communication, and perform the monitoring on the received monitoring video streams. for caching. Then, the data processing device can read the data of the m surveillance video streams received by the data processing device from the cache through the m processes running in parallel by the first processor. Wherein, m is a positive integer greater than 1. Wherein, the picture collected by a video capture device corresponds to one surveillance video stream, and a process run by the first processor is used to read data of one surveillance video stream.
  • the m processes running on the first processor may run in parallel, and the m processes running in parallel are physically or software isolated, which is not limited in this embodiment of the present application.
  • the above m video capture devices may be video capture devices with the same specification parameters, or may be video capture devices with different specification parameters, which is not specifically limited in this embodiment of the present application.
  • the specification parameters include but are not limited to the type of the video capture device, the size of a charge coupled device (CCD) and the like. It can be understood that for video capture devices with the same specification parameters, parameters such as the frame rate of the surveillance video stream, the size and resolution of the image frames in the surveillance video stream are the same.
  • the first processor may preprocess the data of the surveillance video streams read by each of the m processes running in parallel.
  • the first process may preprocess the data of the first monitoring video stream in at least one of the following ways:
  • the first process scales the size of the image frame in the first surveillance video stream to a unified first preset size.
  • the embodiment of the present application does not specifically limit the specific value of the first preset size.
  • the first processor when the first processor reads through the m processes respectively, the m video capture devices with different specification parameters After the data of the surveillance video streams are preprocessed according to the first method, it is helpful for the first processor to perform standardized analysis and processing on the data of the surveillance video streams collected by video collection devices with different specifications and parameters.
  • the first process performs denoising and/or contrast stretching processing on the image frames in the first surveillance video stream, so as to improve the image quality of the image frames in the first surveillance video stream.
  • the first process preprocesses the data in the first surveillance video stream through method 2 , which can improve the image quality of the image frames in the first surveillance video stream, which helps the first processor to analyze and process the data of the first surveillance video stream subsequently.
  • one or two of the foregoing methods may be selected to preprocess the data of the surveillance video stream according to surveillance service needs.
  • the foregoing manner 1 and manner 2 are only exemplary descriptions of preprocessing the data of the surveillance video stream, and do not constitute a limitation to the protection scope of the embodiment of the present application. It can be understood that any technical means that help the first processor to subsequently analyze and process the data of the surveillance video stream shall fall within the scope of protection of the embodiments of the present application.
  • the first processor determines image frames to be processed in the read surveillance video stream.
  • the first processor may respectively determine image frames to be processed in respective preprocessed surveillance video streams through the above m processes running in parallel.
  • the number of image frames to be processed determined by the first processor in any path of monitoring video streams through processes can be one or more, so that the first processor can process multiple preprocessed monitoring video through m processes A number of image frames to be processed are identified in the stream.
  • the following takes the first process run by the first processor to determine the image frame to be processed in its preprocessed first surveillance video stream (corresponding to any surveillance video stream in the embodiment of the present application) as an example , the process of the first processor determining the image frames to be processed in the respective preprocessed surveillance video streams through the above m processes running in parallel is described.
  • the first process may determine the image frames to be processed in the image frames of the first surveillance video stream by means of down-sampling.
  • the first process may determine image frames extracted from every t frames of images in the first surveillance video stream as image frames to be processed.
  • t is a positive integer.
  • the first process can extract an image frame every 3 frames in the first monitoring video stream, such as image frame 1, image frame 5, and image frame 9, to determine is the image frame to be processed.
  • 3 frames of image frames including image frame 2, image frame 3 and image frame 4 are spaced between image frame 1 and image frame 5
  • 3 frame image frames including image frame 6) are spaced between image frame 5 and image frame 9 , image frame 7 and image frame 8).
  • the first process may detect moving objects in the first surveillance video stream through a preset moving object detection algorithm, and map the moving objects detected in the first surveillance video stream to the The image frame of is determined as the image frame to be processed.
  • the image frame corresponding to the moving object may be a continuous multi-frame image frame in the first surveillance video stream, and the frame difference of the continuous multi-frame image frame is greater than the moving object detection threshold.
  • the embodiment of the present application does not specifically limit the moving object detection algorithm and the specific value of the moving object detection threshold.
  • a plurality of moving object detection thresholds are preset in the first processor, and the first process may select among the plurality of moving object detection thresholds based on the number of image frames to be processed in the first processing queue A moving target detection threshold is used as the current moving target detection threshold. In this way, the number of image frames to be processed in the first processing queue to be processed by the second processor can be effectively adjusted, so as to prevent the second processor from being idle or overloaded.
  • the first processing queue reference may be made to the following description, and details are not repeated here.
  • the first process may select a larger moving object detection threshold among a plurality of preset moving object detection thresholds as the current Moving object detection threshold to reduce the number of image frames to be processed determined by the first process.
  • the first process may select a smaller moving object detection threshold among a plurality of preset moving object detection thresholds as the current moving object detection threshold , to increase the number of image frames to be processed determined by the first process. In this way, the adjustment of the number of image frames to be processed in the first processing queue to be processed by the second processor is realized, thereby preventing the second processor from being idle or overloaded.
  • the first process can adjust the size of the moving target detection threshold based on the number of image frames to be processed in the first processing queue, so as to realize the processing of the image frames to be processed by the second processor in the first processing queue.
  • the adjustment of the number of image frames is processed, thereby preventing the second processor from being idle or overloaded.
  • the first process may increase the current moving object detection threshold to reduce the number of image frames to be processed determined by the first process .
  • the first process may lower the current moving object detection threshold to increase the number of image frames to be processed determined by the first process. In this way, the adjustment of the number of image frames to be processed in the first processing queue to be processed by the second processor is realized, thereby preventing the second processor from being idle or overloaded.
  • the first processor buffers the determined image frames to be processed in the first processing queue.
  • the m processes running in parallel in the first processor determine the image frames to be processed, they further perform target detection preprocessing on the determined image frames to be processed, and then preprocess the target detection preprocessed to-be-processed
  • the image frame and the image information of the image frame to be processed are stored in the first processing queue.
  • the image information of the image frame to be processed includes the identification number of the surveillance video stream to which the image frame to be processed belongs.
  • the image information of the image frame to be processed may also include a frame identifier of the image frame to be processed.
  • the first processing queue is a first-in first-out queue.
  • the first process after determining the image frame to be processed in the first surveillance video stream, the first process first performs target detection preprocessing on the determined image frame to be processed , and then cache the image frame to be processed after the target detection preprocessing and the image information of the image frame to be processed in the first processing queue.
  • the second process after determining the image frame to be processed in the second monitoring video stream, the second process first performs target detection preprocessing on the determined image frame to be processed, and then The image frame to be processed after target detection preprocessing and the image information of the image frame to be processed are cached in the first processing queue.
  • the image frame to be processed and the image information of the image frame to be processed determined by the m processes running in parallel in the first processor and subjected to object detection preprocessing are cached in the same processing queue for processing .
  • performing target detection preprocessing on the image frame to be processed can make the image frame to be processed meet the requirements of the input image of the reasoning model used for image reasoning on the image.
  • the inference model reference may be made to the relevant description below, and details are not repeated here.
  • the first process A process running in a processor describes the process of performing target detection preprocessing on image frames to be processed.
  • the first process may perform target detection preprocessing on the first image frame to be processed in at least one of the following ways:
  • the first process may scale the size of the first image frame to be processed to a second preset size.
  • the second preset size is the size of the input image specified by the reasoning model.
  • the first process performs normalization processing on the first image frame to be processed.
  • the normalization process refers to dividing each pixel value in the first image frame to be processed by 255 to obtain the pixel value of the first image frame to be processed after the normalization process. It can be understood that, in order to simplify the operation, the reasoning model requires that the input image is usually a normalized image.
  • the pixel value of pixel a in the first image frame to be processed is 110
  • the pixel value of pixel a in the first image frame to be processed after normalization processing is 110/255, that is, 0.43.
  • the first process fills the first image frame to be processed with black borders, so that the aspect ratio of the first image frame to be processed after filling the black borders meets the aspect ratio requirement of the inference model for the input image.
  • the first process performs dimension expansion on the first image frame to be processed, so that the dimensionally expanded first image frame to be processed meets the dimensionality requirement of the reasoning model for the input image.
  • one or more of the above methods may be selected to perform target detection preprocessing on the image frame to be processed according to the requirements of the inference model for the input image.
  • the manners of performing object detection preprocessing on the image frames to be processed described in manners 1 to 4 above are only exemplary illustrations, and do not constitute limitations on the protection scope of the embodiments of the present application. It can be understood that any technical means to make the image frame to be processed meet the requirements of the reasoning model for the input image should fall within the scope of protection of the embodiments of the present application.
  • each of the m processes running in parallel in the first processor determines the image frame to be processed, it directly caches the image frame to be processed and the image information of the image frame to be processed in the first processor. Process the queue.
  • the first process after determining the image frame to be processed in the first surveillance video stream, the first process directly combines the determined image frame to be processed with the image frame to be processed The image information of the frame is buffered in the first processing queue.
  • the second process after determining the image frame to be processed in the second monitoring video stream, the second process directly transfers the determined image frame to be processed and the image information of the image frame to be processed cached in the first processing queue.
  • the second processor sequentially performs image reasoning on the image frames to be processed acquired from the first processing queue, and sends the image frames to be processed for which image reasoning has been executed to the first processor.
  • An inference model is preset in the second processor, and the inference model is used to perform image inference on the image input into the inference model, so that the target information in the image frame to be processed can be determined according to the image inference result.
  • the target information in the image frame to be processed includes the type and position of the target in the image frame to be processed.
  • the type of the object in the image frame to be processed may be, for example, a person, a cat, a dog, a flame, and the like.
  • the position of the target in the image frame to be processed is the position of the pixel area corresponding to the area where the target is located in the image frame to be processed.
  • the second processor can sequentially perform image reasoning on the image frames to be processed acquired from the first processing queue according to the preset reasoning model.
  • the second processor can perform image reasoning on the image frames to be processed obtained from the first processing queue through n processes running in parallel and the above-mentioned preset reasoning model, so as to determine each The location and type of the pixel. It should be understood that the determined position and type of each pixel in the image frame to be processed here is the inference result after the image inference is performed by the inference model. In this way, the target information in the image frame to be processed can be determined according to the reasoning result.
  • n is a positive integer.
  • the value of n can be determined by the data processing capability of the second processor. For example, the number of concurrently running processes of the second processor may be determined through experiments, so as to maximize the data processing efficiency of the second processor.
  • n is usually smaller than m. That is to say, the number of processes running in parallel in the second processor for performing image reasoning on the image frame to be processed is smaller than the number of processes running in parallel in the first processor for reading the surveillance video stream. That is to say, the number of processes running in parallel in the second processor for performing image reasoning on the image frames to be processed is smaller than the number of channels that the first processor can simultaneously read surveillance video streams.
  • the n processes running in parallel by the second processor can respectively acquire image frames to be processed from the first processing queue, and then perform image reasoning on the respectively acquired image frames to be processed according to a preset reasoning model.
  • the multiple processes may sequentially acquire image frames to be processed from the first processing queue in a preset order, and perform the acquired image frames according to a preset reasoning model. Image inference is performed on the received image frames to be processed.
  • the embodiment of the present application does not specifically limit the preset order.
  • any one of the n processes executes the image reasoning and outputs the reasoning result of the image frame to be processed, it can acquire a new image frame to be processed from the first processing queue again and perform image processing on it. reasoning.
  • the second processor runs 3 processes, namely process 1, process 2 and process 3.
  • process 1, process 2 and process 3 are currently idle, they can sequentially acquire image frames to be processed from the first processing queue in the order of process 1->process 2->process 3 and perform image reasoning on them.
  • process 2 first completes the image inference of the acquired image frame to be processed, it acquires a new image frame to be processed from the first processing queue again, and performs image inference on it.
  • the second processor may use n
  • the process and the preset reasoning model directly perform image reasoning on the image frame to be processed obtained from the first processing queue, so as to determine the position and type of each pixel in the image frame to be processed.
  • the second processor that runs fewer concurrent processes does not need to perform object detection preprocessing.
  • Perform target detection preprocessing on the image frames to be processed obtained from the first processing queue which saves the resources of the second processor and increases the number of image frames to be processed by the second processor, thereby improving the first processing
  • the processor and the second processor jointly analyze and process the monitoring video stream data, that is, the data processing efficiency of the video monitoring system is improved.
  • the second processor can process n processes running in parallel , first perform object detection preprocessing on the image frames to be processed respectively obtained from the first processing queue, and then perform image inference on the image frames to be processed that have completed the object detection preprocessing according to the preset inference model, so that Determine the position and type of each pixel in the image frame to be processed.
  • the process of the second processor performing target detection preprocessing on the image frames to be processed obtained from the first processing queue through n processes running in parallel can refer to the m processes running in parallel in the first processor in S103 The process performs target detection preprocessing on the determined image frames to be processed, which will not be repeated here.
  • the second processor uses n processes running in parallel to perform image inference on the image frames to be processed that have completed target detection preprocessing according to a preset inference model. You can refer to the above-mentioned first possible implementation mode, the second The processor uses n processes running in parallel and a preset reasoning model to perform image reasoning descriptions on the image frames to be processed obtained from the first processing queue, which will not be repeated here.
  • the n processes running in parallel by the second processor send the image frames to be processed that have respectively performed image reasoning to the first processor according to the identification numbers of the surveillance video streams to which the currently processed image frames to be processed belong.
  • the process corresponding to the identification number is placed in the process processing queue, so that the process corresponding to the identification number running in the first processor performs subsequent processing on the image frame to be processed after performing image reasoning.
  • the first processor may perform subsequent processing on the image frames to be processed after the image reasoning has been performed through the m processes running in parallel. It should be understood that the number of processes for the first processor to perform subsequent processing on the image frames to be processed after performing image reasoning is the same as the number of processes for reading m monitoring video streams in S101, and they correspond one-to-one.
  • any one of the n processes running in parallel by the second processor can determine the position and type of each pixel in the image frame to be processed according to the position and type of each pixel in the image frame to be processed obtained by the above-mentioned image reasoning after performing the image reasoning.
  • Object information in the image frame to be processed. This process can be referred to as the process of performing target detection post-processing on the image frames after image reasoning.
  • the process of performing target detection preprocessing, reasoning, and target detection post-processing on the image frame to be processed may be referred to as a process of performing target detection on the image frame to be processed.
  • any process run by the second processor after the process completes the image reasoning of the image frame to be processed, the position and value of each pixel in the image frame to be processed determined by the image reasoning
  • the type is statistically analyzed, and the position of the pixel region with adjacent positions and the same type is determined as the position of the target in the image frame to be processed, and the type of the pixel with the adjacent position and the same type is determined as the type of the aforementioned target.
  • any process determines the target information in the image frame to be processed according to the reasoning result of the image frame to be processed (ie, the position and type of each pixel in the image frame to be processed).
  • the n processes running in parallel by the second processor can send the image frames to be processed for which the target information has been determined respectively to the first processor according to the identification numbers of the surveillance video streams to which the currently processed image frames to be processed belong.
  • the process processing queue of the process corresponding to the identification number the process corresponding to the identification number running in the first processor performs subsequent processing on the image frame to be processed for which the target information has been determined.
  • the first processor may perform subsequent processing on the image frame to be processed for which the target information is determined through m processes running in parallel. It can be seen that the number of processes for the first processor to perform subsequent processing on the image frames to be processed for which the target information is determined is the same as the number of processes for reading m surveillance video streams in S101, and they correspond one-to-one.
  • the first processor processes the received image frame to be processed processed by the second processor according to a preset algorithm, so as to determine whether there is an abnormal event in the surveillance video stream to which the image frame to be processed belongs.
  • the first processor may receive a plurality of image frames to be processed that have been processed by n processes in the second processor through m processes running in parallel, and perform processing on the received plurality of image frames to be processed according to a preset algorithm Processing is performed to determine whether there is an abnormal event in the surveillance video stream to which the multiple image frames to be processed belong.
  • This process may be referred to as a process of performing logical service post-processing on the image frame to be processed.
  • the plurality of image frames to be processed received by the first processor through the m processes running in parallel and processed by the n processes in the second processor may belong to the same monitoring video stream, or may belong to different channels.
  • the monitoring video stream is not limited in this embodiment of the present application.
  • one or more preset algorithms are preset in the first processor.
  • the preset algorithm is pre-designed based on the business monitored by the video surveillance system, and is used to perform calculations on one or more frames of image frames in the surveillance video stream collected by the video collection equipment in the video surveillance system, thereby It can be determined whether there is an abnormal event corresponding to the service monitored by the video monitoring system in the monitoring video stream, thereby realizing intelligent monitoring of the monitored service by the video monitoring system.
  • the abnormal event is that one or more image frames in the monitoring video stream collected by the video acquisition device in the video monitoring system include flames.
  • the pre-designed preset algorithm based on the fire monitoring business can be used to track and analyze one or more frames of image frames in the monitoring video stream collected by the video acquisition equipment in the video monitoring system, so as to determine the Monitors whether flames are included in one or more image frames in a video stream. In this way, the intelligent monitoring of the fire by the video monitoring system can be realized.
  • the monitoring video streams collected by different video capture devices are generally intelligently analyzed and processed using preset algorithms corresponding to their monitoring services.
  • the m processes running in parallel by the first processor may first perform target detection post-processing on the respective received image frames to be processed.
  • target detection post-processing for a detailed description of the post-processing of target detection, reference may be made to the description of the post-processing of target detection in S104 above, which will not be repeated here.
  • the m processes running in parallel by the first processor can determine the target information in the respective received image frames to be processed after performing target detection post-processing on the respective received image frames to be processed.
  • target information For the description of the target information, reference may be made to the detailed description of the target information in S103 above, which will not be repeated here.
  • the first processor running more concurrent processes performs target detection post-processing on the image frames to be processed that have performed image reasoning, so that the second processor running with fewer concurrent processes does not need to execute the processing in S103
  • the target detection post-processing is performed on the image frames to be processed through image reasoning, which saves the resources of the second processor, thereby increasing the number of images to be processed by the second processor, thereby improving the efficiency of the first processor and the second processing.
  • the efficiency of joint analysis and processing of monitoring video stream data is improved, that is, the data processing efficiency of the video monitoring system is improved.
  • the m processes running in parallel by the first processor can determine the target information in the image frame to be processed and the preset algorithm corresponding to the service monitored by the monitoring video stream to which the image frame to be processed belongs. Process whether there is an abnormal event in the surveillance video stream to which the image frame belongs.
  • the first processor executes S106.
  • the m processes running in parallel by the first processor can directly determine the monitoring video stream to which the image frame to be processed belongs according to the target information in the image frame to be processed and the preset algorithm corresponding to the monitoring service of the monitoring video stream to which the image frame to be processed belongs. Whether there is an abnormal event.
  • the first processor executes S106.
  • the first processor controls to output alarm information.
  • the first processor can control the data processing device to output alarm information to the management personnel of the video surveillance system through any mode such as text, voice, image or video stream through its input and output interface (such as a display and an audio module) or a communication interface, In order to enable management personnel to deal with abnormal events in a timely manner.
  • the first processor may mark the target of the abnormal event in the monitoring video stream in which the abnormal event is determined, and instruct the communication interface of the data processing device to transmit the A section of surveillance video stream output marked with the target of the abnormal event is used to alert the management personnel of the video surveillance system and enable the management personnel to deal with the abnormal event in time.
  • the data processing device can separately pool the processing operation of the process running on the first processor and the processing operation of the process running on the second processor, so as to decouple the processes run by the first processor and the second processor respectively.
  • the analysis and processing of different stages of the surveillance video stream enables the first processor to process the number of surveillance video streams in parallel, and is no longer limited by the number of parallel processing of the surveillance video stream by the second processor, thus making the data processing capability and resources
  • the limited second processor does not limit the number of monitoring video streams processed by the data processing device at the same time.
  • the method provided by the embodiment of the present application improves the data processing device to simultaneously process the monitoring video stream.
  • the number of channels that is, the method provided by the embodiment of the present application improves the data processing efficiency of the video surveillance system.
  • the first processor and the second processor decouple the analysis and processing of different stages of the surveillance video stream through their respective running processes, the first processor of the data processing device can flexibly add or delete surveillance video streams to be analyzed and processed.
  • the resources of the second processor can be saved, so that the second processor can use the limited
  • the resource processes more image frames to be processed in the first processing queue, which increases the number of image frames to be processed in the first processing queue processed by the second processor, thereby further improving the performance of the first processor and the second processor in the embodiment of the present application.
  • the efficiency of the two processors jointly processing video monitoring stream data improves the data processing efficiency of the video monitoring system.
  • S101-S103 executed by the first processor can be regarded as the first processing stage of the surveillance video stream
  • S104 executed by the second processor can be regarded as the first processing stage of the surveillance video stream
  • the second processing stage, and S105-S106 executed by the first processor can be regarded as the third processing stage of the surveillance video stream.
  • FIG. 4 shows another schematic diagram of a method for processing video surveillance data provided by an embodiment of the present application.
  • the first processor may execute the above S101-S103 through m processes running in parallel.
  • the m processes may be process 11, process 12, . . . , and process 1m shown in FIG. 4 .
  • the second processor may execute S104 through n processes running in parallel.
  • the n processes may be processes 21, .
  • the first processor may also execute S105-S106 through m processes running in parallel.
  • the m processes may be process 31 , process 32 , . . . , and process 3m shown in FIG. 4 .
  • the embodiment of the present application can divide the video surveillance data processing device into functional modules according to the above method example, for example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module .
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 5 shows a schematic structural diagram of a device 50 for processing video surveillance data provided by an embodiment of the present application.
  • the device 50 may be used to execute the above-mentioned method for processing video surveillance data, for example, to execute the method shown in FIG. 3 .
  • the device 50 may include a first processing unit 51 and a second processing unit 52 .
  • the first processing unit 51 is configured to acquire a plurality of image frames to be processed from multiple surveillance video streams, and buffer the plurality of image frames to be processed in a first processing queue.
  • the second processing unit 52 is configured to perform image reasoning on the plurality of image frames to be processed obtained from the first processing queue, so as to determine target information in the plurality of image frames to be processed.
  • the first processing unit 51 may be used to execute S101-S102
  • the second processing unit 52 may be used to execute S104.
  • the second processing unit 52 is specifically configured to perform image reasoning on the plurality of image frames to be processed through n processes running in parallel.
  • n is a positive integer
  • the value of n is determined according to the data processing capability of the second processing unit 52 .
  • the second processing unit 52 may be used to execute S104.
  • the aforementioned plurality of image frames to be processed are image frames corresponding to moving objects in the surveillance video stream to which the plurality of image frames to be processed belong.
  • the aforementioned plurality of image frames to be processed are image frames extracted from every t frames of images in the surveillance video stream to which the plurality of image frames to be processed belong, and t is a positive integer.
  • the first processing unit 51 is further configured to determine whether there is an abnormal event in the surveillance video stream to which the plurality of image frames to be processed belong according to the target information in the plurality of image frames to be processed; and, if the plurality of image frames to be processed If there is an abnormal event in the surveillance video stream to which the image frame to be processed belongs, the first processing unit 51 controls to output alarm information.
  • the first processing unit 51 may be used to execute S105-S106.
  • the first processing unit 51 is specifically configured to acquire a plurality of image frames to be processed in multiple monitoring video streams through m processes running in parallel, where m is a positive integer greater than 1.
  • the first processing unit 51 may be used to execute S101-S102.
  • the first processing unit 51 is further specifically configured to determine whether there is an abnormal event in the surveillance video stream to which the plurality of image frames to be processed belong according to a preset algorithm and the target information of the plurality of image frames to be processed.
  • the preset algorithm is determined according to services monitored by the surveillance video stream to which the multiple image frames to be processed belong.
  • the first processing unit 51 may be used to execute S105.
  • the above target information includes the type and/or position of the target in the image to be processed.
  • the first processing unit 51 is further configured to perform object detection preprocessing on the plurality of image frames to be processed before buffering the plurality of image frames to be processed in the first processing queue.
  • the target detection preprocessing includes at least one of the following methods: size scaling, normalization processing, black border filling, dimension expansion, and the like.
  • the device 50 provided in the embodiment of the present application includes but is not limited to the above units, for example, the device 50 may further include a storage unit 53 .
  • the storage unit 53 may be used to store the program codes executed by the first processing unit 51 and the second processing unit 52, etc., which is not limited in this embodiment of the present application.
  • the functions implemented by the first processing unit 51 and the second processing unit 52 in the device 50 can be implemented by the processor 11 in FIG. 1 executing the program code in the main memory 12 in FIG. 1 .
  • the function realized by the storage unit 53 may be the same as that realized by the main memory 12 in FIG. 1 .
  • the present application also provides a video surveillance data processing device, the device may be shown in Figure 1, including a processor 11, a main memory 12, a storage medium 13 and a communication interface 14, the device uses In order to implement the video surveillance data processing method shown in FIG. 3 above, efficient processing of the video surveillance data can be realized.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • a software program When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the processes or functions according to the embodiments of the present application are generated in whole or in part when the computer executes the instructions on the computer.
  • a computer can be a general purpose computer, special purpose computer, a computer network, or other programmable apparatus.
  • Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or may contain one or more data storage devices such as servers and data centers that can be integrated with the medium. Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid-state disk (solid-state disk, SSD)) and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

一种视频监控数据的处理方法及装置，涉及视频监控领域，该方法可以提高视频监控数据的处理装置同时分析处理监控视频的路数，从而提高了视频监控系统的数据处理效率。该方法应用于包括有第一处理器和第二处理器的视频监控数据的处理装置中。该方法包括：第一处理器从多路监控视频流中获取多个待处理图像帧，并将该多个待处理图像帧缓存于第一处理队列。第二处理器对从第一处理队列获取的待处理图像帧进行图像推理，以确定待处理图像帧中的目标信息。

Description

视频监控数据的处理方法及装置
本申请要求于2021年05月08日提交国家知识产权局、申请号为202110502427.1、申请名称为“视频监控数据的处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频监控领域，尤其涉及一种视频监控数据的处理方法及装置。
背景技术
随着社会发展、科技进步，各行各业对智能化视频监控的需求越来越强烈。在智能化视频监控系统中，数据处理装置通常会对多个摄像头采集的监控视频进行分析处理，以实现智能化视频监控系统对不同业务的智能监控。例如，智能化视频监控系统可以实现对工地中是否存在没有戴安全帽的工人这一业务的智能监控。
通常，上述数据处理装置分析处理多个摄像头所采集的监控视频时，对于一路监控视频(一个摄像头采集到的监控视频可以称为一路监控视频)而言，数据处理设备采用串行处理方式对其进行以下处理：视频读取、视频图像预处理、对视频图像进行目标检测(包括目标检测预处理、图像推理、模型后处理)、视频图像中的目标跟踪、监控业务的逻辑处理、以及告警信号输出等。
对于多路监控视频而言，由于处理器的运行能力及资源等的限制，使得数据处理设备能同时处理的监控视频的路数极为有限，进而导致智能化监控系统处理监控视频的效率低。
发明内容
本申请提供了一种视频监控数据的处理方法及装置，该方法可以提高视频监控数据的处理装置同时分析处理监控视频的路数，从而提高了视频监控系统的数据处理效率。
为达上述目的，本申请提供如下技术方案：
第一方面，本申请提供了一种视频监控数据的处理方法，该方法由包括有第一处理器和第二处理器的视频监控数据的处理装置执行。该方法可以包括：第一处理器从多路监控视频流中获取多个待处理图像帧，并将该多个待处理图像帧缓存于第一处理队列。第二处理器对从第一处理队列获取的多个待处理图像帧进行图像推理，以确定出该多个待处理图像帧中的目标信息。
通过本申请提供的方法，视频监控数据的处理装置可以通过第一处理队列解耦第一处理器和第二处理器各自执行的运算处理，使得第一处理器并行处理监控视频流的路数，不再受到第二处理器并行处理监控视频流的路数的限制，从而使得当第二处理器的数据处理能力及资源有限时，不会限制视频监控数据的处理装置同时处理监控视频流的路数。也就是说，相比通过视频监控数据的处理装置的通用处理器和图形处理器串行分析处理监控视频流数据时能够同时处理监控视频流的路数，本申请提供的方法提高了视频监控数据的处理装置同时处理监控视频流的路数，即本申请提供的方法提高了视频监控系统的数据处理效率。
在一种可能的设计方式中，上述的第二处理器对从第一处理队列获取的多个待处理图像帧进行图像推理，包括：第二处理器通过并行运行的n个进程对该多个待处理图像帧进行图像推理。其中，n是正整数，n的取值根据第二处理器的数据处理能力来确定。
在另一种可能的设计方式中，上述第一处理器从多路监控视频流中获取多个待处理图像帧，包括：第一处理器通过并行运行的m个进程，在多路监控视频流中获取多个待处理图像帧，m是大于1的正整数。
通过该两种可能的设计方式，第一处理器和第二处理器可以分别并发多个进程来处理多路监控视频流的数据，且第一处理器和第二处理器并发的进程数量无关。也就是说，本申请方法对第一处理器和第二处理器各自执行的运算处理解耦，使得第一处理器并行处理监控视频流的路数，不再受到第二处理器并行处理监控视频流的路数的限制，从而使得当第二处理器的数据处理能力及资源有限时，不会限制视频监控数据的处理装置同时处理监控视频流的路数。
在另一种可能的设计方式中，上述的多个待处理图像帧是该多个待处理图像帧所属监控视频流中的移动目标对应的图像帧。或者，上述的多个待处理图像帧是从该多个待处理图像帧所属的监控视频流中每隔t帧图像抽取的图像帧，t是正整数。
第一处理器可以通过该可能的实现方式，在待分析的监控视频流中确定出需要进行图形推理的待处理图像帧。当第一处理器将待分析监控视频流中的移动目标对应的图像帧确定为待处理图像帧，可以有效避免对该监控视频流中无需进行分析的图像帧(即监控视频流中没有发生变化的图像帧)进行图形推理，从而节省了用于对图像帧进行图形推理的第二处理器的资源，进而使得第二处理器被节省的资源可以用于处理更多路数的监控视频流的数据，即提高了视频监控数据的处理装置同时处理监控视频流的路数，也即本申请提供的方法提高了视频监控系统的数据处理效率。
在另一种可能的设计方式中，上述方法还包括：第一处理器根据多个待处理图像帧中的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件。如果该多个待处理图像帧所属监控视频流中存在异常事件，则第一处理器控制输出告警信息。
在另一种可能的设计方式中，上述第一处理器根据多个待处理图像帧中的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件，具体包括：第一处理器根据预设算法和多个待处理图像帧的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件。其中，预设算法是根据该多个待处理图像帧所属的监控视频流所监控的业务确定的。
通过该两种可能的实现方式，第一处理器可以基于第二处理器的图形推理结果，确定出待处理图像帧所属监控视频流中的异常事件，并向用户告警，从而实现了对视频监控系统对所监控业务的监控。
在另一种可能的设计方式中，上述的目标信息包括待处理图像中目标的类型和/或位置。
在另一种可能的设计方式中，在上述的将该多个待处理图像帧缓存于第一处理队列之前，上述方法还包括：第一处理器对在多路监控视频流中获取的多个待处理图像 帧进行目标检测预处理，以使该多个待处理图像帧满足输入到推理模型的输入图像的要求。其中，第一处理器中预置有该推理模型，且该推理模型用于对该多个待处理图像帧进行图形推理，以确定待处理图像帧中的目标信息。其中，目标检测预处理包括以下至少一种方式：尺寸缩放、归一处理、黑边填充、维度扩展等。
在另一种可能的设计方式中，在上述第二处理器对多个待处理图像帧进行图形推理后，且在第一处理器根据该多个待处理图像帧中的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件之前，上述方法还包括：第一处理器基于第二处理器对该多个待处理图像帧的图形推理结果进行目标检测后处理，以确定出该多个待处理图像帧的目标信息。
通过该两种可能的实现方式，相比通过视频监控数据的处理装置的通用处理器和图形处理器串行分析处理监控视频流数据，本申请提供的方法将目标检测预处理的过程前移，即由具有高并发处理能力的第一处理器执行目标检测预处理，以及将目标检测后处理后移，即由具有高并发处理能力的第一处理器执行目标检测后处理。这样，本申请提供的方法进一步节省了第二处理器的资源，进而使得第二处理器被节省的资源可以用于处理更多路数的监控视频流的数据，即提高了视频监控数据的处理装置同时处理监控视频流的路数，也即本申请提供的方法提高了视频监控系统的数据处理效率。
第二方面，本申请提供了一种视频监控数据的处理装置。该视频监控数据的处理装置用于执行上述第一方面提供的任一种方法。本申请可以根据上述第一方面提供的任一种方法，对该视频监控数据的处理装置进行功能模块的划分。例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。示例性的，本申请可以按照功能将该视频监控数据的处理装置划分为第一处理单元和第二处理单元等。上述划分的各个功能模块执行的可能的技术方案和有益效果的描述均可以参考上述第一方面或其相应的可能的设计提供的技术方案，此处不再赘述。
第三方面，本身请提供了一种视频监控数据的处理设备，该设备包括第一处理器、第二处理器以及存储器，该第一处理器和该第二处理器被配置为调用存储在该存储器中的计算机指令，以使该视频监控数据的处理设备通过第一处理器执行如权利要求1-8中任一项的由第一处理器执行的方法，以及通过第二处理器执行如权利要求1-8中任一项的由第二处理器执行的方法。
第四方面，本申请提供了一种计算机可读存储介质，该计算机可读存储介质包括程序指令，当程序指令在计算机或处理器上运行时，使得计算机或处理器执行第一方面中的任一种可能的实现方式提供的任一种方法。
第五方面，本申请提供了一种计算机程序产品，当其在视频监控数据的处理装置或设备上运行时，使得第一方面中的任一种可能的实现方式提供的任一种方法被执行。
可以理解的是，上述提供的任一种装置、设备、计算机存储介质或计算机程序产品等均可以应用于上文所提供的对应的方法，因此，其所能达到的有益效果可参考对应的方法中的有益效果，此处不再赘述。
本申请在上述各方面提供的实现方式的基础上，还可以进行进一步组合以提供更多实现方式。
附图说明
图1为本申请实施例提供的一种数据处理装置的硬件结构示意图；
图2为本申请实施例提供的一种视频监控系统的框架示意图；
图3为本申请实施例提供的一种视频监控数据的处理方法的流程示意图；
图4为本申请实施例提供的视频监控数据的处理方法的另一种示意图；
图5为本申请实施例提供的一种视频监控数据的处理装置的结构示意图。
具体实施方式
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
在本申请的实施例中，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。
在本申请的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在视频监控系统中，数据处理装置串行分析处理每一路监控视频流的数据时，通常可以通过中央处理单元(central processing unit，CPU)来读取监控视频流，以及对读取到的监控视频中的图像帧进行预处理。接着，数据处理装置可以通过图形处理器(graphics processing unit，GPU)对预处理后的图像帧进行目标检测，以确定图像帧中的目标信息。然后，数据处理装置再通过CPU对检测到的目标信息进行逻辑业务后处理，以确定监控视频流中是否存在异常事件。其中，目标检测、目标信息和逻辑业务后处理等详细描述可以参考下文相关描述，这里不作赘述。
其中，异常事件与视频监控系统所监控的业务对应。例如，如果视频监控系统所监控的业务为火灾监控，则监控视频流中的异常事件即为该监控视频流中的一帧或多帧图像帧中包括火焰。再例如，如果视频监控系统所监控的业务为对工地中未佩戴戴安全帽的工人进行监控，则监控视频流中的异常事件即为该监控视频流中的一帧或多帧图像帧中包括有未佩戴安全帽的人物。等等，不再赘述。
这样的话，数据处理装置通过确定监控视频流中是否存在异常事件，即实现了视频监控系统对所监控业务的智能监控。
然而，GPU的数据处理能力和资源通常都极为有限，因此GPU能够同时处理监控视频流的路数极为有限，因此，CPU和GPU串行分析处理的方式，会导致CPU并行处理监控视频流的路数受到GPU并行处理监控视频流的数量的限制，从而导致通过CPU和GPU串行分析处理的监控视频流的路数极为有限，进而使得数据监控系统的数据处理效率低下。
基于此，本申请实施例提供了一种视频监控数据的处理方法和装置，该方法用于对视频监控系统中的多路监控视频流进行分析处理，以实现对不同业务的智能监控。为简单描述，本申请实施例在下文中将该视频监控数据的处理装置简称为数据处理装置。
具体的，对于视频监控系统中的多路监控视频流而言，本申请实施例提供的方法 可以先通过数据处理装置的第一处理器的多个并行进程在该多路监控视频流中确定待处理图像帧，并将确定出的待处理图像帧缓存至第一处理队列。接着，该方法再通过数据处理装置的第二处理器的多个并行进程对第一处理队列中的待处理图像帧依次进行图像推理，以确定出待处理图像帧中的目标信息。然后，该方法通过数据处理装置的第一处理器的多个并行进程对图像推理后的待处理图像帧进行处理，即可确定出待处理图像帧所属的监控视频流中是否存在异常事件，从而实现视频监控系统对不同业务的智能监控。其中，一个进程用于处理一路监控视频流。
其中，第一处理器运行进程的数量和第二处理器运行进程的数量可以相同，也可以不同。可以看出，通过本申请实施例提供的方法，使得第一处理器并行处理监控视频流的路数，不再受到第二处理器并行处理监控视频流的路数的限制，从而使得数据处理能力及资源有限的第二处理器不会限制数据处理装置同时处理监控视频流的路数。也就是说，相比通过数据处理装置的CPU和GPU串行分析处理监控视频流数据时能够同时处理监控视频流的路数，本申请实施提供的方法提高了数据处理装置同时处理监控视频流的路数，即本申请实施例提供的方法提高了视频监控系统的数据处理效率。
上述的数据处理装置可以是任意具有计算处理能力的计算设备，该计算设备例如可以是通用计算机、笔记本电脑、平板电脑等计算设备。当然，该数据处理装置也可以是服务器，例如是视频监控系统中的监控服务器等，本申请实施例对此不作限定。
参考图1，图1示出了本申请实施例提供的一种数据处理装置的硬件结构示意图。如图1所示，数据处理装置10包括处理器11、主存储器(main memory)12、存储介质13、通信接口14以及总线15。其中，处理器11、主存储器12、存储介质13以及通信接口14之间可以通过总线15连接通信。
处理器11是数据处理装置10的控制中心，可以是通用CPU，也可以是其他通用处理器、数字信号处理器(digital signal processing，DSP)、GPU、神经网络处理单元(neural processing unit，NPU)、张量处理器(tensor processing unit，TPU)、人工智能(artificial intelligent)芯片、专用集成电路(application-specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。
在本申请实施例中，处理器11可以包括第一处理器和第二处理器。其中，第一处理器和第二处理器可以是相同类型的处理器，也可以是不同类型的处理器，本申请实施例对此不作具体限定。
示例性的，第一处理器可以是CPU，第二处理器可以是GPU。
应理解，本申请实施例并不限定上述每个处理器中处理器核的个数。
主存储器12用于存储程序指令，处理器11可以通过执行主存储器12中的程序指令，以实现本申请实施例提供的视频监控数据的处理方法。
在一种可能的实现方式中，主存储器12可以独立于处理器11存在。主存储器12可以通过总线15与处理器11相连接，用于存储数据、指令或者程序代码。处理器11调用并执行主存储器12中存储的指令或程序代码时，能够实现本申请实施例提供的视频监控数据的处理方法。在另一种可能的实现方式中，主存储器12也可以和处理器11集成在一起。
存储介质13可以是易失性存储器或非易失性存储器，或包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data date SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。
通信接口14，用于数据处理装置10与其他设备(如监控摄像头等)通过通信网络连接，所述通信网络可以是以太网，无线接入网(radio access network，RAN)，无线局域网(wireless local area networks，WLAN)等。通信接口14可以包括用于接收数据的接收单元，以及用于发送数据的发送单元。
总线15，可以是工业标准体系结构(industry standard architecture，ISA)总线、外部设备互连(peripheral component interconnect，PCI)总线或扩展工业标准体系结构(extended industry standard architecture，EISA)总线等。该总线可以分为地址总线、数据总线、控制总线等。为便于表示，图1中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
需要指出的是，图1中示出的结构并不构成对数据处理装置10的限定，除图1所示部件之外，该数据处理装置10可以包括比图1所示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
本申请实施例还提供一种视频监控系统。参考图2，图2示出了本申请实施例提供的一种视频监控系统20的框架示意图。
如图2所示，视频监控系统20中包括上述的数据处理装置10，以及包括至少一个视频采集设备，例如图2所示的视频采集设备21、视频采集设备22、…、以及视频采集设备2k，其中，k是正整数。其中，数据处理装置10和该至少一个视频采集设备之间可以通过有线或无线的方式连接通信。
其中，视频采集设备用于采集监控视频流，并将采集到的监控视频流通过有线或无线的通信方式发送至数据处理装置10。这样，数据处理装置10即可对视频采集设备采集到的监控视频流进行分析处理，以实现视频监控系统20对其所监控的业务的智能监控。
下面结合附图，对本申请实施例提供的视频监控数据的处理方法进行详细描述。
参考图3，图3示出了本申请实施例提供的一种视频监控数据的处理方法的流程示意图。该方法可以应用于图2所示的视频监控系统中，并由图1所示的数据处理装置10执行。该方法以包括以下步骤：
S101、第一处理器读取监控视频流。
具体的，数据处理装置可以先通过有线或无线的通信方式，从多个(例如m个， m是正整数)视频采集设备预先或实时接收的m路监控视频流，并对接收到的监控视频流进行缓存。然后，数据处理装置可以通过第一处理器并行运行的m个进程，从缓存中读取数据处理装置接收到的m路监控视频流的数据。其中，m是大于1的正整数。其中，一个视频采集设备采集的画面对应一路监控视频流，第一处理器运行的一个进程用于读取一路监控视频流的数据。
应理解，第一处理器运行的m个进程可以并行运行，且该并行运行的m个进程之间通过物理或软件隔离，本申请实施例对此不作限定。
应理解，上述m个视频采集设备可以是规格参数相同的视频采集设备，也可以是规格参数不同的视频采集设备，本申请实施例对此不作具体限定。其中，规格参数包括但不限定于视频采集设备的类型、电荷耦合器件(charge coupled device，CCD)尺寸等。可以理解的是，具有相同规格参数的视频采集设备，其采集到的监控视频流的帧率、监控视频流中图像帧的尺寸、分辨率等参数均相同。
接着，可选的，第一处理器可以通过并行运行的m个进程，对各自读取到的监控视频流的数据进行预处理。
下面以第一处理器运行的第一进程对第一进程读取到的第一路监控视频流的数据进行预处理为例，对第一处理器通过并行运行的m个进程对各自读取到的监控视频流的数据进行预处理的过程予以说明。具体的，第一进程可以通过以下至少一种方式对第一路监控视频流的数据进行预处理：
方式1、第一进程将第一路监控视频流中的图像帧的尺寸，缩放为统一的第一预设尺寸。这里，本申请实施例对该第一预设尺寸大小的具体取值不作具体限定。
这样的话，如果上述m个视频采集设备是具有不同规格参数的视频采集设备，则当第一处理器通过m个进程对各自读取到的、由具有不同规格参数的m个视频采集设备所采集的监控视频流的数据根据方式一进行预处理后，有助于第一处理器后续对具有不同规格参数的视频采集设备所采集的监控视频流的数据统一进行标准化的分析处理。
方式2、第一进程对第一路监控视频流中的图像帧进行去噪和/或对比度拉伸等处理，以提高第一路监控视频流中图像帧的图像质量。
这样的话，当第一路监控视频流是视频采集设备在夜间或光照条件差环境下采集到的监控视频流，则第一进程通过方式2对第一路监控视频流中的数据进行预处理后，可以使得第一路监控视频流中图像帧的图像质量得以提高，这样有助于第一处理器后续对第一路监控视频流的数据进行分析处理。
应理解，本申请实施例可以根据监控业务需要，选择上述一种或两种方式对监控视频流的数据进行预处理。并且，上述方式1和方式2仅为对监控视频流的数据进行预处理的示例性说明，并不构成对本申请实施例所保护范围的限定。可以理解的是，任意有助于第一处理器后续对监控视频流的数据进行分析处理的技术手段，都应在本申请实施例的保护范围内。
S102、第一处理器在读取到的监控视频流中的确定待处理图像帧。
第一处理器可以通过上述并行运行的m个进程分别在各自预处理过的监控视频流中确定待处理图像帧。其中，第一处理器通过进程在任一路监控视频流中确定的待处 理图像帧的数量可以是一个或多个，这样，第一处理器即可以通过m个进程在多路预处理过的监控视频流中确定出多个待处理图像帧。
为简单描述，下面以第一处理器运行的第一进程在其预处理过的第一路监控视频流(对应于本申请实施例中的任一路监控视频流)中确定待处理图像帧为例，对第一处理器通过上述并行运行的m个进程分别在各自预处理过的监控视频流中确定待处理图像帧的过程予以说明。
在一种可能的实现方式中，第一进程可以通过降采样方式，在第一路监控视频流的图像帧中确定待处理图像帧。
具体的，第一进程可以将从第一路监控视频流中每隔t帧图像抽取的图像帧，确定为待处理图像帧。其中，t是正整数。
示例性的，如果第一路监控视频流中依次包括图像帧1、图像帧2、图像帧3、图像帧4、图像帧5、图像帧6、图像帧7、图像帧8以及图像帧9，且以t的取值是3为例，则第一进程可以将第一路监控视频流中每隔3帧图像抽取的一帧图像帧，例如图像帧1、图像帧5、图像帧9，确定为待处理图像帧。其中，图像帧1和图像帧5之间间隔3帧图像帧(包括图像帧2、图像帧3和图像帧4)，图像帧5和图像帧9之间间隔3帧图像帧(包括图像帧6、图像帧7和图像帧8)。
在另一种可能的实现方式中，第一进程可以通过预设的动目标检测算法检测第一路监控视频流中的移动目标，并将第一路监控视频流中检测出的移动目标所对应的图像帧确定为待处理图像帧。其中，移动目标对应的图像帧可以是第一路监控视频流中连续的多帧图像帧，该连续的多帧图像帧的帧差大于动目标检测阈值。其中，本申请实施例对动目标检测算法和该动目标检测阈值的具体取值不作具体限定。
在一些实施例中，第一处理器中预置有多个动目标检测阈值，第一进程则可以基于第一处理队列中的待处理图像帧的数量，在该多个动目标检测阈值中选择一个动目标检测阈值作为当前的动目标检测阈值。这样，可以有效调节第二处理器所要处理的第一处理队列中待处理图像帧的数量，以避免第二处理器出现过闲或超载情况。其中，第一处理队列的描述可以参考下文描述，这里不作赘述。
示例性的，当第一处理队列中的待处理图像帧的数量大于预设阈值时，则第一进程可以选择预置的多个动目标检测阈值中较大的动目标检测阈值来作为当前的动目标检测阈值，以减少第一进程确定的待处理图像帧的数量。当第一处理队列中的待处理图像帧的数量小于预设阈值时，则第一进程可以选择预置的多个动目标检测阈值中较小的动目标检测阈值来作为当前的动目标检测阈值，以增加第一进程确定的待处理图像帧的数量。这样，即实现了对第二处理器所要处理的第一处理队列中待处理图像帧的数量的调节，进而避免了第二处理器出现过闲或超载情况。
在另一些实施例中，第一进程可以基于第一处理队列中的待处理图像帧的数量，来调节动目标检测阈值的大小，从而实现对第二处理器所要处理的第一处理队列中待处理图像帧的数量的调节，进而避免了第二处理器出现过闲或超载情况。
示例性的，当第一处理队列中的待处理图像帧的数量大于预设阈值时，则第一进程可以调高当前的动目标检测阈值，以减少第一进程确定的待处理图像帧的数量。当第一处理队列中的待处理图像帧的数量小于预设阈值时，则第一进程可以调低当前的 动目标检测阈值，以增加第一进程确定的待处理图像帧的数量。这样，即实现了对第二处理器所要处理的第一处理队列中待处理图像帧的数量的调节，进而避免了第二处理器出现过闲或超载情况。
S103、第一处理器将上述确定的待处理图像帧缓存至第一处理队列。
具体的，第一处理器中并行运行的m个进程各自确定出待处理图像帧后，进一步对各自确定出的待处理图像帧进行目标检测预处理，然后再将目标检测预处理后的待处理图像帧和该待处理图像帧的图像信息存至第一处理队列。
其中，待处理图像帧的图像信息包括该待处理图像帧所属监控视频流的标识号。可选的，待处理图像帧的图像信息还可以包括该待处理图像帧的帧标识。
其中，第一处理队列为先进先出的队列。
示例性的，对于第一处理器运行的第一进程而言，第一进程在第一路监控视频流中确定出待处理图像帧后，先对确定出的待处理图像帧进行目标检测预处理，然后再将目标检测预处理后的待处理图像帧和该待处理图像帧的图像信息缓存于第一处理队列。对于第一处理器运行的第二进程而言，第二进程在第二路监控视频流中确定出待处理图像帧后，先对确定出的待处理图像帧进行目标检测预处理，然后再将目标检测预处理后的待处理图像帧和该待处理图像帧的图像信息缓存于第一处理队列。
可以看出，第一处理器中并行运行的m个进程所确定的、且进行过目标检测预处理的待处理图像帧和该待处理图像帧的图像信息，缓存于同一个处理队列中等待处理。
其中，对待处理图像帧进行目标检测预处理，可以使待处理图像帧满足用于对图像进行图像推理的推理模型的输入图像的要求。其中，推理模型的描述可以参考下文相关描述，这里不作赘述。
下面，以第一处理器中运行的第一进程对第一待处理图像帧(第一进程确定出的待处理图像帧中的任一个待处理图像帧)进行目标检测预处理为例，对第一处理器中运行的进程对待处理图像帧进行目标检测预处理的过程进行说明。
具体的，第一进程可以通过以下至少一种方式对第一待处理图像帧进行目标检测预处理：
方式1、第一进程可以将第一待处理图像帧的尺寸缩放为第二预设尺寸大小。这里，第二预设尺寸大小是推理模型所规定的输入图像的尺寸大小。
方式2、第一进程对第一待处理图像帧进行归一处理。这里，归一处理是指将第一待处理图像帧中的每个像素值除以255，以得到归一处理后的第一待处理图像帧的像素值。可以理解的是，为简化运算，推理模型要求输入的图像通常是归一处理后的图像。
示例性的，如果第一待处理图像帧的像素a的像素值为110，则归一处理后的第一待处理图像帧的像素a的像素值为110/255，即0.43。
方式3、第一进程对第一待处理图像帧进行黑边填充，使得填充黑边后的第一待处理图像帧的长宽比例满足推理模型对输入图像的长宽比例要求。
方式4、第一进程对第一待处理图像帧进行维度扩展，以使维度扩展后的第一待处理图像帧满足推理模型对输入图像的维度要求。
应理解，本申请实施例可以根据推理模型对输入图像的要求，选择上述一种或多 种方式对待处理图像帧进行目标检测预处理。并且，上述方式1-方式4所述的对待处理图像帧进行目标检测预处理的方式仅为的示例性说明，并不构成对本申请实施例所保护范围的限定。可以理解的是，任意为使待处理图像帧满足推理模型对输入图像要求的技术手段，都应在本申请实施例的保护范围内。
可选的，第一处理器中并行运行的m个进程也可以在各自确定出待处理图像帧后，直接将各自确定出的待处理图像帧和该待处理图像帧的图像信息缓存至第一处理队列。
示例性的，对于第一处理器运行的第一进程而言，第一进程在第一路监控视频流中确定出待处理图像帧后，直接将确定出的待处理图像帧和该待处理图像帧的图像信息缓存于第一处理队列。对于第一处理器运行的第二进程而言，第二进程在第二路监控视频流中确定出待处理图像帧后，直接将确定出的待处理图像帧和该待处理图像帧的图像信息缓存于第一处理队列。
S104、第二处理器对从第一处理队列中获取的待处理图像帧依次进行图像推理，并将执行完成图像推理的待处理图像帧发送至第一处理器。
第二处理器中预置有推理模型，该推理模型用于对输入到该推理模型中的图像进行图像推理，从而根据图像推理结果即可以确定出该待处理图像帧中的目标信息。
其中，待处理图像帧中的目标信息，包括待处理图像帧中目标的类型和位置。其中，待处理图像帧中的目标的类型，例如可以是人、猫、狗、火焰等。待处理图像帧中目标的位置即为该目标在待处理图像帧中的所在区域对应的像素区域的位置。
这样的话，第二处理器即可以根据预置的推理模型对从第一处理队列中获取的待处理图像帧依次进行图像推理。
具体的，第二处理器可以通过并行运行的n个进程和上述预置的推理模型，对从第一处理队列中获取的待处理图像帧进行图像推理，以确定出待处理图像帧中每个像素的位置和类型。应理解，这里确定出的待处理图像帧中每个像素的位置和类型，即为推理模型进行图像推理后的推理结果。这样，根据该推理结果即可确定出待处理图像帧中的目标信息。
其中，n是正整数。并且n的取值可以通过第二处理器的数据处理能力确定。例如，可以通过实验确定出第二处理器同时运行的进程的数量，以使第二处理器的数据处理效率最高。
还需要说明的是，n通常小于m。也就是说，第二处理器中并行运行的用于对待处理图像帧进行图像推理的进程数量，小于第一处理器中并行运行的用于读取监控视频流的进程的数量。也即，第二处理器中并行运行的用用于对待处理图像帧进行图像推理的进程数量，小于第一处理器能够同时读取监控视频流的路数。
具体的，第二处理器并行运行的n个进程可以分别从第一处理队列中获取待处理图像帧，然后分别根据预置的推理模型对各自获取到的待处理图像帧进行图像推理。
应理解，如果当前的n个进程中存在多个空闲的进程，则该多个进程可以按照预设顺序，依次从第一处理队列中获取待处理图像帧，并根据预置的推理模型对获取到的待处理图像帧进行图像推理。这里，本申请实施例对该预设顺序不作具体限定。
应理解，该n个进程中的任一个进程执行完图像推理，并输出待处理图像帧的推理结果后，即可再一次从第一处理队列中获取新的待处理图像帧并对其进行图像推理。
作为示例，如果第二处理器运行了3个进程，分别是进程1、进程2和进程3。这样，当进程1、进程2和进程3当前均空闲，则可以按照进程1->进程2->进程3的顺序，依次从第一处理队列获取待处理图像帧并对其进行图像推理。当进程2首先完成所获取到的待处理图像帧的图像推理，则再一次从第一处理队列中获取新的待处理图像帧，并对其进行图像推理。
在第一种可能的实现方式中，如果第一处理队列中的待处理图像帧是已执行过S103中所述的目标检测预处理的图像帧，则第二处理器可以通过并行运行的n个进程和预置的推理模型，直接对从第一处理队列中获取的待处理图像帧进行图像推理，从而确定出待处理图像帧中每个像素的位置和类型。
这种情况下，由于可以运行更多并发进程的第一处理器已经对第一处理队列中的待处理图像帧执行过目标检测预处理，因此运行有较少并发进程的第二处理器无需再对从第一处理队列中获取的待处理图像帧进行目标检测预处理，这样即节省了第二处理器的资源，提高了第二处理器处理待处理图像帧的数量，从而提高了第一处理器和第二处理器联合分析处理监控视频流数据的效率，即提高了视频监控系统的数据处理效率。
在第二种可能的实现方式中，如果第一处理队列中的待处理图像帧是未执行过S103所述的目标检测预处理的图像帧，则第二处理器可以通过并行运行的n个进程，先对各自从第一处理队列中获取到的待处理图像帧进行目标检测预处理，然后再根据预置的推理模型对执行完成目标检测预处理的待处理图像帧进行图像推理，这样即可确定出该待处理图像帧中每个像素的位置和类型。
这里，第二处理器通过并行运行的n个进程对各自从第一处理队列中获取到的待处理图像帧进行目标检测预处理的过程，可以参考S103中第一处理器中并行运行的m个进程对各自确定出的待处理图像帧进行目标检测预处理的描述，这里不再赘述。第二处理器通过并行运行的n个进程，根据预置的推理模型对执行完成目标检测预处理的待处理图像帧进行图像推理的过程，可以参考上述第一种可能的实现方式中，第二处理器通过并行运行的n个进程和预置的推理模型，对从第一处理队列中获取的待处理图像帧进行图像推理的描述，这里不再赘述。
然后，第二处理器并行运行的n个进程，根据各自当前处理的待处理图像帧所属的监控视频流的标识号，将各自已执行完图像推理的待处理图像帧发送至第一处理器中与该标识号对应的进程的进程处理队列中，以使第一处理器中运行的与该标识号对应的进程对执行完图像推理的待处理图像帧进行后续处理。
第一处理器可以通过并行运行的m个进程对执行完图像推理的待处理图像帧进行后续处理。应理解，第一处理器对执行完图像推理的待处理图像帧进行后续处理的进程的数量，与S101中读取m路监控视频流的进程的数量相同，且一一对应。
其中，第一处理器对执行完图像推理的待处理图像帧进行后续处理的描述，可以参考下述S105的描述，这里不予赘述。
可选的，第二处理器并行运行的n个进程中的任一个进程在对执行完图像推理后，可以根据上述图像推理的得到的待处理图像帧中每个像素的位置和类型，确定出待处理图像帧中的目标信息。这一过程，可以称为对执行完图像推理的图像帧进行目标检 测后处理的过程。
应理解，对待处理图像帧进行目标检测预处理、推行推理以及目标检测后处理的过程，可以称为对待处理图像帧执行目标检测的过程。
具体的，对于第二处理器运行的任一个进程而言，该任一个进程完成对待处理图像帧的图像推理后，即对图像推理所确定出的该待处理图像帧中每个像素的位置和类型进行统计分析，并将位置临近且类型相同的像素区域的位置，确定为该待处理图像帧中目标的位置，以及将该位置临近且类型相同的像素的类型确定为前述目标的类型。这样，该任一个进程即根据待处理图像帧的推理结果(即待处理图像帧中每个像素的位置和类型)确定出了待处理图像帧中的目标信息。
这样的话，第二处理器并行运行的n个进程，可以根据各自当前处理的待处理图像帧所属监控视频流的标识号，将各自已确定出目标信息的待处理图像帧发送至第一处理器与该标识号对应的进程的进程处理队列中，以使第一处理器中运行的与该标识号对应的进程对已确定出目标信息的待处理图像帧进行后续处理。
应理解，第一处理器可以通过并行运行的m个进程对确定出目标信息的待处理图像帧进行后续处理。可以看出，第一处理器对确定出目标信息的待处理图像帧进行后续处理的进程的数量，与S101中读取m路监控视频流的进程的数量相同，且一一对应。
其中，第一处理器对确定出目标信息的待处理图像帧进行后续处理的描述，可以参考下述S105的描述，这里不再赘述。
S105、第一处理器根据预设算法对接收到的经第二处理器处理过的待处理图像帧进行处理，以确定该待处理图像帧所属监控视频流中是否异常事件。
具体的，第一处理器可以通过并行运行的m个进程接收经第二处理器中n个进程处理过的多个待处理图像帧，并根据预设算法对接收到的多个待处理图像帧进行处理，以确定出该多个待处理图像帧所属的监控视频流中是否存在异常事件。这一过程，可以称为对待处理图像帧进行逻辑业务后处理的过程。其中，第一处理器通过并行运行的m个进程接收到的、经第二处理器中n个进程处理过的多个待处理图像帧，可以属于同一路监控视频流，也可以属于不同路的监控视频流，本申请实施例对此不作限定。
其中，第一处理器中预置有一种或多种预设算法。其中，预设算法是基于视频监控系统所监控的业务预先设计的，并用于对视频监控系统中的视频采集设备所采集到的监控视频流中的一帧或多帧图像帧进行运算处理，从而可以确定出该监控视频流中是否存在与视频监控系统所监控业务对应的异常事件，进而实现视频监控系统对所监控业务的智能监控。
例如，如果视频监控系统的监控业务是火灾监控，则异常事件即为该视频监控系统中的视频采集设备所采集到的监控视频流中的一帧或多帧图像帧中包括火焰。那么基于监控火灾的业务预先设计的预设算法，可以用于对该视频监控系统中的视频采集设备所采集到的监控视频流中的一帧或多帧图像帧进行跟踪分析，从而确定出该监控视频流中的一帧或多帧图像帧中是否包括火焰。这样，即可实现视频监控系统对火灾的智能监控。
还应理解，不同的视频采集设备可以用于实现不同业务的监控，因此，不同视频 采集设备采集到的监控视频流，一般采用与其监控业务对应的预设算法进行智能分析处理。
在一些实施例中，如果第一处理器并行运行的m个进程所接收到的待处理图像帧，是未执行过目标检测后处理的待处理图像帧，则第一处理器并行运行的m个进程可以先对各自接收到的待处理图像帧进行目标检测后处理。其中，目标检测后处理的详细描述可以参考上述S104中有关目标检测后处理的描述，这里不再赘述。
这样，第一处理器并行运行的m个进程通过对各自接收到的待处理图像帧进行目标检测后处理后，即可确定出各自接收到的待处理图像帧中的目标信息。其中，目标信息的描述可以参考上文S103中目标信息的详细描述，这里不再赘述。
这种情况下，通过运行更多并发进程的第一处理器对执行过图像推理的待处理图像帧进行目标检测后处理，可以使运行有较少并发进程的第二处理器无需在S103对执行过图像推理的待处理图像帧进行目标检测后处理，这样即节省了第二处理器的资源，从而提高了第二处理器处理待处理图像的数量，进而提高了第一处理器和第二处理器联合分析处理监控视频流数据的效率，即提高了视频监控系统的数据处理效率。
然后，第一处理器并行运行的m个进程根据各自确定出的待处理图像帧中的目标信息和该待处理图像帧所属的监控视频流所监控业务对应的预设算法，即可确定该待处理图像帧所属的监控视频流中是否存在异常事件。
当待处理图像帧所属的监控视频流中存在异常事件，则第一处理器执行S106。
在另一些实施例中，如果第一处理器并行运行的m个进程所接收到的待处理图像帧，是已执行过目标检测后处理的待处理图像帧，则第一处理器并行运行的m个进程可以根据各自接收到的待处理图像帧中的目标信息，以及该待处理图像帧所属的监控视频流所监控业务对应的预设算法，直接确定该待处理图像帧所属的监控视频流中是否存在异常事件。
当待处理图像帧所属的监控视频流中存在异常事件，则第一处理器执行S106。
S106、第一处理器控制输出告警信息。
第一处理器可以控制数据处理装置通过其输入输出接口(例如显示器和音频模块)或通信接口，通过文字、语音、图像或视频流等任一种方式向视频监控系统的管理人员输出告警信息，以使管理人员及时对异常事件予以处理。
示例性的，第一处理器可以在确定出异常事件的监控视频流中，对异常事件的目标予以标记，并指示数据处理装置的通信接口通过实时消息传输协议(Real Time Messaging Protocol，RTMP)将标记了异常事件目标的一段监控视频流输出，以实现向视频监控系统的管理人员告警的目的，并使得管理人员及时对该异常事件予以处理。
通过上述方法，数据处理装置可以通过将第一处理器运行进程的处理运算和第二处理器运行进程的处理运算分别池化，以解耦第一处理器和第二处理器通过各自运行的进程对监控视频流的不同阶段的分析处理，使得第一处理器并行处理监控视频流的路数，不再受到第二处理器并行处理监控视频流的路数的限制，从而使得数据处理能力及资源有限的第二处理器不会限制数据处理装置同时处理监控视频流的路数。也就是说，相比通过数据处理装置的CPU和GPU串行分析处理监控视频流数据时能够同时处理监控视频流的路数，本申请实施例提供的方法提高了数据处理装置同时处理监 控视频流的路数，即本申请实施例提供的方法提高了视频监控系统的数据处理效率。
同时，由于第一处理器和第二处理器通过各自运行的进程对监控视频流的不同阶段的分析处理解耦，使得数据处理装置的第一处理器可以灵活增删待分析处理的监控视频流。
此外，在上述的方法中，通过具有更高并发处理能力的第一处理器来执行目标检测预处理和目标检测后处理，可以节省第二处理器的资源，使得第二处理器可以通过有限的资源处理第一处理队列中更多的待处理图像帧，即提高了第二处理器处理第一处理队列中待处理图像帧的数量，从而进一步提高了本申请实施例中第一处理器和第二处理器联合处理视频监控流数据的效率，即提高了视频监控系统的数据处理效率。
可以看出，在上述方法中，对于第一处理器执行的S101-S103可以看作是对监控视频流的第一处理阶段，第二处理器执行的S104可以看作是对监控视频流的第二处理阶段，而第一处理器执行的S105-S106可以看作是对监控视频流的第三处理阶段。
参考图4，图4示出了本申请实施例提供视频监控数据的处理方法的另一种示意图。
如图4所示，在第一处理阶段，第一处理器可以通过并行运行的m个进程执行上述的S101-S103。该m个进程可以是图4所示的进程11、进程12、…、以及进程1m。
在第二处理阶段，第二处理器可以通过并行运行的n个进程执行S104。其中，该n个进程可以是图4所示的进程21、…、以及进程2n，这里，第一处理器和第二处理器之间可以通过第一处理队列连接通信。
在第三处理阶段，第一处理器还可以通过并行运行的m个进程执行S105-S106。其中，该m个进程可以是图4中所示的进程31、进程32、…、以及进程3m。
这样的话，当需要增删第三处理阶段中用于确定异常事件的预设算法时，只需对第三处理阶段的逻辑模块予以替换即可，从而实现了灵活增删用于确定不同监控业务中异常事件的预设算法的目的。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对视频监控数据的处理装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图5所示，图5示出了本申请实施例提供的一种视频监控数据的处理装置50的结构示意图。装置50可以用于执行上述的视频监控数据的处理方法，例如用于执行图3所示的方法。其中，装置50可以包括第一处理单元51和第二处理单元52。
第一处理单元51，用于从多路监控视频流中获取多个待处理图像帧，并将该多个待处理图像帧缓存于第一处理队列。第二处理单元52，用于对从第一处理队列获取的多个待处理图像帧进行图像推理，以确定该多个待处理图像帧中的目标信息。
作为示例，结合图3，第一处理单元51可以用于执行S101-S102，第二处理单元52可以用于执行S104。
可选的，第二处理单元52，具体用于通过并行运行的n个进程对上述多个待处理图像帧进行图像推理。其中，n是正整数，且n的取值根据第二处理单元52的数据处理能力来确定。
作为示例，结合图3，第二处理单元52可以用于执行S104。
可选的，上述的多个待处理图像帧是该多个待处理图像帧所属监控视频流中的移动目标对应的图像帧。或者，上述的多个待处理图像帧是从该多个待处理图像帧所属的监控视频流中每隔t帧图像抽取的图像帧，t是正整数。
可选的，第一处理单元51，还用于根据上述多个待处理图像帧中的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件；以及，如果该多个待处理图像帧所属监控视频流中存在异常事件，则第一处理单元51控制输出告警信息。
作为示例，结合图3，第一处理单元51可以用于执行S105-S106。
可选的，第一处理单元51，具体用于通过并行运行的m个进程，在多路监控视频流中获取多个待处理图像帧，m是大于1的正整数。
作为示例，结合图3，第一处理单元51可以用于执行S101-S102。
可选的，第一处理单元51，还具体用于根据预设算法和上述多个待处理图像帧的目标信息，确定该多个待处理图像帧所属监控视频流中是否存在异常事件。其中，该预设算法是根据该多个待处理图像帧所属的监控视频流所监控的业务确定的。
作为示例，结合图3，第一处理单元51可以用于执行S105。
可选的，上述目标信息包括待处理图像中目标的类型和/或位置。
可选的，第一处理单元51，还用于在将上述多个待处理图像帧缓存于第一处理队列之前，对该多个待处理图像帧进行目标检测预处理。其中，目标检测预处理包括以下至少一种方式：尺寸缩放、归一处理、黑边填充、维度扩展等。
关于上述可选方式的具体描述可以参见前述的方法实施例，此处不再赘述。此外，上述提供的任一种装置50的解释以及有益效果的描述均可参考上述对应的方法实施例，不再赘述。
当然，本申请实施例提供的装置50包括但不限于上述单元，例如装置50还可以包括存储单元53。其中，存储单元53可以用于存储第一处理单元51和第二处理单元52所运行的程序代码等，本申请实施例对此不作限定。
作为一个示例，结合图1，装置50中的第一处理单元51和第二处理单元52所实现的功能，可以通过图1中的处理器11执行图1中的主存储器12中的程序代码实现。存储单元53所实现的功能可以和图1中的主存储器12实现的功能相同。
作为一种可能的实施例，本申请还提供一种视频监控数据的处理设备，该设备可以如图1所示，包括处理器11、主存储器12、存储介质13和通信接口14，该设备用于执行上述图3所示视频监控数据的处理方法，以实现视频监控数据的高效处理。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件程序实现时，可以全部或部分地以计算机程序产品的形式来实现。该计算机程序产品包括一个或多个计算机指令。在计算机上和执行计算机执行指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid-state disk，SSD))等。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (18)

  1. 一种视频监控数据的处理方法，其特征在于，包括：
    第一处理器从多路监控视频流中获取多个待处理图像帧，并将所述多个待处理图像帧缓存于第一处理队列；
    第二处理器对从所述第一处理队列获取的所述多个待处理图像帧进行图像推理，以确定所述多个待处理图像帧中的目标信息。
  2. 根据权利要求1所述的方法，其特征在于，所述第二处理器对从所述第一处理队列获取的所述多个待处理图像帧进行图像推理，包括：
    所述第二处理器通过并行运行的n个进程对所述多个待处理图像帧进行图像推理；其中，所述n是正整数，所述n的取值根据所述第二处理器的数据处理能力来确定。
  3. 根据权利要求1或2所述的方法，其特征在于，所述多个待处理图像帧是所述多个待处理图像帧所属监控视频流中的移动目标对应的图像帧；或者，所述多个待处理图像帧是从所述多个待处理图像帧所属的监控视频流中每隔t帧图像抽取的图像帧，t是正整数。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述方法还包括：
    第一处理器根据所述多个待处理图像帧中的目标信息，确定所述多个待处理图像帧所属的监控视频流中是否存在异常事件；
    如果存在所述异常事件，则第一处理器控制输出告警信息。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述第一处理器从多路监控视频流中获取所述多个待处理图像帧，包括：
    所述第一处理器通过并行运行的m个进程，在所述多路监控视频流中获取所述多个待处理图像帧，m是大于1的正整数。
  6. 根据权利要求4所述的方法，其特征在于，所述第一处理器根据所述多个待处理图像帧中的目标信息，确定所述多个待处理图像帧所属监控视频流中是否存在异常事件，具体包括：
    第一处理器根据预设算法和所述多个待处理图像帧的目标信息，确定所述多个待处理图像帧所属监控视频流中是否存在所述异常事件；其中，所述预设算法是根据所述多个待处理图像帧所属的监控视频流所监控的业务确定的。
  7. 根据权利要求1-6任一项所述的方法，其特征在于，所述目标信息包括待处理图像中目标的类型和/或位置。
  8. 根据权利要求7所述的方法，其特征在于，在所述将所述多个待处理图像帧缓存于第一处理队列之前，所述方法还包括：
    所述第一处理器对所述多个待处理图像帧进行目标检测预处理；其中，所述目标检测预处理包括以下至少一种方式：尺寸缩放、归一处理、黑边填充、维度扩展。
  9. 一种视频监控数据的处理装置，其特征在于，所述装置包括：
    第一处理单元，用于从多路监控视频流中获取多个待处理图像帧，并将所述多个待处理图像帧缓存于第一处理队列；
    第二处理单元，用于对从所述第一处理队列获取的所述多个待处理图像帧进行图像推理，以确定所述多个待处理图像帧中的目标信息。
  10. 根据权利要求9所述的装置，其特征在于，
    所述第二处理单元，具体用于通过并行运行的n个进程对所述多个待处理图像帧进行图像推理；其中，所述n是正整数，所述n的取值根据所述第二处理单元的数据处理能力来确定。
  11. 根据权利要求9或10所述的装置，其特征在于，所述多个待处理图像帧是所述多个待处理图像帧所属监控视频流中的移动目标对应的图像帧；或者，所述多个待处理图像帧是从所述多个待处理图像帧所属的监控视频流中每隔t帧图像抽取的图像帧，t是正整数。
  12. 根据权利要求9-11中任一项所述的装置，其特征在于，
    所述第一处理单元，还用于根据所述多个待处理图像帧中的目标信息，确定所述多个待处理图像帧所属监控视频流中是否存在异常事件；以及用于，如果存在所述异常事件，则控制输出告警信息。
  13. 根据权利要求9-12中任一项所述的装置，其特征在于，
    所述第一处理单元，具体用于通过并行运行的m个进程，在所述多路监控视频流中获取所述多个待处理图像帧，m是大于1的正整数。
  14. 根据权利要求12所述的装置，其特征在于，
    所述第一处理单元，还具体用于根据预设算法和所述多个待处理图像帧的目标信息，确定所述多个待处理图像帧所属监控视频流中是否存在所述异常事件；其中，所述预设算法是根据所述多个待处理图像帧所属的监控视频流所监控的业务确定的。
  15. 根据权利要求9-14中任一项所述的装置，其特征在于，所述目标信息包括待处理图像中目标的类型和/或位置。
  16. 根据权利要求15所述的装置，其特征在于，
    所述第一处理单元，还用于在将所述多个待处理图像帧缓存于第一处理队列之前，对所述多个待处理图像帧进行目标检测预处理；其中，所述目标检测预处理包括以下至少一种方式：尺寸缩放、归一处理、黑边填充、维度扩展。
  17. 一种视频监控数据的处理设备，其特征在于，所述设备包括第一处理器、第二处理器以及存储器，所述第一处理器和所述第二处理器被配置为调用存储在所述存储器中的计算机指令，以使所述设备通过所述第一处理器执行如权利要求1-8中任一项所述的由第一处理器执行的方法，以及通过所述第二处理器执行如权利要求1-8中任一项所述的由第二处理器执行的方法。
  18. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括程序指令，当所述程序指令在计算机或处理器上运行时，使得所述计算机或所述处理器执行权利要求1-8中任一项所述的方法。
PCT/CN2021/138533 2021-05-08 2021-12-15 视频监控数据的处理方法及装置 WO2022237165A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110502427.1A CN115393781A (zh) 2021-05-08 2021-05-08 视频监控数据的处理方法及装置
CN202110502427.1 2021-05-08

Publications (1)

Publication Number Publication Date
WO2022237165A1 true WO2022237165A1 (zh) 2022-11-17

Family

ID=84029223

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138533 WO2022237165A1 (zh) 2021-05-08 2021-12-15 视频监控数据的处理方法及装置

Country Status (2)

Country Link
CN (1) CN115393781A (zh)
WO (1) WO2022237165A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116362564A (zh) * 2023-04-04 2023-06-30 重庆海辉星科技有限公司 酒店公共卫生视频监管结果的生成方法及系统
CN117294903A (zh) * 2023-10-25 2023-12-26 深蓝(深圳)感知科技有限公司 一种多路视频流关键帧标记及推流方法
CN117880626A (zh) * 2024-03-11 2024-04-12 珠海创能科世摩电气科技有限公司 输电线路的视频监控方法、装置、系统、存储介质
WO2024108703A1 (zh) * 2022-11-22 2024-05-30 天地伟业技术有限公司 一种多目视频监控系统搭建方法及电子设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115861033B (zh) * 2022-12-20 2024-09-03 北京远舢智能科技有限公司 一种图像数据处理方法、装置、计算机设备及存储介质
CN116016805B (zh) * 2023-03-27 2023-06-20 四川弘和通讯集团有限公司 数据处理方法、装置、电子设备及存储介质
CN117931458B (zh) * 2024-03-21 2024-06-25 北京壁仞科技开发有限公司 一种推理服务调度方法、装置、处理器及芯片

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062499A (zh) * 2016-11-08 2018-05-22 中国移动通信有限公司研究院 一种人脸识别的方法、系统及平台
CN109922319A (zh) * 2019-03-26 2019-06-21 重庆英卡电子有限公司 基于多核cpu的rtsp协议多视频流并行预处理方法
CN110365942A (zh) * 2019-07-18 2019-10-22 北京工业大数据创新中心有限公司 一种实时视频智能分析方法和系统
US20200334847A1 (en) * 2019-04-22 2020-10-22 Hongfujin Precision Electronics(Tianjin)Co.,Ltd. Image recognition device and method for recognizing images
CN112116636A (zh) * 2019-06-19 2020-12-22 华为技术有限公司 目标分析方法、装置、系统、节点设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062499A (zh) * 2016-11-08 2018-05-22 中国移动通信有限公司研究院 一种人脸识别的方法、系统及平台
CN109922319A (zh) * 2019-03-26 2019-06-21 重庆英卡电子有限公司 基于多核cpu的rtsp协议多视频流并行预处理方法
US20200334847A1 (en) * 2019-04-22 2020-10-22 Hongfujin Precision Electronics(Tianjin)Co.,Ltd. Image recognition device and method for recognizing images
CN112116636A (zh) * 2019-06-19 2020-12-22 华为技术有限公司 目标分析方法、装置、系统、节点设备及存储介质
CN110365942A (zh) * 2019-07-18 2019-10-22 北京工业大数据创新中心有限公司 一种实时视频智能分析方法和系统

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024108703A1 (zh) * 2022-11-22 2024-05-30 天地伟业技术有限公司 一种多目视频监控系统搭建方法及电子设备
CN116362564A (zh) * 2023-04-04 2023-06-30 重庆海辉星科技有限公司 酒店公共卫生视频监管结果的生成方法及系统
CN116362564B (zh) * 2023-04-04 2023-09-08 重庆海辉星科技有限公司 酒店公共卫生视频监管结果的生成方法及系统
CN117294903A (zh) * 2023-10-25 2023-12-26 深蓝(深圳)感知科技有限公司 一种多路视频流关键帧标记及推流方法
CN117880626A (zh) * 2024-03-11 2024-04-12 珠海创能科世摩电气科技有限公司 输电线路的视频监控方法、装置、系统、存储介质
CN117880626B (zh) * 2024-03-11 2024-05-24 珠海创能科世摩电气科技有限公司 输电线路的视频监控方法、装置、系统、存储介质

Also Published As

Publication number Publication date
CN115393781A (zh) 2022-11-25

Similar Documents

Publication Publication Date Title
WO2022237165A1 (zh) 视频监控数据的处理方法及装置
Ali et al. RES: Real-time video stream analytics using edge enhanced clouds
CN110443116B (zh) 视频行人检测方法、装置、服务器及存储介质
US7697026B2 (en) Pipeline architecture for analyzing multiple video streams
EP2688296B1 (en) Video monitoring system and method
CN104320615A (zh) 一种智能视频安防系统及其信号处理方法
JP2023511327A (ja) モデル訓練方法および装置
TWI698121B (zh) 網路裝置、影像處理方法及電腦可讀媒體
Sun et al. mVideo: Edge computing based mobile video processing systems
JP2018198406A (ja) 監視カメラシステムおよび画像処理装置
US7663661B2 (en) Feed-customized processing of multiple video streams in a pipeline architecture
JP7529025B2 (ja) ネットワークの制御方法、および、データ処理システム
Hong et al. Optimizing cloud-based video crowdsensing
US20180332224A1 (en) Integrated Solutions For Smart Imaging
US20200074228A1 (en) Rgbd sensing based object detection system and method thereof
Hou et al. Real-time surveillance video salient object detection using collaborative cloud-edge deep reinforcement learning
CN113393629B (zh) 入侵行为检测方法、装置和多路视频监控系统
CN114125400A (zh) 一种多通道视频分析方法及装置
CN102355575A (zh) 一种具有预警功能的分布式视频监控装置
CN111988397B (zh) 一种基于边缘计算的防震减灾救灾方法和系统
WO2021084944A1 (ja) 情報処理システム、情報処理方法、撮像装置、情報処理装置
WO2018177050A1 (zh) 一种对象识别方法、设备和系统
WO2022237484A1 (zh) 一种推理系统、方法、装置及相关设备
CN113038261A (zh) 一种视频生成方法、装置、设备、系统及存储介质
Paissan et al. People/car classification using an ultra-low-power smart vision sensor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21941727

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21941727

Country of ref document: EP

Kind code of ref document: A1