WO2023138311A1 - 基于亿级像素设备的云端视频监控方法与监控平台 - Google Patents

基于亿级像素设备的云端视频监控方法与监控平台 Download PDF

Info

Publication number
WO2023138311A1
WO2023138311A1 PCT/CN2022/141922 CN2022141922W WO2023138311A1 WO 2023138311 A1 WO2023138311 A1 WO 2023138311A1 CN 2022141922 W CN2022141922 W CN 2022141922W WO 2023138311 A1 WO2023138311 A1 WO 2023138311A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
cloud server
picture
cloud
optical fiber
Prior art date
Application number
PCT/CN2022/141922
Other languages
English (en)
French (fr)
Inventor
袁潮
邓迪旻
Original Assignee
北京拙河科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京拙河科技有限公司 filed Critical 北京拙河科技有限公司
Publication of WO2023138311A1 publication Critical patent/WO2023138311A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/22—Adaptations for optical transmission

Definitions

  • the invention belongs to the technical field of billion-level pixel video processing, and in particular relates to a cloud video monitoring method and monitoring platform based on a billion-level pixel device, a computer terminal device and a storage medium for realizing the method.
  • High resolution represents high quality and high definition.
  • High-definition video images not only have a wide range of applications in military, medical, monitoring, astronomy, etc., but also bring a more comfortable visual experience to entertainment life. In such an era background, it is particularly important to improve the user's video experience.
  • the resolution of video images needs to be continuously improved, and the development of cameras from 4K and 8K to billion-level pixels and billion-level pixel cameras has become a mainstream trend.
  • the imaging resolution exceeds 100 million levels, the amount of data generated also increases sharply; and when using a light field camera to acquire image data, the light field camera can not only collect image information (planar, two-dimensional image information), but also similar to lidar, generate the depth map information corresponding to this picture and the point cloud information corresponding to this depth map through a frame of image collected; the light field camera can also obtain the spatial information and angle information of the light during imaging at the same time, so that the dimensionality of the data increases accordingly.
  • image information planar, two-dimensional image information
  • lidar similar to lidar
  • the remote light field array data needs to be transmitted to the cloud for processing and then rendered and displayed.
  • the heterogeneous system composed of graphics processors and general-purpose processors needs to coordinate with each other when processing high-throughput, high-dimensional image data.
  • it When it is applied to the billion-level pixel video processing based on high-throughput light field camera arrays, it will also cause the data rate and frame rate between the execution end and the transmission end to mismatch, resulting in data delays, freezes, or over-rendering, making the cloud monitoring effect poor.
  • the present invention proposes a cloud video monitoring method and monitoring platform based on a billion-level pixel device, a computer terminal device and a storage medium for realizing the method.
  • a cloud video monitoring method based on a billion-level pixel device includes the following steps:
  • the cloud server adjusts execution parameters of the cloud server and executes video surveillance based on the received forwarding data.
  • the forwarding data includes 2D image data and 3D image data
  • the cloud server is connected to multiple general-purpose processors and multiple graphics processors, and outputs the results of the video monitoring to a visual interface;
  • the step S300 specifically includes:
  • the cloud server displays the 2D image frame and the 3D image frame synchronously on the visual interface based on the format and quantity of the forwarded data received, and adjusts the execution parameters of the cloud server based on the frame parameters of the 2D image frame and/or the 3D image frame.
  • the execution parameters of the cloud server include the number of called general purpose processors and/or graphics processors, and the frequency of called general purpose processors and/or graphics processors.
  • the optical fiber network device includes a data distribution module, and the data distribution module is connected to a plurality of optical fiber transmission channels;
  • the step S200 specifically includes:
  • the picture parameters of the 2D image picture and/or the 3D image picture include picture display frame rate;
  • the execution parameters of the cloud server also include feedback control parameters, and the feedback control parameters are used to control the image acquisition frequency of the billion-level pixel device;
  • Described based on the picture parameter of 2D picture picture and/or 3D picture picture, adjust the execution parameter of cloud server, comprise:
  • a cloud video monitoring platform based on billion-level pixel devices includes a plurality of light field camera arrays, and the plurality of light field camera arrays are arranged around the same monitoring area.
  • the cloud video monitoring platform also includes:
  • a cloud server is connected to a plurality of general purpose processors and a plurality of graphics processors;
  • the optical fiber network equipment includes a data distribution module, the data distribution module is connected to multiple optical fiber transmission channels, and the multiple optical fiber transmission channels include plane data transmission channels and deep point cloud data transmission channels;
  • Each of the light field camera arrays acquires the light field image data of the monitoring area according to a preset frequency, and then sends it to the optical fiber network device;
  • the optical fiber network device forwards the light field image data to the cloud server through the data distribution module;
  • the cloud server performs synchronous processing on the light field image data through the plurality of general purpose processors and the plurality of graphics processors, and outputs the result of the synchronous processing to a visual interface.
  • the optical fiber network device forwards the light field image data to the cloud server through the data distribution module, specifically including:
  • the optical fiber network device identifies plane data and depth point cloud data in the image data through the data distribution module;
  • the plane data is forwarded to the cloud server through the plane data transmission channel
  • the depth point cloud data is forwarded to the cloud server through the depth point cloud data transmission channel at the same time.
  • the cloud server performs synchronous processing on the light field image data through the multiple general-purpose processors and multiple graphics processors, and outputs the result of the synchronous processing to a visual interface, specifically including:
  • the cloud server executes two-dimensional picture visual reconstruction on the received plane data through the plurality of general-purpose processors, and sends the result of the two-dimensional picture visual reconstruction to the graphics processor;
  • the graphics processor performs depth map reconstruction on the received depth point cloud data based on the result of visual reconstruction of the two-dimensional picture.
  • the billion-level pixel device is a light field camera array
  • the image data is light field image data collected by the light field camera array, including planar image data and depth point cloud data.
  • Both the transmission rate and the transmission throughput of the planar data transmission channel are greater than the depth point cloud data transmission channel.
  • the present invention can be implemented as a computer medium, computer program instructions are stored on the computer medium, and by executing the program instructions, a cloud video monitoring method based on a billion-level pixel device described in the first aspect is realized.
  • the present invention can also be embodied as a computer program product, the program product is carried on a computer storage medium, and the processor executes the program, thereby realizing all or part of the steps of the above-mentioned cloud video monitoring method based on a megapixel device.
  • the present invention aims at the large frame rate, high resolution and multi-channel characteristics of the light field image video data of the billion-level pixel device.
  • processing the cloud image server in order to ensure that the cloud image server can process the video data in a timely manner, first identify the plane data and the depth point cloud data in the image data; then forward the plane data and the depth point cloud data to the cloud image server through the corresponding optical fiber transmission channel, and simultaneously display the 2D image picture and the 3D image picture on the visual interface, and adjust the picture parameters of the cloud server based on the 2D image picture and/or 3D image picture. Execution parameters, so as to ensure that the data rate and frame rate of the execution end and the transmission end can match, so as to achieve a better monitoring effect and avoid data delay, freeze or over-rendering.
  • Fig. 1 is a schematic diagram of the main process of a cloud video monitoring method based on a billion-level pixel device according to an embodiment of the present invention
  • Fig. 2 is a schematic diagram of a further specific embodiment of the method described in Fig. 1;
  • Fig. 3 is the architecture diagram of the cloud video monitoring platform based on the billion-level pixel device for realizing the method described in Fig. 1 or Fig. 2;
  • Fig. 4 is a schematic diagram of part of the working principle of the cloud video monitoring platform based on the billion-level pixel device described in Fig. 3;
  • Fig. 5 is a schematic diagram of the structure and principle of a computer electronic device for implementing a cloud video monitoring method according to an embodiment of the present invention
  • Light-field camera also known as plenoptic camera (Plenoptic camera).
  • the light field camera can not only collect image information (planar, two-dimensional image information), but also generate the depth map information corresponding to this picture and the point cloud information corresponding to this depth map through a frame of image collected; the light field camera can also obtain the spatial information and angle information of the light during imaging, and map the pixels in the two-dimensional image into a multi-dimensional (greater than 2-dimensional, such as three-dimensional or four-dimensional) light field for reprojection according to certain rules, and obtain in-focus images of different viewing angles and different phase planes.
  • Depth map information is used to represent the distance between a point in the scene and the camera (viewpoint) in each pixel value of the image.
  • Fig. 1 is a kind of main step diagram of the cloud end video surveillance method based on the megapixel device of an embodiment of the present invention, described step comprises S100-S300, and each step is concretely realized as follows:
  • the cloud server adjusts execution parameters of the cloud server and executes video surveillance based on the received forwarding data.
  • the multiple megapixel devices are multiple light field camera arrays arranged around the same monitoring area, and the image data is the light field image data collected by the light field camera array, including planar image data and depth point cloud data.
  • the forwarding data includes 2D image data and 3D image data
  • the 2D image data may be the planar image data
  • the 3D image data may be the depth point cloud data
  • the light field camera can not only collect image information (planar, two-dimensional image information), but also generate the depth map information corresponding to this picture and the point cloud information corresponding to this depth map through a frame of image collected; the light field camera can also obtain the spatial information and angle information of the light during imaging at the same time, map the pixels in the two-dimensional image into a multi-dimensional (greater than 2-dimensional, such as three-dimensional or four-dimensional) light field for reprojection according to certain rules, and obtain in-focus images of different angles of view and different phase planes.
  • those skilled in the art can also obtain other types of two-dimensional (2D) planar image data from the image data of the light field camera, and 3D image data correspondingly associated with the corresponding 2D planar image data, and the corresponding association means that the image of the target area at that time can be reconstructed after the two are synchronously fused in time series.
  • 2D two-dimensional
  • the hardware structure for realizing the method includes multiple billion-level pixel devices, optical fiber network devices and cloud servers.
  • the optical fiber network equipment includes a data distribution module, and the data distribution module is connected to a plurality of optical fiber transmission channels;
  • the cloud server is connected with a plurality of general purpose processors and a plurality of graphics processors, and outputs the results of the video monitoring to a visual interface.
  • a general-purpose processor includes a CPU, and a graphics processor includes a GPU; the former can perform task scheduling and simple (two-dimensional) data processing, and the latter is dedicated to performing 3D image calculation tasks allocated by the former.
  • FIG. 2 further shows specific implementation forms of each sub-step on the basis of FIG. 1 .
  • the step S200 specifically includes:
  • the data distribution module is connected to multiple optical fiber transmission channels, and the multiple optical fiber transmission channels include plane data transmission channels and deep point cloud data transmission channels;
  • the plane data is forwarded to the cloud server through the plane data transmission channel, and at the same time, the depth point cloud data is forwarded to the cloud server through the depth point cloud data transmission channel.
  • the step S300 specifically includes:
  • the cloud server displays the 2D image frame and the 3D image frame synchronously on the visual interface based on the format and quantity of the forwarded data received, and adjusts the execution parameters of the cloud server based on the frame parameters of the 2D image frame and/or the 3D image frame.
  • the execution parameters of the cloud server include the number of called general purpose processors and/or graphics processors, and the frequency of called general purpose processors and/or graphics processors.
  • the picture parameters of the 2D image picture and/or the 3D image picture include picture display frame rate;
  • the execution parameters of the cloud server further include feedback control parameters, and the feedback control parameters are used to control the image acquisition frequency of the megapixel device;
  • Described based on the picture parameter of 2D picture picture and/or 3D picture picture, adjust the execution parameter of cloud server, comprise:
  • the feedback control parameter is generated when the picture parameter does not satisfy a predetermined condition.
  • the adjustment of the execution parameters of the cloud server based on the frame parameters of the 2D image frame and/or the 3D image frame includes:
  • the frequency of calling the general processor and/or the graphics processor is increased.
  • FIG. 3 is an architecture diagram of a cloud video monitoring platform based on a megapixel device that implements the method described in FIG. 1 or FIG. 2 .
  • the cloud video monitoring platform includes multiple light field camera arrays, and the multiple light field camera arrays are arranged around the same monitoring area.
  • the cloud video monitoring platform also includes a cloud server and optical fiber network equipment.
  • the cloud server is connected to multiple general-purpose processors and multiple graphics processors;
  • the optical fiber network equipment includes a data distribution module connected to multiple optical fiber transmission channels,
  • Each of the light field camera arrays acquires the light field image data of the monitoring area according to a preset frequency, and then sends it to the optical fiber network device;
  • the optical fiber network device forwards the light field image data to the cloud server through the data distribution module;
  • the cloud server performs synchronous processing on the light field image data through the plurality of general purpose processors and the plurality of graphics processors, and outputs the result of the synchronous processing to a visual interface.
  • FIG. 4 further shows a schematic diagram of part of the working principle of the cloud video monitoring platform based on the billion-level pixel device described in FIG. 3 .
  • the multiple optical fiber transmission channels include plane data transmission channels and depth point cloud data transmission channels;
  • the optical fiber network device forwards the light field image data to the cloud server through the data distribution module, specifically including:
  • the optical fiber network device identifies plane data and depth point cloud data in the image data through the data distribution module;
  • the plane data is forwarded to the cloud server through the plane data transmission channel
  • the depth point cloud data is forwarded to the cloud server through the depth point cloud data transmission channel at the same time.
  • the cloud server performs synchronous processing on the light field image data through the multiple general-purpose processors and multiple graphics processors, and outputs the result of the synchronous processing to a visual interface, specifically including:
  • the cloud server executes two-dimensional picture visual reconstruction on the received plane data through the plurality of general-purpose processors, and sends the result of the two-dimensional picture visual reconstruction to the graphics processor;
  • the graphics processor performs depth map reconstruction on the received depth point cloud data based on the result of visual reconstruction of the two-dimensional picture.
  • the transmission rate and throughput of the planar data transmission channel are greater than that of the depth point cloud data transmission channel.
  • the above-mentioned technical solutions of the present invention can be realized automatically based on computer program instructions through computer equipment.
  • the present invention can also be expressed as a computer program product, the program product is carried on a computer storage medium, and the program is executed by a processor, thereby realizing the above technical solution.
  • more embodiments include a computer device, the computer device includes a memory and a processor, the memory stores a computer-executable program, and the processor is configured to perform the following steps:
  • C1 Obtain image data collected by multiple billion-level pixel devices
  • step C1 includes:
  • Each light field camera array collects light field image data of hundreds of millions of pixels in the monitored area according to the preset frequency
  • C2 identifying plane data and depth point cloud data in the billion-level pixel light field image data
  • C3 Forward the plane data through the plane data transmission channel, and forward the depth point cloud data through the depth point cloud data transmission channel at the same time;
  • C4 performing visual reconstruction of the two-dimensional picture on the received plane data through the CPU process, and sending the result of the visual reconstruction of the two-dimensional picture to the GPU process;
  • the GPU process performs depth map reconstruction on the received depth point cloud data based on the result of visual reconstruction of the two-dimensional picture
  • C6 Synchronously display 2D image screen and 3D image screen
  • C7 Based on the picture parameters of the 2D image picture and/or the 3D image picture, adjust the number of CPU processes and GPU processes, and adjust the frequency of the CPU core corresponding to the CPU process and the GPU core corresponding to the GPU process.
  • the technical solution of the present invention is aimed at the large frame rate, high resolution and multi-channel characteristics of the light field image video data captured by the multi-array light field camera, especially the multi-array light field camera.
  • the cloud image server in order to ensure that the cloud image server can process the video data in a timely manner, the plane data and the depth point cloud data in the image data are first identified; /or the screen parameters of the 3D image screen, adjust the execution parameters of the cloud server, so as to ensure that the data rate and frame rate of the execution end and the transmission end can match, so as to achieve a better monitoring effect and avoid data delay, freeze or over-rendering.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本发明提出基于亿级像素设备的云端视频监控方法与监控平台，属于亿级像素视频处理技术领域。方法包括：获取多个亿级像素设备采集的图像数据（S100）；将图像数据通过光纤网络设备转发至云端服务器（S200）；云端服务器调节云端服务器的执行参数并执行视频监控（S300）。云端视频监控平台包括针对同一个监控区域环绕布置的多个光场相机阵列、云端服务器以及光纤网络设备。光纤网络设备包括数据分发模组。云端服务器通过多个通用处理器与多个图形处理器对光场图像数据进行同步处理，并将同步处理的结果输出至可视化界面。本发明考虑到光场图像数据的特点和亿级分辨率要求，采用不同数据传输通道进行数据传输确保视频监控的及时处理。

Description

基于亿级像素设备的云端视频监控方法与监控平台 技术领域
本发明属于亿级像素视频处理技术领域，尤其涉及一种基于亿级像素设备的云端视频监控方法与监控平台、实现所述方法的计算机终端设备以及存储介质。
背景技术
高分辨率(高像素)代表着高质量与高清晰度，高清的视频图像不仅在军事、医学、监控、天文等方面有着广泛的应用,而且也能给娱乐生活带来更舒适的视觉体验。在这样的时代背景下，提高用户的视频体验尤为重要。视频图像的分辨率需要不断提升，由4K、8K再到亿级像素、十亿级像素摄像机的发展已成主流趋势。
然而，当成像分辨率超过亿级时，产生的数据量也剧增；而采用光场相机获取图像数据时，光场相机不仅可以采集到图像信息(平面的、二维图像信息)，还可以类似于激光雷达一样，通过采集的一帧图像生成这张图片对应的深度图信息，以及这张深度图对应的点云信息；光场相机还能同时获取成像时光线的空间信息和角度信息，使得数据的维度也相应增大。在数据维度和数据量同时增大的情况下，如何快速并且及时处理亿级像素下的监控场景数据，成为本领域技术人员亟需处理的技术问题。
同时，在实际应用中，远程光场阵列数据还需要传输至云端进行处理后在进行渲染展示，图形处理器与通用处理器构成的异构系统在处理高通量、高维度的图像数据时需要互相协调彼此的工作状态，当其应用于基于高通量光场相机阵列产生的亿级像素视频处理时，还会导致执行端与传送端的 数据速率、帧率不匹配，产生数据延迟、卡顿或者过度渲染，使得云端监控效果不佳。
发明内容
为解决上述技术问题，本发明提出一种基于亿级像素设备的云端视频监控方法与监控平台、实现所述方法的计算机终端设备以及存储介质。
在本发明的第一个方面，提出一种基于亿级像素设备的云端视频监控方法，所述方法包括如下步骤：
S100：获取多个亿级像素设备采集的图像数据；
S200：将所述图像数据通过光纤网络设备转发至云端服务器；
S300：所述云端服务器基于接收到的转发数据，调节云端服务器的执行参数并执行视频监控。
作为本发明的优点之一，所述转发数据包括2D图像数据和3D图像数据；
所述云端服务器连接多个通用处理器与多个图形处理器，并将所述视频监控的结果输出至可视化界面；
所述步骤S300具体包括：
所述云端服务器基于接收到的转发数据的格式和数量，在所述可视化界面上同步显示2D图像画面和3D图像画面，并基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数。
具体的，所述云端服务器的执行参数包括调用的通用处理器和/或图形处理器的数量，以及调用的通用处理器和/或图形处理器的频率。
作为进一步的优选，所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道；
所述步骤S200具体包括：
通过所述数据分发模组识别所述图像数据中的平面数据和深度点云数据；
将所述平面数据和所述深度点云数据通过对应的光纤传输通道转发至云端服务器。
所述2D图像画面和/或3D图像画面的画面参数包括画面显示帧率；
所述云端服务器的执行参数还包括反馈控制参数，所述反馈控制参数用于控制所述亿级像素设备的图像采集频率；
所述基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，包括：
当所述画面参数不满足预定条件时，生成所述反馈控制参数并且提高调用的通用处理器和/或图形处理器的频率。
在本发明的第二个方面，为实现第一个方面所述的方法，提供一种基于亿级像素设备的云端视频监控平台，所述云端视频监控平台包括多个光场相机阵列，所述多个光场相机阵列针对同一个监控区域环绕布置。
具体的，所述云端视频监控平台还包括：
云端服务器，所述云端服务器连接多个通用处理器与多个图形处理器；
光纤网络设备，所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道，所述多个光纤传输通道包括平面数据传输通道和深度点云数据传输通道；
每个所述光场相机阵列按照预设频率采集得到所述监控区域的光场图像数据后，发送至所述光纤网络设备；
所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至 所述云端服务器；
所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面。
作为进一步的改进，所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至所述云端服务器，具体包括：
所述光纤网络设备通过所述数据分发模组识别所述图像数据中的平面数据和深度点云数据；
将所述平面数据通过平面数据传输通道转发至云端服务器，同时将所述深度点云数据通过深度点云数据传输通道转发至云端服务器。
所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面，具体包括：
所述云端服务器通过所述多个通用处理器对接收到的平面数据执行二维画面可视化重建，并将所述二维画面可视化重建的结果发送给所述图形处理器；
所述图形处理器基于所述二维画面可视化重建的结果对接收到的深度点云数据进行深度图重建。
在上述两个方面的技术方案中，所述亿级像素设备为光场相机阵列，所述图像数据为所述光场相机阵列采集的包含平面图像数据和深度点云数据的光场图像数据。
所述平面数据传输通道的传输速率和传输通量均大于所述深度点云数据传输通道。
本发明的上述技术方案可以通过计算机设备，基于计算机程序指令自动 化实现。
因此，在本发明的第三个方面，本发明可以实现为一种计算机介质，计算机介质上存储有计算机程序指令，通过执行所述程序指令，实现第一个方面所述的一种基于亿级像素设备的云端视频监控方法。
同样的，在本发明的第四个方面，本发明还可以表现为一种计算机程序产品，所述程序产品承载于计算机存储介质，通过处理器执行所述程序，从而实现上述基于亿级像素设备的云端视频监控方法的全部或者部分步骤。
本发明针对亿级像素设备的光场图像视频数据大帧率、高分辨率以及多通道的特点，在进行云端图像服务器处理时，为了确保云端图像服务器对于视频数据的处理能够及时，先识别所述图像数据中的平面数据和深度点云数据；然后将所述平面数据和所述深度点云数据通过对应的光纤传输通道转发至云端图像服务器，并在可视化界面上同步显示2D图像画面和3D图像画面，并基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，从而确保执行端与传送端的数据速率、帧率均能够匹配，从而达到较好的监控效果，避免数据延迟、卡顿或者过度渲染。
本发明的进一步优点将结合说明书附图在具体实施例部分进一步详细体现。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的 前提下，还可以根据这些附图获得其他的附图。
图1是本发明一个实施例的一种基于亿级像素设备的云端视频监控方法的主体流程示意图；
图2是图1所述方法的进一步具体实施例示意图；
图3是实现图1或图2所述方法的基于亿级像素设备的云端视频监控平台的架构图；
图4是图3所述基于亿级像素设备的云端视频监控平台的部分工作原理示意图；
图5是本发明一个实施例的用于实现云端视频监控方法的计算机电子设备的结构与原理示意图
具体实施方式
下面，结合附图以及具体实施方式，对发明做出进一步的描述。
在介绍本发明的各个实施例之前，先介绍与本申请有关的部分现有技术。
光场相机(Light-field camera)，也称为全光相机(Plenoptic camera)。光场相机不仅可以采集到图像信息(平面的、二维图像信息)，还可以通过采集的一帧图像生成这张图片对应的深度图信息，以及这张深度图对应的点云信息；光场相机还能同时获取成像时光线的空间信息和角度信息，将二维图像中的像素按照一定规则映射为多维(大于2维，例如三维或者四维)光场进行重新投影，得到不同视角和不同相平面的对焦图像。
深度图信息，用于在图像的每一个像素值中，表示场景中某点与摄像机(视点)的距离。
在上述基础上，接下来介绍本发明的各个实施例。
首先，参见图1。图1是本发明一个实施例的一种基于亿级像素设备的云 端视频监控方法的主要步骤图，所述步骤包括S100-S300，各个步骤具体实现如下：
S100：获取多个亿级像素设备采集的图像数据；
S200：将所述图像数据通过光纤网络设备转发至云端服务器；
S300：所述云端服务器基于接收到的转发数据，调节云端服务器的执行参数并执行视频监控。
作为本发明更具体的实现场景的例子，所述多个亿级像素设备为针对同一个监控区域环绕布置的多个光场相机阵列，所述图像数据为所述光场相机阵列采集的包含平面图像数据和深度点云数据的光场图像数据。
所述转发数据包括2D图像数据和3D图像数据；
作为一个示意性的例子，2D图像数据可以是所述平面图像数据，3D图像数据可以是所述深度点云数据。
然而，可以理解的是，光场相机不仅可以采集到图像信息(平面的、二维图像信息)，还可以通过采集的一帧图像生成这张图片对应的深度图信息，以及这张深度图对应的点云信息；光场相机还能同时获取成像时光线的空间信息和角度信息，将二维图像中的像素按照一定规则映射为多维(大于2维，例如三维或者四维)光场进行重新投影，得到不同视角和不同相平面的对焦图像。
因此，本领域技术人员还可以从光场相机图像数据中获得其他类型的二维(2D)平面图像数据，以及与相应的2D平面图像数据对应关联的3D图像数据，对应关联意味着二者在时序同步融合后可以重建出当时的目标区域图像。
为实现所述方法的硬件结构包括多个亿级像素设备、光纤网络设备以及云端服务器。所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道；
所述云端服务器连接多个通用处理器与多个图形处理器，并将所述视频监控的结果输出至可视化界面。
可以理解，通用处理器包括CPU，图形处理器包括GPU；前者可执行任务调度以及简单的(二维)数据处理，后者专用于执行接收前者分配的3D图像计算任务。
图2在图1基础上进一步展现了各个子步骤的具体实现形式。
所述步骤S200具体包括：
通过所述数据分发模组识别所述图像数据中的平面数据和深度点云数据；
所述数据分发模组连接多个光纤传输通道，所述多个光纤传输通道包括平面数据传输通道和深度点云数据传输通道；
将所述平面数据和所述深度点云数据通过对应的光纤传输通道转发至云端服务器。
具体的，将所述平面数据通过平面数据传输通道转发至云端服务器，同时将所述深度点云数据通过深度点云数据传输通道转发至云端服务器。
所述步骤S300具体包括：
所述云端服务器基于接收到的转发数据的格式和数量，在所述可视化界面上同步显示2D图像画面和3D图像画面，并基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数。
所述云端服务器的执行参数包括调用的通用处理器和/或图形处理器的数量，以及调用的通用处理器和/或图形处理器的频率。
所述2D图像画面和/或3D图像画面的画面参数包括画面显示帧率；
在一个实施例中，所述云端服务器的执行参数还包括反馈控制参数，所述反馈控制参数用于控制所述亿级像素设备的图像采集频率；
所述基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，包括：
当所述画面参数不满足预定条件时，生成所述反馈控制参数。
在另一个实施例中，所述基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，包括：
当所述画面参数不满足预定条件时，提高调用的通用处理器和/或图形处理器的频率。
在图1-图2基础上，参见图3。图3是实现图1或图2所述方法的基于亿级像素设备的云端视频监控平台的架构图。
在图3中，所述云端视频监控平台包括多个光场相机阵列，所述多个光场相机阵列针对同一个监控区域环绕布置。
所述云端视频监控平台还包括云端服务器与光纤网络设备。
所述云端服务器连接多个通用处理器与多个图形处理器；
所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道，
每个所述光场相机阵列按照预设频率采集得到所述监控区域的光场图像数据后，发送至所述光纤网络设备；
所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至所述云端服务器；
所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面。
图4进一步展现图3所述基于亿级像素设备的云端视频监控平台的部分工作原理示意图。
在图4中，所述多个光纤传输通道包括平面数据传输通道和深度点云数据传输通道；
所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至所述云端服务器，具体包括：
所述光纤网络设备通过所述数据分发模组识别所述图像数据中的平面数据和深度点云数据；
将所述平面数据通过平面数据传输通道转发至云端服务器，同时将所述深度点云数据通过深度点云数据传输通道转发至云端服务器。
所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面，具体包括：
所述云端服务器通过所述多个通用处理器对接收到的平面数据执行二维画面可视化重建，并将所述二维画面可视化重建的结果发送给所述图形处理器；
所述图形处理器基于所述二维画面可视化重建的结果对接收到的深度点云数据进行深度图重建。
在上述各个实施例中，所述平面数据传输通道的传输速率和传输通量均大于所述深度点云数据传输通道。
本发明的上述技术方案可以通过计算机设备，基于计算机程序指令自动化实现。同样的，本发明还可以表现为一种计算机程序产品，所述程序产品承载于计算机存储介质，通过处理器执行所述程序，从而实现上述技术方案。
具体的，参见图5，更多的实施例包括一种计算机设备，所述计算机设备包括存储器和处理器，所述存储器存储有计算机可执行程序，所述处理器被配置为执行如下步骤：
C1：获取多个亿级像素设备采集的图像数据；
更具体的，所述步骤C1包括：
每个光场相机阵列按照预设频率采集得到监控区域的亿级像素光场图像数据；
C2：识别所述亿级像素光场图像数据中的平面数据和深度点云数据；
C3：通过平面数据传输通道转发所述平面数据，同时通过深度点云数据传输通道转发所述深度点云数据；
C4：通过CPU进程对接收到的平面数据执行二维画面可视化重建，并将所述二维画面可视化重建的结果发送给GPU进程；
C5：GPU进程基于所述二维画面可视化重建的结果对接收到的深度点云数据进行深度图重建；
C6：同步显示2D图像画面和3D图像画面；
C7：基于2D图像画面和/或3D图像画面的画面参数，调节CPU进程和GPU进程的数量，同时调节CPU进程对应的CPU核以及GPU进程对应的GPU核的频率。
本发明的技术方案，针对亿级像素设备，尤其是多阵列光场相机拍摄得到的光场图像视频数据大帧率、高分辨率以及多通道的特点，在进行云端图像服务器处理时，为了确保云端图像服务器对于视频数据的处理能够及时，先识别所述图像数据中的平面数据和深度点云数据；然后将所述平面数据和所述深度点云数据通过对应的光纤传输通道转发至云端图像服务器，并在可视化界面上同步显示2D图像画面和3D图像画面，并基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，从而确保执行端与传送端的数据速率、帧率均能够匹配，从而达到较好的监 控效果，避免数据延迟、卡顿或者过度渲染。
最后应当说明的是：以上实施例仅用以说明本发明的技术方案而非对其限制，尽管参照上述实施例对本发明进行了详细的说明，所属领域的普通技术人员应当理解：依然可以对本发明的具体实施方式进行修改或者等同替换，而未脱离本发明精神和范围的任何修改或者等同替换，其均应涵盖在本发明的权利要求保护范围之内。
本发明未特别明确的部分模块结构，以现有技术记载的内容为准。本发明在前述背景技术部分提及的现有技术可作为本发明的一部分，用于理解部分技术特征或者参数的含义。本发明的保护范围以权利要求实际记载的内容为准。

Claims (9)

  1. 一种基于亿级像素设备的云端视频监控方法，所述方法包括：
    S100：获取多个亿级像素设备采集的图像数据；
    S200：将所述图像数据通过光纤网络设备转发至云端服务器；
    S300：所述云端服务器基于接收到的转发数据，调节云端服务器的执行参数并执行视频监控；
    其特征在于：
    所述转发数据包括2D图像数据和3D图像数据；
    所述云端服务器连接多个通用处理器与多个图形处理器，并将所述视频监控的结果输出至可视化界面；
    所述步骤S300具体包括：
    所述云端服务器基于接收到的转发数据的格式和数量，在所述可视化界面上同步显示2D图像画面和3D图像画面，并基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数。
  2. 如权利要求1所述的一种基于亿级像素设备的云端视频监控方法，其特征在于：
    所述云端服务器的执行参数包括调用的通用处理器和/或图形处理器的数量，以及调用的通用处理器和/或图形处理器的频率。
  3. 如权利要求1所述的一种基于亿级像素设备的云端视频监控方法，其特征在于：
    所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道，所述多个光纤传输通道包括平面数据传输通道和深度点云数据传输通道；
    所述步骤S200具体包括：
    通过所述数据分发模组识别所述图像数据中的平面数据和深度点云数据；
    将所述平面数据通过平面数据传输通道转发至云端服务器，同时将所述深度点云数据通过深度点云数据传输通道转发至云端服务器。
  4. 如权利要求1-3任一项所述的一种基于亿级像素设备的云端视频监控方法，其特征在于：
    所述亿级像素设备为光场相机阵列，所述图像数据为所述光场相机阵列采集的包含平面数据和深度点云数据的光场图像数据。
  5. 如权利要求1所述的一种基于亿级像素设备的云端视频监控方法，其特征在于：
    所述2D图像画面和/或3D图像画面的画面参数包括画面显示帧率；
    所述云端服务器的执行参数还包括反馈控制参数，所述反馈控制参数用于控制所述亿级像素设备的图像采集频率；
    所述基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，包括：
    当所述画面参数不满足预定条件时，生成所述反馈控制参数。
  6. 如权利要求2所述的一种基于亿级像素设备的云端视频监控方法，其特征在于：
    所述2D图像画面和/或3D图像画面的画面参数包括画面显示帧率；
    所述基于2D图像画面和/或3D图像画面的画面参数，调节云端服务器的执行参数，包括：
    当所述画面参数不满足预定条件时，提高调用的通用处理器和/或图形处理器的频率。
  7. 一种基于亿级像素设备的云端视频监控平台，所述云端视频监控平台包括多个光场相机阵列，所述多个光场相机阵列针对同一个监控区域环绕布置，其特征在于，所述云端视频监控平台还包括：
    云端服务器，所述云端服务器连接多个通用处理器与多个图形处理器；
    光纤网络设备，所述光纤网络设备包括数据分发模组，所述数据分发模组连接多个光纤传输通道，所述多个光纤传输通道包括平面数据传输通道和深度点云数据传输通道；
    每个所述光场相机阵列按照预设频率采集得到所述监控区域的光场图像数据后，发送至所述光纤网络设备；
    所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至所述云端服务器；
    所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面；
    所述光纤网络设备通过所述数据分发模组将所述光场图像数据转发至所述云端 服务器，具体包括：
    所述光纤网络设备通过所述数据分发模组识别所述光场图像数据中的平面数据和深度点云数据；
    将所述平面数据通过平面数据传输通道转发至云端服务器，同时将所述深度点云数据通过深度点云数据传输通道转发至云端服务器。
  8. 如权利要求7所述的一种基于亿级像素设备的云端视频监控平台，其特征在于：
    所述云端服务器通过所述多个通用处理器与多个图形处理器对所述光场图像数据进行同步处理，并将所述同步处理的结果输出至可视化界面，具体包括：
    所述云端服务器通过所述多个通用处理器对接收到的平面数据执行二维画面可视化重建，并将所述二维画面可视化重建的结果发送给所述图形处理器；
    所述图形处理器基于所述二维画面可视化重建的结果对接收到的深度点云数据进行深度图重建。
  9. 如权利要求7所述的一种基于亿级像素设备的云端视频监控平台，其特征在于：
    所述平面数据传输通道的传输速率和传输通量均大于所述深度点云数据传输通道。
PCT/CN2022/141922 2022-01-18 2022-12-26 基于亿级像素设备的云端视频监控方法与监控平台 WO2023138311A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210052910.9 2022-01-18
CN202210052910.9A CN114363530B (zh) 2022-01-18 2022-01-18 基于亿级像素设备的云端视频监控方法与监控平台

Publications (1)

Publication Number Publication Date
WO2023138311A1 true WO2023138311A1 (zh) 2023-07-27

Family

ID=81091657

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/141922 WO2023138311A1 (zh) 2022-01-18 2022-12-26 基于亿级像素设备的云端视频监控方法与监控平台

Country Status (2)

Country Link
CN (1) CN114363530B (zh)
WO (1) WO2023138311A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363530B (zh) * 2022-01-18 2022-08-30 北京拙河科技有限公司 基于亿级像素设备的云端视频监控方法与监控平台

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190005605A1 (en) * 2007-11-23 2019-01-03 PME IP Pty Ltd Multi-user multi-gpu render server apparatus and methods
CN111614975A (zh) * 2020-05-08 2020-09-01 北京拙河科技有限公司 一种亿级像素视频播放方法、装置、介质及设备
US20210321081A1 (en) * 2020-04-09 2021-10-14 Looking Glass Factory, Inc. System and method for generating light field images
CN114363530A (zh) * 2022-01-18 2022-04-15 北京拙河科技有限公司 基于亿级像素设备的云端视频监控方法与监控平台

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9715761B2 (en) * 2013-07-08 2017-07-25 Vangogh Imaging, Inc. Real-time 3D computer vision processing engine for object recognition, reconstruction, and analysis
US9420238B2 (en) * 2014-04-10 2016-08-16 Smartvue Corporation Systems and methods for automated cloud-based 3-dimensional (3D) analytics for surveillance systems
EP3301926A1 (en) * 2016-09-30 2018-04-04 Thomson Licensing A method and a device for reconstructing a point cloud representative of a scene using light-field data
CN111954896A (zh) * 2018-04-12 2020-11-17 凸版印刷株式会社 光场图像生成系统、图像显示系统、形状信息取得服务器、图像生成服务器、显示装置、光场图像生成方法及图像显示方法
CN113192182A (zh) * 2021-04-29 2021-07-30 山东产研信息与人工智能融合研究院有限公司 一种基于多传感器的实景重建方法及系统
CN113891072B (zh) * 2021-12-08 2022-02-11 北京拙河科技有限公司 基于亿级像素数据的视频监测与异常分析系统与方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190005605A1 (en) * 2007-11-23 2019-01-03 PME IP Pty Ltd Multi-user multi-gpu render server apparatus and methods
US20210321081A1 (en) * 2020-04-09 2021-10-14 Looking Glass Factory, Inc. System and method for generating light field images
CN111614975A (zh) * 2020-05-08 2020-09-01 北京拙河科技有限公司 一种亿级像素视频播放方法、装置、介质及设备
CN114363530A (zh) * 2022-01-18 2022-04-15 北京拙河科技有限公司 基于亿级像素设备的云端视频监控方法与监控平台

Also Published As

Publication number Publication date
CN114363530A (zh) 2022-04-15
CN114363530B (zh) 2022-08-30

Similar Documents

Publication Publication Date Title
US10506223B2 (en) Method, apparatus, and device for realizing virtual stereoscopic scene
US8723951B2 (en) Interactive wide-angle video server
JP2005339313A (ja) 画像提示方法及び装置
CN113301439A (zh) 视频数据的基于内容的流分割
US10250802B2 (en) Apparatus and method for processing wide viewing angle image
JP7101269B2 (ja) ポーズ補正
WO2023138311A1 (zh) 基于亿级像素设备的云端视频监控方法与监控平台
CN109040601B (zh) 一种多尺度非结构化的十亿像素vr全景摄影系统
CN110675506A (zh) 实现多路视频融合的三维增强现实的系统、方法及设备
US20140362099A1 (en) Image processing apparatus and image processing method
US9466148B2 (en) Systems and methods to dynamically adjust an image on a display monitor represented in a video feed
US11783445B2 (en) Image processing method, device and apparatus, image fitting method and device, display method and apparatus, and computer readable medium
CN113873264A (zh) 显示图像的方法、装置、电子设备及存储介质
WO2007060497A2 (en) Interactive wide-angle video server
CN113674354B (zh) 一种三维重建方法及系统
DE102019215387A1 (de) Zirkularfischaugenkameraarrayberichtigung
WO2023138217A1 (zh) 基于gpu融合处理的超清画质图像数据处理方法与装置
CN115002345B (zh) 一种图像校正方法、装置、电子设备及存储介质
CN114170567A (zh) 监测区域亿级像素的光场相机ai分析系统及其方法
US9325963B2 (en) Device and method for rendering and delivering 3-D content
CN113099206A (zh) 图像处理方法、装置、设备及存储介质
CN111277797A (zh) 一种用于安防监视的vr立体成像系统
Hu et al. LiveVV: Human-Centered Live Volumetric Video Streaming System
US20230291865A1 (en) Image processing apparatus, image processing method, and storage medium
WO2022244131A1 (ja) 画像データ生成装置、表示装置、画像表示システム、画像データ生成方法、画像表示方法、および、画像データのデータ構造

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22921732

Country of ref document: EP

Kind code of ref document: A1