WO2023029567A1 - 一种传感器采集的多种数据的可视化方法及其系统 - Google Patents
一种传感器采集的多种数据的可视化方法及其系统 Download PDFInfo
- Publication number
- WO2023029567A1 WO2023029567A1 PCT/CN2022/092130 CN2022092130W WO2023029567A1 WO 2023029567 A1 WO2023029567 A1 WO 2023029567A1 CN 2022092130 W CN2022092130 W CN 2022092130W WO 2023029567 A1 WO2023029567 A1 WO 2023029567A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- metadata
- code stream
- full
- motion video
- image
- Prior art date
Links
- 238000007794 visualization technique Methods 0.000 title claims abstract description 18
- 230000006835 compression Effects 0.000 claims abstract description 34
- 238000007906 compression Methods 0.000 claims abstract description 34
- 238000004891 communication Methods 0.000 claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 19
- 238000009877 rendering Methods 0.000 claims abstract description 9
- 230000005540 biological transmission Effects 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 7
- 238000012800 visualization Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 abstract description 6
- 230000000007 visual effect Effects 0.000 abstract description 6
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000013079 data visualisation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
- H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/426—Internal components of the client ; Characteristics thereof
- H04N21/42607—Internal components of the client ; Characteristics thereof for processing the incoming bitstream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4348—Demultiplexing of additional data and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N21/4355—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reformatting operations of additional data, e.g. HTML pages on a television screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8126—Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
Definitions
- the invention relates to the technical field of wireless communication, in particular to a visualization method and system for various data collected by sensors.
- the back-end data processing unit can realize more application scenarios based on the data collected by sensors. But at the same time, the back-end data processing unit has increasingly high requirements for sensor data transmission, such as: the need to ensure data reliability during data transmission, the need to ensure the diversity of data types and synchronization between different types of data during data transmission sex etc.
- the present invention provides a visualization method and system for various data collected by sensors, which solves the problem of less visualization content and effective information in the traditional visualization methods based on data collected by sensors.
- the technical solution of the present invention is to adopt a visualization method of various data collected by a sensor, including: obtaining image compression code stream and metadata; encapsulating the metadata into the image frame by frame in a time-synchronized manner Compress the preset field of the code stream, and generate a full-motion video code stream containing the metadata; transmit the full-motion video code stream to the receiving end through a communication link; analyze the full-motion video code stream and Render to full motion video including metadata.
- parsing and rendering the full-motion video code stream into a full-motion video containing metadata includes: parsing the full-motion video code stream frame by frame into image frames and the time-synchronized metadata, Wherein, when each frame of the image frame is analyzed, by analyzing the element information contained in the metadata that is time-synchronized with the image frame, after the image frame is rendered, the element information is superimposed on the and presenting in the image on the display interface; repeating the above-mentioned steps of generating the image including element information to generate the full-motion video composed of the image.
- parsing the full-motion video stream frame by frame into image frames and the time-synchronized metadata includes: parsing the full-motion video stream into multiple frames containing the An image frame of metadata; extracting the metadata contained in the image frame based on the frame start field, and obtaining the full-motion video stream composed of the image frame or the compressed image without the metadata stream.
- the method for visualizing various data further includes: when the sending end transmits the full-motion video code stream to the receiving end through a communication link, if the communication link includes at least one transmission node , the transmission node is capable of parsing the full-motion video stream into image frames and the time-synchronized metadata; after modifying the metadata, the modified metadata is time-synchronized Encapsulate into the preset field of the image frame, and generate a full-motion video code stream containing the modified metadata; continue to transmit the full-motion video code stream containing the modified metadata through a communication link to the receiving end.
- superimposing the element information on the image and presenting it on a display interface includes: directly rendering the element information by parsing the element information contained in the metadata that is time-synchronized with the image frame loading to the display interface; or when parsing out the element information, calculate part of the element information, render and load the processed element information to the display interface.
- acquiring the image compression code stream and metadata includes: outputting, based on the same reference clock circuit, a first control signal for collecting the image compression code stream and a first control signal for collecting the metadata with the same reference clock. second control signal; acquiring the image compression code stream and the metadata including absolute time based on the first control signal and the second control signal.
- the metadata includes at least GNSS positioning data.
- the preset field is an SEI field.
- the present invention provides a visualization system for various data collected by a sensor, including: a sending end, configured to, after obtaining image compression code stream and metadata, encapsulate the metadata frame by frame in a time-synchronized manner In the preset field of the image compression code stream, and after generating the full dynamic video code stream containing the metadata, the full dynamic video code stream is transmitted to the receiving end through the communication link; the receiving end is used to The full-motion video code stream is parsed and rendered into a full-motion video containing metadata.
- the transmission node can parse the full-motion video stream into image frames and the metadata of time synchronization thereof; for the metadata After the modification, the modified metadata is encapsulated into the preset field of the image frame in a time-synchronized manner, and a full-motion video stream containing the modified metadata is generated; the modified The full-motion video code stream of the metadata continues to be transmitted to the receiving end through the communication link.
- the sending end includes: a data collection unit, configured to collect the metadata and the image compression code stream; and a data processing unit, configured to generate a full-motion video code stream including the metadata.
- the primary improvement of the present invention is to provide a visualization method for various data collected by the sensor, by encapsulating the metadata frame by frame into the preset field of the image compression code stream in a time-synchronized manner, and generating the
- the full-motion video stream of metadata enables the receiving end to parse the full-motion video stream into image frames and their time-synchronized element information, and render the image frames and their time-synchronized element information on the display interface in real time , so as to realize the diversity of visual content, and ensure the validity of element information due to the synchronization of image frames and metadata.
- Fig. 1 is the simplified flow chart of the visualization method of the various data collected by the sensor of the present invention
- Fig. 2 is a simplified unit connection diagram of the visualization system of various data collected by sensors of the present invention.
- a visualization method for various data collected by a sensor including: obtaining image compression code stream and metadata; encapsulating the metadata into the preset image compression code stream frame by frame in a time-synchronized manner field, and generate a full-motion video code stream containing the metadata; transmit the full-motion video code stream to the receiving end through a communication link; parse and render the full-motion video code stream into a file containing the metadata Full motion video.
- the preset field can be: when the communication transmission protocol used is H264 or H265, the preset field can be SEI (Supplemental Enhancement Information, supplementary enhancement information) field; the communication transmission protocol used is TS (MPEG2 (Moving Picture Experts Group, moving image expert group) Transport stream, transport stream) encapsulation protocol, the default field is a custom field; the type of metadata information changes according to the type of device equipped with sensors, for example: when the device is a ship, Metadata can include device status data, and device status data includes at least GNSS (Global Navigation Satellite System, Global Navigation Satellite System) data, wind direction data, and heading data; when the device is an aircraft, the metadata includes at least the aircraft POS (Position and Orientation System, positioning and orientation system) data, aircraft state data, load sensor type data, pod POS data, pod state data and image processing board data, etc.
- GNSS Global Navigation Satellite System
- GPS Global Navigation Satellite System
- the metadata includes at least the aircraft POS (Position and Orientation System,
- the metadata information can include positioning, viewing angle Direction, pitch angle, field of view, tower height, channel, transmission bandwidth, device ID (Identity document, identification number) and other information.
- the POS data of the carrier aircraft at least includes the data of the yaw angle of the carrier aircraft, the data of the pitch angle of the carrier aircraft, the data of the roll angle of the carrier aircraft, the latitude and longitude data of the carrier aircraft, the height data of the carrier aircraft, the distance data of the carrier aircraft compared with the starting point, Compared with the azimuth data of the starting point and the flight speed data of the carrier aircraft.
- Pod POS data at least include visible light horizontal field of view data, visible light vertical field of view data, infrared horizontal field of view data, infrared vertical field of view data, camera focal length data, pod heading Euler angle data, pod pitch O Pull angle data, heading frame angle data, pitch frame angle data, roll frame angle data, target longitude, latitude and height data, target speed data, target speed azimuth data and estimated distance data of the target compared to the carrier aircraft.
- parsing and rendering the full-motion video code stream into a full-motion video containing metadata includes: parsing the full-motion video code stream frame by frame into image frames and the time-synchronized metadata, While parsing a frame of the image frame, by parsing the element information contained in the metadata that is time-synchronized with the image frame, after rendering the image frame, superimposing the element information on the image and presenting In the display interface; repeating the above-mentioned steps of generating the image including element information to generate the full-motion video composed of the image.
- element information is defined as the data rendered on the display interface in the form of text parameters, images, etc., for users to know information such as equipment status and task status; generating element information based on metadata may include: GNSS data extraction vehicle based Coordinate element information; when the type of the vehicle equipped with the sensor is a UAV, based on the GNSS data and the field of view data, the coordinates of the UAV’s field of view center coordinates and other types of geographical elements are calculated, and based on the UAV’s field of view Data and pitch and yaw angles to calculate the specific field of view area of the video screen on the map, etc.
- the method may further include: when rendering the image frame into an image and presenting it on a display interface, superimposing the element information on an area of the display interface that does not belong to the image.
- the present invention uses the data transmission method of synchronously encapsulating metadata into the image compression code stream, so that the receiving end can re-extract pure image frames and full-motion video from the full-motion video code stream without destroying the compressed code stream.
- the metadata synchronized with the image frame is extracted from the preset field of the code stream, which ensures the synchronization of the two data and the purity of each other, so that the receiving end can not only realize the application of real-time data visualization based on the full-motion video It can also implement multiple application scenarios such as image mosaic service based on two types of data that are synchronized with each other, and load status analysis based only on metadata.
- the metadata and the image frame are synchronized by using the data transmission method that synchronously encapsulates the metadata in the image compression code stream. Therefore, the receiving end analyzes And the image rendered to the display interface is completely synchronized with the element information, ensuring that the element information can effectively represent various information of the image.
- superimposing the element information on the image and presenting it on the display interface includes: directly rendering the element information by analyzing the element information contained in the metadata that is time-synchronized with the image frame loading to the display interface; or when parsing out the element information, calculate part of the element information, render and load the processed element information to the display interface.
- the element information that can be directly rendered and loaded to the display interface is element information that can be directly obtained from metadata, for example: element information that can directly extract the coordinates of the vehicle based on the GNSS data in the metadata; Part of the element information loaded by rendering is element information that needs to be calculated by at least two kinds of metadata.
- the UAV center coordinates of the field of view that can be calculated based on GNSS data and field of view data need to be based on the UAV
- the field of view data and the pitch and yaw angle data can only calculate the specific field of view area of the video screen on the map.
- parsing the full-motion video stream frame by frame into image frames and the time-synchronized metadata includes: parsing the full-motion video stream into multiple frames containing the An image frame of metadata; extracting the metadata contained in the image frame based on the frame start field, and obtaining the full-motion video stream composed of the image frame or the compressed image without the metadata stream.
- the method for visualizing various data further includes: when the sending end transmits the full-motion video code stream to the receiving end through a communication link, if the communication link includes at least one transmission node, The transmission node is capable of parsing the full-motion video stream into image frames and the time-synchronized metadata; after modifying the metadata, encapsulating the modified metadata in a time-synchronized manner into the preset field of the image frame, and generate a full-motion video code stream containing the modified metadata; continue to transmit the full-motion video code stream containing the modified metadata to the the receiving end.
- the present invention uses a data transmission method that synchronously encapsulates metadata into an image compression code stream, so that when there is a transmission node in the communication link, the transmission node can re-start from the full-motion video code stream without destroying the compressed code stream.
- Extract the pure image frame and the metadata synchronized with the image frame from the preset field of the full-motion video code stream so that the transmission node can not only realize various application scenarios based on the image frame and metadata, but also through the
- the metadata and image frames are re-encapsulated and then transmitted to the receiving end, which ensures the modifiability and diversity of application scenarios when the full-motion video stream is transmitted in the communication link.
- acquiring the image compression code stream and metadata includes: based on the same reference clock circuit, respectively outputting a first control signal for collecting the image compression code stream and a first control signal for collecting the metadata with the same reference clock
- Two control signals acquiring the image compression code stream and the metadata including absolute time based on the first control signal and the second control signal.
- the present invention uses the reference clock signal output by the same reference clock circuit as the first control signal and the second control signal, so that the time stamps contained in the image compression code stream and metadata all refer to the same clock source, therefore, the image compression code stream Timestamps and metadata timestamps can be considered absolute time to each other within systems of the same clock source.
- the present invention does not specifically limit the method of generating the first control signal and the method of generating the second control signal.
- the way of generating the control signal is different.
- the payload processing subunit of the aircraft can output the first control signal and the second control signal respectively based on the same reference clock circuit after receiving the instruction from the ground station; After the instruction issued by the cloud, the first control signal and the second control signal are respectively output based on the same reference clock circuit. Therefore, the present invention only limits that the first control signal and the second control signal are generated based on the same reference clock circuit, and does not limit the specific generation method.
- the present invention encapsulates the metadata frame by frame into the preset field of the image compression code stream in a time-synchronized manner, and generates a full-motion video code stream containing the metadata, so that the receiving end can capture the full-motion
- the video stream is parsed into image frames and their time-synchronized element information, and the image frames and their time-synchronized element information are rendered on the display interface in real time, thereby realizing the diversity of visual content, and due to the synchronization of image frames and metadata To ensure the validity of element information.
- the present invention provides a visualization system for various data collected by sensors, including: a sending end, which is used to synchronize the metadata according to time after obtaining image compression code stream and metadata Encapsulate frame by frame into the preset field of the image compression code stream, and after generating the full dynamic video code stream containing the metadata, transmit the full dynamic video code stream to the receiving end through the communication link; the said The receiving end is configured to parse and render the full-motion video code stream into a full-motion video containing metadata.
- the transmission node can parse the full-motion video code stream into image frames and the metadata of time synchronization thereof; After the modification, the modified metadata is encapsulated into the preset field of the image frame in a time-synchronized manner, and a full-motion video stream containing the modified metadata is generated; the modified metadata will be included The full-motion video stream of metadata continues to be transmitted to the receiving end through the communication link.
- the sending end includes: a data collection unit, configured to collect the metadata and the image compression code stream; a data processing unit, configured to generate a full-motion video code stream including the metadata.
- RAM random access memory
- ROM read-only memory
- EEPROM electrically programmable ROM
- EEPROM electrically erasable programmable ROM
- registers hard disk, removable disk, CD-ROM, or any other Any other known storage medium.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
Claims (11)
- 一种传感器采集的多种数据的可视化方法,其特征在于,包括:获取图像压缩码流和元数据;将所述元数据按照时间同步的方式逐帧封装至图像压缩码流的预设字段中,并生成包含所述元数据的全动态视频码流;将所述全动态视频码流通过通信链路传输至接收端;将所述全动态视频码流解析并渲染为包含元数据的全动态视频。
- 根据权利要求1所述的多种数据的可视化方法,其特征在于,将所述全动态视频码流解析并渲染为包含元数据的全动态视频,包括:将所述全动态视频码流逐帧解析为图像帧及其时间同步的所述元数据,其中,在每解析出一帧所述图像帧的同时,通过解析所述元数据包含的与所述图像帧时间同步的要素信息,渲染所述图像帧后,将所述要素信息叠加至所述图像中并呈现于显示界面中;重复上述生成包含要素信息的所述图像的步骤,生成所述图像构成的所述全动态视频。
- 根据权利要求2所述的多种数据的可视化方法,其特征在于,将所述全动态视频码流解析为图像帧及其时间同步的所述元数据,包括:基于帧起始字段将所述全动态视频码流解析为多帧包含所述元数据的图像帧;基于帧起始字段提取所述图像帧包含的所述元数据,并获得所述图像帧构成的所述全动态视频码流或不包含所述元数据的所述图像压缩码流。
- 根据权利要求1所述的多种数据的可视化方法,其特征在于,所述多种数据的可视化方法还包括:在发送端将所述全动态视频码流通过通信链路传输至所述接收端后,若所述通信链路中包含至少一个传输节点,所述传输节点能够将所述全动态视频码流解析为图像帧及其时间同步的所述元数据;对所述元数据进行修改后,将修改后的所述元数据按照时间同步的方 式封装至所述图像帧的预设字段中,并生成包含修改后的所述元数据的全动态视频码流;将包含修改后的所述元数据的全动态视频码流继续通过通信链路传输至所述接收端。
- 根据权利要求2所述的多种数据的可视化方法,其特征在于,将所述要素信息叠加至所述图像中并呈现于显示界面中,包括:通过解析所述元数据包含的与所述图像帧时间同步的要素信息,直接将所述要素信息渲染加载至所述显示界面;或解析出所述要素信息时,对所述要素信息进行计算,将处理后的所述要素信息渲染加载至所述显示界面。
- 根据权利要求1所述的多种数据的可视化方法,其特征在于,获取图像压缩码流和元数据,包括:基于同一参考时钟电路分别输出具有同一参考时钟的用于采集所述图像压缩码流的第一控制信号和用于采集所述元数据的第二控制信号;基于所述第一控制信号和所述第二控制信号获取包含绝对时间的所述图像压缩码流和所述元数据。
- 根据权利要求2所述的多种数据的可视化方法,其特征在于,所述元数据至少包括GNSS定位数据。
- 根据权利要求1所述的多种数据的可视化方法,其特征在于,所述预设字段为SEI字段。
- 一种传感器采集的多种数据的可视化系统,其特征在于,包括:发送端,用于获取图像压缩码流和元数据后,将所述元数据按照时间同步的方式逐帧封装至图像压缩码流的预设字段中,并生成包含所述元数据的全动态视频码流后,将所述全动态视频码流通过通信链路传输至接收端;所述接收端,用于将所述全动态视频码流解析并渲染为包含元数据的全动态视频。
- 根据权利要求9所述的多种数据的可视化系统,其特征在于,所述通信链路包含至少一个传输节点的情况下,所述传输节点能够将所述全动态视频码流解析为图像帧及其时间同步的所述元数据;对所述元数据进行修改后,将修改后的所述元数据按照时间同步的方式封装至所述图像帧的预设字段中,并生成包含修改后的所述元数据的全动态视频码流;将包含修改后的所述元数据的全动态视频码流继续通过通信链路传输至所述接收端。
- 根据权利要求9所述的多种数据的可视化系统,其特征在于,所述发送端包括:数据采集单元,用于采集所述元数据和所述图像压缩码流;数据处理单元,用于生成包含所述元数据的全动态视频码流。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020247006129A KR20240051136A (ko) | 2021-08-30 | 2022-05-11 | 센서에 의해 수집되는 다양한 데이터의 가시화 방법 및 그 시스템 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111006671.5 | 2021-08-30 | ||
CN202111006671.5A CN115731633A (zh) | 2021-08-30 | 2021-08-30 | 一种传感器采集的多种数据的可视化方法及其系统 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023029567A1 true WO2023029567A1 (zh) | 2023-03-09 |
Family
ID=85291013
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/092130 WO2023029567A1 (zh) | 2021-08-30 | 2022-05-11 | 一种传感器采集的多种数据的可视化方法及其系统 |
Country Status (3)
Country | Link |
---|---|
KR (1) | KR20240051136A (zh) |
CN (1) | CN115731633A (zh) |
WO (1) | WO2023029567A1 (zh) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101309411A (zh) * | 2008-07-07 | 2008-11-19 | 浙江大学 | 一种利用深度信息的多通道视频流编码方法 |
CN104219492A (zh) * | 2013-11-14 | 2014-12-17 | 成都时代星光科技有限公司 | 无人机图像传输系统 |
US20150172726A1 (en) * | 2013-12-16 | 2015-06-18 | Samsung Electronics Co., Ltd. | Method for real-time implementation of super resolution |
CN106534857A (zh) * | 2016-11-02 | 2017-03-22 | 深圳市维海德技术股份有限公司 | 码流传输方法及基于usb的摄像机 |
CN110989840A (zh) * | 2019-12-03 | 2020-04-10 | 成都纵横自动化技术股份有限公司 | 数据处理方法、前端设备、后端设备及地理信息系统 |
CN111294648A (zh) * | 2020-02-20 | 2020-06-16 | 成都纵横自动化技术股份有限公司 | 一种无人机空地视频传输方法 |
CN111327860A (zh) * | 2020-01-21 | 2020-06-23 | 成都纵横自动化技术股份有限公司 | 一种图数同步传输方法和电子设备 |
CN111554080A (zh) * | 2020-07-13 | 2020-08-18 | 成都纵横自动化技术股份有限公司 | 一种光电吊舱数据采集系统 |
-
2021
- 2021-08-30 CN CN202111006671.5A patent/CN115731633A/zh active Pending
-
2022
- 2022-05-11 WO PCT/CN2022/092130 patent/WO2023029567A1/zh active Application Filing
- 2022-05-11 KR KR1020247006129A patent/KR20240051136A/ko active Search and Examination
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101309411A (zh) * | 2008-07-07 | 2008-11-19 | 浙江大学 | 一种利用深度信息的多通道视频流编码方法 |
CN104219492A (zh) * | 2013-11-14 | 2014-12-17 | 成都时代星光科技有限公司 | 无人机图像传输系统 |
US20150172726A1 (en) * | 2013-12-16 | 2015-06-18 | Samsung Electronics Co., Ltd. | Method for real-time implementation of super resolution |
CN106534857A (zh) * | 2016-11-02 | 2017-03-22 | 深圳市维海德技术股份有限公司 | 码流传输方法及基于usb的摄像机 |
CN110989840A (zh) * | 2019-12-03 | 2020-04-10 | 成都纵横自动化技术股份有限公司 | 数据处理方法、前端设备、后端设备及地理信息系统 |
CN111327860A (zh) * | 2020-01-21 | 2020-06-23 | 成都纵横自动化技术股份有限公司 | 一种图数同步传输方法和电子设备 |
CN111294648A (zh) * | 2020-02-20 | 2020-06-16 | 成都纵横自动化技术股份有限公司 | 一种无人机空地视频传输方法 |
CN111554080A (zh) * | 2020-07-13 | 2020-08-18 | 成都纵横自动化技术股份有限公司 | 一种光电吊舱数据采集系统 |
Also Published As
Publication number | Publication date |
---|---|
CN115731633A (zh) | 2023-03-03 |
KR20240051136A (ko) | 2024-04-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9639935B1 (en) | Apparatus and methods for camera alignment model calibration | |
US20110007150A1 (en) | Extraction of Real World Positional Information from Video | |
WO2023029551A1 (zh) | 一种基于多无人机的图像拼接方法及其系统 | |
AU2018230039B2 (en) | Real time frame alignment in video data | |
US11363214B2 (en) | Local exposure compensation | |
CN111695541A (zh) | 基于机器视觉的无人机森林防火系统及方法 | |
WO2019118319A1 (en) | High dynamic range processing on spherical images | |
CN112313596A (zh) | 基于飞行器的巡检方法、设备及存储介质 | |
CN110710199B (zh) | 智能摄像机、图像处理装置以及数据通信方法 | |
WO2023029567A1 (zh) | 一种传感器采集的多种数据的可视化方法及其系统 | |
CN108141564B (zh) | 用于视频广播的系统和方法 | |
JP5864371B2 (ja) | 静止画自動生成システム、静止画自動生成システムにおける作業者用情報処理端末及び指示者用情報処理端末、及び判定装置 | |
WO2023029588A1 (zh) | 一种应用于gis的动态视频呈现方法及其系统 | |
WO2023029712A1 (zh) | 一种数据传输、解析方法及数据传输系统 | |
CN114326764A (zh) | 基于Rtmp传输的智慧林业无人机定点直播方法及无人机系统 | |
AU2018230040B2 (en) | Symbology encoding in video data | |
Yun et al. | Edge Media Server for Real-Time 4K Video Streaming with Multiple 5G-Enabled Drones | |
JP2017220728A (ja) | 検証システム、情報処理装置、検証方法 | |
CN116820118A (zh) | 一种数据处理方法、装置、电子设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22862710 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20247006129 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022862710 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2022862710 Country of ref document: EP Effective date: 20240402 |